Skip to content

[content] Life Sciences guide: scheduled Globus transfers to Data Depot - #285

Merged
Guangzhen-Jin merged 5 commits into
mainfrom
feature/globus-scheduled-transfers
Sep 11, 2026
Merged

Guangzhen-Jin merged 5 commits into
mainfrom
feature/globus-scheduled-transfers

Conversation

@aseetharam

Copy link
Copy Markdown
Collaborator

Draft for review — not ready to merge.

New Life Sciences guide on scheduling recurring Globus transfers from cluster scratch to Data Depot, so long-running projects survive the scratch purge. Written for the DeWoody lab consultation but applies generally; this question comes up repeatedly.

Review request

@rwilfong — this is your area. Four things I could not resolve on my own, marked [NEEDS CLARIFICATION] in the page:

  1. Collection names. docs/snippets/globus.md and docs/userguides/depot/storage/transfer.md disagree. What is the correct scratch collection name per cluster, and is the Depot name current?
  2. Consent persistence. Are the Purdue scratch and Depot collections GCSv5 with persistent consent? This decides whether the re-authentication advice is troubleshooting or a planned step, and it is the difference between a timer that survives a six-month project and one that quietly stops.
  3. Depot redundancy wording. I described Depot as redundantly backed up across two sites; the existing user guide says redundant arrays in multiple campus datacenters and is explicit that snapshots are not backups. Which framing do you want?
  4. Web app flow. Does anything at Purdue differ from generic Globus documentation — label wording, option placement, an RCAC-specific step?

Please look hardest at the !!! danger box on delete-on-destination. If a reader enables that option on a scratch-to-Depot timer, the next run after a purge deletes their Depot backup while every task reports success. It is the one place on the page where wrong guidance destroys data. I want your eyes on the wording and on whether the option label matches what Purdue users actually see.

Optional: screenshots

Archetype D calls for screenshots, and this page has none. If you are taking screenshots for Tuesday's session anyway, three would help most:

  1. Transfer & Timer Options expanded, showing where Schedule Start and Repeat live
  2. The sync-level selector with the delete-on-destination checkbox visible and unchecked
  3. The Activity > Tasks view of a completed timer run

No pressure if you would rather not. Every screenshot needs meaningful alt text under the WCAG mandate and goes stale when Globus changes its UI, so I would rather have three good ones than a dozen.

Verification

  • mkdocs build --strict passes with no new warnings against the baseline.
  • Nav entry added under Life Sciences > Guides.

Not ready to merge until

  • The four clarifications are resolved and the draft banner is removed.

aseetharam and others added 3 commits August 27, 2026 11:38
First draft of a Life Sciences guide on scheduling recurring scratch-to-Depot
Globus transfers, for researchers whose projects outlast the scratch purge
window. Nothing on the site covered Globus timers; a repo-wide search for
"globus timer", "timer create" and "recurring transfer" returned no hits.

Optimised for review rather than publication: carries a draft banner and five
[NEEDS CLARIFICATION] markers for the Globus owner, covering collection names,
GCSv5 consent persistence, Purdue-specific web-app flow, and a conflict between
the brief and the Depot user guide over whether Depot is "backed up".

Reuses existing coverage by linking rather than restating: Globus basics from
the Depot transfer guide, purge policy and paths consistent with the
scratch_purge / scratch_space snippets, purchasing from the Depot overview.

Nav entry added and breadcrumbs.json regenerated.
@rwilfong

Copy link
Copy Markdown
Collaborator

Hi Arun,

  1. The collection names differ for each cluster. Gilbreth and Bell have separate scratch directories on Globus (e.g., Purdue Bell Cluster - Scratch) while Anvil, Negishi, and Gautschi do not (e.g., Gautschi Cluster Collection).
  2. I believe so. Once authentication is given the first time to access the collections and in an initial transfer, there is persistent consent. Users do not need to authorize further unless they explicitly remove their consent from the clusters. I haven't worked with long-running timers between GCS collections, so let me check with the Globus folks.
  3. I like the redundant storage arrays in multiple campus data centers wording - it seems to be more conservative. Snapshots are not backups but are available for accidental changes or deletions.
  4. Nope. There's no glaring differences that I've come across using the Purdue Globus site vs the main Globus one other than the branding is Purdue branding and it defaults to Purdue authentication.

For the !!! danger box, this is the exact label that users see in both instances of Globus (Purdue-branded and Globus). It comes unchecked by default and should stay that way! I agree with the wording of the warning.

I updated the docs/snippets/globus.md and docs/userguides/depot/storage/transfer.md to reflect the scratch differences for each Globus collection and what each cluster's is.

@rwilfong

Copy link
Copy Markdown
Collaborator

After a discussion with Globus Support, the only way to truly automate a Timer is to use a Globus service account. That would require setting up guest collections, the confidential client, the timer. It gets a little tricky because if anything goes wrong with the timers, the email notification will go to {client_ID}@clients.auth.globus.org rather than their Purdue email, so there's danger of a silent failure.

I think having their Purdue account create the timer with notifications on for inactive/failed transfers is a better solution since this is more short-term. They'll get notified if it's inactive and they can go in and authenticate.

Do you have any thoughts, Arun?

@rwilfong

Copy link
Copy Markdown
Collaborator

@aseetharam I added a few images that show the Timer & Transfer Options including the scheduling start and repeat for the timer, and an example of an established timer.

I left a few [Needs Clarification] that will need to be cleared before publishing. Let me know if there's anything else you'd like me to add!

@aseetharam

Copy link
Copy Markdown
Collaborator Author

Thanks @rwilfong, this covers everything! Agreed on the personal account with notifications. From what I've heard service accounts are headache, so I'll leave that out entirely rather than mention it as an option.

Your point on persistent consent changes that section. I had re-authentication under Troubleshooting as though it were expected. If consent only breaks when someone explicitly revokes it, that's an edge case, so I'm rewriting it that way.

Two small fixes on your commits: the new image paths are relative and Life Sciences moved to absolute (/assets/images/lifesciences/...) under invariant 5, and I'm expanding the alt text to describe what each screenshot shows for the WCAG mandate.

Pushing now and taking it out of draft. The lab session is tomorrow. If Globus comes back with anything different on long-running GCS-to-GCS timers we can amend the page after.

@aseetharam
aseetharam marked this pull request as ready for review August 31, 2026 15:27

@Guangzhen-Jin Guangzhen-Jin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@Guangzhen-Jin
Guangzhen-Jin merged commit 9ddd16a into main Sep 11, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants