Summary
ASPEN's aspen and CARLISLE's carlisle wrapper scripts each independently
define a nearly-identical set of bash functions for structured logging and
pipeline state tracking (log_info/log_step/log_ok/log_warn/log_error/
log_next/log_divider, json_escape, print_versions,
write_pipeline_state_marker, run_jobby_best_effort, and (after
CCBR/ASPEN#119 lands) a _progress_monitor background-loop function). This is
duplicated code that will drift over time.
Both wrappers already depend on ccbr_tools at runtime (via
module load ccbrpipeliner) for jobby. ccbr_tools's pyproject.toml
already has a [tool.setuptools] script-files mechanism that installs
arbitrary .sh/.bash/.R/.py scripts directly onto PATH on
pip install — this is the natural place to host a shared shell library for
exactly this kind of cross-pipeline utility code.
Proposed work
- Add a new shared shell library, e.g.
scripts/ccbr_pipeline_logging.sh,
containing generic/parameterized versions of the functions above.
write_pipeline_state_marker and run_jobby_best_effort must be
parameterized (env vars or args) for PIPELINE_NAME, PIPELINE_VERSION,
and the snakemake log path — ASPEN keeps snakemake.log at the workdir
root, CARLISLE keeps it under logs/snakemake.log. Do not hardcode
either convention.
- Add the new file to
[tool.setuptools] script-files in pyproject.toml so
it's installed onto PATH when ccbr_tools/ccbrpipeliner is loaded.
- Also include a generic
rescript/syncscripts helper (re-sync
workflow/scripts into an existing workdir's scripts/ folder) — this only
needs $PIPELINE_HOME/$WORKDIR, no pipeline-specific parameterization.
- Document a minimum required
ccbr_tools/ccbrpipeliner module version
contract for consumers of this library.
Related
⚡ Generated using AI ⚡
Summary
ASPEN's
aspenand CARLISLE'scarlislewrapper scripts each independentlydefine a nearly-identical set of bash functions for structured logging and
pipeline state tracking (
log_info/log_step/log_ok/log_warn/log_error/log_next/log_divider,json_escape,print_versions,write_pipeline_state_marker,run_jobby_best_effort, and (afterCCBR/ASPEN#119 lands) a
_progress_monitorbackground-loop function). This isduplicated code that will drift over time.
Both wrappers already depend on
ccbr_toolsat runtime (viamodule load ccbrpipeliner) forjobby.ccbr_tools'spyproject.tomlalready has a
[tool.setuptools] script-filesmechanism that installsarbitrary
.sh/.bash/.R/.pyscripts directly ontoPATHonpip install— this is the natural place to host a shared shell library forexactly this kind of cross-pipeline utility code.
Proposed work
scripts/ccbr_pipeline_logging.sh,containing generic/parameterized versions of the functions above.
write_pipeline_state_markerandrun_jobby_best_effortmust beparameterized (env vars or args) for
PIPELINE_NAME,PIPELINE_VERSION,and the snakemake log path — ASPEN keeps
snakemake.logat the workdirroot, CARLISLE keeps it under
logs/snakemake.log. Do not hardcodeeither convention.
[tool.setuptools] script-filesinpyproject.tomlsoit's installed onto
PATHwhenccbr_tools/ccbrpipelineris loaded.rescript/syncscriptshelper (re-syncworkflow/scriptsinto an existing workdir'sscripts/folder) — this onlyneeds
$PIPELINE_HOME/$WORKDIR, no pipeline-specific parameterization.ccbr_tools/ccbrpipelinermodule versioncontract for consumers of this library.
Related
published).
JSON into ASPEN) landing first, so the extracted library targets a stable
implementation rather than a moving one.
⚡ Generated using AI ⚡