Repository navigation

Updates to pipeline & tools for 2026 - #74

Draft

mihow wants to merge 22 commits into

mainfrom

melabbas/pipeline-and-tools

Commits on Jan 20, 2026
- Initial plan
  Copilot
  committed
  93cab14
- Migrate from Poetry to uv: Convert pyproject.toml to PEP 621 format and update README
  
  Copilot
  and
  mihow
  committed
  ef5e263
- Co-authored-by: mihow <158175+mihow@users.noreply.github.com>
  Copilot
  committed
  8b741f5
- Add GitHub Actions CI workflow for uv-based testing
  
  Copilot
  and
  mihow
  committed
  76e46e3
- Update Python requirement to >=3.10, simplify CI to test only Python 3.12, and add research extras testing
  
  Copilot
  and
  mihow
  committed
  f43ff1c
Commits on Mar 8, 2026
- chore: exclude data, plans, and SLURM log files from version control
  
  mohamedelabbas1996
  and
  claude
  committed
  d866870
- config: add GBIF credentials env vars and fiftyone optional dependency
  
  mohamedelabbas1996
  and
  claude
  committed
  e3e4239
- chore: update lockfiles after adding fiftyone optional dependency
  
  mohamedelabbas1996
  and
  claude
  committed
  3bde89c
- chore: standardize SLURM headers and venv activation in existing job scripts
  
  mohamedelabbas1996
  and
  claude
  committed
  b6b87e3
- feat: include verbatimScientificName in DwCA data loading
  
  mohamedelabbas1996
  and
  claude
  committed
  6cf3004
- feat: allow clean-dataset to run without prior verification step
  
  mohamedelabbas1996
  and
  claude
  committed
  5ea3c7c
- feat: add download-gbif module and CLI command for GBIF DwCA downloads
  
  mohamedelabbas1996
  and
  claude
  committed
  ca59db2
- feat: add fetch-and-pack module for quota-safe chunked webdataset creation
  
  mohamedelabbas1996
  and
  claude
  committed
  927e923
- feat: add FiftyOne webdataset visualization module
  
  mohamedelabbas1996
  and
  claude
  committed
  94e849e
- feat: add Vermont species pipeline SLURM scripts
  
  mohamedelabbas1996
  and
  claude
  committed
  dbd2ee3
- feat: add GBIF download and DwCA inspection job scripts
  
  mohamedelabbas1996
  and
  claude
  committed
  d567949
- feat: add FiftyOne visualization job scripts
  
  mohamedelabbas1996
  and
  claude
  committed
  44ff278
- feat: add fetch-and-pack chunked pipeline and test scripts
  
  mohamedelabbas1996
  and
  claude
  committed
  9e35b9a
- chore: add archived and legacy pipeline scripts
  
  mohamedelabbas1996
  and
  claude
  committed
  6ebcd6f
- feat: add webdataset inspection script with class distribution stats
  mohamedelabbas1996
  committed
  dd146e1
Commits on Jun 4, 2026
- Merge remote-tracking branch 'origin/main' into melabbas/pipeline-and-tools
  mihow
  committed
  d03746d
- test: compare command help against whitespace-normalized output
  
  mihow
  and
  claude
  committed
  230987e

Navigation Menu

Repository navigation

Updates to pipeline & tools for 2026 - #74

Updates to pipeline & tools for 2026#74
mihow wants to merge 22 commits into
mainfrom
melabbas/pipeline-and-tools

Commits on Jan 20, 2026

Initial plan

Migrate from Poetry to uv: Convert pyproject.toml to PEP 621 format and update README

Co-authored-by: mihow <158175+mihow@users.noreply.github.com>

Add GitHub Actions CI workflow for uv-based testing

Update Python requirement to >=3.10, simplify CI to test only Python 3.12, and add research extras testing

Commits on Mar 8, 2026

chore: exclude data, plans, and SLURM log files from version control

config: add GBIF credentials env vars and fiftyone optional dependency

chore: update lockfiles after adding fiftyone optional dependency

chore: standardize SLURM headers and venv activation in existing job scripts

feat: include verbatimScientificName in DwCA data loading

feat: allow clean-dataset to run without prior verification step

feat: add download-gbif module and CLI command for GBIF DwCA downloads

feat: add fetch-and-pack module for quota-safe chunked webdataset creation

feat: add FiftyOne webdataset visualization module

feat: add Vermont species pipeline SLURM scripts

feat: add GBIF download and DwCA inspection job scripts

feat: add FiftyOne visualization job scripts

feat: add fetch-and-pack chunked pipeline and test scripts

chore: add archived and legacy pipeline scripts

feat: add webdataset inspection script with class distribution stats

Commits on Jun 4, 2026

Merge remote-tracking branch 'origin/main' into melabbas/pipeline-and-tools

test: compare command help against whitespace-normalized output