Skip to content

Add Initial SIST Regression Test Suite - #32

Merged
harryswift01 merged 5 commits into
mainfrom
16-feature-create-the-sist-regression-suite
Aug 7, 2026
Merged

Add Initial SIST Regression Test Suite#32
harryswift01 merged 5 commits into
mainfrom
16-feature-create-the-sist-regression-suite

Conversation

@harryswift01

Copy link
Copy Markdown
Member

Summary

This PR adds the initial SIST regression-testing infrastructure to preserve the agreed historical scientific behaviour while future maintenance and modernisation work is carried out.

The suite rebuilds the SIST C++ executables from source, runs the documented pBR322 competition workflow, and compares the generated scientific results against the recorded baseline outputs.

Changes

Regression test suite:

  • Added end-to-end pytest coverage for the documented pBR322 competition workflow.
  • Added CLI smoke tests to confirm the command completes successfully and produces the expected output file and sections.
  • Added numerical regression tests covering all 4,361 sequence positions.
  • Added an isolated temporary repository copy so the tests rebuild and execute SIST without modifying the developer checkout.
  • Registered regression and slow pytest markers.

Baseline data and provenance:

  • Added the pbr322.toy.fa regression input dataset.
  • Added reference output generated using the inherited SIST executable.
  • Added reference output generated after rebuilding the executable from the repository source.
  • Documented the relationship between the inherited and source-built outputs.
  • Confirmed that all scientific values shared by both outputs match exactly, excluding the variable runtime field.
  • Preserved the additional reporting fields present only in the inherited executable for provenance.

Project and CI configuration:

  • Added project metadata and optional testing and documentation dependencies through pyproject.toml.
  • Set the initial maintained project version to 0.0.1.
  • Added pull-request CI for regression tests and documentation builds.
  • Added a scheduled weekly regression workflow.
  • Added failure-artifact uploads for regression test output.
  • Updated .gitignore for Python caches, test artefacts and local build directories.

Impact

  • Establishes a reproducible scientific baseline before changes are made to the legacy implementation.
  • Detects unexpected changes to SIST command-line behaviour and numerical results.
  • Verifies that the C++ executables can be rebuilt from the tracked source during CI.
  • Protects the existing documentation by building it for every pull request.
  • Provides a foundation for adding more datasets, transition modes and supported platforms later.
  • Does not intentionally change the existing SIST scientific calculations or command-line behaviour.

@harryswift01 harryswift01 self-assigned this Aug 5, 2026
@harryswift01 harryswift01 linked an issue Aug 5, 2026 that may be closed by this pull request
6 tasks
@harryswift01
harryswift01 requested a review from jimboid August 6, 2026 14:51

@jimboid jimboid left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Introducing the framework for running regression testing in CI inline with best practices. This is a foundational PR to introduce the building blocks upon which further additions to the code will be built. Specifically here adding regression tests that will ensure the integrity of the original code is maintained, project configurations to enable distribution of codes via conda.

@harryswift01
harryswift01 merged commit 984a25b into main Aug 7, 2026
2 checks passed
@harryswift01
harryswift01 deleted the 16-feature-create-the-sist-regression-suite branch August 7, 2026 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Create the SIST 1.0.0 regression suite

2 participants