Skip to content

test(cocalc): adversarial checks for backup --restored-before, in CI - #16

Merged
wstein-agents merged 2 commits into
cocalc/mainfrom
cocalc/tests-b6-restored
Nov 1, 2026
Merged

wstein-agents merged 2 commits into
cocalc/mainfrom
cocalc/tests-b6-restored

Conversation

@wstein-agents

Copy link
Copy Markdown

Adds tests/cocalc/b6_adversary.py and runs it in CoCalc CI. It checks backup --restored-before TIME --parent ID adversarially.

Oracle: restoring the new snapshot must give exactly the tree that was on disk when the backup ran. Each scenario prepares a restored tree, backs it up with the option and again without it (the control), then compares both restores with the disk.

  • In contract (must not diverge):
    • baseline;
    • writes shortly after TIME, and right after it (inside the 2 s margin);
    • an interrupted restore that is then rerun;
    • links and special files;
    • chmod passes, before and after TIME.
  • Out of contract (reported, expected to diverge): a non-empty restore target holding stale files, the wrong --parent, and a killed restore that was not rerun.
  • Hard checks:
    • a future --restored-before is rejected;
    • no control backup ever diverges;
    • no in-contract scenario diverges.
  • Negative controls:
    • the baseline must actually reuse files, otherwise the option did nothing and the passes prove nothing;
    • stale_nonempty_target and wrong_parent must show a divergence, which proves the oracle can see stale reuse;
    • a binary without the option exits 2.

Local run against the published v0.11.4-cocalc.4 (x86_64 binary sha256 7d5c3020…4414): exit 0, no problems. In-contract scenarios reused 18–70 files with 0 divergent; the two out-of-contract cases each have 1 divergent file; the future TIME was rejected. With v0.11.4-cocalc.3 the script exits 2 ("binary has no --restored-before").

rustic-tests added 2 commits November 1, 2026 09:52
…ative controls

The baseline must reuse files (else the option did nothing), and the two
out-of-contract scenarios must show the divergence the oracle exists to see.
@wstein-agents
wstein-agents merged commit e5fa0ac into cocalc/main Nov 1, 2026
25 checks passed

Copy link
Copy Markdown
Author

Merged at the exact approved head 42848b9d60fcc1be9be3f8bb234f5d160d92955f (merge commit e5fa0aca18d2f179f3eec61aa71dd90cf225b449).

  • Merge bar for this fork (William, 10-31): "i think it is fine to merge with @review approval because that is just codex."
  • @review: APPROVE at exact head 42848b9d. That covers the full diff against cocalc/main 7396f0e2, and @review independently checked that the harness passed in all six jobs.
  • CI: all required checks green at the head (run 41333856278: 24 pass, 1 skipped).
  • Two documentation nits (the docstring's exit summary, and the wording in s_write_after_time_no_margin) follow in a small PR from @rustic-tests.

Owner: @ops-4 on alpha.cocalc.ai, project ops (c3337130)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant