Skip to content

Plan lab workflow reliability and scientific comparison #91

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
MrScripty wants to merge 1 commit into from
Closed

Plan lab workflow reliability and scientific comparison #91

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions docs/plans/lab-workflow-reliability/execution-ledger.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Lab workflow reliability execution ledger

## 2026-10-19 — Plan preparation

- Parent requested a standards-compliant implementation plan only, with a
separate documentation branch/worktree, independent review, validation and
draft publication. No product implementation or merge is authorized here.
- Inspected local status/worktrees, repository tree, current main and PR89/90.
Main is `300f12df6f48e028fd9379911703f7c89337952a`; PR89/90 remain open at
`33069a859836953960a06dfd6001cba54aba0aa0` /
`429d0c4ff44dba16bc95313d2a0980169dbc9b9f`.
- Created `docs/lab-workflow-reliability-plan` at main in the separate
`/workspace/Tuldok-lab-workflow-plan` worktree. Existing worktrees and refs
were preserved. Only this plan directory is an allowed write set.
- Coordinated with the integration lead in
[PR90 comment](https://github.com/MrScripty/Tuldok/pull/90#issuecomment-6272255247).
Lead retains PR89/90 and all integration/merge authority.
- Inspected completed D1, current material/preflight/publication/CLI/evidence/
lineage ownership and PR90's RES-1. Wrote six implementation milestones;
no new design-discovery phase or product worker was started.
- Read adopted Coding Standards at `dcc56f26e884ade260770beceba2501d3746200d`.
The available skill is standards-authoring-specific and explicitly excludes
ordinary application work. No repository AGENTS.md/local skills were found.
Application-purpose Engine invocation is unavailable; exact source structural
validation and manual review are recorded separately, without Engine certification.
- Independent review and actual validation results are recorded in
[reports](reports/validation.md). All product claims A1–A8 remain pending.
- Independent GPT-6.1 Sol High review found one blocking draft gap: legacy
completed-state gates made recovered evidence inaccessible. Revised C2/M1/T2
with an exact attached publication-result Ref/new readers and explicit legacy
failed/promotion-refusal semantics. Also made scientific per-case criteria and
recovery operation budgets explicit. Focused re-review cleared all findings;
[the exact subject hashes and limits](reports/independent-plan-review.md) remain
distinct from future implementation acceptance.
- A disconnect notice was investigated: actual commands and source reads
succeeded, all draft hashes were intact, and no work was lost. Reported this
to lead/parent in [PR90](https://github.com/MrScripty/Tuldok/pull/90#issuecomment-6272545966).
- Final plan-creation validation passed: unchanged upstream structural checker,
all23 relative links/anchors, new-file whitespace and all8 paths confined to
the new documentation directory. No application/runtime/catalog test ran.
The coherent documentation commit contains the plan, exact contract/test
handoff, issues, ledger and review/validation evidence. The draft PR identifies
its final Git SHA; branch/worktree remain retained for lead review, with no merge.
15 changes: 15 additions & 0 deletions docs/plans/lab-workflow-reliability/issues.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Lab workflow reliability issues

| ID | Severity / evidence | Objective and owner | Disposition / verification / revisit |
| --- | --- | --- | --- |
| RES-1 | High availability/safety boundary; PR90 retained findings describe advisory-lock timeout terminating `_loop`, while `_start` retains a dead assigned thread. | A3; Runs coordinator owner. | Fix in M2 after integration baseline is verified. Inject actual PostgreSQL lock timeout and active custody failure; prove unhealthy/readiness fence, no duplicate coordinator or GO, safe drainage/reconciliation. Never count an exited coordinator as idle performance. |
| PUB-1 | High lost-work cost; lab reports completed training with diagnostic evidence beyond selected shard bounds. Exact private failure is not present here. | A1/A2; evidence profile and Runs publication owners. | M1 classifies one preserved failure before changing profiles. Bounded indexed producer output plus explicit same-Run publication continuation; invalid historical outputs stay invalid. Re-plan if mandatory data cannot fit admitted limits. |
| DEP-1 | Integration dependency; PR89/90 are open at the recorded SHAs, not merged. | All milestones; repository lead. | Verify current merged baseline/CI and remove duplicate tasks before implementation. Plan author changes neither branch. Revisit on any semantic dependency change. |
| PERF-2 | Measurement need; historical artifact timings are single-workload observations, preflight/restore still costly. | A5; performance owner. | M3 exposes phases; M4 measures cold/warm/history-size costs before retaining cache or batch machinery. No invented target/general speedup. |
| SCI-1 | Missing actual compatible baseline/candidate pair and fresh CUDA/scientific evidence. | A7 and any hardware-specific claim; lab/evidence owner. | Deterministic compatible/mismatch UI fixtures may proceed, labeled correctly. Lab-specific acceptance stays unavailable until exact artifacts, case/split/units/metrics and hardware procedure exist. No training rerun is authorized by plan creation. |
| TOOL-1 | Standards tooling limitation; no MCP tools exposed; application-purpose invocation returns unavailable. | Plan validation; documentation author. | Use exact checked-in structural validator and manual policy/contract review, retaining tool outcome. Do not claim full Engine or application compliance certification. Revisit if a supported application connection becomes available. |
| REVIEW-R1 | Blocking draft-contract gap; existing artifact/evidence/candidate readers require completed Run state. | Recovery usability/compatibility; Runs and evidence/selection owners. | Resolved in the reviewed plan: exact versioned attached result Ref for new readers, old failed projections unchanged, candidate/package promotion and new recovered-input bindings unsupported initially; T2/T7 cover all dispositions. Product work remains pending. |
| REVIEW-R2 | Review recommendation: scientific per-case pass/fail was not explicit beside training pass-counts. | Scientific interpretation; criterion/evidence owner. | Resolved in C6/T7 with owned criterion Ref, units/threshold/direction, qualified/diagnostic distinction and pass/fail/unknown; no invented criterion. |
| REVIEW-R3 | Review recommendation: publication recovery needed explicit independent operation bounds. | Resources/recovery; Runs owner. | Resolved in C2/T2 with finite wall/I/O/copy/concurrency/history limits, paused exhaustion and unchanged original compute envelope. Implementation must verify those proposed limits. |

No unrelated product issue is silently repaired by this documentation branch.
Loading
Loading