Skip to content

Separate host-share response drainage from invalidation acknowledgement timing - #112

Draft
mrchypark wants to merge 1 commit into
Augani:mainfrom
mrchypark:fix/hostshare-invalidation-drain-budget
Draft

mrchypark wants to merge 1 commit into
Augani:mainfrom
mrchypark:fix/hostshare-invalidation-drain-budget

Conversation

@mrchypark

Copy link
Copy Markdown

A host-share invalidation currently spends the same short deadline waiting for active filesystem responses and waiting for guest notification acknowledgements. An already-admitted host operation lasting more than that deadline can trigger VM-fatal recovery before notification submission. This occurred during an installed data-image workload; the exact slow operation and inode were not identified.

Give initial response drainage a separate bounded 30-second recovery policy. After that drain, start one short acknowledgement deadline for the whole notification prefix. Preserve the existing global response barrier, opcode admission, total-active reset accounting and WRITE publication fence. Later chunk barriers consume the same remaining acknowledgement deadline rather than receiving another 30 seconds. Keep cancellation and reset fail-closed, reject submissions crossing a notification transport reset, and align enclosing reverse-exchange/activation-request budgets.

The global barrier is deliberate: an experimental finite-request-set approach allowed a READ payload and LOOKUP identity captured while an older physical WRITE was pending to publish after acknowledgement. The conservative barrier requires crossing responses to publish before notification. No admission phases or descriptor bypass are introduced.

Source validation:

  • Actual latest-main sources compiled in an isolated package: 45 VirtioFS/frontend tests passed, followed by four additional checks for blocking unlock through the global drain, two existing server lock/interrupt methods, and nested timing budgets. The two server methods and timing method were extracted with supporting fixtures; this is not a full original package test pass.
  • Coverage includes physical WRITE-before-syscall blocking, crossing READ/LOOKUP ordering and fresh post-ACK data/identity, successful ACK after a 1.2-second held WRITE, independent missing-ACK expiry, one deadline across chunks, drain expiry, cancellation, reset, dirty-WRITE fencing and queue/publication regressions.
  • Local actual sources compiled in an isolated package. Final ordering/chunk/lock checks passed; all six budget/cancellation/reset cases passed after correcting a test assertion about preserved used-ring bytes after reset. The production 30-second fail-closed drain test passed in an earlier local run.

Installed validation limits:

The combined installed candidate also contained the independently proposed root/lifecycle change and local signing/build integration. Two ordinary GUI cycles passed stable checks after initial API timeouts. One clean stop/restart retained selected Docker/PVC payload hashes. The exact original native Linux e2image workload then completed without the earlier coherence crash, and the helper remained alive. Boot full e2fsck exited 0; data full e2fsck exited 4 for a disconnected directory inode. The checker exited 4 and was not OOM-killed.

This demonstrates completion of the formerly interrupted workload in one combined-candidate attempt, not complete filesystem acceptance or independent standalone-branch runtime qualification. Zero complete engine/filesystem acceptance cycles passed. The inode inconsistency exists in the frozen input as well as the checker copy. Its timestamps correlate with an earlier crash, but object age does not establish when disconnection occurred and the cause remains unproven; no completed earlier full data fsck establishes a clean baseline.

Thirty seconds is a recovery policy, not a measured host-I/O latency guarantee. Continuous non-WRITE arrivals may still exhaust the global drain budget; fair-queue nonstarvation is unresolved. Opcode admission remains unchanged; the server interrupt-cancellation test is not a new guarantee that interrupts pass a closed publication gate.

Full-package verification remains incomplete. Earlier filtered attempts failed before tests in unchanged MachineManager code; no clean upstream-main compile was run, so baseline causality is not established. An isolated build of the entire unchanged server test file also hit an unrelated expression typechecking limit; only its two relevant lock methods were extracted for the focused run.

Submitted as a draft for source review. The combined installed Dory candidate did not pass complete engine/filesystem acceptance; the data fsck failure and its cause remain unresolved. No further Dory runtime qualification or repair is claimed.

Companion: #111 — independent daemon-startup/whole-VM-shutdown proposal. Source files do not overlap; installed observations above apply to the combined candidate, not independent qualification of either branch.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant