Skip to content

fix(launcher-data): put back or sweep what interrupted saves leave, by one rule - #257

Open
Furroxide wants to merge 1 commit into
devfrom
fix/launcher-atomic-write-leftovers
Open

Furroxide wants to merge 1 commit into
devfrom
fix/launcher-atomic-write-leftovers

Conversation

@Furroxide

Copy link
Copy Markdown
Owner

Outcome

#226 gave the launcher's own files one recovery for interrupted saves and left three follow-ups. This closes all
three. Every file the launcher and the CLI save now goes through one writer, with one naming rule for what a stopped
save leaves behind, one recovery and one sweep.

1. The developer tools no longer put back or delete an author's <file>.bak

topiaforge new mod, restore, pack, mod add|remove, and the VPM and world tools save through the developer
repository. Its writer always moved the old file aside as <file>.bak, and its recovery ran before every read of
every file it read: project files, .csproj files, README.md, Unity package manifests, templates.

  • A file there, with <file>.bak beside it: the .bak was deleted, on a read alone. An author's own
    MyMod.csproj.bak was gone after any command that read MyMod.csproj.
  • A file missing, with <file>.bak beside it: the .bak was put back as the file.
  • Every save: the old file replaced an existing <file>.bak, which was then deleted.

The developer tools now save through the launcher's writer, moved to lib/src/atomic_file_writes.dart. A backup is
named <file>.<process id>-<microseconds>.bak, as the launcher names it, and #226's recovery, moved to
lib/src/atomic_files.dart with its rules unchanged, puts it back. A <file>.bak is never deleted.

  • Backups from earlier developer tools stay readable. When an older CLI's save stopped between its two moves, it
    left <file>.bak and, beside it, its new version as <file>.<process id>-<microseconds>.tmp, because it stopped
    before moving that file in. So a missing file with a <file>.bak and such a temporary file beside it is put back,
    as the old recovery did. A <file>.bak without one is the author's own and is left alone. The launcher's own files
    never read a <file>.bak, as before.
  • The writer now replaces the file in one move first, as the launcher's writer does, and moves the file aside
    only when that move fails. A developer save on Linux or macOS therefore never leaves the file missing, which fix(launcher-data): harden interrupted-save recovery for launcher files #226's
    put-back relies on there.
  • The project search still walks up from the folder it is given and looks for an interrupted save in each folder
    on the way. A folder above the project that cannot be listed is passed over, as before, rather than refused.

2. The launcher log and the game-compatibility cache

Neither one holds player state.

  • The log. A rotation stopped part way left the log moved aside as a backup, and the next line began a new log,
    so the old one was orphaned. The log is now put back before it is read, appended to or added to a diagnostic
    bundle. Once per launcher or CLI process, old leftovers beside a log that is there are swept. A backup that another
    program holds never stops logging: the log goes on in a new file, and the backup waits for a later look.
  • The cache (<game>/BepInEx/TopiaForge/compat-status.json) is put back the same way before it is read, and old
    leftovers beside it are swept. Its verdict is still checked against the hash of GameCode.dll.

3. Temporary files are swept

A save that stopped after writing its <file>.<token>.tmp left that file forever. Every save now first sweeps the
old temporary files of its own file, under these limits:

  • only names of exactly that form, for that file;
  • only regular files: a link is never followed or deleted, and neither is a folder;
  • never a file that a save in this process is writing;
  • only once the save that made it began an hour ago, by its token, and the file has not changed for an hour, by its
    modification time, so a save still running in another process keeps its own. That is the same bound as the
    multiplayer store's sweep (feat(launcher-data): add the multiplayer network clients and store #214);
  • a leftover that another program holds is left for a later sweep, and the save goes on.

docs/Troubleshooting.md gains a section, "Files beside a saved file".

Deviations from the plan

Compatibility and security

  • Public API/schema/save/package impact: nothing exported changes. The new internal libraries are
    atomic_files.dart and atomic_file_writes.dart, and local_launcher_repository/atomic_backup_recovery.dart
    has moved into them. Developer backups are now named <file>.<token>.bak. No file format changes.
  • Filesystem/network/archive/process or credential impact:
    • A save now lists its file's folder once, to sweep. The sweep deletes only the writer's own names, only regular
      files, and only after an hour.
    • The developer tools no longer delete or replace <file>.bak.
    • A held developer backup now reads as <file name> could not be read just now, so nothing was changed., with
      fix(launcher-data): harden interrupted-save recovery for launcher files #226's cause and next step. Before, an unreadable backup gave an error naming its full path.
    • Nothing new is logged.
  • Migration or rollback: none is needed, since an older CLI's interrupted saves are still recovered. A revert
    restores the old behaviour, including the deletion of <file>.bak.

Verification

On a Windows 11 x64 machine, on dev at 22d68d1 and then rebased onto d89f295 (its #248 changes no Dart file), with the pinned Flutter 3.44.6 and its Dart 3.12.2:

  • New tests, 18, plus one changed:
    • test/atomic_write_sweep_test.dart, 7:
      • the naming rule, for .bak and .tmp alike;
      • the sweep: only its own old temporary files, and backups only where asked;
      • a link named like a leftover is neither followed nor deleted;
      • both writers sweep before they save;
      • a held leftover waits, and the save goes ahead (Windows);
      • the temporary file of a save in this process is never swept, however old it looks.
    • test/atomic_write_leftovers_test.dart, 11. Developer files: an author's <file>.bak kept through reads and
      saves, and never put back; the newest of several backups put back; a held backup refused with nothing changed;
      the project search passing over a folder it cannot list (POSIX); a save that leaves no leftovers. The log: never
      reading the launcher's own <file>.bak; a stopped rotation put back; old leftovers swept and recent ones kept;
      a held backup never stopping logging (Windows). The cache: put back and swept.
    • developer_persistence_hardening_test.dart's recovery test now builds what an older CLI's interrupted save
      really leaves, the .tmp included.
  • Proof on dev's code: with dev's lib, 8 of the 10 integration tests that ran on Windows fail. The other
    two are guards. dev never touched the log's backup, and never read a <file>.bak for the launcher's own files;
    both must stay that way now that the log is put back and the developer files read an older CLI's backup.
  • Linux, Ubuntu 24.04 under WSL2, as the unprivileged nobody account so folder modes hold: the new files,
    developer_persistence_hardening_test.dart, and fix(launcher-data): harden interrupted-save recovery for launcher files #226's recovery, race, manager-state and persistence tests:
    45 pass, and 3 skipped (the Windows-only held-file tests).
  • Mutations, 17 of 17 caught, each restored; 15 on Windows, and the last 2 under WSL as nobody:
    • the naming rule: a token without its clock reading counts; another file's leftover counts; a backup counts as
      a temporary file and the other way round; a <file>.bak counts without the temporary file beside it; the
      launcher's own files read a <file>.bak;
    • the sweep: the temporary file of a running save swept; a leftover that changed recently swept; a save that
      began recently loses its leftover; a link followed and deleted;
    • the log: its backup never put back; old backups never swept;
    • the cache: never put back; never swept;
    • the developer tools: an older CLI's backup no longer read; the project search stopping at a folder it cannot
      list;
    • the writers: the developer tools' saves sweep nothing; the launcher's saves sweep nothing.
  • packages/launcher_data: 1,064 pass, 10 skipped. dart analyze --fatal-infos and
    dart format --set-exit-if-changed: clean. Other programs kept this machine heavily loaded. Under that load, two
    earlier full runs timed out at 30 s on the known flaky doctor test, "doctor reports an exact-SDK resolution
    failure as blocking". That test takes 14 to 19 s alone, on this branch and on dev alike; fix(launcher-data): stop losing LAN queries, and make the LAN and doctor tests deterministic #250 makes it
    independent of the machine's speed.
  • apps/topiaforge_cli: 514 pass, 4 skipped. The Release build is in place (dotnet build TopiaForge.slnx -c Release -warnaserror, 0 warnings, on the same C# tree). Under the same load, the default 30 s limit timed out
    55 tests, nearly all of them release tests that never touch launcher_data. With --timeout 4x, 511 passed. The
    last three passed in a rerun without a time limit, 10 of 10 in their files. One of the three is the template
    lifecycle test, which takes every template through new, restore and pack.
  • python -I .github/scripts/check_topiaforge_residue.py and check_readme_counts.py: pass. Line-count audit: no
    non-generated Dart file over 500 lines; the largest new file has 320.

Checklist

  • I read AGENTS.md and preserved CLEAN/Bloc/Unity/TopiaForge UI boundaries.
  • I added focused regression coverage for meaningful defects.
  • I ran the relevant build, analyze, test, format, package, and line-count gates.
  • I added provenance, license text, and modification notes for new third-party material. (None added.)
  • I did not include secrets, proprietary game assemblies, generated build directories, or unrelated changes.

…y one rule

The developer tools saved through their own writer, which moved the old
file aside as <file>.bak, and recovered before every read: a read
deleted an author's own <file>.bak beside a file, a missing file got
that .bak put back in its place, and every save replaced it. They now
save through the launcher's writer and recover through #226's rules,
moved into shared libraries: only <file>.<pid>-<micros>.bak counts. A
<file>.bak left by an older CLI's interrupted save is still put back,
recognised by the .tmp that save always leaves beside it.

The launcher log and the game-compatibility cache, which hold no player
state, are put back after a stopped save and have old leftovers swept.
Every save first sweeps its own file's old temporary files: only the
writer's names, only regular files, never one a save in this process is
writing, and only once both its token and its last change are an hour
old.

Signed-off-by: Furroxide <221987073+Furroxide@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown

Unity PR artifacts are ready.

Artifact Link
VPM package listing Download artifact

Workflow run: Unity Source Validation #674
Commit: 4486780

Artifacts are temporary and expire according to the workflow retention policy.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant