World layers: caves, world border, water limit — and game_data reorganisation - #14
Closed
valentinps wants to merge 11 commits into
Closed
World layers: caves, world border, water limit — and game_data reorganisation#14valentinps wants to merge 11 commits into
valentinps wants to merge 11 commits into
Conversation
Nothing in a save (or docs.json) records a cave, so the source is the cooked world-partition export: the game's own cave atmosphere volumes (the fog/lighting regions it swaps you into underground -- 108 of them, some with authored names), the BP_CaveFloor tunnel splines, the cave-only foliage clusters and the placed cave rock kit. extract_caves.py unions all four, closes the gaps between one cave's separately-authored chambers, and traces the outline of each connected component into the committed game_data/sav_data/caves.json (84 caves, 61 KB). The core embeds that table and emits it as a new static payload step, and the frontend draws every ring in one line bucket under a "Caves" sidebar row (default hidden like the other world layers), with the cave's name, area and altitude range in the tooltip. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two invisible edges, both authored level geometry with no representation in a save: - The damaging border is 15 FGDamageOverTimeVolume actors whose DoT class comes from /World/Hazard/WorldPerimeter/. Eleven are vertical walls (four axis-aligned, three rotated blocks cutting the NE/NW/SW corners, four more as an outer backstop); the safe side is a 7-vertex polygon, 7915 x 7049 m. The other four span the whole world in XY and are altitude limits instead: damage above +2000 m and below -244 m. - The water limit is the union of the 270 FGWaterVolume actors (all carry mResourceClass Desc_Water_C, i.e. swimmable and extractor-valid). The rendered ocean is a different thing entirely: 31 water-plane patches, the biggest 51 x 34 km, so the sea you can see runs ~70 km while the water you can touch stops within ~8 km -- on the west side the real water ends 334 m before the damage wall, with visual-only ocean in between. The perimeter polygon is written as an exact half-plane intersection of the wall boxes and cross-checked both ways against an independent rasterize-and- trace of the same volumes (they agree to 40 cm shy of a cell). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds a "Map Limits" sidebar category (default hidden, like the other world layers) with one row per ring: the world border in red, the water limit in cyan, each a line bucket with the facts in its tooltip -- the altitudes the damage slabs start at for the border, and "8.2 km of real water inside a 70 km ocean" for the water. The rings ride along as a new static payload step fed by the embedded worldBounds.json, the same shape as the spawners step. Two things worth knowing for the next static layer: - ring z is 0 (sea level), not the volumes' real -9.6..+10.4 km span: a line's z only drives the altitude filter and depth sorting, so the honest span would just make the tooltip claim the border sits 10 km up. - synthetic ids need a ':' in them. Bulk id arrays get the instance-name prefix re-added client-side (slim_payload_value / expandPayloadIds), which silently turned "worldPerimeter" into "Persistent_Level:PersistentLevel. worldPerimeter" and broke the color lookup keyed off it. The frontend now keys off a separate `kinds` array, and a test pins the ':' rule. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The caves outlines, the damaging world border and the water limit are three answers to the same question -- what is the terrain actually like here? -- and none of them belongs to the save being viewed, so they now share one sidebar category instead of two. Three rows, three colors, one checkbox each; "Map Limits" is gone as a name. Payload steps stay separate (caves + mapLimits), so the collectors and their tests are untouched apart from the step count going to 20.
The old split was game_data/sav_data/ (committed) vs game_data/generated/
(gitignored), which read as "world tables vs docs.json extracts" but was
really "the IP line we drew", and membership had drifted: creatureSpawners
(coordinates, like the committed slugs) was gitignored, half the hand-written
tables sat in sav_data/ and half at game_data/ root, and two generated files
had no consumer at all.
Now:
game_data/curated/ committed hand-maintained inputs only
game_data/generated/docs/ gitignored Docs.json extracts
game_data/generated/world/ gitignored level-export tables
game_data/generated/ gitignored map render + tiles
so a checkout is either extracted (extract_all.py) or unpacked
(package_game_data.py unpack), never half-committed. game_data/README.md
states the rule where the folders are; sav_data/ is gone, and so are
sav_data_gaps.json (no consumers) and the flat generated/*.json layout.
Two things had to be fixed for "nothing generated is committed" to actually
hold, both of which were latent bugs in the old layout:
- freeDroppedItems.json recovered each pickup's item class from its OWN last
output (FModel can't decode the cooked FInventoryItem struct). Delete the
file and all 703 pickups lose their item. That knowledge now lives in the
committed curated/pickupItems.json, an input; --items-from-save writes what
it learns back into it.
- every table's key order was inherited from the previously committed file,
so a fresh machine produced different bytes than an incremental re-run.
Tables are now written sorted: same dump in, byte-identical files out.
Verified by deleting generated/{docs,world}/ and re-running extract_all --
every table came back byte-identical to the pre-reorg ones.
Also: package_game_data packs generated/ recursively so the new subfolders
need no change there, but an archive built before this commit carries no
world/ tables (they used to be committed), so CI now points at game-data-v3
-- publish the rebuilt zip under that tag or CI cannot compile the crate.
sav_core's build script now says which command to run when the data is
missing instead of failing on the first include_str!.
NOTICES/README: the distribution clause said game-derived data is "not
distributed here" while CI downloaded exactly that from a release. Reworded
to say what is true -- none of it is in the repository, and the archive is a
convenience build of your own extraction.
The row now reads "Caves (approximate)", hovering it explains why, and a cave's tooltip carries an "Outline: Approximate" line where someone reading a size off it will see it. The number is honest about its source: an outline is the game's cave fog volume unioned with the cave geometry inside it, which runs generous by tens of meters and can miss a bare tunnel with no cave props in it. That finds a cave; it does not survey one, and the layer should not imply otherwise. row.hint is new on leaf rows -- the long version of a caveat a label can only gesture at, shown as a title on hover.
…atching Three separate frictions in the search bar, all of them cases where the answer was already on screen but out of reach. A collectable item's suggestion row now carries the same show/hide eye a building or layer row has, flipping that pickup's own Collectables rows. Searching "Mercer Sphere" honestly means two things -- what is stashed in inventories, and what is still lying out there -- and only the first had an answer. The eye drives the real sidebar checkboxes rather than a second state of its own, and appears on exactly the six items that have a layer at all: the three slugs, Somersloop, Mercer Sphere, Hard Drive. Every individual location in an item's result list gets a crosshair button that flies the map to that one machine and marks it with a solid pink pin (every other pin on the map is white-filled, so this one cannot be mistaken for scenery). Coordinates told you where a building was; this takes you there. It is deliberately independent of "Show only these on map" -- "where is THIS one" is worth asking without blanking every other layer for context -- and hands over to the same banner, so Details reopens the list exactly where it was. Finally, a query's words may now match anywhere in a label, in any order, so "coal node" finds "Coal Ore (Resource Node)". The parenthesised kind suffix is a real part of those names and exactly how people say them, but no contiguous substring spans it, so the old strict test could never reach past it. Tighter matches still rank first, so "iron" keeps surfacing Iron Plate ahead of Reinforced Iron Plate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The map deliberately stops short of the right edge -- the altitude rail is docked there -- but the panels that float over it are body-level siblings of #map, so their left:50% was the WINDOW's center: half a rail-width (32px) too far right, at every viewport size. On its own that drift is invisible; under the search pill, which is genuinely centered on the map (#topBar reserves the same rail width in its right padding), the filter banner sat visibly off. One --map-center-x for all four of them (filter banner, selection panel, editor toolbar, editor hint), explained once where it's defined. Also moves a group header's "× 28" badge from the right edge to right after the building name: "Constructor × 28" is one phrase and now reads as one. The label gives up its flex stretch and the badge takes over the gap-filling, so the count column stays exactly where it was -- aligned with every non-grouped row's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Search: collectable layer toggles, per-location "Show on map", word-order matching
A planning tool for "where should the belts/trains/power actually run".
Reachable only by typing its name in the search bar (a new "tool" catalog
entry, so it costs no permanent UI), it takes a set of points -- clicked
empty map, clicked buildings, the whole rectangle selection in bulk, or
typed X/Y metres -- and draws the shortest set of links joining them.
Two link styles, both exact and both O(n log n) (map/static/map/emst.js):
Straight Euclidean MST. Sits inside the Delaunay triangulation, so a
sweep-hull triangulation cuts the candidate set from n^2/2
edges to under 3n and Kruskal does the rest.
X / Y only Rectilinear (L1) MST -- NOT the straight tree with corners
drawn on: changing the metric changes which points the optimal
tree even connects. Sits inside the octant graph, and inside
one octant the L1 distance collapses to a linear function of
the far endpoint, so four Fenwick sweeps find all eight
nearest neighbours at once.
Optionally one point is a DESTINATION, and a slider trades total network
length against how far every point has to travel through the tree to reach
it (Prim-Dijkstra: Prim with the key alpha * tree-distance + edge weight).
alpha = 0 is Prim, so still the exact MST; alpha = 1 is Dijkstra over a
candidate set augmented with the destination's own edges, so every point
joins it directly -- the exact minimum of summed trip distance. Between the
two it is a heuristic trade, and the panel says so rather than implying an
optimum that is NP-hard to find. On 5,700 real machines, 40% along the
slider costs 9% more network and cuts total travel by 47%.
Everything is computed in map pixels: the world projection is one uniform
scale with no rotation, so an optimal tree there is optimal in metres, and
"along X" on screen is "along X" in the world.
tools/check_emst.js is the gate. A short candidate set still yields a
spanning tree, just a silently longer one, so every result is diffed
against brute-force Prim over grids, circles, collinear runs, duplicates
and clusters -- plus, with a destination, that alpha=0 is still the MST,
that alpha=1 is the exact star, and that the trip distances the panel
reports match a walk of the tree it actually drew. It caught an inverted
in-circle sign on its first run. 100,000 points solve in about a second.
Also here, both found by using the tool:
- selection.js: selecting a PIN (vehicles, collectables, players, crash
sites) marked the bare coordinate, which is where the tail points -- the
pin itself, floating above it, looked untouched. Now ringed, with the
same geometry the hover highlight already uses.
- The same object can no longer be added twice: points carry the object's
save id, so clicking a 100m foundation's far corner counts as the same
object rather than a second point. Clicking it again removes it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Optimal network finder (EMST): shortest links joining a set of points
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three static world layers the in-game map never draws, plus the
game_data/reorganisation that landing them forced.
Draft on purpose — this needs a green CI run before it is merged, because
it is the first build that bootstraps entirely from the game-data archive
rather than from committed tables (see "Requires" below).
The World layer
One "World" sidebar category, default hidden like the other world layers,
with three independently-toggleable rows:
records a cave, so this is traced from the cooked level data: the game's own
cave atmosphere volumes (the fog/lighting regions it swaps you into
underground — 108 of them, ~25 carrying authored names like
Atmosphere_SwampCave), theBP_CaveFloortunnel splines, the cave-onlyfoliage clusters and the placed cave rock kit, unioned and contoured. The
row says "approximate" and hovering it says why: the outline runs generous
by tens of meters and can miss a bare tunnel. It finds a cave; it does not
survey one.
(
FGDamageOverTimeVolumecarrying a/World/Hazard/WorldPerimeter/DoTclass), as a 7-vertex polygon, 7,915 × 7,049 m, with the altitudes the
damage starts at (+2,000 m / −244 m) in the tooltip. Three of the eleven
walls are rotated, cutting the NE/NW/SW corners — the NE one is what fences
off that big empty landmass in the top-right of the map render.
(the union of 270
FGWaterVolumeactors, allmResourceClass Desc_Water_C).The rendered ocean is a different thing entirely: 31 water-plane patches,
the biggest 51 × 34 km. The sea you can see spans ~70 km; the water you can
touch stops within ~8 km, and on the west side it ends 334 m before the
damage wall — a strip that looks like ocean and holds nothing.
Both new tables ride along as static payload steps, same shape as the
existing spawners step.
game_data: one rule — if a script wrote it, it is not in git
The old split (
sav_data/committed vsgenerated/gitignored) read as"world tables vs Docs.json extracts" but was really the IP line, and
membership had drifted:
creatureSpawners.json(coordinates, like thecommitted slugs) was gitignored, half the hand-written tables sat in
sav_data/and half atgame_data/root, and two generated files had noconsumer at all.
Two latent bugs had to be fixed for that rule to actually hold:
freeDroppedItems.jsonread its own last output. FModel cannot decodethe cooked
FInventoryItemstruct, so the extractor recovered each pickup'sitem class from the previously committed table — delete it and all 703
pickups lose their item. That knowledge is now a committed input,
curated/pickupItems.json;--items-from-savewrites what it learns backinto it.
machine produced different bytes than an incremental re-run. Tables are now
written sorted: same dump in, byte-identical files out.
Verified by deleting
generated/{docs,world}/and running the fullextract_all.py: every table came back byte-identical to the pre-reorg ones.sav_core's build script now names the command to run when the data ismissing, instead of failing on the first
include_str!.Requires
game-data-v3(published,--latest=falseso the desktop updater'sreleases/latest/download/latest.jsonis untouched). The v2 archive predatesthe split and carries no
world/tables — they used to be committed — so CIcannot compile against it. The workflow in this branch points at v3.
What CI is being asked to prove
alone and compile
sav_core+ the wasm build.sav_coretests still pass, including the new ones pinning the caverings, the map-limit rings and the synthetic-id
:rule.