Extraction output (`.ext`) is not byte-reproducible across runs at constant configuration (follow-up to #304)

[clementchmlt]

Summary

At constant configuration — same magic build, same PDK tree, same input layout,
same command tokens, and the same working-directory path — extract all
followed by ext2spice produces .ext files that are not byte-identical
between runs, while the SPICE netlists written from those same .ext files
are byte-identical, run after run.

Because the final outputs are stable on this design, nothing downstream flags the
difference. That stability is a property of the netlisting stage, not of the
extraction stage, so it is not something a different design can be relied on to
preserve.

This is a follow-up to #304. In that thread the extresist error was fixed in
8.3.478, and the reporter suggested closing it and opening a separate issue for
the reproducible ordering of extract output; no such issue appears to exist.
#30 reports a related but distinct symptom — the .spice output itself
varying. In the measurement below the .spice output did not vary.
#538 is specific to extresist, which is not enabled here.

Versions

component	version	upstream commit
magic	8.3.679	4fab5040a65c5218c42be283820aaf0184df9391
netgen	1.5.323	bb8a6108b93b2d05538a976876f014f6c3a0269c
open_pdks	1.0.606	026824c7969ce6f4fc9678e6ca04b0a06a596c4b

Platform: linux/arm64 container, base image debian@sha256:abd67ffcfa541b485a3dff59865ab629aa048a6c613e639d36e7456b0b229241, gcc 12.2.0,
make -j1. Command token, unchanged from the audited chain:

magic -dnull -noconsole -rcfile <PDK>/sky130A/libs.tech/magic/sky130A.magicrc <script.tcl>

Input

Public repository, pinned commit — the design is used here only as a
reproducible input. Nothing in this report is a statement about that
repository or its contents.

• repository: https://github.com/agrancagnolo/soc_for_cubesat_scalability
• commit: ceaf9090060a700d04e00fd0708dbe4e38893dbc
• layout view: gds/adc_top.gds.gz (sha256 of the file: 2c5b24b0be059a567cf36b91487a35c54d4d2dd851c08a470a90da8ba887d42e)
• top cell: adc_top

What was measured

1 — Audit measurement. 370 units across 13 repositories were each run
twice through the same chain at constant configuration. 1 unit out of
370 shows a byte difference on a surface that survives path and timestamp
normalisation — magic's log. That unit was then isolated with three further
runs, two of them at an identical container path: the log still differs
between those two, so it is not a path artefact. Comparing the intermediate
files of two of those runs — which sat at different paths, so this count mixes
the two factors — 100 of 104 .ext files differ byte for byte, while both
.spice outputs and the LVS report are identical.

The reproducer below was built precisely to remove that ambiguity: it holds the
path constant.

2 — Minimal reproducer (below). 3 sequential runs, same container path,
distinct host directories, nothing else changed:

surfaces compared	differing across the 3 runs
.ext — 104 produced	102
.log — 1 produced	1
.spice — 2 produced	0

103 of 107 compared surfaces do not carry the same fingerprint across the 3 runs.

In the reproducer the 2 extracted SPICE netlists are identical across all 3 runs. As a check that the reduced reproducer really runs the same chain, extraite_h.spice, extraite_f.spice come out byte-identical to the netlists the audit published for this unit, at every run.

Shape of the difference

Across the 102 differing .ext files, 100 have the same number of
lines in both runs, and the diff hunks touch these record keywords only:

record keyword	diff lines
cap	1173171

In other words the same records are emitted, in a different order — and within a
record the two node names can appear swapped. Redacted, one hunk looks like:

-cap "<net A>" "<net B>" <value>
+cap "<net B>" "<net A>" <value>
+cap "<net C>" "<net B>" <value>
-cap "<net B>" "<net C>" <value>

Node names and values are withheld: they are design content, and this report
publishes none of it. They are recoverable in five minutes from the pinned input
above by anyone who wants them.

Reproducer

Build the PDK and the tool at the versions above, then run the script below
twice into two different directories that are both mounted at the same path
inside the container, and compare:

# extract.tcl
gds readonly true
gds rescale false
gds read adc_top.gds.gz
load adc_top
select top cell
extract do local
extract all
ext2spice lvs
ext2spice
ext2spice hierarchy off
ext2spice
quit -noprompt

# run twice, same container path, different host directories
for r in R1 R2; do
  mkdir -p "$PWD/$r"
  docker run --rm -v "$PWD/$r:/repro" -v "$PWD/design:/design:ro" \
             -v "$PDK_ROOT:/pdk:ro" <image> sh -c '
      cd /repro && cp /pdk/sky130A/libs.tech/magic/sky130A.magicrc ./.magicrc
      cp /design/adc_top.gds.gz . && export PDK_ROOT=/pdk PDK=sky130A
      magic -dnull -noconsole -rcfile ./.magicrc /repro/extract.tcl > magic.log 2>&1'
done
diff <(sha256sum R1/*.ext | sed 's|R1/||') <(sha256sum R2/*.ext | sed 's|R2/||')

What this report does and does not claim

• It reports what varies, in terms of file fingerprints. It does not
  assert a mechanism: no claim is made here about where in the code the ordering
  originates.

• It does not report a wrong result. On this design the extracted netlist and
  the LVS verdict are stable; the concern is that the intermediate representation
  is not, so a byte-level comparison of extraction output cannot be used as a
  regression check.

• The measurement comes from a published audit of open analog designs on open
  PDKs, whose artefacts are frozen. They are not yet reachable at a public
  URL; they are published with the preprint, and anything in them can be
  checked against the fingerprints below.

• articleA/runs/i8/L0_COMPARAISONS.jsonl — sha256:df337486f8e44437b2d7bb3ce18f585e9804b7f43ca5e0a8d77e25bd1843da69

• articleA/runs/i8/L0_ECART_CARACTERISATION.json — sha256:4686ba26e224ace4e7bbf99c588f60bc7e22793e566a2d43c3449fea3c7d9aad

• articleA/runs/i8/L0_ECART_LOCALISATION.json — sha256:5bd756bf246bf46612e4dce0b9e312e1ba7e1ac71f2102a216e293f6dd2439e9

• articleA/runs/i15/REPRODUCTEUR.json — sha256:400ecb0ec30acd566280111e41a6ec77a28b373135dc173e5f758b6b7589b9ed

• run_id 2026-08-11-6ba358591bbc-d95b6275e61a-ef75f0565d8b, freeze_sha d95b6275e61afa217ec5d513df541cd8191ae25da616e0215943b6b323d1d503

• Any reply, including one disputing the measurement, will be recorded
  verbatim in the audit artefact and summarised in the paper. A measurement is
  not silently changed in response to a reply; if a reply reveals a real error,
  the correction and both values are published.

[dlmiles]

Observable ordering change with output artifacts from magic can be dictated by the hash key used internally which can be based on the malloc memory address location of items in the design.

Does disabling ASLR https://en.wikipedia.org/wiki/Address_space_layout_randomization improve your chances of getting more consistent ordering for repeated runs on the same hardware, same inputs, same build, same system packages, etc...

It is on the list (just after multi-threaded output generation) that a backend thread performs output data collection/collation and can order the final output in a stable manner (as per what you want to configure). Since it just becomes an I/O operation to stitch the final output file into order (when you already processed and indexed all the segments to produce an ordering key, then stored the data into a temporary fileID - one large file, with offset and segment length), you are just left with a problem of replaying the order. On modern NVME systems such random file I/O is pretty fast. I do have a tool that does this after the fact to a certain scale, so the output can be compared with diff for the purpose of automated testing and also play around with that is needed for the output data collection/collation pass. You can probably knock something up in Python (or an LLM can) that does the same on the scale you need, you walk one pass to generate order-key/offset/length, then make a 2nd pass to shuffle the data. An ordering key could be the unique hierarchy/device path.

---

man setarch

setarch --addr-no-randomize magic ...