orbit-accuracy-experiments

Experiments on orbit accuracy and uncertainty, run through the Space Data Network's WASM modules with the space-data-module-sdk harness. The results are the evidence behind the accuracy claims in the Evidence-Supported ASO Catalog whitepaper, and behind what the Fast Conjunction Screening whitepaper says about the states it screens.

Rules

1. Modules compute; this repository counts. Every orbit computation — propagation, frames, time scales, errors in RTN — runs in an SDN module from space-data-network-modules. Code here moves records in and out, fits statistical models to the modules' outputs, and computes statistics.
2. Plan first. Each experiment has a PLAN.md that states its hypotheses, data windows, methods, endpoints and decision rules. Its config.json holds the same numbers for the code. The plan is frozen by a commit before its test window is read, and the code refuses the test window until then.
3. Every number has a manifest. A run writes runs/<run-id>/manifest.json: command, config hash, commits of this repository and of the modules, each module's WASM SHA-256, package versions, runtime, and the SHA-256 of every input file. Reports are generated from a run's metrics, never typed.
4. Element sets stay on this machine. Space-Track data and per-sample tables derived from it live in runs/ (ignored). Only aggregates, manifests and reports are committed, under results/.
5. Failures are results. A hypothesis that fails is reported with the same care as one that passes.

Layout

Path	Contents
harness/	Module loading and provenance, record framing, archive and reference readers, statistics
experiments/<id>/	PLAN.md, config.json, and numbered steps
test/	End-to-end checks of the harness against public vectors and the real archive
site/	The GitHub Pages site: the experiments run live in the browser
runs/	Run outputs (ignored)
results/	Committed run summaries
data/	Inputs published for checking where their terms allow it: each data/<id>/SOURCES.md lists them, docs/data-licenses.md has every source's terms and attribution

Experiments

ID	Question	Status
V1	How close does FFM (Full Force Model), seeded from a precise orbit, stay to it over three days, force model by force model?	Reported (results): SLR spheres 4.78 m median at 24 h and 13.9 m at 72 h with every force (E-f); GPS 28.5 m at 24 h with a cannonball model.
E1	Can a GPS element set's state at epoch be corrected, with an honest uncertainty, from information available at publication?	Reported (results): correction at epoch cuts the GPS element-set 3D RMS from 2.04 km to 0.79 km (R = 61 % [53, 67]); it does not carry to an unseen orbital plane (6 %), and its covariance fails the coverage gate
E2	Can catalog history (and E1's corrected element sets) give VCM-equivalent products — state, B/BDOT/AGOM and a covariance that stays realistic when propagated?	Reported (results): the batch fit (F2, 3-day span) gives realistic GPS covariance at 0–1 day (mean d²/3 1.2 and 1.05) and fails at 3 days, driven by two satellites with likely unflagged maneuvers; ten catalog-history covariance variants compared
E2b	Are consecutive element sets' errors correlated, do covariance products that model it stay realistic, how do the literature baselines (Osweiler, Thompson et al., Geul et al., Verhaeghe et al.) score, and what does covariance intersection do with one provider's sets?	Reported (results): consecutive sets' along-track errors are correlated at 0.61–1.00 (train, every regime; E2's independence correction is never positive definite); the measured covariance beats E2's best variant on 1-day energy score in GPS, LEO-POD and SLR-LEO but is consistent only in SLR at some horizons; the literature baselines are overconfident (mean d²/3 from about 10 to 10⁷); covariance intersection collapses onto one set (CI/SEL 0.90–1.00) while naive fusion is overconfident
E3	How accurate is a catalog combined from several public sources, against precise orbits, and where does combining stop helping?	Reported (results): in GPS, selection across sources gives 3.5 cm at issue and 30 m / 104 m / 336 m median at 1 / 3 / 7 days; elsewhere only Space-Track is scorable (0.3–1.2 km median)
E4	How closely do products built only from public catalog data match each operator's published ephemeris and its covariance, and where truth exists, which is closer to it?	Reported (results): public-data products sit 0.6–5 km from operator ephemerides at their first horizon (Starlink 2.9 km, GPS/GLONASS ~1.2 km, Planet 0.6 km); operators' own predictions are ~0.1 m from truth for GPS/GLONASS
E5	How much of HASDM's density accuracy can a temperature calibration built only from public densities recover (a stand-in for the Dynamic Calibration of the Atmosphere), and does it improve 1–7-day LEO propagation?	Reported (results): on held-out satellites the calibrated JB2008 cuts the spread of ln(observed/model) from 0.155 to 0.056 (−64 % [57, 70]); the forecast keeps 9 % [2, 18] for a day, then nothing; 3-day propagation is not improved (median ratio 1.04 [0.96, 1.10]). Against HASDM (CHAMP, GRACE-A, 2003 and 2008): JB2008 0.148, the one-satellite stand-in 0.112, HASDM 0.078.
E6	Can public observations (SatNOGS Doppler, amateur optical; ILRS laser ranging as the anchor) make an element-set catalog more accurate than OMM + SGP4?	Reported (results): laser ranges anchor full-force FFM fits at 7 m (0 d) to 42 m (7 d) against SGP4's 0.47–0.69 km (ratio 0.015–0.061); SatNOGS Doppler from waterfall images makes the ISS worse than SGP4 (2.7–10.7×; per-pass frequency bias, drift and timing lag); too few amateur optical lines of truth objects (18) to fit
E8	Can a temperature correction estimated from the orbital decay of many catalogued objects (their OMM histories) stand in, at zero latency, for HASDM's Dynamic Calibration of the Atmosphere, and does it improve LEO propagation?	Reported (results): fitted to 48 objects (4 per 50-km perigee bin), it cuts the spread of ln(observed/model) on held-out Swarm and GRACE-FO from 0.155 to 0.142 (8.4 % [−3.9, 18.3]; 25 % needed), most in storms (24.7 % [17.4, 29.3]); as a forecast it is worse than JB2008 (−16.8 % at 0–1 d, operational drivers), it does not change 1–7-day propagation (3-day ratio 1.022 [0.936, 1.128]), and on the 2003 and 2008 HASDM windows it is worse than JB2008 (0.270 against 0.148; HASDM 0.078). E5's density-calibrated stand-in stays far better (0.056) but its inputs arrive 49–92 days late; the element sets arrive within hours.
E13	Can a DCA-form correction to JB2008, solved from public operator states alone (Planet's daily states, SpaceX's ephemerides), reproduce Swarm and GRACE-FO density variations and improve 1–3-day orbit prediction of objects that did not calibrate it?	Reported for Planet (development split, test A and test B); Part S (SpaceX) and the combined Planet + Starlink solve are still collecting, to 2026-10-20 (results): from 36 Doves' daily drag (orbital-energy form), a common mode explains R1 = 0.247 [0.137, 0.347] of the spread of ln(observed/JB2008) in test A (0.25 needed: not supported) and 0.401 [0.283, 0.501] in test B (supported); a held-out Dove's variation R3 = 0.378 and 0.254 (supported); as a nowcast and forecast it does not improve 1–3-day orbit prediction of Swarm (median error ratio 1.02 and 0.92 in A, 1.01 and 1.00 in B) or of held-out Doves (0.98 and 0.97; 1.00 and 1.01), and it does not follow storms. Exploratory, on one NRLMSISE-00 reference for both fleets (the arcs rescaled by the ratio of drag works, agreeing with direct NRLMSISE-00 fits to 0.04 % at most): the same fleet explains 0.48 [0.41, 0.55], 0.47 [0.35, 0.57] and 0.48 [0.41, 0.56] of the spread in dev, A and B, a larger spread to explain rather than a smaller residual (0.075, 0.095 and 0.088 against 0.069, 0.086 and 0.071 on JB2008). Interim Part S (437 version pairs, two days): SpaceX's stated position sigma is close to the realised error for coasting (decaying) satellites (median realised / stated 3D error 1.12, though 27 % of transverse components fall beyond 3 sigma) and 3.2 times too small for station-keeping ones.

Sources for a high-accuracy catalog from public and login-only data — access, terms, latency and contribution — are inventoried in docs/hac-sources.md.

The parity target for the high-fidelity products is docs/vcm-parity.md: every Vector Covariance Message field, its PRW carrier, FFM's implementation and the evidence for it.

Running

Requirements: Node 22 or later; a space-data-network-modules checkout with built dist/isomorphic/module.wasm artifacts; the SDN archive's gp_history and converted reference states.

npm ci
export SDN_MODULES_ROOT=../../main-packages/space-data-network-modules   # default when this repo sits in the stack
node experiments/e1-gps-epoch/steps/10-baseline.mjs --window a0
npm test

Paths default to config.json and can be overridden with --modules, --archive and --reference or the SDN_MODULES_ROOT, SDN_GP_HISTORY and SDN_REFERENCE_STATES environment variables.

The site

digitalarsenal.github.io/orbit-accuracy-experiments runs the modules in the visitor's browser through the same SDK harness, on a CesiumJS globe: FFM from any V1 seed (compared against recorded results), any of the 72 Orekit cases, and a VCM read and propagated with its covariance. Every file it reads can be downloaded with its SHA-256. site/README.md has the build.

License

MIT for the code and the results (LICENSE). Third-party data stays under its sources' terms: docs/data-licenses.md quotes each source's terms, says whether this repository may reproduce it (in data/, as a release asset, on the site) and gives the attribution it asks for. Results use every source an experiment read, whatever its license; a source whose terms do not allow redistribution is cited by URL and SHA-256 instead. The SatNOGS-derived files in data/e6/ are CC BY-SA 4.0. site/dist/provenance.json lists each site file's source and terms.