Published measurement output from vibeic/vibe-ic, together with the exact design-input snapshots needed to interpret those results. The tooling, external benchmark corpus, and reusable IP live in their own repositories.
| path | what it is |
|---|---|
evaluation/<benchmark>/input/ |
retained design inputs; not an evaluation result cell |
evaluation/<benchmark>/v<version>_<llm>/ |
canonical open-benchmark result cells, keyed by benchmark, plugin version, and authoring model |
ic/<design>/v<version>_<pdk>/ |
canonical IC campaign output, one directory per (design, plugin version, PDK) |
protocol_parity/ |
deterministic Phase-1 protocol inputs and parity evidence; this is not an authoring-LLM benchmark result |
PUBLISHING.md |
the publishing contract these directories must satisfy |
BENCHMARK_IC_CAMPAIGN_STATUS.md |
campaign state at the time of the snapshot |
For IC physical-result cells, only cells that PASSED and conform to the layout are published. Three conditions, each measured rather than judged:
| condition | instrument |
|---|---|
the directory is named v<version>_<PDK> |
PUBLISHING.md's layout rule |
its RESULT.md says PASS |
the cell's own verdict |
benchmark_evidence_structure_check.py exits 0 |
the repo's own validator |
Four cells meet all three. Everything else was removed rather than shipped:
a caravel_user_project cell whose own RESULT.md says FAIL and which fails the
structure check; every clean_run_* directory; and every loose phase1/, phase2/,
phase3/, reports/, sim/, steps/ tree that sat directly under an IC instead of
inside a cell. PUBLISHING.md allows exactly input/ and v*/ at that level.
A published IC result that did not pass is not evidence of convergence, and mixing it with converged cells makes the passing ones harder to trust.
Open-benchmark results follow a separate retention rule described in
evaluation/README.md: retain only the newest documented
complete result for each (benchmark, Vibe-IC version, authoring LLM) identity. A
result without an attributable plugin version or model is not placed in a named cell.
The removed snapshot paths remain recoverable from the source snapshot commit recorded
in SNAPSHOT.txt.
Design inputs are retained beside their result families, but outside version/model result cells:
- IC inputs:
ic/<design>/input/ - Open-benchmark inputs retained from source runs:
evaluation/<benchmark>/input/ - Protocol inputs:
protocol_parity/<protocol>/input/
External benchmark material lives in vibeic/benchmark-external; reusable IP and its
pinned submodule commits live in vibeic/IP. Neither sibling repository is folded into
this results repository.
vibeic/vibe-ic now holds only the plugin. Three trees that were never part of it
came here instead, so that their growth stops being the plugin's problem:
| tree | what it is |
|---|---|
ic/, evaluation/ |
benchmark results, and the design inputs that produced them |
Two sibling trees got their own repositories rather than being folded in here, because a results repository that also holds external material and IP pointers is three things wearing one name:
| repository | what it is |
|---|---|
| vibeic/benchmark-external | external benchmark material |
| vibeic/IP | the four IP submodule pointers and their pinned commits |
These repositories are now the canonical homes, so future data growth is separate from
the plugin. Historical copies and paths in the original vibe-ic history remain useful
for provenance, but are not the publishing target.
The design inputs moved with their result families rather than being discarded: an input is the one part of a benchmark run that cannot be regenerated from the result artifact.
Snapshot of vibeic/vibe-ic at the commit recorded in SNAPSHOT.txt. History was
deliberately not carried over — this is a fresh tree, not a filtered rewrite, so commit
SHAs here do not correspond to anything in vibe-ic. To read how a result came to be, use
vibe-ic's own history for the same path.
DANGLING_SYMLINKS.txt records 31 dangling symlinks that existed in the source tree at
snapshot time. Their loose steps/ trees are not part of the current canonical result
layout, so the current tracked tree contains zero symlinks. The inventory remains as
migration provenance; it is not a claim that those paths still exist here.