The unpack submodule check needs a way to declare a submodule knowingly unpopulated

[zackees]

#1401 added a post-unpack check that fails a package when .gitmodules declares a directory that came out empty. It has now fired four times, and three of those four were on submodules nothing reads.

The record so far

package	submodule	actually needed?	outcome
esp8266	libraries/LittleFS/lib/littlefs etc.	yes — lfs.h include failed	true positive (#1380/#1398)
samd-core	Adafruit_TinyUSB_Arduino, Adafruit_ZeroDMA	no — no sketch includes either	5 boards red (#1400/#1418)
silabs-core	extra/core-api	no — fbuild installs ArduinoCore-API itself via ensure_arduino_api()	2 boards red (#1421)
ch32v-core	libraries/Adafruit_TinyUSB_Arduino	no — libraries are never enumerated for ch32v	9 boards red (#1420)

Sixteen boards went red for a condition that was harmless in fifteen of those cases. The check found one real bug and produced three build-stopping false alarms.

What the check is missing

#1401 reasoned about two states and named both deliberately:

• no .gitmodules → clean
• declared submodule whose directory is missing entirely → not reported, that is an incomplete extract

There is no third state for declared, present, empty, and knowingly fine — which is what silabs, samd and ch32v all are. The check has no way to distinguish "this core is broken" from "fbuild sources this another way, on purpose."

That is not a hypothetical distinction. For silabs it is written into the code path: ensure_arduino_api() installs ArduinoCore-API into cores/silabs/api/ precisely because the submodule is not populated. The check fails a package whose own manager is already handling the gap.

Proposal

Let a package declare which declared-but-empty submodules are expected, with a reason. Roughly:

PackageBase::new(...)
    .expect_empty_submodule("extra/core-api", "supplied by ensure_arduino_api() into cores/silabs/api")

Then:

• an empty directory not on the list still fails, so ESP8266 core cached without the littlefs submodule; any <LittleFS.h> include fails #1380's ESP8266 case stays caught
• a listed one passes, and the reason string is the documentation
• a listed entry that is not declared in .gitmodules, or that came out populated, should itself be an error — otherwise the list rots silently as upstream changes

That last rule matters: it is what stops the escape hatch becoming a way to permanently silence a real regression.

Why not the alternatives

• Check at use rather than at unpack. Closest to correct, but feat(packages): fail at unpack when a core arrives without its submodules #1401 argues persuasively that a compile-time failure inside a core's own headers is unguardable from the sketch side — __has_include passes because the header exists and only what it includes is missing. Keeping detection at package time is right; it just needs the third state.
• Fix every core's URL. Works where a prepared artifact exists (esp8266, samd). It does not generalize: ch32v has no release assets and the pin postdates the last release (ch32v-core is fetched without its TinyUSB submodule — all 9 CH32V boards are red #1420), and silabs' asset is 448 MB to populate a directory nothing reads (silabs-core: the check fails on extra/core-api, which fbuild deliberately supplies itself #1421).
• Downgrade to a warning. Loses ESP8266 core cached without the littlefs submodule; any <LittleFS.h> include fails #1380, which is the one true positive.

Related

• feat(packages): fail at unpack when a core arrives without its submodules #1401 — the check
• samd core is fetched without its submodules (TinyUSB, ZeroDMA) — same class as #1380 #1400 / fix(samd): fetch the core from Adafruit's bundle so submodules come with it #1418 — SAMD, fixed by switching artifact
• ch32v-core is fetched without its TinyUSB submodule — all 9 CH32V boards are red #1420 — ch32v, no artifact available
• silabs-core: the check fails on extra/core-api, which fbuild deliberately supplies itself #1421 — silabs, artifact available but wrong to use

Aside: the nightly hides this on quiet days

Nightly Platforms reported success on 2026-09-05 with its entire matrix skipped — the guard correctly found no commits in 24h, and the run's only passing job was the guard itself. That is intended behaviour, but it means 16 red boards on 09-04 were followed by a green 09-05 that had built nothing. Worth knowing when reading nightly history; not necessarily worth changing.