🐛 legacy usage compound sparse unique index collapses to one document per organization, ever

[PierreBrisorgueil]

What — BillingUsage's (organizationId, weekKey) unique+sparse index (used for meter-mode replay protection) is a compound index. MongoDB's sparse-index exclusion rule for compound indexes only skips a document when every indexed field is missing — since organizationId is always present, a legacy (non-meter) usage document (which never sets weekKey) still gets indexed, with weekKey treated as null. A second legacy document for the same organization — regardless of month — collides on {organizationId, weekKey: null} and is rejected as a duplicate key.

Impact — BillingUsageRepository.increment() catches the duplicate-key error and retries without upsert; the retry's filter ({organizationId, month}) matches nothing (the existing doc is for a different month), so it resolves to null with no error surfaced. Net effect: under meterMode: false (the default), a downstream consumer's legacy usage counters can only ever be recorded for the first month an organization is active — every subsequent month's increment() call silently no-ops and the write is lost.

Reproduction (against a real MongoDB, using the real repository functions):

await repo.increment(orgId, '2199-01', 'executions', 5); // creates the doc
await repo.increment(orgId, '2199-02', 'executions', 7); // returns null — nothing written

Confirmed: after both calls, exactly one document exists for the organization (the January one); February's counters are gone.

Discovered via — writing fresh-database boot tests for #3990 (a distinct, already-fixed bug: startMongoose() not awaiting index builds, and an invalid $exists:false partial filter on the other usage index). This defect is independent of both: it exists regardless of index-build timing or the partial-filter fix, and reproduces even with the index built correctly and awaited.

Suggested fix direction (not evaluated in depth — needs its own design pass): either (a) make the weekKey index a genuine partial index (partialFilterExpression: { weekKey: { $exists: true } } instead of sparse: true, mirroring the pattern used elsewhere in this codebase for "only index docs where field X is present") so legacy documents are excluded regardless of other fields' presence, or (b) reconsider the compound index's field order/shape. Needs a migration for already-deployed databases either way, plus a decision on whether the silent null return from increment() should instead throw/log when a duplicate-key retry matches nothing (currently indistinguishable from a legitimate replay no-op).

Out of scope for #3990 — filing separately per this repo's non-obvious-bug documentation convention.