Run source-compatible unit tests without a build

[kriszyp]

⊙ Problem

Ordinary tests need build-free source execution; export-rewriting fixtures need mutable compiled modules.

Stacked on #562; merge that foundation first.

🤔 Retain legacy mutation fixtures?

Keep legacy mutation coverage compiled; #js declares it.

💡 Solution

package.json makes #src select source under typestrip, including JavaScript, and denies #js there. Both aliases select dist under ordinary Mocha. The launcher partitions Mocha's selection into separate processes.

⚖️ Alternatives

Automatic helper selection favors the launcher over fixed lists.

🔧 Changes

Product and architecture tour

Choose the runtime

Command	Selection	Build
npm run test:unit:typestrip	Source-compatible core suites	No
npm run test:unit:typestrip:resources	Source-compatible resource suites	No
npm run test:unit:typestrip:main	Complete core selection, source then compiled	Yes
Ordinary Mocha commands	Complete compiled selection	Yes

typeStripSuites.mts recognizes #js, local helpers and static child fixtures. runTypeStripTests.mts keeps Mocha's options, rejects ambient native conditions for mixed runs, and preserves compiled global --file setup in one process. Source-only refuses to omit compiled setup. Watch and parallel mode use project Mocha; file and external reporters run per lane.

Exports stay compiled; speed is unmeasured.

Completion and storage

mocha.init.js records completion, totals and files; missing completion and signals fail. --fail-zero checks the combined total.

perFileDatabaseTeardown.js observes cached ESM databases, preserving initial directories and removing reopened residue. nativeModules.mts rejects compiled cache entries at file boundaries and completion. Utility-only suites leave storage unloaded. packageBindings.mts preserves core's blob extensions and addon identity. rocksdbBindings.mts delegates to it. testUtils.js uses real native initialization; compiled keeps its stub. .mts avoids rewire's .ts hook.

Additional changes

alasqlFunctionImporter.ts default-imports the CommonJS date-helper object; a native namespace import did not expose its date functions. DESIGN.md records that boundary. unit-test.yml adds Node 22.18.0/24.0.0 and current 22/24/26 source CI with no build or Harper install and type-checks the harness. tsconfig.harness.json scopes that check.

AGENTS.md maps the source subset and built complete selection; CONTRIBUTING.md documents commands and limits; DESIGN.md records the lane contract; DESIGN.md indexes both notes. hnsw-real-store.cjs uses its existing source family consistently; no benchmark was run.

✅ Verification

Build, types, lint, formatting and 28 harness tests pass. Mixed core: 3,829 source + 3,207 compiled, 207 pending. Windows selection on Linux: 5,008 passed. All five hosted native jobs passed.

Broad validation and CI limits

Without dist, core 3,829/26 pending, resources 3,234/36 and indexes 645/7 pass on Node 22.18/24.0/26. Compiled core 7,036/207 passed before the final backup fixture fix; all 78 backup tests pass with a relative root. Compiled resources: RocksDB 4,283/54; LMDB 3,255/722. Integration: 2,389 passed, 18 skipped, one TODO before later harness-only fixes. Deployment timing assertion passed the repeat.

CI Bun shard 6, Next.js adapters (20/22/24) and Windows runtime/admission failures match #562's logs. Windows passes on current main; these are parent-branch failures. Other jobs are still running.

Focused regression evidence

typeStripSuites covers selection/completion, inheritance, reporters, workers, late storage and interrupts. preTestPrepNative checks real initialization/isolation. perFileDatabaseTeardown tests guarded completion, both engines, closed residue and prior data. branchDatabase and closeLoadedDatabases use the shared named schema handler. longLivedTransactions resolves its worker helper absolutely.

schema.test.js scopes its PID stub; declaration-time mutation split database roots. pidInvariant.cjs fails without the PID fix.

gitSSHMaterialization restores its root and retains compiled logger mutation. storageExhaustion drains loader warnings before its fatal-console oracle. rocksdbBackup.test.js pins its child's canonical backup key and absolute parent path. logConfigFixture.js rewrites the compiled logger.

🤔 Local command deadlines

CI bounds steps. Local per-test timeouts are disabled; use --timeout or interrupt.

🤔 Final review tradeoffs

Source-only covers compatible suites. Use project Mocha; separate file reporters. Five helpers are pinned native; other marker changes appear in skip counts.

Review limits

Optional Composer output failed format repair; prompt truncation carries. Native coverage remains an accepted policy nit.

PR QA: Do legacy SQL date functions agree across cold HTTP and export workers in both runtimes? — queued @ 357adfb, qa-pr-harper-3199-357adfb

Developer docs are included; public API and storage format are unchanged.

Remaining test and fixture files

Import groups: source=#src; compiled=#js; bindings match core; JS is retained. Paths: unitTests; bridge=dataLayer/harperBridge.

• apiTests — source: multi-threaded-test.mjs.
• bin — compiled: agentCli, backup, cliOperations-test.mjs, cliOperations, deploySetup, install, login, status, workloadIdentity.
• bin — source + bindings: blobMigration, migrationCanonicalStructures, migrationMetadataPrefix, migrationStagingRecovery.
• bin — source: copyDbIntegrity.
• components — compiled: ComponentV1, EntryHandler, OptionsWatcher, Scope, componentBootOutcomes, componentLoader, componentLoaderWalk, deployActivation, deployLifecycle, rootConfigPublication, startOnMainThreadOnce.
• components — source: componentLoaderBranches, deployCertification, deployComponentBranchedDatabases, envOperations, envOperationsValidation, packageComponentOperation, reservedComponentName, secretOperationsValidation.
• components/mcp — compiled: rateLimit, transport.
• components/mcp/tools — compiled: operations.
• components/status — compiled: ComponentStatus.
• config — compiled: componentEnvPrepass, configUtils-runtimeEnvVars, configUtils, harperConfigEnvVars-config, harperConfigEnvVars-defaults, harperConfigEnvVars-edge-cases, harperConfigEnvVars-set, harperConfigEnvVars-union, harperConfigEnvVars, rootConfigWatcher.
• dataLayer — compiled: SQLSearch, bulkLoad, delete, export, hdbInfoController, insert, readAuditLog, schemaDescribe, update.
• dataLayer — source + bindings: backupProducerRoles, backupStreamAbort-fixture.cjs, restoreMarker.
• bridge/ResourceBridge — compiled: resourceDeleteRecordsBefore, resourceSearchByConditions.
• bridge/bridgeUtility — source + JS: insertUpdateReturnObj.
• bridge/lmdbBridge/lmdbMethods — compiled: lmdbCreateAttribute, lmdbCreateRecords, lmdbCreateSchema, lmdbCreateTable, lmdbDeleteAuditLogsBefore, lmdbDeleteUtility, lmdbDropAttribute, lmdbDropSchema, lmdbDropTable, lmdbGetDataByHash, lmdbGetDataByValue, lmdbSearchByConditions, lmdbSearchByHash, lmdbSearchByValue, lmdbUpdateRecords, lmdbUpsertRecords.
• bridge/lmdbBridge/lmdbUtility — compiled: lmdbCreateTransactionsEnvironment, lmdbSearch-LimitOffset, lmdbSearch, lmdbWriteTransaction.
• dataLayer — source: restoreGeneration.
• fixtures/nativeTeardown — fixture: afterRescan.mjs, check.mjs, close.mjs, create.mjs, writeSettled.mts.
• fixtures/unitRuntimes — fixture: compiled.js, earlyExit.js, fails.js, fileReporter.cjs, laterTeardown.cjs, noDatabases.js, parallel.js, setup.mjs, signal.js, source.js, unfinishedReporter.cjs, waitForInterrupt.js.
• perFileDatabaseTeardownFixtures — source + bindings: checksDatabases.js.
• resources — source: adoptedHandleBookkeeping, arrayPutCallerImpact, arrayPutElementFailure, arrayPutNormalization, auditDedupRetention, auditEntryRecordFlags, auditFloor, auditWalkRetentionFloor, blob, blobCompression, branchDeclare-thread.js, cache-probe.bench.js, certificateVerificationTables, clusterSchemaMerge, conditionDeleteVisibility, conditionsArrayMutation, copyApply, create-thread.js, create, createTableCatalogOrder-thread.js, databaseAliasIdentity-thread.js, databaseGeneration, dbisEncodeLeak, defineResource, defineTable-registration, deleteTransactionLogsBeforeRocks, derivedIndexRuntimeNativeBackend, derivedIndexRuntimeRocks, dropTableCrossWorkerWrite-worker.js, dropTableCrossWorkerWrite, dropTableGeneration, dropTableGhost, dropTableLifecycle, dualClockAuditRecord, evictionLmdb, expirationWarning, expiresAtAttribute, fullTextDerivedIndexes, getRecordCount, hnswDerivedIngest.bench.js, hnswSchemaOptions, idAllocation, immediateTransactionWrites, indexBackfillConvergence-crash.js, indexBackfillConvergence, indexCanonicalOptions, indexOrphanApplyOrder, indexOrphanReplicaApply, indexOrphanSameKeyTxn, indexOrphanStagingOrder, indexRebuildThreadConsistency-thread.js, indexRebuildThreadConsistency, localOnly-crash.js, localOnly, nativeFullTextTableSearch, nestedSelectMissingValues, operationContextTransactionLeak, operationContextUser, operationScopedWriteAuth, permissions, post-trailing-slash, query-array-scoping, query-tier1, queryCountBytes, queryCountVector, rangeReadActivity, recordCacheStableBuffer, recordLock-thread.js, recordLock, recordLockCluster, releasedSlotScope, reloadMarker, replayCommitBoundaries-crash.js, replayCommitBoundaries, replayStructures, replicatedDeleteRedelivery, replicationSeqCursor, residencyConfirmation, resource-operation, retryCounterAuditLeak, sameKeySaveOrder, searchByIndexOptions, searchPlannerRebuildingIndex, snapshotFalse, sortAlignedCondition, staleLastValueEncoding, structureForkOnMigration, subscribeProgress, subscribeResume, subscriptionBufferedState, subscriptionEntryRead, subscriptionLateCommit, subscriptionLatePatch, subscriptionPreviousCountScanBound, subscriptionRegistrationBoundary, subscriptionReplay, subscriptionSuperseded, subscriptionValueIdentity, tableClearSecondaryIndex, thisNodeId, tokenUseTableConvergence, transactionBroadcastEndDuringDelivery, transactionBroadcastKeyWalk, transactionCommitLatency, transactionQueueDepth, updateAttributesLock-thread.js, validation, vectorIndex-thread.js, vectorIndex, vectorIndexFormat, vectorIndexPlane-thread.js, vectorIndexPlane, writeTargetReads, writerResidency.
• resources/analytics — compiled: read.
• resources/analytics — source + bindings: write.
• resources — compiled + bindings: auditLog, commitConflictDeadline, createTableCatalogOrder, originClosedFloor.bench.js, transaction.
• resources — source + bindings: caching-rocks-database.bench.js, caching-rocks-database, caching, databaseAliasIdentity, databases, derivedIndexCoverage, estimateRangeCondition, estimatedEntryCount, evictionBatch, evictionBusy, immediateTransactionConflictRetry, indexBuildAbandonment, lingeringWriteCommit, originClosedFloor-thread.js, originClosedFloor, outstandingCommitTracking, primaryRocksMetadataRepair, query, queryCount, requestPathRetryExhaustion, resumeAfterMidScopeCommit, rocksCompressionOpen, schemaMigrationFragility, sourceApplyConflictRetry, txn-tracking, updateAttributesLock.
• resources — compiled: crud, debugLongTransactionsReads, derivedIndexRuntime.bench.js, graphqlSchemaDiagnostics, jsResource, liveSubscriptionReauth.bench.js, loadEnv, recordEncoder, tracked.
• resources/models — source: calibrationStore, decideOutcome, decideWrite, decisionStore, embedBulkWrite, noMatch, scoringLimit, sourceIdentity.
• resources/scheduler — source: engineElection.
• security — compiled: auth-fastify, auth, authCredentialDeferral, impersonation, keys, role, tlsFileWatch, tokenAuthentication, user.
• security/authn/oidc — compiled: tokenExchange.
• security/certificateVerification — compiled: certificateVerificationSource, crlVerification, index, pkijs-ed25519-patch.
• server — source: durableSessionResume, durableSessionSaveStartTime, startupDependencies-fixture.cjs.
• server/ipc/utility — source: ipcUtils.
• server/itc/utility — source + JS: ITCEventObject.
• server/jobs/fixtures — fixture: reportIncarnation.cjs.
• server/jobs — compiled: jobRunner, jobs.
• server — compiled: liveSubscriptionAuth, nodeName, securityHeaders, transactionLogCooling, udsMirror.
• server/serverHelpers — compiled: Request, serverUtilities.
• server/serverHelpers — source: contentTypes, refusedOperation, requestTimePlugin, serverErrorHandler.
• server/status — compiled: status.
• server/threads — source: certificationGate, manageThreadsLateExports, pinnedWorkerELU, portMetadata-fixture.cjs, preloadSafeMode, restartThrottle, stuckWorker-fixture.cjs, stuckWorkerDiagnostics, workerDataProviders, workerPortMetadata, workerSourceMaps.
• server/threads/fixtures — fixture: certificationShutdownHarness.cjs.
• server/threads — compiled: typestrip-runtime.test.mjs, watchDirFallback.
• sqlEngine — compiled: router.
• sqlTranslator — compiled: processAST, sql_statement_bucket.
• utility — compiled: OperationFunctionCaller, common_utils, globalSchema, installation, lifecycle, mount_hdb, operation_authorization, packageUtils, signalling.
• utility/environment — compiled: environmentManager, systemInformation.
• utility/functions — source: geo.
• utility/lmdb — compiled: OpenDBIObject.
• utility/logging — compiled: readLog.
• utility — source: processManagement.
• validation — compiled: configValidator, fileLoadValidator, installValidator, readLogValidator, role_validation, schemaMetadataValidator, validationWrapper.

_{🤖 Generated by OpenAI Codex; posted via @kriszyp.}

Release target: v5.4 — stacked on #562.

Related PRs: #562 overlaps (stacked foundation), #1835 overlaps (blob bindings), #2155 overlaps (eviction bindings), #3121 overlaps (backup fixtures), #3137 overlaps (subscription logger), #3187 overlaps (handler aliases)

Complexity: complicated — worker completion and module identity.

Deferred findings

• 1eada620d2e4e40d minor unclassified — .github/workflows/unit-test.yml — Native timeout caps lack hosted native measurements — filed Measure hosted native unit CI deadlines #3196
• 19f8b08d43209eae minor performance — unitTests/typeStripSuites.mts — Shared helpers are reparsed separately for every suite — filed Measure repeated parsing in unit runtime selection #3197
• c163b5329a470bd8 minor correctness — unitTests/mocha.init.js — Completion interception patches only project-installed Mocha — filed Track completion when a distinct Mocha installation runs units #3198

_{Review-Coverage: authored=codex; ran=gemini,claude; adjudicated=domain; blocked=cursor-composer(failed); declined=cursor-grok,cursor-kimi,cursor-muse,muse; rounds=9; full=1 @ 357adfb}

_{Review-Attention: study ~14m (sensitive: unit-test.yml; decisions: do-less-alternative, mutable-export-fixtures, completion-interception) @ 357adfb}

[gemini-code-assist]

Code Review

This pull request establishes a dual-lane unit testing architecture that separates native source-compatible tests (utilizing Node.js native type stripping via the typestrip condition) from legacy compiled tests. It introduces a custom test launcher (runTypeStripTests.mts) to partition suites, updates package.json imports and scripts, and implements a nativeCoreGuard to prevent source processes from loading the compiled core. Additionally, the database teardown and Mocha initialization processes are refactored to support ESM-only environments. No review comments were provided, so there is no further feedback to address.