Repository navigation
Migronaut v1: Migrate tooling, enhance security, and refactor core functionality - #1
Merged
Merged
Conversation
Replace @biomejs/biome with oxlint + oxfmt across scripts, deps, and docs. Line width kept at 100 cols (.oxfmtrc.json, .editorconfig now agree); format:check added to prepublishOnly since there's no CI gate yet. - lint/format/format:check scripts now target src bin tests - .oxlintrc.json left as-is (user's standard template); no-any, no-console, and import-type consistency are now review-only conventions, documented as such in ARCHITECTURE.md and CLAUDE.md - fixed real oxlint findings surfaced by the new rule set: no-void in core/lock.ts, missing braces (curly) in core/config.ts, and a Promise executor param name in run.test.ts - removed the dead biome-ignore suppression comment in tests/unit/config.test.ts - biome.json removed; .npmignore now excludes the oxlint/oxfmt configs instead
Standardize on pnpm@10.34.5 (packageManager field), matching every other repo in the org. npm's looser hoisting was masking a real vitest/vite peer mismatch that pnpm's stricter resolution turned into a hard runtime failure — fixed by adding vite as an explicit devDependency instead of relying on vitepress's transitive copy. - add "packageManager": "pnpm@10.34.5" next to engines - swap package-lock.json for pnpm-lock.yaml - approve build scripts for esbuild/mongodb-memory-server (persisted in the new pnpm-workspace.yaml's allowBuilds, pnpm 10.x's actual mechanism for this, not package.json as originally assumed) - fix: add vite ^8 as a direct devDependency — vitest 4 needs vite ^6||^7||^8 as a peer, but only vitepress's bundled vite@5 was ever resolved under npm; pnpm's strict linking surfaced this as a Startup Error until pinned explicitly - update package.json/vercel.json build-tooling scripts and the execSync call in tests/integration/runtime-ts.test.ts from npm to pnpm; consumer-facing npm install/npx instructions in README/docs/ blog are left untouched — we don't dictate a library consumer's package manager - update CLAUDE.md/ARCHITECTURE.md command references (pnpm run / pnpm exec) accordingly
Remove release-automation tooling in favor of a manual version/ changelog flow, and add CI so lint/format/build/test actually gate every push and PR instead of relying on local runs. - remove bumpp and changelogen devDependencies (changelogen was already dead — wired to no script); simplify release to just `pnpm publish`, versioning and CHANGELOG.md entries now done by hand; ARCHITECTURE.md's release section updated to match - add .github/workflows/ci.yml: a lint job (oxlint, oxfmt --check, and a build smoke-check) plus a test job matrixed across Node 20/22/24 running coverage + typecheck - matrix intentionally excludes Node 18 despite the package's own engines floor staying >=18 — vitest 4 requires Node ^20||^22||>=24 and simply can't run the suite there
….ts + tsd Convert migronaut's own source from TypeScript to plain CommonJS, matching the @alexify/kerberos no-build philosophy: src/ and bin/ ship as-is, with a single hand-written index.d.ts at the package root as the sole source of type truth, verified by tsd instead of tsc. - src/**/*.ts, bin/migronaut.ts → CJS (.js), require/module.exports - New root index.js + index.d.ts; src/types/index.ts folded into index.d.ts - bin/migronaut.js runs directly under plain node — no compile step for authors or consumers - Test runner: Vitest → node:test + c8 (266 tests); tests/types/index.test-d.ts added for tsd - package.json: drop "type": "module", tsup/vitest/tsx; add c8/tsd; drop the build/typecheck scripts (none needed) - CI: replace the tsup build-smoke step with `node bin/migronaut.js --help`; drop the separate typecheck job - No more dual CJS/ESM build — package is CommonJS-only (ESM consumers still work via Node's interop) - CLAUDE.md updated to describe the new layout and conventions Migration-file loading (user .ts/.mjs/.cjs migrations via src/utils/loader.js) is unchanged — this only affects how migronaut itself is packaged.
Add scripts/size.js (esbuild-based, to report raw/min/gzip bundle size for the library entry (index.js) and CLI entry (bin/migronaut.js), with peer dependencies (mongodb, mongoose) excluded as external. - package.json: new "size" script; esbuild added to devDependencies; "scripts" dir added to lint/format/format:check targets - CI: lint job's smoke step now runs `pnpm size` instead of `node bin/migronaut.js --help`, so bundle-size numbers stay visible in CI logs on every run - CLAUDE.md: documented `pnpm run size` and the widened lint/format scope
…copy Now that migronaut's own source is plain CommonJS (not TypeScript), "TypeScript-first" is a misleading description — the toolkit still ships full type coverage for consumers via the hand-written index.d.ts, but that's not the same claim as being authored in TypeScript. - package.json "description" - README.md tagline - src/cli/index.js — CLI --help description - blog/04-distribution-kit.md — r/typescript post title suggestion
Replace every runtime dependency with small in-tree modules; package.json
no longer has a `dependencies` key at all — only the mongodb (required)
and mongoose (optional) peers remain.
- dotenv → src/utils/env.js: native util.parseEnv on Node >= 20.12 with a
built-in single-line fallback parser (quotes, `export ` prefix,
full-line and inline # comments), override:false semantics
- chalk → src/utils/colors.js: green/yellow/red/cyan/dim, binary
detection FORCE_COLOR > NO_COLOR > TERM=dumb > isTTY, stripAnsi
- ora → src/cli/spinner.js: start(text)/stop(), complete no-op off-TTY
- commander → src/cli/args.js: compatible subset (one level of
subcommands, --no-x negation, short aliases, camelCase option keys,
<x>/[x] positionals, optsWithGlobals, generated help/version) — zero
diffs in the 10 command files; combined short flags (-fy) unsupported
- cli-table3 → hand-rolled box-drawing renderer in src/cli/table.js with
ANSI-aware column widths
- zod → built-in table-driven validateConfig (CONFIG_KEYS) in
src/core/config.js; same ConfigInvalidError with {path, message} issues
MigronautLogger is now pino-compatible ({debug, info, warn, error,
child?}): success→info and dim→debug across src/, resolveLogger adapts a
pino instance (child({component: 'migronaut'}) bound once, every call
guarded so logging can never break a run) and the direct
config.logger.dim() call in loadConfig is routed through resolveLogger.
Bundle size: CLI 658→105 KB raw (85→17 KB min+gzip), library entry
278→75 KB (37→12 KB). Tests: +59 unit tests (env, colors, args, spinner,
table, validateConfig, logger adapter incl. real pino), 341 total, c8
gate 90/85/90 green, tsd updated. pino added as a devDependency; stale
.npmignore deleted; README, CLAUDE.md, ARCHITECTURE.md, CHANGELOG and
docs updated for the zero-dependency policy.
Rewrite hot-path and CLI code across src/ to use imperative loops instead of .map/.forEach/.filter/.reduce/.find/.some (fusing filter+map/filter+reduce chains into single passes, building Map/Set directly instead of via .map()-into-constructor); parallelize independent awaits with Promise.all (status/dry-run row building, migrate-mongo import writes and checksum resolution, config-file discovery) while keeping the sequential up/down migration loops untouched; and switch all fs calls (existsSync/readFileSync/ writeFileSync/mkdirSync/readdirSync) to node:fs/promises with ENOENT try/catch instead of exists-then-act checks. .sort()/.reverse() are left as-is (no sane loop equivalent), and the migration-execution/rollback order is unchanged — this is a pure internal refactor, no public API, error type, or return shape changes. Also fixes a regression caught during verification: #runUp's path-traversal check (MigrationInvalidNameError) had been accidentally wrapped inside the new fs.access try/catch, which swallowed it into a misleading MigrationFileNotFoundError. Test call sites for now-async utils (computeChecksum, verifyChecksum, applyEnvFile, nextSequenceIndex, createMigrationFile, createConfigFile, mapMigrateMongoDocs) are updated mechanically (await / assert.rejects) with no style changes to the tests themselves.
…onaut Replace the leftover "mongo-migrate-kit"/"mmk" wordmarks baked into the brand assets with "migronaut", keeping the original icon artwork untouched. - logo-icon-b*.svg/.png: split icon paths from the old text glyph paths and swapped in a real <text> "migronaut" element (same color/position) - website_standard_*, monochrome_*, transparent_logos_*: erased the old raster text and re-composited "migronaut" in matching style/background - favicon_migronaut_*: dropped the text entirely (too small to fit "migronaut" where "mmk" used to be) and kept the icon only - logo-icon-a*, npm_logo_*: unchanged (icon-only, never had brand text)
Add bench/bench.js, a zero-dependency node:perf_hooks harness measuring ops/sec for migronaut's hottest paths (computeChecksum, loadMigrationFile, Changelog, MigrationLock), modeled on the kerberos project's bench setup. DB-bound scenarios spin up a throwaway mongodb-memory-server replica set for the run — no new runtime or dev dependencies added. Wire it up as `pnpm bench` (manual only, not run in CI, matching the `size` script's informational-only philosophy), add bench/ to the lint/format globs, and document results in README's new Benchmarks section plus a one-line mention each in CLAUDE.md and ARCHITECTURE.md.
Implements Stage 1 (Security) of the code-review plan. Seven fixes, all
with negative tests; nothing here is a refactor.
- template: interpolate user values via JSON.stringify instead of raw
single-quoted literals. A crafted --db/--uri could previously inject
executable code into the generated config, which is import()ed on the
next run; an apostrophe in a password also produced an unparseable
file. (S1)
- config: skip __proto__/constructor/prototype in mergeDefined. JSON.parse
creates __proto__ as an own enumerable property, so a JSON config could
override values through the merged object's prototype. (S2)
- config: parseBoolean now fails closed — unrecognized values throw
ConfigInvalidError instead of silently becoming false, so a typo like
MIGRONAUT_STRICT=on can no longer disable checksum enforcement. (S4)
- template/migrator: mask credentials in the URI before writing a config
file, add an in-file note and an init-time warning. Regex-based, not
new URL(), so multi-host URIs are not mangled. (S5)
- lock: move toLockInfo into lock.js and apply it to
LockAlreadyHeldError context as well, so the internal owner token and
_id never reach error output or unlock --json. (S6)
- migrator: reject non-string filenames in up/down/redo/dryRun before any
query — kit.down({$ne: null}) was a query-operator injection into
findOne({name}). Validate collection names (no $/NUL, not system.*) for
the configured collections and for import --from/--to, checked before
connecting so a bad flag costs no round trip or lock. (S7)
- template: restrict --template to .ts/.js/.cjs/.mjs under 1 MB, so an
arbitrary file cannot be read and copied into the migrations dir. (S10)
Behavior change: previously-ignored malformed boolean env vars now abort
the run. This is the point of the fix — those runs were silently losing a
safety setting.
…n lock loss Runner now writes the changelog record inside the migration's own transaction (via withTransaction + onSuccess), closing the gap where a crash between commit and record left a migration applied but unrecorded. runWithLock aborts via signal on lock loss or MigratorKit.stop()/SIGINT/SIGTERM, checked between migrations. Also: atomic redo under one lock, hook guarantees (afterAll on failure, HookFailedError), --strict on bulk up, --batch validation, checksum CRLF/BOM normalization, configurable environment, and related cleanup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Runner now writes the changelog record inside the migration's own transaction (via withTransaction + onSuccess), closing the gap where a crash between commit and record left a migration applied but unrecorded. runWithLock aborts via signal on lock loss or MigratorKit.stop()/SIGINT/SIGTERM, checked between migrations. Also: atomic redo under one lock, hook guarantees (afterAll on failure, HookFailedError), --strict on bulk up, --batch validation, checksum CRLF/BOM normalization, configurable environment, and related cleanup.
…xit codes
Errors now carry the original Error as `cause`, so a failed migration's stack
still points into user code (`--verbose` prints it). Log methods take an
optional fields argument — `{runId, migration, direction, batch, durationMs}` —
passed in pino's `(fields, msg)` order to pino-style loggers. A per-run runId
doubles as the lock's owner token and is stored on every changelog record, so a
leftover lock traces back to the migrations it held.
CLI: exit codes now identify the failure (3=lock held, 5=connection, 6=config,
7=migration failed, …; unmapped stays 1), `--json` failures include error
context and the migrations applied before the failure, and new `--verbose` /
`--quiet` / `--no-color` / `--env-file` / `--no-env` flags. Status table
measures CJK width correctly and renders descriptions.
Behavior change: debug output is now opt-in via `--verbose`.
Startup: `mongodb` is now required inside connect(), so --help/--version/init/
create load zero driver modules (0.09s → 0.04s). ensureIndexes creates
{status,batch} and {batch} alongside the unique name index, in one round trip,
once per instance (opt out with ensureIndexes: false) — the highest-batch lookup
no longer needs a blocking in-memory SORT.
Queries: #nextBatch uses an indexed getMaxBatch sort+limit instead of loading the
whole changelog (~8400 vs ~17 ops/sec on 1k records); redo, down --steps and
dry-run use a new getApplied projection. list('pending') gets its own path that
never reads or hashes applied files, so pendingMigrations() is a cheap probe again.
Scale: new zero-dep mapLimit paces per-file reads (16) and import writes (8),
replacing unbounded Promise.all fan-out. create resolves config leniently, so a
failing secret-manager factory no longer blocks an offline command.
…events clientOptions reaches the driver (TLS, AWS IAM/X.509, proxies, pool sizing), and an injected `client` is reused and never closed — migrations can share the app's own pool. `uri` becomes optional when one is supplied. up --to <file> applies through that file; down --to <file> reverts everything after it, so up --to X / down --to X is a round trip to the same state. New `migronaut audit`: read-only checks of config, connectivity, transaction support, indexes, lock state, checksum drift and runtime, as a checklist or --json (exit 1 on failure, warnings are advisory). New `migronaut lock` shows the holder. MigratorKit is now an EventEmitter (run:*, migration:*, lock:*), all sharing the run's runId; a throwing listener cannot break a run. Adds per-migration timeoutMs (best-effort — the run stops and the lock frees), a shipped migronaut.schema.json for JSON configs, and opt-in reloadMigrations.
Split the test scripts: integration tests now run with --test-concurrency=1, since each boots its own replica set and running them together collided on ports and starved the timing-sensitive lock tests. `pnpm test` is green and warning-free, stable across three consecutive runs. Config and migration generators now follow the project's module system (package.json "type"), so a generated file no longer makes Node reparse it and print MODULE_TYPELESS_PACKAGE_JSON on every command. The repo's own config moves to CommonJS for the same reason. Fixed the docs build, which had been failing: dropped an unused top-level vite dependency that conflicted with VitePress's own, and renamed the VitePress config to .mts so it loads as ESM. CI: read-only permissions, concurrency cancellation, per-job timeouts, unit tests across ubuntu+windows × Node 18/20/22/24 (the declared engines floor and Windows path handling were untested), plus jobs for docs:build and a new check:dts that compiles index.d.ts on its own. Adds SECURITY.md (with an explicit threat model), CONTRIBUTING.md, CODE_OF_CONDUCT.md, publishConfig.access, a pinned mongodb-memory-server, and tests for two-process lock contention, SIGKILL/SIGTERM behavior, mongoose, and the package entry point. Corrects the docs author, the package name in llms.txt, and removes stray scratch migrations.
…laky test The transactions guide still described the pre-fix sequence — commit, then write the changelog record — which is exactly the gap that was closed. It now says the changelog record commits inside the migration's own transaction, and spells out what the non-transactional path still leaves open. The SIGTERM test waited for the lock, then asserted one migration had been applied. But an abort is checked before each migration, so a signal landing between acquiring the lock and starting the first file is a legitimate zero-migration abort — the test raced that window and failed roughly one run in four. It now waits for the migration's own marker document. Five for five under the load that used to reproduce it. Adds command pages for audit and lock, documents --to on both up and down with explicit anchors (VitePress keeps the em dash in generated ids, so the cross-links would have been dead), and replaces the stale "0 or 1" exit-code tables in up/down/redo with the real codes. Also a troubleshooting entry for the MODULE_TYPELESS_PACKAGE_JSON warning, a misplaced JSDoc block in migrator.js, and CLAUDE.md's test count and script list.
--no-color only set NO_COLOR, but FORCE_COLOR outranks it in the detection order, so the flag was a silent no-op wherever FORCE_COLOR is exported — CI runners and many local shells. It now clears FORCE_COLOR too: that precedence is right between two ambient environment signals, but an explicit flag on this invocation outranks both. The test that covers this inherited the parent env, so whether it exercised the case at all depended on the developer's shell — it passed here and failed locally on the same commit. It now pins FORCE_COLOR explicitly and also asserts the baseline, so it fails against the old behavior every time rather than sometimes.
- Redact connection-string credentials in every error message, --json payload
and --verbose stack; strip terminal control chars from DB-sourced table cells
- Fix the lock-loss window: hard renewal deadline within the TTL, staleness
judged in server time ($$NOW) instead of comparing host clocks
- Pass the abort signal to down()/import, fail typed on a closed-stdin
confirmation (was: hang, then exit 0), exit cleanly on EPIPE
- De-duplicate the up/down execution loop, extract audit/import runners,
rebuild all 12 CLI commands on a shared defineCommand envelope (--quiet,
unified CONFIG_INVALID preflights, dry-run --to/--batch)
- Kill the --strict N+1 under the lock, add a covering {status,name} index,
project import reads; add TRANSACTIONS_UNSUPPORTED (exit 15) for standalone
- Run-level event metrics (durationMs, counts, migration:skipped), runId on all
summary log lines, structured lock warnings
- Raise engines to >=22.18: drop the .env fallback parser, shrink CI matrices
to [22, 24]; add per-file coverage gate, ci-success job, nightly bench, +30
integration tests and 60+ type asserts
BREAKING CHANGE: Node >= 22.18.0 is now required (package unpublished — folded
into the 1.0.0 changelog, no version bump).
…lookups - Merge double filter/some passes into one loop: run:end counters, audit drift+pending and fail/warn tallies, down --to validation + collection - Measure table cell widths and settle column maxima in a single pass (was: two map passes plus a max scan, n+1 intermediate arrays per render) - Index CLI options in a memoized Map (--long/-s → option) instead of a linear scan per argv token - Build name Sets directly from records (no intermediate .map() arrays); allocate the unknown-config-keys list lazily via for..in - Drop per-call closures: extensions.some(endsWith) → loop with break, redactDeep array branch → preallocated indexed loop Runtime code only (src/, bin/) — tests and bench keep the readable style; MongoDB cursor .find() calls are untouched (not array methods).
…odes
- fix: unbounded lock-wait loop on NaN timeouts; redo resolves its target
under the lock; a throwing afterAll no longer masks the run's failure;
dry-run down reuses the real down selection (rejects imported records);
stale stop() no longer aborts a later run; post-timeout rejections
can't crash the CLI
- security: terminal-escape sanitization in the default logger/spinner;
event payloads carry redacted error strings; uncaughtException handler;
.env __proto__ guard + 1 MB cap; warning on TLS-weakening clientOptions
- perf: {status, appliedAt, name} index; server-side down --steps/--to
selection; bulk import writes; per-instance checksum cache
- cli: --json is a global flag; init --format <js|ts|json> replaces the
overloaded init --json; every error code maps to a dedicated exit code
(EXIT_CODES exported from the package root); status --pending/--limit,
--check exits 2; terminal-width-aware tables; a logger from the config
file is respected instead of being clobbered
- obs: run:end counts on the failure path; connect/config debug logging;
lock:acquired carries ttlMs/acquireMs; waitedMs/attempts in the
runMigrations summary
- tests: one shared replica set via --test-global-setup (~78 s integration
run instead of 17 boots); MONGOMS_VERSION pinned; new audit, shared,
run, schema-sync and SIGINT suites (409 unit + 224 integration)
- docs: fixed the dist/ recipe, hook signatures, exit-code tables and the
Node 22.18 floor everywhere; the JSON schema is now served at its
$schema URL and pinned by a sync test
…ayer Add MIGRONAUT_FORCE_COLOR/MIGRONAUT_NO_COLOR and MIGRONAUT_USER above the ecosystem-standard NO_COLOR/FORCE_COLOR/TERM and USER, which stay honored underneath. Replace the hand-written env if-chain with an ENV_KEYS table pinned against CONFIG_KEYS by a test, add the five missing scalar options, and make every value fail closed with the variable named instead of coercing to NaN or being silently ignored. Docs: fix pnpm/yarn install commands (wrong package name), correct the Node floor to >=22.18 on six pages, drop the nonexistent .env fallback parser and the false "multiline unsupported" claim, document audit()/stop()/EXIT_CODES, normalise the four migrate-mongo tables, add a vs-mongo-migrate-kit page, and switch the README badges to the kerberos style.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces several foundational improvements to the project's configuration, automation, and documentation. It adds new configuration files for code formatting and linting, establishes a comprehensive CI workflow, and updates the
.npmignoreandCHANGELOG.mdfiles to better reflect the current state and capabilities of the project.Tooling and automation enhancements:
.github/workflows/ci.ymlto automate linting, type checking, unit/integration testing, documentation builds, and coverage reporting, with matrixed Node.js and OS support..editorconfigto standardize editor behavior across contributors, enforcing UTF-8, 2-space indentation, LF line endings, and more.Linting and formatting:
.oxlintrc.jsonfor linting configuration (using oxlint with plugins and detailed rule sets) and.oxfmtrc.jsonfor consistent code formatting rules. [1] [2]Packaging and documentation:
.npmignoreto ensure only necessary files are published to npm, removing source, test, config, and CI files from the package.CHANGELOG.mdwith a detailed, structured summary of features, hardening, automation, and documentation, reflecting the current state and full capabilities of the project.