Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 58 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -343,6 +343,17 @@ There is no released version section until then; everything below lands in it.

### Security

- **Scopes bound what a peer writes, not only what it declares.** An
`update()` took the scope of its delta, so with a field-based `sync.scope`
(`data.$type`, as the docs show) it was minted global and reached every
peer; an op now takes the scope of the record as it leaves it, read after
validation. And a host took the scope an op declared on trust, so a peer
granted `Other` could overwrite a `Doc` record by sending an op with no
scope: a host's `push` now passes the grant to `applyRemote` as `bound`,
and an op whose record is outside it — before or after the op, as the host
scopes it — is refused (`outOfScope` in the answer and the apply result,
`refused` with `reason: 'scope'`). Found while comparing with RxDB.

- **Record ids are checked at every entrance.** An id is
`[A-Za-z0-9_@-][A-Za-z0-9._@-]{0,127}`: the local writers reject anything
else with `TypeError` `code: 'INVALID_ID'` (naming `getId` when it minted
Expand Down Expand Up @@ -384,6 +395,48 @@ There is no released version section until then; everything below lands in it.

### Fixed

- **Offline writes past retention reach the host.** A node that wrote more
ops than `retention.ops` before reaching a host had the oldest compacted,
and the host took none of them — nor any later one of that stream —
without a word. A node that dials now keeps its own ops until every
enabled registered node holds them (`listNodes()` shows `acked`,
`OpLog.retain(keep)` is the hook), and a host that lacks ones the log no
longer holds is reported once per hello as `UNDELIVERABLE`.
- **A restored directory no longer forks its node in silence.** A node
restored from an older copy minted again seqs its peers already held, and
they dropped its new writes as duplicates. At hello it now reports
`REWOUND`, moves its streams past what the host holds, and bootstraps from
the host to get back what the lost ops wrote.
- **Logged is committed.** A record file (or chain block) that failed after
the log append rejected the write while its op stayed logged and
replicated, and the writer read the record as missing until a reopen. The
write now stands: the entry is held in memory, the batch is replayed onto
the files at the next open, and the storage reports `UNPERSISTED`. A
peer's batch is held the same way.
- **A torn record file is `INTEGRITY`**, not a bare `SyntaxError` from
`get()`/`has()`; `repair(id)` rebuilds it.
- **A collection answers for its own records.** `collection.get(id)`
returned a record of another collection, and `update`, `delete` and
`record` acted on one; `get` is `null` there now, the others reject with
`NOT_FOUND`. An entity named like a storage member (`get`, `close`, `log`…)
replaced it, or broke `open()` with a bare `TypeError`; it is refused at
`open()`.
- **`cursor.one()` leaves the cursor as it was** instead of a `limit(1)` the
next `fetch()` kept.
- **Indexed objects match whatever their key order.** `$eq`/`$in` through
an index, and unique keys, compared JSON text; they compare canonical keys
now, as the predicate does.
- **`merge()` and `diff()` walk own keys and never take `__proto__`** as a
field (a JSON-parsed delta set the result's prototype).
- **`import { open } from '@alexify/syncom'` works in Node.js ESM.** The root
re-exported through spreads of locals, which Node's CommonJS lexer cannot
read; both barrels spell their exports out.
- **Docs that disagreed with the code:** failing listeners and watch runs
log at `warn`, not `debug`; an op that skips ahead is dropped as a gap, not
set aside; there is no read-only open for a second tab; a host relays an
op its validator or `beforeApply` skipped; a nested object is one
register; the event tables list `log`, `refused` and every `error` code.

- **The debt that had no decision attached.** The `Emitter`, the logger
and `loadKeys` lived twice (once per platform half); they are
`src/emitter.js`, `src/logger.js` and `src/keys.js` now, shared, and
Expand Down Expand Up @@ -563,6 +616,11 @@ There is no released version section until then; everything below lands in it.

### Changed

- **Browser budget 41 → 42 KB.** The fixes above — scopes checked on the
record, own ops kept until delivered, `REWOUND`/`UNDELIVERABLE`,
`UNPERSISTED`, scoped collections, canonical index keys — add about 1.2 KB
min+gzip; the browser entry is 41.9 KB.

- **A write no longer costs more as the store grows.** The record catalog is a
whole file, and it was saved every `catalog.flushEvery` (64) puts whatever
its size, so every write paid for a sixty-fourth of every row: inserts into
Expand Down
35 changes: 28 additions & 7 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,8 +121,15 @@ docs/ VitePress site
counted, not logged, not relayed. Payload shape is `applyOp`'s business.
- **Scopes are enforced by the host.** `unit().push` drops out-of-scope ops
before `applyRemote` (`refused` event, `syncom.ops.rejected{scope}`);
`snapshot`/`changes` serve `grant(asked, granted)`. `applyRemote`'s
`scopes` option only gates `advance` adoption, and adoption needs
`snapshot`/`changes` serve `grant(asked, granted)`. The scope an op declares
is the peer's word: `push` also passes the grant as `bound`, and
`applyRemote` refuses an op whose RECORD — before the op and after it, as
this node's `sync.scope` reads it — is outside it (`outOfScope`, not logged,
not relayed: a gap, like a forged op; relaying it as skipped would hand the
write to every peer of the declared scope). A local op takes the scope of
the record as it leaves it (`scopeData` in `src/commit.js`, re-read after
validation), never of a `set`'s delta alone. `applyRemote`'s `scopes`
option only gates `advance` adoption, and adoption needs
`adoptFrom === op.origin` — the host a feed is read from. Never adopt on the
push path.
- **`ops.js` walks own keys only.** `Object.keys` + `isKey` (`__proto__` is
Expand Down Expand Up @@ -183,7 +190,17 @@ docs/ VitePress site
The storage opens its log with `sealApplied: true` and every write path
calls `log.applied(from, to)` after `EntryStore.write`/`#persistBatch`; a roll then
seals nothing from the oldest batch not yet applied onwards. A new path
that appends must mark what it applied, or the tail grows forever.
that appends must mark what it applied, or the tail grows forever. The
append is the commit point: a chain block or record file that fails after
it is never thrown through — `core.persist` (`Storage#land`) and
`persistLocal` hold the entries in memory (`EntryStore.hold`, never
evicted), leave the batch unapplied for the next open to replay, and
report `UNPERSISTED`. A node that dials keeps its own ops past retention
until every enabled registered node in their scope holds them
(`OpLog.retain(keep)`, `node.acked` — the own keys of a peer's vector,
replaced at hello, raised by push answers); a host behind what the log
retains is `UNDELIVERABLE`, and a host ahead of the node's own stream at
hello is `REWOUND` (adopt the host's own keys, then bootstrap from it).
- **A failed page is a break in the stream.** `#receive` runs pages one after
another per peer (`peer.inbound`), numbered by `peer.feed`; a rejection goes
to `#pageFailed`, which closes the feed (pages of the old feed are dropped),
Expand Down Expand Up @@ -229,9 +246,11 @@ docs/ VitePress site
exists to declare that difference. Do not "fix" it by making the Node half
async. The one async function of the Node half is `verifyAsync` — the
engine's verify, `crypto.verify` with a callback, on libuv's thread pool —
and it is not on the package root: both barrels drop it (`const {
verifyAsync, ...crypto }`), and `src/signing.js` is its only caller. In the
browser it is `verify` itself.
and it is not on the package root: both barrels leave it out by name, and
`src/signing.js` is its only caller. In the browser it is `verify` itself.
The barrels spell every export out — a required module spread whole, the
rest by name, never a spread of a local — because that is the shape Node's
cjs-module-lexer reads for an ESM importer (`tests/package/esm.test.js`).
- **Signed ops: the id is the key, the signature is the header plus a salted
digest.** `nodeId` = the Ed25519 public key (`isKeyId`: 43 base64url chars,
canonical last char — a non-canonical one is the same key under another
Expand Down Expand Up @@ -296,7 +315,9 @@ docs/ VitePress site
when the cursor is ordered and limited (the rows come back sorted, `fetch`
must not sort again) or an `Every` otherwise; `Catalog.candidates` narrows
through `$and`/`$or`, and `ids()`/`indexed()` hand out cached, shared
arrays and sets — never edit what they return. Unique keys are checked after validation and
arrays and sets — never edit what they return. Indexed and unique values are
keyed canonically (`keyOf`: object keys sorted at every depth), as the
predicate compares them. Unique keys are checked after validation and
the fold, before `#log.append` (local: throw) and after `applyOp` on the
apply path (remote: `conflict` event, never a skip — both records stay).
- **`$text` is the catalog's.** An entity's `search` fields are joined into
Expand Down
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,8 +78,8 @@ that fails the build when exceeded.

| Entry | raw | min | min+gzip | budget |
| ----- | ---:| ---:| --------:| ------:|
| main entry — browser (@alexify/syncom) | 284.2 KB | 118.2 KB | 40.7 KB | 41.0 KB |
| main entry — node (@alexify/syncom) | 268.3 KB | 117.0 KB | 40.2 KB | — |
| main entry — browser (@alexify/syncom) | 296.0 KB | 122.3 KB | 41.9 KB | 42.0 KB |
| main entry — node (@alexify/syncom) | 279.9 KB | 121.3 KB | 41.6 KB | — |
| wrpc adapter — browser (@alexify/syncom/wrpc) | 3.4 KB | 1.4 KB | 0.7 KB | 2.0 KB |
| contracts — browser (@alexify/syncom/contracts) | 3.7 KB | 1.6 KB | 0.8 KB | 1.0 KB |
| idb backend — browser (@alexify/syncom/idb) | 3.7 KB | 1.7 KB | 0.8 KB | 2.0 KB |
Expand Down Expand Up @@ -169,6 +169,8 @@ nineteen in one table of forty features, each cell decided from the published so
syncom is a database that merges field by field, not a text CRDT. It syncs with no service to
run, its hosts validate and scope what peers send, and its log is verifiable and purgeable. It
has no text or list CRDT and no UI bindings, and it is a pre-release, not on npm yet.
RxDB, the closest of them, is compared [feature by feature](https://syncomjs.vercel.app/guide/vs-rxdb)
in 300 rows, from its code and every page of its docs.

## Benchmarks

Expand Down Expand Up @@ -266,7 +268,7 @@ _Apple M3 Max · 16 cores · Node v24.14.1 · median of 5 × 1 s samples (5 roun
| TinyBase<br><sub>tinybase@10.0.1</sub> | MergeableStore + IndexedDB persister + custom synchronizer | 56.4 KB | 23.8 KB | |
| Yjs<br><sub>yjs@13.6.33, y-indexeddb@9.0.12, y-protocols@1.0.7</sub> | yjs + y-indexeddb + y-protocols/sync | 80.2 KB | 24.5 KB | |
| PouchDB<br><sub>pouchdb-browser@9.0.0</sub> | pouchdb-browser | 117.8 KB | 40.4 KB | pouchdb-browser carries its HTTP adapter (the CouchDB client) with it |
| **syncom**<br><sub>@alexify/syncom@0.0.1</sub> | open() over OPFS + the wrpc link adapter | 119.9 KB | 41.3 KB | storage, schema, queries, the op log and the sync protocol are one entry |
| **syncom**<br><sub>@alexify/syncom@0.0.1</sub> | open() over OPFS + the wrpc link adapter | 124.1 KB | 42.6 KB | storage, schema, queries, the op log and the sync protocol are one entry |
| RxDB<br><sub>rxdb@17.6.0, dexie@4.4.6, rxjs@7.8.2</sub> | rxdb + Dexie storage + replication | 282.3 KB | 90.1 KB | |
| Fireproof<br><sub>@fireproof/core@0.24.19</sub> | @fireproof/core | 784.2 KB | 204.0 KB | |
| OrbitDB<br><sub>@orbitdb/core@4.0.0, helia@6.1.4, libp2p@3.5.1, @libp2p/gossipsub@15.0.23, blockstore-idb@4.0.1</sub> | @orbitdb/core + helia + libp2p, IndexedDB blockstore | 1185.6 KB | 353.7 KB | createHelia brings Helia's default libp2p configuration with it |
Expand Down
18 changes: 11 additions & 7 deletions bench/report/model.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -135,9 +135,13 @@ export const compareKinds = (compare) => {

const PER_WHAT = { 'ops/s': 'per operation', 'records/s': 'per record' };

export const compareModel = (compare, scenarioId, scale = 'log') => {
// The libraries a chart shows: all of them, or the ids of `only` (a page about one alternative).
const pick = (libraries, only) => (only ? libraries.filter((library) => only.includes(library.id)) : libraries);

export const compareModel = (compare, scenarioId, scale = 'log', only = null) => {
const scenario = compare.scenarios.find((candidate) => candidate.id === scenarioId) ?? compare.scenarios[0];
const rows = compare.libraries
const libraries = pick(compare.libraries, only);
const rows = libraries
.filter((library) => scenario.results[library.id])
.map((library) => {
const result = scenario.results[library.id];
Expand All @@ -151,7 +155,7 @@ export const compareModel = (compare, scenarioId, scale = 'log') => {
});
})
.sort(byValueDesc);
const missing = compare.libraries.filter((library) => !scenario.results[library.id]);
const missing = libraries.filter((library) => !scenario.results[library.id]);
const reasons = missing.map((library) => {
const failed = scenario.failed?.[library.id];
const why = failed ? `failed: ${failed.replace(/^\w*Error:\s*/, '')}` : scenario.unsupported?.[library.id];
Expand All @@ -170,8 +174,8 @@ export const compareModel = (compare, scenarioId, scale = 'log') => {
};
};

export const coldStartModel = (compare) => {
const rows = compare.libraries
export const coldStartModel = (compare, only = null) => {
const rows = pick(compare.libraries, only)
.filter((library) => compare.coldStart?.[library.id])
.map((library) => {
const phases = compare.coldStart[library.id];
Expand All @@ -197,9 +201,9 @@ export const coldStartModel = (compare) => {
};
};

export const sizeModel = (size) => {
export const sizeModel = (size, only = null) => {
const kb = (bytes) => `${(bytes / 1024).toFixed(1)} KB`;
const rows = size.libraries
const rows = pick(size.libraries, only)
.map((library) => ({
id: library.id,
label: library.name,
Expand Down
6 changes: 3 additions & 3 deletions bench/results/size.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"generatedAt": "2026-10-22T19:01:27.716Z",
"generatedAt": "2026-10-23T19:12:12.497Z",
"esbuild": "0.28.2",
"libraries": [
{
Expand All @@ -9,8 +9,8 @@
"packages": [
"@alexify/syncom@0.0.1"
],
"min": 122785,
"gzip": 42313,
"min": 127030,
"gzip": 43609,
"note": "storage, schema, queries, the op log and the sync protocol are one entry"
},
{
Expand Down
1 change: 1 addition & 0 deletions docs/.vitepress/config.mts
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,7 @@ export default withMermaid({
items: [
{ text: 'Why syncom?', link: '/guide/why' },
{ text: 'Compared with alternatives', link: '/guide/comparison' },
{ text: 'Compared with RxDB', link: '/guide/vs-rxdb' },
{ text: 'Benchmarks', link: '/guide/benchmarks' },
{ text: 'Getting Started', link: '/guide/getting-started' },
{ text: 'Architecture', link: '/guide/architecture' },
Expand Down
6 changes: 4 additions & 2 deletions docs/.vitepress/theme/components/BenchCompare.vue
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,11 @@ import { compareCaption, compareKinds, compareModel } from '../../../../bench/re
import BenchBars from './BenchBars.vue';
import Segmented from './Segmented.vue';

const props = withDefaults(defineProps<{ kind?: string; line?: string }>(), {
// `only`: the library ids to show (a page about one alternative); all of them when absent.
const props = withDefaults(defineProps<{ kind?: string; line?: string; only?: string[] }>(), {
kind: 'insert',
line: 'disk',
only: undefined,
});

const kinds = compareKinds(compare);
Expand All @@ -27,7 +29,7 @@ const scales = [
{ id: 'log', title: 'Log' },
{ id: 'linear', title: 'Linear' },
];
const model = computed(() => compareModel(compare, scenario.value, scale.value));
const model = computed(() => compareModel(compare, scenario.value, scale.value, props.only ?? null));
const caption = compareCaption(compare);
</script>

Expand Down
7 changes: 4 additions & 3 deletions docs/.vitepress/theme/components/BenchFigure.vue
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,13 @@ import size from '../../../../bench/results/size.json';
import { coldStartModel, machineCaption, sizeModel } from '../../../../bench/report/model.mjs';
import BenchBars from './BenchBars.vue';

const props = defineProps<{ chart: 'coldstart' | 'size' }>();
// `only`: the library ids to show (a page about one alternative); all of them when absent.
const props = defineProps<{ chart: 'coldstart' | 'size'; only?: string[] }>();

const figure = computed(() =>
props.chart === 'coldstart'
? { model: coldStartModel(compare), caption: machineCaption(compare.machine) }
: { model: sizeModel(size), caption: `esbuild ${size.esbuild} · ${size.generatedAt.slice(0, 10)}` },
? { model: coldStartModel(compare, props.only ?? null), caption: machineCaption(compare.machine) }
: { model: sizeModel(size, props.only ?? null), caption: `esbuild ${size.esbuild} · ${size.generatedAt.slice(0, 10)}` },
);
</script>

Expand Down
12 changes: 12 additions & 0 deletions docs/.vitepress/theme/custom.css
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,18 @@
left: 9px;
}

/* A page about one alternative (/guide/vs-rxdb): two libraries share the
* width and wrap their longer cells instead of scrolling sideways. */
.vp-doc .sc-matrix-duel th,
.vp-doc .sc-matrix-duel td {
font-size: 13px;
}
.vp-doc .sc-matrix-duel th:not(:first-child),
.vp-doc .sc-matrix-duel td:not(:first-child) {
width: 50%;
min-width: 15rem;
}

/* The page of the matrix uses the whole width next to the sidebar instead of
* the reading column, and drops the outline on the right. */
@media (min-width: 960px) {
Expand Down
Loading
Loading