Skip to content

Commit 0bbfca6

Browse files
docs: document dump importers, add design doc and benchmark script
- ADMIN_API.md: dump_url semantics and the new dump_importer field. - USER_GUIDE.md: creating a database from a SQLite dump, importer selection and the --dump-import-* flags. - STREAMING_DUMP_IMPORT_DESIGN.md: the design the implementation follows, with 'as built' notes where it was refined (422 for unknown importer, size limit also applies to terminated statements, exact UTF-8 error position, DDL text differences between importers) and first results. - scripts/bench-dump-import.sh: runs both importers against one server, samples RSS, compares exported data and integrity_check.
1 parent 0adb160 commit 0bbfca6

4 files changed

Lines changed: 900 additions & 0 deletions

File tree

‎docs/ADMIN_API.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,9 +21,30 @@ body:
2121
```json
2222
{
2323
"dump_url"?: string,
24+
"dump_importer"?: "buffered" | "streaming",
2425
}
2526
```
2627

28+
`dump_url` initializes the new namespace from a SQLite SQL dump (`sqlite3 db .dump` or this
29+
server's `GET /dump` output). Supported schemes are `file:` (an absolute path on the server
30+
host) and `http(s):`. The dump must run inside a transaction and end with `COMMIT`; `ATTACH` is
31+
rejected.
32+
33+
`dump_importer` selects how the dump is loaded (it requires `dump_url`):
34+
35+
- `buffered` (historical): the whole dump is read into memory, parsed, then executed. Memory
36+
usage is proportional to the dump size.
37+
- `streaming`: statements are framed with `sqlite3_complete()` and executed while the dump is
38+
still being read. Memory usage is bounded by the server's queue settings plus the largest
39+
single statement (see `--dump-import-*` flags); a statement larger than
40+
`--dump-import-max-statement-size` is rejected with `413`.
41+
42+
When omitted, the server's `--dump-importer` setting (`SQLD_DUMP_IMPORTER`, default `buffered`)
43+
applies. Both importers produce the same data; the streaming importer additionally stores the
44+
schema SQL exactly as written in the dump, whereas the buffered importer stores the parser's
45+
normalized rendering. A dump whose *data* contains the word "attach" is rejected by the buffered
46+
importer (substring check) but accepted by the streaming one (statement-level check).
47+
2748
```HTTP
2849
DELETE /v1/namespaces/:namespace
2950
```

0 commit comments

Comments
 (0)