Skip to content

v3: pinned image, peer snapshots, fix data-wiping maintenance jobs - #4

Open
TheTrunk wants to merge 12 commits into
masterfrom
v3-pinned-image
Open

TheTrunk wants to merge 12 commits into
masterfrom
v3-pinned-image

Conversation

@TheTrunk

Copy link
Copy Markdown
Member

Built and smoke-tested by CI; published as ghcr.io/runonflux/blockbook-docker:v3.0.0. See the commit messages and README.

Every new container built RocksDB and blockbook from trezor/blockbook master
and downloaded whatever daemon master named, on Debian 11. Upstream moved on:
Bitcoin Cash Node 29 needs glibc 2.34 and exits at once, Zcash moved to Zebra
with no download URL, geth templates gained {{.Env.RPCBindHost}} (ETC started
with "lookup null"), and instances of one app ran different versions.

- Build blockbook, RocksDB and all coin configs into the image at a pinned
  commit on Debian 12; copy zebrad from zfnd/zebra. overrides.json pins
  fluxd 9.1.0, BCHN 29.2.0, DigiByte 8.26.2 and live addnodes for BTG, DGB
  and VIA (their shipped seeds are dead). Re-render configs and fetch the
  pinned daemon when the image changes.
- corruption.sh wiped the whole backend on routine peer messages (invalid
  header received, InvalidChainFound); only real database errors now.
- log_clean emptied LevelDB/RocksDB write-ahead logs (any *.log over 20 MB),
  which corrupts the databases; only named logs now.
- The cron jobs looped forever on $HOME and piled up ~1,650 processes per
  container; one /etc/cron.d file with flock now.
- -dbcache=500 gave RocksDB a 500 byte cache; default 512 MiB, 4 workers.
- heal.sh: reconsiderblock when the daemon has marked the real chain invalid
  (blockbookflux instances stuck at 2263328 since January).
- Peer snapshots: synced instances serve a hard-link snapshot on port 1337 to
  the app's other instances, which seed from it instead of block 0.
- PRUNE_BACKEND=1: Bitcoin's blocks (886 GB) exceed the 820 GB Flux app
  limit, so bitcoind runs with manual pruning kept just behind blockbook
  (prune.sh). patches/0001 (blockbook -prunedbackend) gets a confirmed
  transaction from the block the index places it in and downloads a pruned
  block again from an archival peer (getblockfrompeer), so transactions,
  histories, tx-specific and blocks stay complete. History pages fetch in
  parallel, at most 8 downloads run at once, the tx cache is emptied past
  TXCACHE_MAX_GB (20), and the index stores vsizes for when no peer answers.
  Without the extended index (~245 GB of Trezor's 1.1 TB Bitcoin index) the
  index is ~640 GB and fits. New-block websocket notifications resolve
  inputs from the index instead of the backend.
- corruption.sh also clears an index blockbook refuses to open (another
  coin, other index settings), and on a pruned backend rebuilds the backend
  with it when no peer can provide the index.
- CI builds, runs an end-to-end regtest smoke test, and pushes v* tags to
  ghcr.io.
…eanly: close the ZMQ socket on its own thread, re-read the height after an unclean stop
… sync with one worker; cap blockbook's Go heap; v3 releases also move the v3 and v3.0.0 tags
…the Trezor terms link

423f085 predates the explorer rollout, so Flux instances served the old UI
while the Zelcore servers in the same app pools served the new one.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant