Skip to content

docs(adr): rustbgpd over gRPC (supersedes ADR 001) + staged loco-rs migration - #150

Open
lance0 wants to merge 3 commits into
mainfrom
docs/adr-rustbgpd-loco
Open

lance0 wants to merge 3 commits into
mainfrom
docs/adr-rustbgpd-loco

Conversation

@lance0

@lance0 lance0 commented Sep 28, 2026

Copy link
Copy Markdown
Owner

Records the two architecture decisions from the 2026-09-28 research sweep. Docs only — no code, no behaviour change.

ADR 023 — Drive rustbgpd over gRPC instead of embedding its crates (supersedes ADR 001)

ROADMAP.md planned to embed rustbgpd-rib + rustbgpd-transport in-process. That is not available:

  • crates/rib/Cargo.toml:2 and crates/transport/Cargo.toml:2 are publish = false (likewise api, telemetry, policy, evpn, event-history, bmp, mrt, cli, bfd).
  • rustbgpd's own docs/reference/embedding.md:579: "Never publish as a library: transport, api, evpn, evpn-linux, the daemon binary"; :577 — rib is "not ready to be a stable external API in alpha"; :325-326 and :688-689 name gRPC ("Shape A") as the recommended production embedding.
  • The peer/RIB orchestration (PeerManager, RibManager::run, listener/config wiring) is daemon-private; src/lib.rs exports nothing outside bench-internals.
  • Independently: embedding would drop the fail-open property of ADR 003, which depends on prefixd dying with the speaker process so the session drops and routers age rules out.

The decision: keep the FlowSpecAnnouncer trait (ADR 007) and swap the GoBGP gRPC client for a rustbgpd one — announce → InjectionService.AddFlowSpec, withdraw → DeleteFlowSpec, list_active → RibService.ListFlowSpecRoutes, session_status → NeighborService.ListNeighbors. rustbgpd ships examples/ddos-mitigation/config.toml describing exactly this topology and naming prefixd. Caveat recorded: those FlowSpec RPCs are explicitly_outside_v1 (docs/reference/v1-stable-surface.json:180), so the version gets pinned and its changelog tracked.

ADR 024 — Migrate to loco-rs in stages (MSRV bump accepted)

Owner decision: loco is the target framework and the MSRV bump is fine. The ADR records the costs accepted (MSRV 1.85 → 1.94, SeaORM 2.0 on sqlx 0.9 alongside the current sqlx 0.8 during the transition, a framework with breaking changes in both 1.1 and 1.2) and the parity decisions that survive migration:

  • auth stays axum-login + tower-sessions (ADR 008) — loco ships JWT/API-key only
  • the WebSocket feed stays a prefixd route — loco has no WS/SSE layer
  • metrics, rate limiting and request IDs stay prefixd middleware
  • OpenAPI stays utoipa + the custom /openapi.json route (loco-openapi still targets loco-rs ^0.16)
  • config keeps its five-file loader and hot-reload invariant
  • prefixdctl stays a standalone binary
  • reconciliation stays a tokio interval task — loco's scheduler shells out per firing

Phases: (0) MSRV + pin, (1) loco shell mounting the existing axum::Router via after_routes — zero behaviour change, full suite green, (2) data layer resource-by-resource behind the Repository seam, (3) handlers → controllers, (4) batteries (Postgres job queue for alerting), (5) cleanup. Estimated 100-170 person-days for the orthodox full path, so the sequencing rule is explicit: do not start phases 2-3 while the rustbgpd swap is mid-flight.

Also in this PR

  • ADR 001 → status Superseded by ADR 023, original text kept as the historical record.
  • ADR 007 → trait sketch corrected to the shipped signatures (list_active/session_status, Result<()>); the planned RustBgpdAnnouncer is named as such rather than as an existing type.
  • ROADMAP.md → rustbgpd milestone rewritten for the gRPC shape (additive announcer → parity/validation → remove GoBGP), new loco-rs migration milestone, and the dependency-cadence gate no longer names GoBGP exclusively.
  • ADR index rows for 023/024, ADR counts in README/AGENTS updated (19→24, 22→24), and the two decisions added to AGENTS.md's Key Design Decisions.

No CHANGELOG entry on purpose: this is decision documentation, and both open PRs (#148, #149) already touch [Unreleased] — adding a third writer invites conflicts.

Linear: LAN-1945 (swap decision, with LAN-1947/1948 as children) and LAN-1946 (loco decision, with LAN-1960 as the Phase 0+1 step); LAN-1957 re-scoped to the loco job queue.

…tion

ADR 023 supersedes ADR 001: the GoBGP sidecar is replaced by rustbgpd driven
over gRPC (InjectionService.AddFlowSpec/DeleteFlowSpec, RibService.ListFlowSpecRoutes,
NeighborService.GetNeighborState) behind the unchanged FlowSpecAnnouncer trait.
Crate embedding - what ROADMAP planned - is not available: rib/transport/api are
publish = false, rustbgpd's own embedding doc says "never publish as a library:
transport, api, ...", and the peer/RIB orchestration is daemon-private. Embedding
would also drop the fail-open property of ADR 003, which relies on the speaker
being a separate process.

ADR 024 records the decision to migrate to loco-rs in stages with the MSRV bump
accepted: a loco shell hosting the existing axum router first, then data layer,
controllers and batteries, with explicit parity decisions (auth, WebSocket,
metrics, config hot reload, prefixdctl, scheduler).

ROADMAP: the rustbgpd milestone is rewritten for the gRPC shape (additive
announcer -> parity/validation -> remove GoBGP), and a loco-rs migration
milestone is added. ADR 007's trait sketch is corrected to the shipped
signatures (list_active/session_status) and the ADR count references updated.
Two corrections from the rustbgpd-side review of the cited API surface:

- Drop the EventService.WatchEvents "benefit": rustbgpd's event history covers
  unicast only, and prefixd does not consume a pushed stream at all - its
  reconciliation is the ADR 011 poll-and-converge loop (30s default, reading the
  FlowSpec view). Recorded explicitly so nobody builds a dependency on events.
- Replace it with what the controller contract actually guarantees: AddFlowSpec
  is an upsert that always succeeds (reconciliation re-announce is safe), while
  deleting an absent rule returns NOT_FOUND and must be treated as drift.
rustbgpd decided (LAN-1961, option B) to qualify and promote the three
controller RPCs into the v1 inventory before v1.0, scoped and staged behind the
advertised view (LAN-1962) and the documented add/delete contract (LAN-1963).
Record that in ADR 023 so the version pin reads as a transitional measure with a
review trigger on every rustbgpd minor, not as a permanent constraint.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant