Skip to content
pedro-pscunhaPublic

About

No description, website, or topics provided.

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

keylink

Typed peer-to-peer RPC over iroh. You define a service; callers get async methods. Everything underneath — NAT traversal, framing, schema versioning, authorization — is invisible from the call site.

keylink::service! {
    service Inventory;

    rpc  GetItem(sku: String) -> Option<Item>;
    rpc  Reserve(sku: String, qty: u32) -> ReserveOutcome;
    tell Note(line: String) -> ();          // fire-and-forget, no reply

    recv Fetch(sku: String) -> Datasheet;   // a file streams down beside the reply
    send Upload(name: String) -> Stored;    // and up beside the request
    blob Manual(sku: String) -> Sheet;      // an address; iroh-blobs moves the bytes
}
let node   = keylink::Node::bind().await?;
let client = inventory::Client::connect(&node, peer_id).await?;

let item = client.get_item("ADU-014".into()).await?;   // this is the whole API

No IP address, no port, no framing, no serialization call, no ALPN string, no version number appears in your code.

Status: early, and nothing here has ever run in production. The API will move. Read Status before you build on it.

Install

Not on crates.io yet, so a git dependency is the only way in. keylink needs Rust 1.91 or later, which is the floor its own dependency graph imposes.

[dependencies]
keylink = { git = "https://github.com/pedro-pscunha/keylink" }

default = ["blocking", "fs", "env", "persist-file"]. Build with --no-default-features for a browser: all four need an OS facility a tab does not have. Three say so with keylink's own compile_error! rather than failing deep inside a dependency, and persist-file stops earlier still, inside tokio.

feature default what it adds
blocking yes The synchronous client, and Node::enter. Needs OS threads.
fs yes Body::from_file. Needs a filesystem.
env yes Node::bind_from_env and Allowlist::from_env. Needs a process environment.
persist-file yes persist, plus keylink-persist's local-file adapter.
persist no The Persist observer over the port alone. No adapter, and it compiles for a browser.
web no Body::from_web_file and Body::from_web_stream. Meaningful on wasm32-unknown-unknown alone.
blobs-fs no Blobs::persistent, which pulls redb and reflink-copy. Blobs::memory needs neither, and is the one to reach for first.

The workspace

crate what it is
keylink The library. service!, Node, Server, the generated clients, actors and the registry.
keylink-persist A port for actor state that outlives its process, and its local-file adapter.
keylink-http The HTTP/JSON gateway, for callers that speak neither Rust nor the Keylink protocol.
keylink-openapi OpenAPI 3.1 and JSON Schema for a gateway surface. Opt-in, and its schema dependency reaches neither crate above.
demo The two-machine walkthrough under Try it. Not published.
fixtures/imported-ledger A service defined in one crate and consumed by another, so the generated seam is tested across a crate boundary. Not published.

Documentation

  • keylink-http/README.md — the gateway's route table, its error codes and its limits
  • keylink-openapi/README.md — generating an OpenAPI 3.1 document from a mounted surface
  • CONTRIBUTING.md — the gate, the wire-contract pin, the public API rules, and when --no-verify is legitimate
  • mise run docs builds the rustdoc, which is the complete API reference
  • Examples — every one of them is run by mise run gate, so each is the shape that works rather than the shape a document remembers. The authoritative list is [tasks.examples] in mise.toml; commands are under Try it.
example what it is about
reliable_call effectively-once, composed from a key, a Peer and attaching_for
payment_approval #[defer], Allowlist::only, and both durability ports of one store
replay_guard the retry a memory ledger cannot see, and Applied inside the state
durable_ledger Persist as an observer, storing inside every turn
firmware_depot blob, send and tell in one service
cold_store a blob store on a disk, and the flush that commits it
closure_share one grant for a whole tree, have/want, and taking the grant back; one process or two machines
match_room a keyed registry, two callers on one parked reply, and a feed clock
substation supervision, and what a caller parked across a restart is told
trading_desk one scenario, run locally and over QUIC, against two backpressure policies
foundry the compose patterns: pool, factory, keyed topic, names, group, schedule
gatehouse admission, per-method rules from TOML, and revoking a live connection
toll_booth middleware, and the two budgets that answer alike and mean opposite things
control_room events, an inspector and a console reading a node over QUIC
noticeboard pushing to peers, which is the direction a request cannot travel
bridge driving a service from a wire keylink did not design, over one hand-written codec
toolroom the synchronous API, from a main with no runtime in sight
waypoint a hand-written Dial, and a handle that finds a peer that moved
wire_contract what moves the ALPN hash, what does not, and what a move costs at dial time
fault_lines eight error types, one condition, and the vocabulary above them
inventory the HTTP gateway on one singleton surface
gateway every served surface at once, plus the OpenAPI documents

The RFCs carry the design reasoning, including the decisions that were reversed.

  • RFC-01 — iroh-blobs in keylink
  • RFC-02 — runtime mount and unmount of protocols
  • RFC-03 — keylink in the browser
  • RFC-04 — an actor model for keylink
  • RFC-05 — idempotency: what settles a key, and what does not
  • RFC-06 — observers, and durable state as an adapter, plus an implementation brief that shares its number
  • RFC-07 — the turn model: one dispatch path, a typed mailbox, two inputs
  • RFC-08 — serving-side idempotency: which methods accept a key, and where its reply lives

What it handles

Addressing

Peers are dialed by Ed25519 public key. iroh hole-punches directly when it can and relays when it cannot; neither side names a location.

Concurrency

Every call takes its own QUIC stream, so calls run in parallel and a slow handler blocks nobody. There is no client mutex, no request-id correlation and no cancellation bookkeeping — dropping the stream cancels the call.

Fire-and-forget

A tell travels on a unidirectional stream and returns a proof of admission — Queued locally, Transmitted remotely — never the declared type: there is no reply stream to decode one from. The Ok means the bytes reached the transport and nothing more — not delivery, not handling, and no order against a call. It does not accept an idempotency key either, and the compiler is what says so: with_idempotency hands back an IdempotentClient, and a tell is not among its methods. The ledger answers a repeat with the recorded reply and a tell has no reply to record, so a key attached to one could only be discarded. There is no reliable tell. Anything that has to arrive is an rpc.

Large payloads

A body is not framed: its extent is the rest of the stream and its flow control is QUIC's. A 4 GiB file costs zero framing bytes and one chunk of peak memory. A body that ends early is an error, never a short file.

Content addressing

A blob method returns a Grant; iroh-blobs moves the bytes, with verified resume and deduplication. keylink owns the mount, because the mount iroh-blobs' own documentation teaches serves every hash to every peer (and, up to iroh-blobs 0.103.0, accepted writes too). Returning a grant to a peer records it for that exact (peer, root) pair, and a blobs request is admitted only against a grant — so every entry was created by an RPC that already passed its service's allowlist and layers.

A Grant is one blob (id.into()) or a closure (seq.into()). A closure is a hash sequence built with Blobs::add_seq, and it is one grant that covers the sequence and every blob it lists, however many that is. The peer reaches a member through the root, by position; a member asked for by its own hash is refused unless it was granted on its own. A closure is one level deep, so a tree with sub-sequences is flattened into one sequence that lists every blob. A closure authorizes every hash it lists: list only what the recipient may read, and never hashes a peer supplied.

handle.fetch_into(&store) fetches a closure on one dial: the root, then only what the store lacks of the members, 4096 members to a window. It answers with a Fetched { read, kept }: the payload bytes read from the holder, and the verified bytes the store already held and did not ask for. Resume goes down into a member: the store's bitfield says which 16 KiB chunk groups it holds, and only the others are requested, so after a cut no verified group is sent twice. A 4 MiB member cut after exactly 100 groups resumed reading 2,556,004 of its 4,194,404 bytes and keeping 1,638,400. A request carries at most 64 KiB of ranges; a blob, a closure's root, or a window of members with more fragments than that is cut across several requests and never refetched whole. kept counts each distinct blob once for the whole call and never counts bytes the call itself read (a 32-byte key and table overhead for each distinct member, O(distinct members), where planning memory is not). Blobs::missing (and handle.missing(&store)) answer what is lacking, from the fetcher's own store.

Blobs::reader(id) reads a blob the node holds, as the same BlobReader a holder's blob opens as: pieces of at most a chunk group, a 256 KiB payload window at a time (a little over twice that in memory while a window is verified), each window verified against the address before any of it is handed out. A blob the store does not hold in full, absent or part way, is refused with Error::BlobNotHeld before a byte is read, and one whose stored data or hashes no longer verify fails with Error::BlobCorrupt at the window where they differ. A store that cannot be read at all is Error::Io, not corruption. (The file store's own export already validates down from the address and refuses data that no longer matches; the reader's decode is a second, independent verification.) Blobs::read_to_bytes(id, limit) is the same for a small blob into memory.

A closure has no limit on how many members it lists. Planning memory and request size are bounded, the work is not linear: iroh-blobs 0.104 walks offsets one at a time on both ends, so a window starting at member k costs k steps and a whole closure about N^2 / 4096 of them. A 300,000-member closure with alternating gaps takes 74 s from a cold start on an idle machine and 216 to 283 s on a busy one (mise run closure-300k, not in the gate), 12,289 members take 8 s (in it). A blob listed twice in one window is requested once. The fetcher plans each window of 4096 members from that slice of the root alone, so what it holds in memory is a window's, and a window's request has at most 4096 entries, far under iroh-blobs' message limit (a compile-time check). The root is checked for a whole number of addresses before it is downloaded, from the size the holder proves with the last chunk (BlobFault::NotASequence). The holder refuses a request whose furthest run starts past the members the root lists, read from the root's size (one metadata read), because a request can name an offset near u64::MAX in a few bytes and iroh-blobs walks offsets one at a time. A run that starts at a member and goes on is fine, so an open-ended request for a whole closure passes. Missing has no serde, so no rpc carries a list of what a store lacks to a peer unless a handler converts it by hand.

A grant lasts until it is revoked. Blobs::revoke(peer, root) and Blobs::revoke_everywhere(root) refuse the next request; a request already admitted finishes. Blobs::revoke_peer(peer) also closes the peer's live blobs connections, so a transfer in flight stops, and it refuses the grant of a call whose handler was already running when it landed (RemoteError::GrantsWithdrawn). Bytes already delivered stay with the fetcher. Remove a peer from admission first, then call revoke_peer: removing it from an allowlist does not touch its blob grants. Both DynamicAdmission and Trust close every RPC connection of a removed peer. For DynamicAdmission, apply the admission change before calling revoke_peer. For delegated authority, bind Trust::with_blobs(&blobs) before serving calls, using the same Blobs handle as the node's services; Trust then performs both steps.

The ledger holds at most GRANT_LIMIT grants (172,032) across every peer, and Blobs::set_grant_limit changes it. A grant past the limit is answered RemoteError::GrantsExhausted { limit }, and nothing is evicted to make room. A peer has an entry only while it holds a grant, so one count bounds both tables, and a table is rebuilt when its length falls under half the most it has held, so what a revoke leaves allocated stays bounded. The cost is measured with a counting allocator, on aarch64 macOS: the dense shape (a few peers, many grants each, grant_allocation) is 64.2 bytes per grant filling to 100,000 and at worst 128.5 under trade-one-for-one churn; the sparse shape (one grant per peer) is 386.3 at 1,000 peers and 320.7 at 100,000. The default limit's size, about 64 MiB, uses the worst of these, the sparse 386.3. Grants die with the process.

The closure_share example runs it end to end, in one process or between two machines.

Schema safety

postcard is not self-describing: swapping two same-shaped enum variants silently changes their meaning. A service's ALPN is <crate>/<Service>/<version label>/<shape hash>. The label names a version; the derived hash still decides whether two declarations agree. A field added or renamed anywhere in the contract moves the hash. Renaming a type or moving its module does not.

With no common contract, the dial returns Error::SchemaMismatch(Box<Mismatch>). An admitted peer answers a per-service probe in the same handshake, so the error names both windows. A probe that does not finish within five seconds is closed and reported as unknown. If both sides claim one label with different shapes, it names that too. A peer that serves neither a matching version nor the probe is reported as unknown, not as a guessed version.

Contract windows

Declare version 3; immediately after service Inventory;; leaving it out means version 1. Add previous 2 { hash frozen::inventory::VERSION.hash(); keyed_hash frozen::inventory::KEYED_VERSION.hash(); ... } after every current declaration. Repeat the original wire declarations in their original order, including attributes: serde metadata can change the schema hash, and configuration attributes gate the declarations and conversion arms together. A shipped application pins both literal hashes read from that release's VERSION and KEYED_VERSION; examples in this repository name the frozen module's VERSION and KEYED_VERSION so changes to keylink's own wire remain free.

Mark each changed or removed method changed. Implement the generated version_2::Convert for version_2::Step: all four request/reply upgrade/downgrade functions are required. Unchanged methods are carried automatically. More blocks extend the window through adjacent conversion steps, up to 64 versions including current. An oversized generated window fails to compile; hidden manual construction also refuses the 65th entry. Blocks must descend by label, and a frozen singleton or keyed declaration that no longer matches its pin fails to compile. Use crate:: paths inside blocks rather than super::, since the frozen types live one generated module deeper. See demo/src/main.rs for the complete implementation.

One handler serves every version. Authorization, layers and the idempotency ledger see only current types. Blob grants follow the address carried by the negotiated reply. The server selects its newest common version in one handshake. client.version() reports it; inventory::window() reports the build's supported versions. A newer client talking to an older server converts both ways. Request conversion failure is Error::UnsupportedRequest: no stream opened and nothing was sent. TellError::Unsupported has the same pre-send guarantee. Error::UnsupportedResponse means a reply arrived but could not be converted: the call was sent and may have executed. Server request conversion fails before policy, key claim and dispatch as RemoteError::UnsupportedRequest; a server reply conversion failure is RemoteError::UnsupportedResponse and does not promise that nothing ran. Use idempotency before retrying a reply-conversion failure.

Conversions own the policy: adding a field needs an explicit upgrade default, and downgrading may lose it. Bodies and blob content are application bytes and are not versioned. Blob metadata may convert, but its complete Content identity (blob or closure kind and root) must not change: a live conversion that changes it fails as UnsupportedResponse, and such a stored reply becomes Forgotten during recovery. Durable replies carry their full contract version (label and hash): recovery upgrades a known older reply to the current shape, and a malformed, unknown or unconvertible reply becomes Forgotten, never a newly executable call. Older connections run the downgrade again on replay; use deterministic conversions when byte-for-byte replies matter.

An idempotency key is bound to the received message bytes in the version the caller spoke, not to a re-encoding of the upgraded request. The binding also includes the negotiated version label and authorizing principal/holder. A retry over another negotiated label returns IdempotencyRefusal::RequestMismatch, even when its request bytes happen to be identical. Peer can therefore return this refusal after an upgrade negotiates a newer label; it does not rerun the operation. Reconcile the original operation rather than blindly issuing a new key. Same-version retries still share the service's ledger across connections and restarts.

Keylink's own envelope and errors are in every hash. Until its wire is stable, two binaries must use the same keylink wire revision even when their application versions differ. A window does not promise compatibility across keylink wire changes.

Authorization

iroh proves who a peer is; it never decides whether they are welcome. Allowlist does, and fails closed: it has no Default, admitting everyone must be typed out, and from_env errors at startup rather than bouncing every peer at runtime.

trust::Trust admits peers from signed delegation chains rooted in explicitly trusted keys. Every child narrows resources, operations, validity, and delegation depth. Push chains with insert, or let peers offer their own through trust::present before dialing. Revocable::apply records signed revocations or replaces roots before closing connections that no longer qualify.

Bind trust.with_blobs(&blobs)? before issuing grants or serving calls, with the same store used by the node's services. Every clone shares that binding; binding a different store is refused. Losing any established authority proof withdraws all of that holder's old grants, even if another chain still admits it or it has no live RPC connection. This is deliberately conservative: content grants do not record which chain authorized them. Changes that return an exhaustion error and pending revocations established by insert, verify, open, admission, or preflight have the same effect. Lost RPC admission closes first, then blob grants and in-flight grant permits are revoked. Apply trust::Change::Expire on a timer to remove expired authority and its grants; a blob fetch does not check certificate time.

trust::Signed<T> carries a forwardable value and its delegation chain. A verifier supplies a Purpose, trusted roots, and a time; open verifies signatures and authority before decoding the value. Signatures are domain-separated from certificates and revocations. Postcard bytes must round-trip exactly, so nondeterministically ordered values may be refused. Use Trust::open when revocations must count.

trust.guard::<Request, _>(WithinCaveats).resource(extract).install(...) requires admission and checks principal, service, method, and resource before idempotency, every layer, and actor lookup. A keyed resource starts with the actor id; an extractor refines it to actor/name. Singleton services use the extracted name. An application's trust::Access adds policy after caveats. A policy outage is AccessUnavailable, not a forbidden caller. The runnable recipe is keylink/examples/trust_chain.rs.

Retry safety

A call that fails mid-flight leaves the caller unable to say whether it ran. client.with_idempotency(key) removes the choice between a double booking and no booking: the serving side records the reply under (peer, key) and answers a repeat from that record without entering the handler again. Each record also binds a blake3 fingerprint of the received request-message bytes, excluding transport and retry controls, including method, actor and arguments, plus the negotiated contract label, selected principal and holder. Reusing the key for different request bytes, contract label or authority returns IdempotencyRefusal::RequestMismatch, even if the new request is otherwise authorized. Durable records retain the same binding.

The word that does the work is ran. A layer that refuses a call before it reaches the handler leaves the key claimable, because nothing happened; one that refuses after leaves Interrupted, because what happened is genuinely unknown. A record that expires or is evicted leaves a tombstone, so a forgotten key is refused rather than silently run a second time — for as long as the tombstone pool holds it, which is limit 4 below.

use std::time::Duration;

use keylink::prelude::*;
use keylink::Idempotency;

// wait for the call already running under this key instead of being refused
client.with_idempotency(Idempotency::attaching_for(key, Duration::from_secs(5)))
      .reserve("ADU-014".into(), 2).await?;

client.key_state(key).await?   // Fresh means the ledger holds nothing for this key

The record is in memory by default and dies with the process. Records::Durable puts it on a disk instead, at two fsyncs per keyed call. Composed with a reconnecting handle it is effectively-once for one rpc, which is the recipe below; RFC-05 has the reasoning.

Departure

Every other failure here arrives as the answer to a question you asked. A dead peer does not — you find out on your next call, however long after the fact that is. client.watch() is a future that resolves when the connection ends, and it distinguishes the three cases that matter:

use keylink::prelude::*;
use keylink::Departure;

match client.watch().await {
    Departure::Closed  => info!("peer said goodbye"),
    Departure::Local   => {}                  // we shut down; nothing happened
    Departure::Lost(e) => reconnect(e).await, // it broke
}

Reconnecting handles

Dial-per-message is wasteful and holding one client forever is fragile. Peer<C> connects on first use, reuses the connection, and reconnects exactly once when a call reports Transport or Closed. Remote, local I/O, and per-stream errors return without replay:

let peer: Peer<inventory::Client> = Peer::new(&node, addr);
let item = peer.call(|c| {
    let sku = sku.clone();                             // `call` takes an FnMut
    async move { c.get_item(sku).await }
}).await?;

Once, not until it works. Retrying forever turns a peer that is genuinely gone into a caller that never returns.

Composition

Many services share one node, and a keylink service can sit beside a foreign iroh protocol via Server::router(). Registering two services on one ALPN is a configuration error, not a last-one-wins race.

Layers

A Middleware sees the request typed, so a per-method rule is a match on the real enum. Make the impl generic over Req: Rpc and one layer covers every service you ever define. QueueTimeout ships with the crate.

Actors, when shared state is the awkward part

The same service! block also generates an Actor trait taking &mut self, so state lives on a task instead of behind a Mutex:

use keylink::prelude::*;

let warehouse = inventory::actor(Warehouse::seeded());  // in this process
warehouse.reserve("ADU-014".into(), 2).await?;          // Err = Unanswered

let server = warehouse.serve(&node, allow)?;            // and to other machines

Serving it changes nothing on the wire — peers use the same inventory::Client, on the same ALPN. What changes is that calls no longer overlap, which is the whole trade: a maintainer can add an .await inside a handler that mutates state. Under the concurrent model that same edit fails the build, because clippy::await_holding_lock is deny-level here and a std::sync guard is what &self state needs.

Two costs, both structural. A turn is never cancelled: dropping the call does not stop the handler, and a deadline answers the caller without ending the turn. Two remedies, both below: a deadline bounds the caller's wait and sheds work that has not started, and Deferred::away takes a slow step out of the turn so it does not hold the actor. And an actor must not call itself: keylink detects a re-entrant local call by actor identity and refuses it instead of relying on mailbox depth.

Local mailbox policy

The simple actor, setup(..).observe(..).actor(), setup(..).feeds(..).actor(), and supervised doors park at depth 1 by default. The depth is configurable and is not pinned at 1. Use the generated setup(..).backpressure(..).actor() or restarting(..).backpressure(..).actor() door—or LocalActor::spawn_deep directly—and choose what a full mailbox does:

use std::num::NonZeroUsize;
use keylink::Backpressure;

let depth = NonZeroUsize::new(32).expect("non-zero depth");
let warehouse = inventory::setup(Warehouse::seeded())
    .backpressure(Backpressure::Park(depth))
    .actor();

Backpressure::Park(depth) waits for a slot; Backpressure::Shed(depth) refuses immediately with Full. A deeper mailbox lets callers queue further ahead. It does not make self-calls safe, bound the memory held by parked senders, or remove the need to budget enough time for shutdown to drain queued turns. Keyed actors use RegistryBuilder::mailbox_depth(NonZeroUsize) as their default. A generated .backpressure(..).mountable() setting overrides that default for one actor. Park waits, and Shed returns MailboxFull.

A send method cannot be served this way at all, and serve stops compiling for a service declaring one. Its handler reads the upload inside the turn, so a peer that declares a large body and then stops writing holds the actor until the connection idle timeout, answering nothing else meanwhile. Bulk transfer goes on a second service, registered beside the actor on the same node.

Watching a turn, from outside

Middleware intercepts a message on the wire. An Observer watches a turn against the state — after the method, before the reply. It sees the state and how the turn went, and never a request or a reply payload; a layer sees the opposite. Neither grows toward the other.

use keylink::prelude::*;
use keylink::Ended;

let ledger = ledger::setup(state).observe(saver).actor();
ledger.post(1).await?;             // the observer ran before this returned

match ledger.ended().await {
    Ended::Stopped              => {}
    Ended::Observer { name, .. } => error!("{name} ended it; the turn was not committed"),
    _ => {}
}

Returning Err ends the actor. The refused caller is told Unanswered::Turn(Withheld::Refused { name }), so it learns which observer without a second lookup, and ended() adds that observer's own error — Withheld::Gone means "gone" and must not be stretched to mean "gave up on durability". A refused turn reaches a peer as RemoteError::Withheld(Withheld::Refused { name }) rather than NoSuchActor, which would say "there is nothing here, stop asking" about a turn that ran. The peer decodes the same Withheld the local caller matches on, rather than reading a prefix off the front of a string.

An observer that panics is a third answer and not that one

The value is withheld either way — a veto says "do not commit", a crash says nobody read back whether it committed — but both are told Withheld::ObserverPanicked { name }, which is a variant of its own and not a shade of the one above it. Reporting a crash as a refusal named a decision nobody made.

The actor owns its observers, so nothing is shared and no Sync bound appears anywhere. after_turn returns a boxed future rather than being an async fn, which is what lets an implementation finish with the state before it awaits — an implementation that holds it across an await needs A: Sync, and one that does not, does not.

A second input

A mailbox is one way into an actor. A feed is the other. Declare feed Restocked(item: Restock); and the generated Actor trait grows async fn on_restocked(&mut self, item: Restock) — no Caller, no return value, because nobody called and there is nothing to reply to — and a Stream is wired into the same loop at spawn, one argument per declared feed:

let warehouse = inventory::setup(Warehouse::seeded())
    .feeds(inventory::Feeds::new(deliveries)) // one argument per declared feed
    .actor();

Feed items and mailbox messages are interleaved fairly: with no off-turn work waiting, the mailbox takes half the turns and each of n feeds takes 1/2n; with finished off-turn work ready too (next section), each of the three inputs takes a third. A feed is local only and never on the wire, so declaring one does not move the ALPN. A feed that ends, and a feed handler that panics, both end the actor — a fault with no caller to resume it into would otherwise be dropped onto a state the handler may have left half applied — and Ended names which happened.

Slow work, off the turn

A turn holds its actor until it returns, so a handler that spends an hour on a check holds the actor for an hour. #[defer] lets the handler return without answering: reply.later() parks the caller. Deferred::away finishes the job. It sends the slow work out of the turn, and the work's output comes back as a new turn on the same actor, with the caller still waiting:

#[defer]
rpc Land(change: Change) -> Landed;

async fn land(&mut self, _cx: Caller<'_>, reply: Reply<Landed>, change: Change)
    -> Replied<Landed>
{
    let plan = self.prepare(&change);          // in the turn, fast
    let (deferred, replied) = reply.later();
    deferred.away(self, run_checks(plan));     // off the turn, slow
    replied
}

impl Back<Checked, Landed> for Repo {
    async fn back(&mut self, checked: Checked, reply: Deferred<Landed>) {
        match self.compare_and_swap(checked) { // in a turn again
            Ok(landed) => reply.send(landed),  // gated by this turn's commit
            Err(stale) => reply.away(self, recheck(stale)),
        }
    }
}

The work belongs to the actor. It starts only after the launching turn commits, so an observer that refuses that turn stops it from ever starting, and a durable actor's work starts only after the turn that chose it is on disk. It is cancelled when the actor ends. stop, a refusal and a restart wait for the cancelled work to be gone, so the parked caller is told Gone or Restarting by its own incarnation and never by the next one. abort does not wait: it destroys the actor now, and the caller is still told Gone, but the actor's end can be published while the cancelled work is being destroyed. In a browser the wait is weaker for every end: the cancelled future is dropped on the executor's next poll, which can be after the end is published. Only the future passed to away belongs to the actor: a task it spawns, a spawn_blocking closure it awaits and anything it starts outside the process are not cancelled with it, and need their own cancellation. The back turn is an ordinary turn: the panic guard, the observers and the commit all apply, and other turns ran while the work was away, so back re-checks what the launching turn assumed. The work itself holds no reference to the state. It is not durable: a crash loses it, and the actor reconciles what was pending on restart. stop does not wait for it to finish. Calling away outside a turn, or with another state type than the running turn's, drops the work and tells the caller Dropped on the first call.

A deadline ends the wait

ServiceBuilder::deadline(budget) answers RemoteError::DeadlinePassed to any call with no answer after budget, and a caller can set a shorter one of its own with with_deadline, which travels in the request. What becomes of the work depends on the call. A running actor turn finishes, and its reply is discarded. A keyed call is awaited to the end for as long as its connection survives, and its key settles as what the call really returned, so a retry with the same key attaches to it or replays its answer and the work runs once. An unkeyed call to a concurrent Handler is cancelled: its future is dropped where it stands.

let reserved = client
    .with_idempotency(Idempotency::attaching_for(key, Duration::from_secs(30)))
    .with_deadline(Duration::from_secs(2))
    .reserve("ADU-014".into(), 2)
    .await;
// Err(Error::DeadlinePassed { .. }) here is the caller's own timer. Retry with the same key.

The budget is also stamped on the call as a queueing bound, as QueueTimeout does, so a message that is still queued when the caller stops waiting never starts and its key stays claimable. The caller's own error is Error::DeadlinePassed, and it knows nothing about the far side: the call may not have arrived, may be running, or may have finished with its reply lost. A deadline does not bound the body of a recv reply, and the blocking clients do not take one. A service-only deadline also cannot end a call whose own upload source stalls: the client writes the whole request body before it reads any reply, so a send call whose Body stops producing never reads the DeadlinePassed already written. The caller's with_deadline wraps the whole exchange, upload included, and does end it.

In a browser a deadline is clamped to i32::MAX milliseconds (about 24.8 days), the longest wait its timer can express: a longer one would wrap to an immediate expiry, so it waits that long instead.

Many instances behind one ALPN

Keyed actors, addressed by an id inside the request, live in inventory::Registry. Build one, call registry.mount(id, inventory::spawn(state)), and serve it with registry.serve(&node, allow). Peers reach it through inventory::RegistryClient. That path once cost per-method layers, per-method authorization, blob methods and observers, because its payload was opaque to keylink. It no longer does: a keyed request is Addressed<Request> and its reply is the service's own typed Response, and a keyed mount runs the same turn loop a local Handle does — so layers, per-method authorization, blob grants, observers, the panic guard and Observer::finish all reach it. The idempotency ledger was never on that list; what was missing was a client that could send a key, and RegistryClient::with_idempotency is it.

What the keyed path still costs: one QUIC connection per actor service — not one for a node's whole fleet, each with its own handshake, admission decision, idempotency ledger, keep-alive and congestion window. registry.ended(&id) now returns a non-inserting future for a mounted instance, so a listener can await the entry's final loop-or-teardown Ended; registry.stop(&id, budget) remains the separate question of whether that shutdown drained before its budget. The budget is not optional: every caller names how long it is prepared to wait before the actor is killed instead.

A typed local keyed reference

inventory::at_local(&registry, id) addresses a mounted actor in this process and returns inventory::LocalRef. Its generated rpc, tell, and blob methods use the registry route and the actor's normal turn loop, returning transport-free LocalError; send and recv are absent because a local reference has no QUIC body stream. Missing ids, capacity and closed mailboxes remain distinct from a turn that ran, and the same lifecycle, factory and eviction machinery is used as for peer delivery.

One function, either surface

A function that runs against a local actor in a test and a remote one in production could not be written: a local call answers Unanswered, a served one answers Error, and no Result type is both. service! also generates inventory::Calls, a trait carrying the service's rpc methods with one error type — Failure, which puts a Withheld in the same arm whichever side withheld the value. It is implemented for Handle, ActorRef, Client and IdempotentClient, so one async fn stock<C: inventory::Calls>(surface: &C) -> Result<u64, Failure> runs against all four. The concrete surfaces are untouched — their own methods still answer Unanswered and Error, and Failure is reachable only through the bound, which is the act of saying I have chosen not to know which surface I hold. impl Calls + Governed narrows it to the surfaces a Dispatch stands in front of, which refuses a Handle: it means the layers and the allowlist ran, and it does not mean at-most-once, for which the honest instrument is IdempotentClient named in the signature.

It carries rpc and none of the other four kinds, because rpc is the only kind all four surfaces declare: an ActorRef and an IdempotentClient have no tell, a Handle has no send or recv, ActorRef has no validly servable send, and blob answers a Grant locally against a BlobHandle — a content address plus the node that holds it — remotely. One behaviour diverges across the bound, and it is documented rather than papered over: a handler panic unwinds into the calling task against a Handle, and arrives as Failure::Turn(Withheld::Panicked { .. }) against a served surface. Failure cannot express an unwind, and TurnLost argues at length why it should not try.

Generated surface matrix

These are deliberate capability boundaries, not missing wrappers. A blank cell means the method is not generated on that surface.

generated surface rpc tell send recv blob error vocabulary
Handle<A> yes yes yes Unanswered; NotQueued for tell
LocalRef<'_> yes yes yes LocalError; LocalTellError for tell
ActorRef<'_> yes yes yes Error
Client yes yes yes yes yes Error; TellError for tell
IdempotentClient yes yes yes yes Error
blocking::Client yes yes BlockingError<Error>; BlockingError<TellError> for tell
blocking::IdempotentClient yes BlockingError<Error>

A successful local tell returns Queued: the mailbox admitted the message, but its turn has not run. A successful remote tell returns Transmitted: the bytes reached the caller's transport, but the peer may never receive, authorize, route, queue, or handle them. Use rpc ... -> () when completion or an idempotency record matters. That acknowledged form is why tell stays absent from ActorRef and both idempotent clients: adding it there would hide failures those surfaces exist to make observable.

The generated Calls trait carries only rpc, and is implemented for Handle, ActorRef, Client, and IdempotentClient; it intentionally does not erase LocalRef's transport-free LocalError or the blocking client's synchronous execution model. Governed is present on the three served async surfaces (ActorRef, Client, and IdempotentClient), not on a bare local handle or local keyed reference.

Sync and async

keylink::blocking is a narrow singleton client for programs with no runtime. Its plain generated client exposes unary rpc and tell methods. Its blocking::IdempotentClient exposes only rpc: idempotency excludes tell, and blocking excludes send, recv, and blob. Keyed addressing remains async. The blocking module owns a private multi-threaded runtime — mandatory, because a current-thread runtime only advances iroh's keepalives during block_on, so the connection would stall between calls. Calling it from inside a runtime returns BlockingError::BlockingInAsync from generated methods instead of deadlocking. Node::enter lends that runtime to the calling thread, which is what a synchronous program needs to construct an actor at all.

The browser

The crate compiles and links for wasm32-unknown-unknown with --no-default-features. The web feature adds Body::from_web_file and Body::from_web_stream, so a <input type=file> streams up the same way a path does on a server. blocking, fs and env refuse the target by compile_error! — each needs an OS facility a browser does not have. mise run wasm-browser loads the module into headless Chrome and runs keylink/tests/browser.rs there: the ALPN derivation, a postcard round trip of the reply type, both web constructors, and the compose boundaries. A page cannot open a UDP socket, so no connection runs in a tab and none is claimed; see Status.

Reliable delivery, and its seven limits

Every piece of effectively-once for an rpc is already here. Composed, they are a recipe rather than a feature:

  1. Put the key on the work, not on the attempt. client.with_idempotency(key) records the reply under (peer, key). Every retry of that one piece of work carries that one key; different work gets a different key.
  2. Call through Peer<C>. It reconnects once when the call reports Transport or Closed and re-issues the call against the new connection. Application and local errors are not replayed. The ledger belongs to the served Service, not to the connection, so the retry meets the same record.
  3. Ask to attach. Idempotency::attaching_for(key, budget) waits for a call already in flight under the key instead of being refused with InFlight — the difference between a retry that helps and a retry that races. The budget is capped by the served Limits::attach, which defaults to MAX_ATTACH, 30 s, and the cap is the serving side's, not yours: the budget arrives from the network and a peer asking to be held for a year is asking for a QUIC stream to be held for a year. So a caller with a 90-second handler cannot wait it out — at the default it is refused at 30 s with the work still running, which is InFlight and still the truth. Ask again, or ask key_state. A service that will not admit an attacher at all passes Attaching::Forbidden to Limits::attach, which answers InFlight at once and spends no waiting place — distinct from Attaching::UpTo(Duration::ZERO), which admits the attacher and holds it for no time.

keylink/examples/reliable_call.rs is that recipe, compiled by the gate and run over two nodes. keylink/tests/reliable.rs::a_retried_call_across_a_reconnect_runs_exactly_once is the proof: the work runs, the serving side closes the connection that was carrying its answer, and the retry — over a second handshake, against the same ledger — is answered from the record without entering the handler again.

That is effectively-once for one rpc. Here is what it does not cover.

Limits 2, 3 and 5 are now partly the serving side's choice rather than only keylink's: ServiceBuilder::keyable decides which methods accept a key at all, so a key on any other is refused as NotKeyable instead of filling the ledger — which also voids limit 2's remedy on every method outside the policy, because cx.call() is then None and an Applied there never fires. And ServiceBuilder::records can decline to hold the reply at all — reaching limit 5's Unrecordable refusal by configuration rather than by the method's shape, which keeps at-most-once and gives up the answer on purpose so that a caller's payload is not held for the 600 s of limit 3. It narrows limit 4's window from MAX_TOMBSTONES plus MAX_RECORDS of churn to MAX_TOMBSTONES alone. And ServiceBuilder::limits replaces the numbers themselves: RECORD_TTL, MAX_RECORDS, MAX_RECORDED_BYTES, MAX_RECORDED_FRAME and MAX_TOMBSTONES are the defaults of Limits fields rather than fixed values, which is how a service bounds its own memory instead of only its exposure. Not every constant these limits print: MAX_HEADER is the wire's own cap and APPLIED_BOUND bounds a set in the caller's state, and neither is a Limits field. The margin limit 4 names is the one to keep — it is a property of the default, so narrowing it takes on the degradation that limit says a caller cannot detect. See docs/rfc-08-serving-side-keys.md.

  1. There is no reliable tell. A tell has no reply stream, so nothing can ack it, nothing can refuse it, and it does not accept a key — there is no reply to record and answer a repeat with. with_idempotency returns an IdempotentClient, which has no tell method, so this limit is one the compiler enforces rather than one you have to remember. Work that has to arrive is an rpc.

  2. The ledger is memory by default, so across a restart the guarantee is yours to keep — or you buy it. Records::Memory dies with the process. Records::Durable writes each record through keylink-persist's entry port and survives one, which costs two fsyncs per keyed call: one before the handler is entered and one before its reply is written.

    Durable state is what made the default sharp rather than mild. While nothing survived a restart, re-running a retried call was correct — a fresh process had a fresh state. It stopped being correct the moment state became durable. A process that fsyncs turn n, records the reply in an in-memory ledger, and dies before the frame reaches the peer will, on restart, load a state that already contains n, find an empty ledger, and run n again. That is the pairing Records::Durable exists for, and it is why a service with durable state should think twice before leaving the ledger at the default.

    What no store makes durable is the in-flight half. A key claimed when the process died comes back Interrupted — unknown — because the caller it belonged to held a live stream, and no disk carries one of those forward.

    A store that refuses a write is answered, not swallowed, and the two refusals are on opposite sides of the handler. If it cannot write the claim, nothing ran and the caller is told NotClaimed — the same key is safe to send again. If it cannot write the settle, the call ran and its reply is discarded rather than sent, which is Unrecordable; a reply this ledger cannot replay is a reply the caller would ask for again. Neither is Interrupted, which means the handler produced nothing at all.

    Records::Durable on a service that accepts no idempotency key is refused at startup, as ConfigError::DurableWithoutKeys. No ledger is built, so the store would be held and never written — and its exclusive claim on the log file would refuse the next service that meant to use it.

    So effectively-once is a property of one process's lifetime, not of a key — for as long as the only thing that knows about the key is the ledger. cx.call() changes that: its CallId binds the peer and key to the same received-message, contract-label and principal/holder commitment as the ledger. Applied<T> is a bounded set of those bound calls, so an old answer cannot cross requests or principals after the ledger forgets its slot. Retries must preserve message bytes, negotiated label and authority; another contract encoding is another bound call. CallId v2 does not migrate old IDs; a reset of pre-v2 applied-ID state is required. Put one in your own state, and the dedupe is three lines:

    let call = cx.peer().and_then(Context::call);
    if let Some(&balance) = self.applied.get(call) { return balance; }
    self.balance += amount;
    self.applied.insert(call, self.balance);

    The method has to be one your service keys. Keyable decides that, and on a method outside the policy a keyed call is refused NotKeyable before the handler runs — so cx.call() is None there always, and an Applied in that handler never fires and never says so. Deduplicating in a handler means putting its method in the policy.

    Insert in the turn that applies, as above, and never in a turn that merely hears about the work: the two together are one snapshot and one fsync, and that is the whole reason the set lives in your state rather than in keylink's. A #[defer] method's parking turn is the trap — park the CallId beside the Deferred and let the releasing turn record it, or a restart in between leaves a record with no work behind it. Applied<()> is the valueless form for exactly that shape, where there is no answer to replay.

    Applied implements Serialize/Deserialize and no Schema, so it cannot reach an ALPN and postcard::to_allocvec(self) keeps compiling when you add the field. Take that encoder rather than writing your own: a hand-rolled snapshot that omits the field compiles, passes every test, and starts empty after every restart, which is the silent at-least-once this whole mechanism exists to prevent.

    APPLIED_BOUND is 4096 calls — and the unit is calls, not seconds. It means "4096 other pieces of keyed work may land between yours and its retry". Turning that into a duration needs your adapter's turn rate: over keylink-persist's file store a turn is ~7.6 ms, so ~130/s and roughly 31 seconds — but the same turn with no fsync behind it is ~13 µs, where 4096 entries is well under a second. Measure and divide, and treat the count as the guarantee.

    That 7.6 ms is a macOS number and it does not carry to Linux. std::fs::File::sync_data issues F_FULLFSYNC on macOS, which asks the drive to flush its own write cache — measured at about 12x a plain libc::fsync, 5498 µs against 462 µs. On Linux the same call is fdatasync, so a Linux deployment is faster by roughly that order and every figure derived from 7.6 ms in this file is pessimistic there. Measure on the platform you will run on.

    And the set is a term in that turn rate, so measure at the bound you mean to run. Persist re-encodes the whole state every completed turn, so an Applied costs its entries on every turn whether one changed or not — 4096 entries is ~92 µs, which is 1.2% of an fsync turn and 6.5× a non-syncing one. That ratio divides by 14.0 µs, not by the ~13 µs above: both halves come from one run, and dividing by the tabled figure instead gives 7.2×. APPLIED_BOUND's rustdoc has the measured table and says why the two differ. Raise the bound for slow callers; lower it for cheap turns.

    The busiest window is the one right after the restart, which concentrates arrivals — every peer's backoff fires at once. What drains the set, though, is only first-time keyed work: a recognised retry returns before it touches the eviction order, so a herd of pure retries costs no places at all. A reconnecting client supplies both, and it is the new work that spends the bound.

    What this does not buy, in full. The duplicate still enters the handler and still costs a turn — about 7.6 ms of fsync on macOS, less on Linux, on a state it did not change — because only a durable ledger could refuse it before dispatch, and that one is structural: the record lives in your state, so reaching it means dispatching to your state. And the dedupe is yours to write, because the correct version knows what your answer is and whether replaying it beats refusing.

    Separately: key_state(key) answers Fresh after a restart on a Records::Memory service, so a peer learns nothing about a key your domain is about to dedupe. This paragraph used to add as the tree stands today, and predicted that "a durable record store would answer it again, with nothing here changing". It does, and nothing here changed: on Records::Durable the query answers Done after a restart. What did not come back is the cost above it — a duplicate entering the handler is a consequence of the record living in your state, and no store on a disk alters that.

    It also rests on the calling peer keeping its identity. The peer is in the hash, so a client that reconnects on a fresh keypair is a different caller, is not recognised, and runs the work again — silently, and indistinguishably from a first call, which is the degradation limit 4 below names as the one a caller cannot detect. A long-lived SecretKey on the calling side is the precondition, not a deployment preference.

    keylink/tests/persist.rs::a_retried_call_across_a_restart_is_applied_once is the proof, over two real nodes and the real file store: the process dies, the restarted ledger answers Fresh, the retry is dispatched, and the work does not happen twice.

  3. A record answers for 600 s. That is RECORD_TTL, alongside volume bounds of MAX_RECORDS — 1024 entries — and MAX_RECORDED_BYTES — 1 MiB of held reply in total, across every peer, not per key. Past any of them the reply is dropped and the retry is refused as Forgotten: at-most-once still holds, and the answer does not. Read the result out of your own domain; do not re-run the work.

    cx.call() is no help here, and it is worth saying so. A Forgotten repeat is refused before dispatch, so there is no turn, no Context, and nothing for a handler to recognise — the advice above is the caller's to act on, in its own domain, and limit 2's remedy is the serving side's. The two limits look adjacent and are not.

  4. Past MAX_TOMBSTONES — 8192, eight times the record bound — a key is Fresh again and the retry runs. This is the one edge where the ledger's guarantee degrades: effectively-once, to at-most-once, to at-least-once. It is also the one state a caller cannot detect, because Fresh is what "never used" looks like too. The wide margin over the record bound is what makes it unlikely rather than impossible.

    The real bound, beside it. The margin is over this ledger's traffic, so what it protects is the window between a call and its retry on one live process — and a restart resets it to nothing, which is limit 2. Two bounds, then, and they are different quantities: MAX_TOMBSTONES is how many other keys may pass before this one is vacant again, and APPLIED_BOUND is how many other keys may pass before your own state forgets it. The second is the one that survives a restart, and it is the one you can raise.

    Undetectable degradation has two doors, and this is the second: a caller on a fresh keypair also gets Fresh from Applied, for the same reason and with the same silence.

  5. A reply that carries a payload is never recorded. A recv method answers with a body beside its reply, and a body is streamed rather than held, so the repeat is refused as Unrecordable instead of being answered with a header whose payload never arrives — a permanent property of the method, not an eviction. A blob reply is not affected: its bytes move on iroh-blobs and the reply itself is just a header carrying an address, so it records and replays like any other.

    So is any reply frame over MAX_RECORDED_FRAME — 1 MiB, 1048576. This line used to name MAX_RECORDED_BYTES and print no figure, which was wrong twice: the ledger's ceiling is a bound on one frame, and the pool is a separate number that happens to share its value today.

    The ceiling equals MAX_HEADER, the wire's own cap, and that is not a coincidence. MAX_HEADER bounds the postcard payload, while the ledger stores the whole frame including its four-byte length prefix — so a maximal payload arrives as 1048580 bytes, which exceeds the 1 MiB pool on its own. A reply that cannot fit the pool under any conditions is permanently unrecordable, so Unrecordable is the right answer for it and the ceiling stops at the payload cap to say so. Raise the pool and that changes: a bigger pool can hold a maximal frame, and then the ceiling gains the four bytes.

    The number is worth setting lower than the default, and Limits::frame_bytes is how: one large reply that fits the pool can evict many small ones. There is no floor to quote, because the smallest useful value is the encoding of one service's own response type.

  6. Interrupted is permanent, and it is the answer that needs a decision. A call whose handler was entered and produced no reply — a dead connection, a shutdown — settles the key as Interrupted. Whether the work took effect is genuinely unknown and no record can make it known, so retrying that key is refused forever, by design. What a caller does: reconcile against its own domain, then issue a new key. client.key_state(key).await asks without running anything, which is the only way to learn a state you are not willing to discover by executing.

  7. A feed has no delivery guarantee at all. It is a local stream into an actor rather than a call: nobody sent it, nothing acks an item, nothing refuses one, and no key applies. It never reaches the wire. A feed that dies takes the actor with it, which is the only report a feed can make.

HTTP/JSON gateway

Use the separate keylink-http package when a caller cannot use Rust or the Keylink protocol. It serves generated rpc, send, recv and blob declarations on a singleton surface, and rpc, recv and blob on a keyed actor surface. tell has no route, and a keyed surface serves no send. Its README carries the route table and the error codes with their statuses; this section is the summary.

keylink-openapi is a separate opt-in package that generates an OpenAPI 3.1 document and a standalone JSON Schema from the same surface the gateway is mounted with. Its schema dependency reaches neither keylink nor keylink-http. The gateway calls the existing generated Client, so the Keylink server has one serving pipeline.

Run the example:

cargo run -p keylink-http --example inventory

The program prints a loopback URL. Call it with an argument object:

curl -i -X POST "$URL/rpc/Add" \
  -H 'Authorization: Bearer allow' \
  -H 'Content-Type: application/json' \
  --data '{"left":2,"right":3}'

The success body is only the declared return value. The response above is 5, not a Keylink Response enum. A zero-argument RPC uses {}. A unit return uses null. HTTP field names use the Rust argument names. Serde attributes on nested domain types still apply.

Outer middleware must verify the HTTP identity and insert a Principal in the request extensions. Principal is a stable, non-secret namespace. The gateway then calls the mandatory async Access policy before it reads the body. The Keylink server authenticates the gateway node's EndpointId, not the HTTP principal.

Set finite request and success limits with JsonLimits. Both limits are inclusive. Error envelopes have a separate fixed limit. JavaScript clients must keep integers in the safe integer range, or use JSON-safe domain DTOs.

An optional Idempotency-Key is valid for one logical call. The gateway hashes it with the principal namespace before it reaches Keylink. Two principals cannot replay each other's result. Do not reuse one key for another method or argument set. Replay records live for the Keylink server process lifetime; they are not a durable exactly-once record.

Errors use this shape:

{"error":{"code":"forbidden","message":"this caller may not invoke this method"}}

The stable codes are invalid_request, invalid_idempotency_key, idempotency_not_supported, invalid_stream_metadata, range_not_supported, unauthenticated, forbidden, method_not_found, actor_not_found, method_not_allowed, blob_not_found, idempotency_conflict, blob_gone, request_too_large, request_stream_too_large, unsupported_media_type, range_not_satisfiable, idempotency_capacity, response_too_large, response_stream_too_large, upstream_error, upstream_unavailable and upstream_timeout. The package README carries the same set with each status. the_readme_documents_every_error_code reads both files and fails when a code the gateway can answer with is missing from either, so this list cannot fall behind the enum. It compares codes alone: no test checks a status or a description, and those are reviewed by hand.

A client disconnect drops the gateway-owned Keylink call future. Work that already entered the backend can continue, and an idempotent call can settle as Interrupted. The gateway does not promise rollback.

The gateway does not expose tell, feeds, __Status, WebSockets, actor lifecycle, actor discovery, Keylink administration, or CORS. send, recv, blob, keyed actors and OpenAPI generation all exist — the package README carries the route table.

Try it

export KEYLINK_SECRET=$(openssl rand -hex 32)      # 64 hex characters, one identity
cargo run -p demo -- id                            # the id peers dial

# machine A
KEYLINK_SECRET=<64-hex> KEYLINK_ALLOWED_IDS=<peer-id> cargo run -p demo -- serve

# machine B
cargo run -p demo -- connect <endpoint-id> [file-to-upload]
cargo run -p demo -- connect-sync <endpoint-id>    # the same, without a runtime

Set KEYLINK_SECRET before you read an id. Every subcommand binds through Node::bind_from_env, which falls back to a random identity when the variable is unset — so demo -- id without it prints an address that the next command will not answer on. Each side keeps its own secret and puts the other side's printed id in KEYLINK_ALLOWED_IDS.

With one machine and no internet, set KEYLINK_OFFLINE=1 on both sides: the demo binds without discovery, prints a <id>@<ip:port> line, and accepts that form as the dial target.

Build both application versions from one commit to try the window. Leave the server on the default build (version 3), then connect from the other machine with cargo run -p demo --features demo/previous-contract -- connect <id>. Swap roles: serve with --features demo/previous-contract, then connect with the default build. Both directions print speaking 2 and finish with contract: OK. The newer client fills aisle: None and reports the version-3-only Count as Unsupported when talking to 2.

cargo run -p demo --features demo/swap-variants -- connect <id> builds a peer whose Currency variants are swapped — identical shapes, so postcard would decode Brl as Usd silently. The handshake refuses it instead.

The demo's Warehouse is served as an actor, so it also stands as the worked example of that model against an unmodified client. Its upload is a second service on a second ALPN, registered beside it on the same node, because a send method cannot be served as an actor — the split every actor with a bulk transfer now has to make.

One grant for a whole tree, between two machines on n0's public relays and discovery (a bare EndpointId is enough to dial):

cargo run -p keylink --example closure_share -- serve                  # machine A, prints its id
cargo run -p keylink --example closure_share -- fetch <endpoint-id>    # machine B

fetch ends with closure_share: OK or exits non-zero. With no arguments the same flow runs in one process, which is what mise run examples does.

Working on it

./scripts/install-hooks.sh   # once per clone — the hooks are not automatic
mise run gate                # the whole gate. There is no other one

CI is off by decision, and mise run gate is the gate. The workflow was deleted from main rather than left sitting there switched off, because a workflow nobody runs still looks like coverage — and the repository's Actions setting is off as well, so two independent things hold it. CONTRIBUTING.md measures both. The pre-push hook runs gate, so the check happens before a push instead of after one — and ./scripts/install-hooks.sh is what makes that true for a clone, which is why it is the first line above.

What that costs, stated rather than discovered: the gate now runs on one machine, on one OS. CI ran Linux, and a developer on a Mac no longer has anything checking that the tree builds on the other one. Nothing in the gate is weaker than it was — wasm-guards, msrv and deny moved into gate in the same change that removed the workflow, so the three checks only CI ran are still run — but they are run wherever you are, not everywhere.

wasm-guards is also stronger than the loop it replaces. The workflow asserted only that blocking, fs and env fail to build for wasm, which is vacuous: they fail anyway, deep inside tokio, once the feature turns on something a browser has no facility for. Disabling the fs guard left the CI check green. The task now requires the failure to quote keylink's own compile_error!, which is the claim the workflow's comment made and did not check.

What the gate needs on the machine

Install mise and rustup, then mise install. That fetches the five pinned tools — cargo-deny, ripgrep, sccache, wasm-bindgen-test-runner and chromedriver — and rust-toolchain.toml adds the wasm32-unknown-unknown target and the llvm-tools component. The second is not decoration: ring links its C through ar, and Apple's writes an archive index rust-lld cannot read.

Four things mise does not install, each named here because a missing one fails somewhere unhelpful:

  • Google Chrome, for wasm-browser. The chromedriver pin is coupled to the installed browser's major version, so a Chrome that auto-updates past the pinned one reddens the task. Move the pin, never bypass the task.
  • python3, which five gate steps are written in — anchors, anchors-selftest, headings, replay-anchors and task-lists.
  • curl, which both HTTP smokes drive the running gateway with.
  • A second Rust toolchain, 1.91, for msrv. About 200 MB on first use. That cost used to sit on CI; it is paid once per clone now, because an MSRV nobody compiles is a promise broken silently.

The steps run cheapest-failure-first, and [tasks.gate] in mise.toml is the list — not repeated here, because a second copy is a copy that goes stale. mise run pre-commit is the cheap subset a commit pays, and that one is pinned: mise run task-lists fails when any copy of it disagrees with mise.toml.

See CONTRIBUTING.md for the wire-contract pin, the public API rules, and when --no-verify is legitimate. The RFCs record the decisions, including the reversed ones.

Status

Early. The API will move. Not published to crates.io.

Contract windows support frozen application versions with explicit conversions, for singleton services and keyed registries. The application owns conversion policy; keylink's own wire is not yet versioned.

Greenfield: nothing here has ever run in production, so an ALPN move costs nothing and no compatibility shim is owed. Changes are taken in their correct form rather than their compatible one — every call site moves and the old path is deleted. That stops being true the day somebody deploys this, which is the argument for fixing the sharp things now.

A moved contract hash is therefore not a cost, and nothing here is designed around avoiding one. No change is scheduled, grouped or reshaped to keep a pin still, and a pin move is not reported as a price paid — a rename, a new variant and a reshaped struct are all free. The two tests that pin the literals are #[ignore]d, so not even the bookkeeping is owed. What still runs is the pair of claims about the mechanism rather than the value: that two type sets derive two different contracts, and that a peer built from other types is refused at the handshake. This holds until keylink has a stable version, which is when the pins come back and reversing it is the point of reaching one.

Known gaps, in rough priority order:

  • Trust is a local view. Revocations reach other nodes only through the application's feeder. Accepted records are permanent and never pruned. Each signer and proven upstream issuer has an accepted-revocation budget; exhausting a key permanently refuses its subtree rather than evicting tombstones. Path accounting also bounds churn through freshly delegated revoker keys. Adding new trusted roots supplies new operator-authorized budgets; it does not reset existing keys. Idle expired connections close only when the application applies trust::Change::Expire; expiry is checked on every call regardless. Already-dispatched calls finish after revocation. The host clock is trusted, with no skew allowance.

  • Blob withdrawal is an explicit binding. Trust::with_blobs must receive the same store as the services before any grants are issued. Without it, closing RPC admission does not retract grants. With it, any lost authority proof withdraws all of a known holder's grants, including narrowed and disconnected holders. Expired grants wait for the application's Change::Expire timer. Delivered bytes cannot be taken back.

  • Signed values are replayable. The application supplies freshness in T. Its shape is not part of a service contract hash; drift is refused at open. Idempotency records remain per device key, not per delegated principal.

  • Trust closes every tracked connection for a revoked peer. Connections are tracked independently across service ALPNs. The open presentation protocol has byte, depth, per-holder and total retained-chain caps, but no rate limit. New entries fail loudly at the total cap; feeders must handle capacity refusals. Pending unauthorized revocations are bounded separately; accepted tombstones are bounded by issuer-path budgets and are never evicted. Browsers can verify chains and signed values, not serve guarded QUIC connections.

  • No connection has ever run in a browser. mise run wasm-browser loads the module into headless Chrome and runs every test in keylink/tests/browser.rs there — the schema hash, a postcard round trip of WireResponse, both web body constructors, and the compose patience and lossy-broadcast boundaries — so the module is no longer merely linked. What is untested in a tab is everything that needs a socket: no Node, no Server, no Client, no actor turn. A page cannot open a UDP socket, so QUIC is out of reach whatever keylink does. Threaded wasm is refused outright: the local mailbox holds a thread-affine task handle behind a Send + Sync type.

  • A turn is never cancelled. A deadline bounds the caller's wait and sheds work that has not started; slow work goes off the turn with Deferred::away. A deadline's answer to a caller does not stop a turn that is running, so it sheds nothing once the turn has started, and a wedged keyed turn keeps its key Running past the caller's deadline, bounded by the per-peer running cap. It does record the turn's real reply under the key, so the retry receives it. QueueTimeout and a deadline both refuse a message that has not started, and leave its key claimable, because nothing ran.

  • Protocols cannot be added or removed after spawn. iroh-examples' custom-router shows a runtime-mutable alternative worth adopting.

  • Idempotency records are in memory by default, so a retry across a restart runs the handler again. Records::Durable keeps them on disk instead, at the storage cost and with the in-flight recovery rule Reliable delivery above describes, along with the other limits of the guarantee.

  • A keyed instance's Registry::ended(&id) future must be subscribed while the id is mounted; it returns None for an id that is not live and never inserts one. The future reports the entry's final loop-or-teardown ending, while Registry::stop(&id, budget) separately reports whether that shutdown drained or had to kill it.

  • Supervision restarts one actor, and no strategy reaches its siblings. LocalActor::spawn_supervised, LocalActor::spawn_supervised_deep, and run_keyed_supervised take an Intensity and put a fresh incarnation on the same task and the same mailbox, so a caller whose reply died mid-restart is told Restarting rather than Gone. There is no Supervisor type, no OneForAll/RestForOne and no parent/child tree; RFC-07 §9.8 refuses each one and gives the reason. compose::Group is the part of that caveat which was built — shared fate across several actors, retired in reverse insertion order — and its two verdicts are Forget and StopAll, so nothing there restarts a sibling either. Restarting is opt-in, so the rest of this bullet is the unsupervised default, and under a supervisor it is what one incarnation costs. What a fault costs depends on which input caused it: a panicking rpc turn costs that message, the actor survives, and the caller is told rather than left to infer it. A panic in a feed handler ends the actor, and so does a feed that simply stops — nobody is waiting on a feed item, so dropping its fault would carry on over a half-applied state. Ended says which happened. RFC-04 §8 has the original argument against supervision; RFC-07 §5.1 has the argument for an ending that names itself.

  • Error::SchemaMismatch is classified from the rendered error string, because the TLS alert code is several private layers down. Narrow and tested, but it would break if iroh reworded its message.

Contributing

Read CONTRIBUTING.md first, then run ./scripts/install-hooks.sh so mise run gate fires before your push rather than after it. There is no CI to catch what the hook misses — see Working on it for why, and for what that costs.

License

Licensed under either of

at your option.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

About

No description, website, or topics provided.

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages