diff --git a/CHANGELOG.md b/CHANGELOG.md index d336c92c9..758cf8758 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,11 @@ Crypto: no chain writes; the resolver path remains read-only. See `plans/2026-08-05-eth-crypto-bindings.md`. +SMP protocol: +- `ROWN` command (v23): the names an address owns, and whether the account has + been used at all, so a recovered wallet seed can be scanned for the accounts + already in use. See `plans/2026-09-22-names-owned-by.md`. + # 6.5.1 Version 6.5.1.0 diff --git a/plans/2026-09-22-names-owned-by.md b/plans/2026-09-22-names-owned-by.md new file mode 100644 index 000000000..432f42120 --- /dev/null +++ b/plans/2026-09-22-names-owned-by.md @@ -0,0 +1,90 @@ +# Owned-by: the names an address owns + +A recovered seed does not say how many of its accounts have been used. Until it +does, a restored device cannot take a fresh account without risking one its +owner already has, and cannot find the names it already holds. This adds the +question the protocol could not ask: what does this address own. + +This is Workstream B of the SimpleX names v2 plan, after the Ethereum crypto +primitives. The client consumer is the wallet recovery scan in simplex-chat. + +## What this delivers + +``` +GET /v2/owned-by/
?offset=N resolver, JSON OwnedNames +ROWN
-> ROWND SMP v23 (nameOwnedSMPVersion) +ownedSimplexNames agent, over ownedNames in the server +``` + +The agent returns the relay it used alongside the answer, so a caller can pass +it back as used and ask the next account elsewhere. A relay that cannot answer +at all - too old for ROWN, unreachable, or with no resolver of its own behind it +- is not the end of the lookup: it asks another, so a scan does not fail +wholesale on the one relay it happened to draw during a version rollout. Sending every account of a +seed to one relay would tell that relay the accounts belong to one wallet; the +scan therefore asks one account per relay, and `getNextNameServer` avoids the +hosts already used where the configured set allows. `proxiedSMPRelayVersion` is +pinned to the current version so ROWN can be proxied at all — a scan that falls +back to a direct session hands the relay its address with its IP. The pin is +necessary but not sufficient: under the default `SPMUnknown` proxy mode a +configured names server is a known host, so the scan reaches it directly, as +RSLV already does. + +Enumeration is read off the ERC-721 registrar (`balanceOf`, +`tokenOfOwnerByIndex`, `labelOf`), not from registration logs: the token is the +name, so a name acquired by transfer counts. Each name is then answered with the +`NameResponse` resolving it gives, so the client that scans can list the names +and act on them without asking again — the reason the per-name cost is worth +paying. The registrar does not maintain enumeration on expiry, so a name past +its grace is still enumerated; it answers as available, naming nothing the +account holds, so it is left out while still counting towards `inUse`. + +## What `inUse` actually answers + +`inUse` is `holds a name now, or nonce > 0, or balance > 0`, on the one chain +the resolver is configured with. Holding a name is only one way for an account +to be in use: an account that was funded, or ever sent a transaction, is in use +with no name, and a scan that reads names alone hands out an account its owner +is already using. Within that scope the signal is sound — an EOA's balance can +only fall through its own transaction, which bumps the nonce, so "funded then +drained" still reads as used, and an EIP-7702 delegation leaves the authority's +nonce incremented, so it needs no separate code check. + +It does not see: + +- **Accounts holding only other tokens.** An EOA can hold ERC-20s or NFTs with + nonce 0 and no ether. ERC-20 has no reverse index, so answering this needs + `eth_getLogs` over Transfer topics for all history, which providers cap, or an + external indexer. Out of scope here. +- **Accounts used on another chain.** The same key is the same address on every + L2 and sidechain; this chain's nonce and balance stay 0. +- **History, as opposed to current state.** `balanceOf` is present ownership. A + name registered for an account by a controller — which leaves the owner at + nonce 0 — and later transferred away, or lapsed and re-registered by someone + else, leaves no trace. + +The cost is bounded. A name-holding account is always found, so no account that +holds a name is ever mislabelled. A missed account matters only as a gap-limit +reset: the scan can stop up to `scanGapLimit` indexes early and not reach a name +beyond that. That is recovery incompleteness, recoverable by scanning again on a +better signal, not loss. In the other direction a dust transfer marks an account +used; the indexes come from hardened derivation with no published xpub, so they +cannot be enumerated to be dusted. + +## What is deliberately absent + +- **No token or indexer integration**, per the limits above. +- **Nonce and balance are not carried over SMP.** The resolver reports them, and + `OwnedNames` keeps only the names, the flag and the cursor, so a client cannot + yet see why an account is in use. +- **No caching, and no batching.** One owned-by is a `balanceOf` per configured + TLD, then for each name the same reads resolving it takes - roughly fifteen + `eth_call`s - plus the nonce and balance, all issued one at a time, so it is + forked on + the server the same way RSLV is and nothing is memoised. The page size trades + against both the relay's response cap and its timeout, and it bounds names per + registrar rather than per page, so a second TLD doubles a full page. +- **Enumeration is not atomic.** `balanceOf` and each `tokenOfOwnerByIndex` are + separate calls at `latest`, so a transfer between them can duplicate or skip an + entry, and successive pages can straddle blocks. Pinning every call to one + block number is the fix if it ever matters for a recovery scan. diff --git a/protocol/simplex-messaging.md b/protocol/simplex-messaging.md index a80d211b2..3350b391e 100644 --- a/protocol/simplex-messaging.md +++ b/protocol/simplex-messaging.md @@ -70,6 +70,7 @@ Version 21, 2026-07-05 - [Resolver commands](#resolver-commands) - [Resolve name command](#resolve-name-command) - [Name record response](#name-record-response) + - [Names owned by an address command](#names-owned-by-an-address-command) - [Transport connection with the SMP router](#transport-connection-with-the-SMP-router) - [General transport protocol considerations](#general-transport-protocol-considerations) - [TLS transport encryption](#tls-transport-encryption) @@ -86,7 +87,7 @@ It's designed with the focus on communication security and integrity, under the It is designed as a low level protocol for other application protocols to solve the problem of secure and private message transmission, making [MITM attack][1] very difficult at any part of the message transmission system. -This document describes SMP protocol version 22. Versions 1-5 are discontinued. The version history: +This document describes SMP protocol version 23. Versions 1-5 are discontinued. The version history: - v1: binary protocol encoding - v2: message flags (used to control notifications) @@ -109,6 +110,7 @@ This document describes SMP protocol version 22. Versions 1-5 are discontinued. - v20: public namespaces resolver (RSLV command, RNAME response) — direct or forwarded via PFWD - v21: server public information in handshake - v22: `RNAME` says whether a name can be registered, not only what it resolves to +- v23: the names an address owns (ROWN command, ROWND response) — direct or forwarded via PFWD ## Introduction @@ -1453,7 +1455,8 @@ companion REST resolver process (`scripts/resolver/snrc-resolve.py`) that queries the SNRC contract on Ethereum; alternative backings (different chains, DHT, etc.) are valid as long as they expose the documented HTTP shape (`GET /v2/resolve/` returning a `NameRegistration` on 200 for every -registration shape, 400 for unknown TLDs, 502 for upstream failures) or +registration shape, 400 for unknown TLDs, 502 for upstream failures, and from +SMP v23 `GET /v2/owned-by/
?offset=N` returning an `ownedNames`) or substitute a different transport returning the same JSON. The resolver API is versioned separately from this protocol: `/v1/resolve/` returns a bare `NameRecord` and is what relays before v22 call as `/resolve/`. @@ -1623,6 +1626,52 @@ The names router caps the resolver response it will accept stays within the SMP proxied transmission budget of 16224 bytes; a response over the cap is `ERR NAME RESOLVER`. +#### Names owned by an address command + +`ROWN` asks which names an address holds, and whether the account has been used +at all, so a device restoring a wallet seed can find the accounts already in +use. It is unauthenticated and accepted direct or forwarded, as `RSLV` is, from +v23. + +```abnf +rown = %s"ROWN" SP address offset +address = length "0x" 40HEXDIG ; EIP-55 checksummed, length-prefixed +length = 1*1 OCTET ; 42 +offset = 4*4 OCTET ; Word32, network byte order: where to resume listing +``` + +```abnf +rownd = %s"ROWND" SP ownedNames +``` + +`ownedNames` is a UTF-8 JSON object consuming the remainder of the transmission. + +| Field | JSON type | Constraints | +|---|---|---| +| `lastBlockTs` | number | the oldest block any read behind this answer saw | +| `names` | array | the names the address holds, each the `nameResponse` resolving it answers with | +| `inUse` | boolean | whether the account has been used at all | +| `nextOffset` | number | cursor to resume from, absent when the listing is complete | + +Each name is answered in full, as `RSLV` answers it, so a client can list the +names and act on them without resolving each one again. Enumeration is not +maintained on expiry, so the registrar still enumerates a name past its grace; +it answers as `available` and names nothing the account holds, so it is left out +of `names` while still counting towards `inUse`. + +As with `RNAME`, a router reads through a node of its own, which can lag. Every +name carries the block it was read at, and `lastBlockTs` is the oldest of those +and of the enumeration's own block, so a client can tell a badly lagging router +even from an answer that carries no names — which is the answer a recovery scan +acts on. + +`inUse` is what the registry could see on its own chain — the account's nonce, +its balance, and every name it holds, which is not only the names listed here: +a later page of a held account still reports `inUse` true with an empty `names`. Holding a name is only one way for an account +to be in use, and an account used only for other tokens, or on another chain, +reads as unused, so a client MUST NOT treat `inUse` as false meaning the account +has never been used. + ## Transport connection with the SMP router ### General transport protocol considerations diff --git a/scripts/resolver/README.md b/scripts/resolver/README.md index f8f5dde8b..dafe139c5 100644 --- a/scripts/resolver/README.md +++ b/scripts/resolver/README.md @@ -93,7 +93,7 @@ curl -s http://127.0.0.1:8000/v2/resolve/foobar.testing | jq | reth p2p | `:30303` tcp/udp | Ethereum sync (open on firewall) | | nimbus p2p | `:9000` tcp/udp | beacon sync (open on firewall) | | nimbus REST | `127.0.0.1:5052` | beacon API | -| **resolver** | `127.0.0.1:8000` | SNRC REST (`/v2/resolve`, `/resolve`, `/health`) | +| **resolver** | `127.0.0.1:8000` | SNRC REST (`/v2/resolve`, `/v2/owned-by`, `/resolve`, `/health`) | ## Caveats @@ -121,12 +121,13 @@ standalone for local dev (no Docker), via [`uv`](https://docs.astral.sh/uv/): uv run scripts/resolver/service/snrc-resolve.py # defaults to local reth + mainnet .testing ``` -Three routes, versioned separately from the protocol so each only changes when +Four routes, versioned separately from the protocol so each only changes when its own shape does: | Route | Called by | Answers | |---|---|---| | `/v2/resolve/` | routers from SMP v22 | a `NameRegistration` | +| `/v2/owned-by/
` | routers from SMP v23 | the names an address owns | | `/resolve/` | routers before SMP v22 | a name record, flat | | `/health` | anyone | readiness | @@ -165,6 +166,35 @@ label, so a hashed query cannot be answered with a name. See A subname reports the expiry and grace of the 2LD above it, since that is what bounds its lifetime. A subname nobody created reports as not registered. +### v2: `/v2/owned-by/
?offset=N` + +The body is the SMP protocol's `OwnedNames`: the names the address holds, each +the same `NameResponse` `/v2/resolve` answers with, and `inUse`, whether the +account has been used at all. The router decodes it as is and forwards it, so +the fields are specified with the wire, in the **Names owned by an address +command** section of +[`protocol/simplex-messaging.md`](../../protocol/simplex-messaging.md). + +Answering each name in full is what lets a caller list and act on them without a +second request for each. `lastBlockTs` is the oldest block any of the reads saw, +so an answer with no names still says how far behind the node was. Enumeration comes off the registrar's ERC-721 index, so +a name acquired by transfer counts; one past its grace is still enumerated but +answers as available, so it is left out while still counting towards `inUse`. + +`offset` is the cursor to resume from, and it counts per registrar, so a page +holds up to `SNRC_MAX_OWNED` names for each configured TLD. `nextOffset` is the +next cursor, or null when the listing is complete. `inUse` covers every name the +address holds, not only the page, so a later page reports it true with an empty +`names`. + +| Status | Meaning | +|---|---| +| 200 | the names the address holds, with `inUse` | +| 400 | `badAddress`, `badOffset`, `noRegistrarConfigured` | +| 502 | `upstreamError` | + +Error bodies carry `address`, a fixed `error` code and a `message`. + ### v1: `/resolve/` What routers before SMP v22 call. Its shape is unrelated to v2's: the record is @@ -374,3 +404,8 @@ here. To override any of them, set `SNRC_REGISTRY_`, `SNRC_REGISTRAR_` or `SNRC_CONTROLLER_` on the `resolver` service in `docker-compose.yml`, or as env vars when you run the script directly. + +`SNRC_MAX_OWNED` (default 16) caps the names `/v2/owned-by` returns per +registrar per page. A router will not read a body over 16000 bytes, and a client +cannot ask for a smaller page, so keep it times the number of configured TLDs +well under that. diff --git a/scripts/resolver/service/snrc-resolve.py b/scripts/resolver/service/snrc-resolve.py index 42881a2a8..be61ce1f0 100755 --- a/scripts/resolver/service/snrc-resolve.py +++ b/scripts/resolver/service/snrc-resolve.py @@ -44,6 +44,8 @@ SNRC_CONTROLLER_ SimplexController (proxy) for the TLD; reservations, and through its `prices()` oracle what registering costs (default: mainnet for .testing, empty for .simplex) + SNRC_MAX_OWNED Names /v2/owned-by returns per registrar per page + (default: 16) SNRC_PORT Listen port (default: 8000) SNRC_BIND Bind address (default: 0.0.0.0) @@ -69,7 +71,7 @@ import sys import time from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer -from urllib.parse import unquote, urlparse +from urllib.parse import parse_qs, unquote, urlparse from urllib.request import Request, urlopen from eth_hash.auto import keccak @@ -124,6 +126,9 @@ ZERO_ADDR = "0x0000000000000000000000000000000000000000" +# Names per registrar per /owned-by page; times the configured TLDs, a full page of NameResponses must fit the 16000 bytes a relay reads. +MAX_OWNED = max(1, int(os.environ.get("SNRC_MAX_OWNED", "8"))) + # The registry prices in attoUSD (1e-18 USD); the protocol carries US cents. @@ -417,6 +422,14 @@ def encode_uint(value: int) -> str: return value.to_bytes(32, "big").hex() +def encode_address(address: str) -> str: + return (12 * "00") + address[2:].lower() + + +def is_address(address: str) -> bool: + return len(address) == 42 and address.startswith("0x") and all(c in "0123456789abcdef" for c in address[2:].lower()) + + def encode_text_call(node: bytes, key: str) -> str: sel = selector("text(bytes32,string)") head = node.hex() + (0x40).to_bytes(32, "big").hex() @@ -875,12 +888,89 @@ def resolve(name: str): } +def owned_by(address: str, offset: int = 0): + """Every name an address holds, across every configured TLD, with the + account's on-chain footprint. + + Enumeration is read off the ERC-721 registrar, so a name acquired by + transfer counts. Each name is answered with the NameResponse /v2/resolve + gives for it, so a caller can list and act on them without asking again. + `lastBlockTs` is the oldest block any of those reads saw, so a caller can + tell a resolver lagging critically behind even when it holds no names. + `inUse` is what a recovery scan asks, derived from this chain's nonce and + balance and every name it holds, not only the page returned: an account + holding only other tokens is not seen, and neither is one used on another + chain. + """ + if not is_address(address): + return 400, { + "address": address, + "error": "badAddress", + "message": "expected a 0x-prefixed 20-byte address", + } + + if offset < 0: + return 400, { + "address": address, + "error": "badOffset", + "message": "offset is a position in the listing", + } + + configured = {t: r for t, r in REGISTRARS.items() if r} + if not configured: + return 400, { + "address": address, + "error": "noRegistrarConfigured", + "message": "no registrar is configured on this resolver", + "configuredTlds": [], + } + + # the oldest block behind the answer: this one, or any name's own + last_block_ts = chain_now() + names, truncated, total_held = [], False, 0 + for tld, registrar in configured.items(): + held = decode_uint(eth_call(registrar, selector("balanceOf(address)") + encode_address(address))) + total_held += held + last = min(offset + MAX_OWNED, held) + if last < held: + truncated = True + for i in range(offset, last): + token = decode_uint( + eth_call(registrar, selector("tokenOfOwnerByIndex(address,uint256)") + encode_address(address) + encode_uint(i)) + ) + label = registered_label(registrar, token) + if not label: + continue + status, body = registration(label + "." + tld) + # one past its grace answers as available, naming nothing it holds + if status == 200 and body["registration"]["type"] == "registered": + names.append(body) + if body["lastBlockTs"] is not None: + last_block_ts = min(last_block_ts, body["lastBlockTs"]) + + nonce = decode_uint(rpc("eth_getTransactionCount", [address, "latest"])) + balance = decode_uint(rpc("eth_getBalance", [address, "latest"])) + names.sort(key=lambda n: n["registration"]["nameRecord"]["name"]) + return 200, { + "address": address, + "lastBlockTs": last_block_ts, + "names": names, + "nonce": nonce, + "balance": str(balance), + # every name held, not just this page, or page two reads as owning nothing + "inUse": total_held > 0 or nonce > 0 or balance > 0, + "offset": offset, + "nextOffset": offset + MAX_OWNED if truncated else None, + "checkedTlds": sorted(configured), + } + + # ---------- HTTP layer ---------- class Handler(BaseHTTPRequestHandler): def do_GET(self): # noqa: N802 - http.server contract - path = urlparse(self.path).path - parts = [unquote(p) for p in path.split("/") if p] + parsed = urlparse(self.path) + parts = [unquote(p) for p in parsed.path.split("/") if p] if parts == ["health"]: self._respond(200, {"ok": True, "rpc": RPC, "registries": REGISTRIES, **head_block()}) @@ -898,6 +988,20 @@ def do_GET(self): # noqa: N802 - http.server contract self._respond(status, body) return + if len(parts) == 3 and parts[0] == "v2" and parts[1] == "owned-by": + address = parts[2].strip().lower() + try: + offset = int(parse_qs(parsed.query).get("offset", ["0"])[0]) + except ValueError: + self._respond(400, {"address": address, "error": "badOffset", "message": "offset is a decimal position in the listing"}) + return + try: + status, body = owned_by(address, offset) + except Exception as e: # surface upstream errors as 502 + status, body = 502, upstream_error({"address": address}, e) + self._respond(status, body) + return + # /v1/resolve is an alias: relays before SMP v22 call /resolve if parts[:2] == ["v1", "resolve"] and len(parts) == 3: parts = ["resolve", parts[2]] @@ -926,7 +1030,7 @@ def do_GET(self): # noqa: N802 - http.server contract { "error": "noSuchRoute", "message": "not found", - "routes": ["/health", "/v2/resolve/", "/v1/resolve/", "/resolve/"], + "routes": ["/health", "/v2/resolve/", "/v2/owned-by/
", "/v1/resolve/", "/resolve/"], }, ) @@ -952,7 +1056,7 @@ def main(): ) for tld, addr in REGISTRIES.items(): sys.stderr.write(f" .{tld:<8s} = {addr or '(not configured)'}\n") - sys.stderr.write(" GET /v2/resolve/ GET /v1/resolve/ GET /health\n") + sys.stderr.write(" GET /v2/resolve/ GET /v2/owned-by/
GET /v1/resolve/ GET /health\n") try: server.serve_forever() except KeyboardInterrupt: diff --git a/scripts/resolver/service/test_snrc_resolve.py b/scripts/resolver/service/test_snrc_resolve.py index 4b0555ed1..fd417e57c 100644 --- a/scripts/resolver/service/test_snrc_resolve.py +++ b/scripts/resolver/service/test_snrc_resolve.py @@ -1035,5 +1035,156 @@ def test_an_available_name_says_so_too(self): self.assertEqual(res["lastBlockTs"], self.now) self.assertEqual(res["registration"]["type"], "available") +class OwnedByTests(unittest.TestCase): + """`/v2/owned-by` answers what a recovery scan asks: which names an account + holds, and whether the account has been used at all. Enumeration is read + off the registrar's ERC-721 index, so a name acquired by transfer counts + the same as one registered here.""" + + REGISTRAR = "0xef47eb4384b46c89e4482a677c2cbcbd2a6fd85a" + ADDR = "0x70997970c51812dc3a010c7d01b50e0d17dc79c8" + GRACE = 90 * 86400 + # keccak-256("alice") + ALICE = 0x9C0257114EB9399A2985F8E75DAD7600C5D89FE3824FFA99EC1C3EB8BF3B0501 + + BLOCK_TS = 1813000000 + + def setUp(self): + self._saved = (snrc.REGISTRARS, snrc.eth_call, snrc.rpc, snrc.registration, snrc.chain_now) + snrc.REGISTRARS = {"testing": self.REGISTRAR, "simplex": ""} + snrc.rpc = lambda method, params: "0x0" + snrc.chain_now = lambda: self.BLOCK_TS + # RegistrationV2Tests covers resolving one; owned-by picks which + snrc.registration = lambda name: (200, self.response(name, "registered")) + + def tearDown(self): + snrc.REGISTRARS, snrc.eth_call, snrc.rpc, snrc.registration, snrc.chain_now = self._saved + + def response(self, name, type_, block_ts=None): + reg = {"type": type_} + if type_ == "registered": + reg["nameRecord"] = {"name": name} + return {"lastBlockTs": self.BLOCK_TS if block_ts is None else block_ts, "registration": reg} + + def _chain(self, held, label=b"alice"): + def call(to, data): + sel = data[:10] + if sel == snrc.selector("balanceOf(address)"): + return snrc.encode_uint(held) + if sel == snrc.selector("tokenOfOwnerByIndex(address,uint256)"): + return snrc.encode_uint(self.ALICE) + if sel == snrc.selector("labelOf(uint256)"): + return "0x" + snrc.encode_uint(32) + snrc.encode_uint(len(label)) + label.hex() + "00" * ((-len(label)) % 32) + raise AssertionError("unexpected call " + sel) + return call + + def names(self, body): + return [n["registration"]["nameRecord"]["name"] for n in body["names"]] + + def test_a_held_name_is_answered_as_a_full_registration(self): + """The caller acts on the names straight away, so each is the same + NameResponse resolving it would give.""" + snrc.eth_call = self._chain(1) + status, body = snrc.owned_by(self.ADDR) + self.assertEqual(status, 200) + self.assertEqual(self.names(body), ["alice.testing"]) + self.assertEqual(body["names"][0]["registration"]["type"], "registered") + self.assertIn("lastBlockTs", body["names"][0]) + + def test_a_name_past_its_grace_is_not_listed_as_held(self): + """Enumeration keeps it until someone re-registers, but it answers as + available, which names nothing the account still holds.""" + snrc.eth_call = self._chain(1) + snrc.registration = lambda name: (200, self.response(name, "available")) + _, body = snrc.owned_by(self.ADDR) + self.assertEqual(body["names"], []) + self.assertTrue(body["inUse"]) + + def test_holding_a_name_is_in_use(self): + snrc.eth_call = self._chain(1) + _, body = snrc.owned_by(self.ADDR) + self.assertTrue(body["inUse"]) + + def test_an_account_that_sent_a_transaction_is_in_use_with_no_name(self): + """The case a names-only scan gets wrong: an account in use, holding + nothing, would be handed out again.""" + snrc.eth_call = self._chain(0) + snrc.rpc = lambda method, params: "0x3" if method == "eth_getTransactionCount" else "0x0" + _, body = snrc.owned_by(self.ADDR) + self.assertEqual(body["names"], []) + self.assertEqual(body["nonce"], 3) + self.assertTrue(body["inUse"]) + + def test_a_funded_account_is_in_use_with_no_name_and_no_nonce(self): + snrc.eth_call = self._chain(0) + snrc.rpc = lambda method, params: "0x0" if method == "eth_getTransactionCount" else "0xde0b6b3a7640000" + _, body = snrc.owned_by(self.ADDR) + self.assertEqual(body["balance"], "1000000000000000000") + self.assertTrue(body["inUse"]) + + def test_an_untouched_account_is_not_in_use(self): + snrc.eth_call = self._chain(0) + _, body = snrc.owned_by(self.ADDR) + self.assertFalse(body["inUse"]) + self.assertIsNone(body["nextOffset"]) + + def test_more_names_than_a_page_carry_the_cursor_to_resume_from(self): + snrc.eth_call = self._chain(snrc.MAX_OWNED + 1) + _, body = snrc.owned_by(self.ADDR) + self.assertEqual(body["nextOffset"], snrc.MAX_OWNED) + self.assertEqual(len(body["names"]), snrc.MAX_OWNED) + + def test_a_malformed_address_is_refused(self): + status, body = snrc.owned_by("0xnothex") + self.assertEqual(status, 400) + self.assertEqual(body["error"], "badAddress") + + def test_a_page_past_the_end_still_reports_the_account_in_use(self): + """inUse is a property of the account, not of the page: an empty later + page must not read as an account that owns nothing.""" + snrc.eth_call = self._chain(1) + _, body = snrc.owned_by(self.ADDR, snrc.MAX_OWNED) + self.assertEqual(body["names"], []) + self.assertTrue(body["inUse"]) + + def test_a_label_that_is_not_utf8_is_reported_not_fatal(self): + """The registrar stores the label bytes unchecked, so one bad label + must not take down the whole listing.""" + snrc.eth_call = self._chain(1, label=b"\xff\xfe") + status, body = snrc.owned_by(self.ADDR) + self.assertEqual(status, 200) + self.assertEqual(len(body["names"]), 1) + + def test_an_account_with_no_names_still_reports_the_block_it_was_read_at(self): + """inUse false is the answer a scan acts on, so it has to carry how + stale the registry it came from is.""" + snrc.eth_call = self._chain(0) + _, body = snrc.owned_by(self.ADDR) + self.assertEqual(body["names"], []) + self.assertEqual(body["lastBlockTs"], self.BLOCK_TS) + + def test_the_oldest_block_of_every_read_is_the_one_reported(self): + """A name resolved against a node further behind is what the caller + has to judge the whole answer by.""" + snrc.eth_call = self._chain(1) + snrc.registration = lambda name: (200, self.response(name, "registered", self.BLOCK_TS - 600)) + _, body = snrc.owned_by(self.ADDR) + self.assertEqual(body["lastBlockTs"], self.BLOCK_TS - 600) + + def test_a_negative_offset_is_refused(self): + snrc.eth_call = self._chain(1) + status, body = snrc.owned_by(self.ADDR, -1) + self.assertEqual(status, 400) + self.assertEqual(body["error"], "badOffset") + + def test_no_configured_registrar_is_an_error_not_an_empty_answer(self): + """An empty list would read as "this key owns nothing", which is the + one answer a scan must not invent.""" + snrc.REGISTRARS = {"testing": "", "simplex": ""} + status, body = snrc.owned_by(self.ADDR) + self.assertEqual(status, 400) + self.assertEqual(body["error"], "noRegistrarConfigured") + + if __name__ == "__main__": unittest.main() diff --git a/src/Simplex/Messaging/Agent.hs b/src/Simplex/Messaging/Agent.hs index e9a7379b4..c6d68e20e 100644 --- a/src/Simplex/Messaging/Agent.hs +++ b/src/Simplex/Messaging/Agent.hs @@ -67,6 +67,7 @@ module Simplex.Messaging.Agent deleteConnShortLink, getConnShortLink, resolveSimplexName, + ownedSimplexNames, getConnLinkPrivKey, deleteLocalInvShortLink, changeConnectionUser, @@ -219,13 +220,15 @@ import qualified Simplex.Messaging.Crypto.Ratchet as CR import qualified Simplex.Messaging.Crypto.ShortLink as SL import Simplex.Messaging.Encoding import Simplex.Messaging.Encoding.String +import Simplex.Messaging.Eth.Address (Address) +import Simplex.Messaging.Names.Record (OwnedNames) import Simplex.Messaging.Notifications.Protocol (DeviceToken, NtfRegCode (NtfRegCode), NtfTknStatus (..), NtfTokenId, PNMessageData (..), pnMessagesP) import Simplex.Messaging.Notifications.Types import Simplex.Messaging.Parsers (defaultJSON, parse) import Simplex.Messaging.Protocol ( BrokerMsg, Cmd (..), - ErrorType (AUTH), + ErrorType (AUTH, NAME), MsgBody, MsgFlags (..), NameResponse, @@ -465,6 +468,11 @@ resolveSimplexName :: AgentClient -> NetworkRequestMode -> UserId -> SimplexDoma resolveSimplexName c nm userId domain = withAgentEnv c $ resolveSimplexName' c nm userId domain {-# INLINE resolveSimplexName #-} +-- | Names an address owns, with the relay used, so a scan can ask the next account elsewhere. +ownedSimplexNames :: AgentClient -> NetworkRequestMode -> UserId -> [SMPServer] -> Address -> Word32 -> AE (SMPServer, OwnedNames) +ownedSimplexNames c nm userId used addr offset = withAgentEnv c $ ownedSimplexNames' c nm userId used addr offset +{-# INLINE ownedSimplexNames #-} + getConnLinkPrivKey :: AgentClient -> ConnId -> AE (Maybe C.PrivateKeyEd25519) getConnLinkPrivKey c = withAgentEnv c . getConnLinkPrivKey' c {-# INLINE getConnLinkPrivKey #-} @@ -1272,9 +1280,25 @@ deleteLocalInvShortLink' c (CSLInvitation _ srv linkId _) = withStore' c $ \db - resolveSimplexName' :: AgentClient -> NetworkRequestMode -> UserId -> SimplexDomain -> AM NameResponse resolveSimplexName' c nm userId domain = do - resolverSrv <- getNextNameServer c userId + resolverSrv <- getNextNameServer c userId [] resolveName c nm userId resolverSrv domain +ownedSimplexNames' :: AgentClient -> NetworkRequestMode -> UserId -> [SMPServer] -> Address -> Word32 -> AM (SMPServer, OwnedNames) +ownedSimplexNames' c nm userId used addr offset = tryRelays ownedNamesRelays used + where + tryRelays attempts tried = do + srv <- getNextNameServer c userId tried + ((srv,) <$> ownedNames c nm userId srv addr offset) `catchError` \e -> + if attempts > 1 && cannotAnswer e then tryRelays (attempts - 1) (srv : tried) else throwE e + -- cannot answer, as against answers: too old for ROWN, unreachable, or with no resolver + cannotAnswer e = temporaryOrHostError e || case e of + SMP _ (NAME _) -> True + _ -> False + +-- | Relays one owned-names lookup asks before it gives up: during a version rollout the first one picked may have no ROWN. +ownedNamesRelays :: Int +ownedNamesRelays = 3 + changeConnectionUser' :: AgentClient -> UserId -> ConnId -> UserId -> AM () changeConnectionUser' c oldUserId connId newUserId = do SomeConn _ conn <- withStore c (`getConn` connId) diff --git a/src/Simplex/Messaging/Agent/Client.hs b/src/Simplex/Messaging/Agent/Client.hs index 8cbb4c17b..637ef0c16 100644 --- a/src/Simplex/Messaging/Agent/Client.hs +++ b/src/Simplex/Messaging/Agent/Client.hs @@ -70,6 +70,7 @@ module Simplex.Messaging.Agent.Client secureGetQueueLink, getQueueLink, resolveName, + ownedNames, getNextNameServer, enableQueueNotifications, EnableQueueNtfReq (..), @@ -258,6 +259,8 @@ import Simplex.Messaging.Crypto.BBS (BBSPresHeader (..), BBSPublicKey) import Simplex.Messaging.Crypto.Entitlement (EntitlementCredential, EntitlementProof, generateEntitlementProof) import Simplex.Messaging.Encoding import Simplex.Messaging.Encoding.String +import Simplex.Messaging.Eth.Address (Address) +import Simplex.Messaging.Names.Record (OwnedNames) import Simplex.Messaging.Notifications.Client import Simplex.Messaging.Notifications.Protocol import Simplex.Messaging.Notifications.Transport (NTFVersion) @@ -2029,14 +2032,23 @@ resolveName c nm userId server domain = resolveViaProxy smp proxySess = proxyResolveName smp nm proxySess domain resolveDirectly smp = directResolveName smp nm domain +-- | Names an address owns, from one names-capable relay, proxied as resolveName is. +ownedNames :: AgentClient -> NetworkRequestMode -> UserId -> SMPServer -> Address -> Word32 -> AM OwnedNames +ownedNames c nm userId server addr offset = + snd <$> sendOrProxySMPCommand c nm userId server "" "ROWN" NoEntity ownedViaProxy ownedDirectly + where + ownedViaProxy smp proxySess = proxyOwnedNames smp nm proxySess addr offset + ownedDirectly smp = directOwnedNames smp nm addr offset + -- | Pick a names-capable server for the user (the agent owns server selection, -- accounting for the names role). nameSrvs is opt-in (a plain list); empty means -- no server resolves names - a declared agent error, never a fallback. -getNextNameServer :: AgentClient -> UserId -> AM SMPServer -getNextNameServer c userId = +-- Used servers are avoided where the set allows, operator first: one operator asked about every account learns the wallet. +getNextNameServer :: AgentClient -> UserId -> [SMPServer] -> AM SMPServer +getNextNameServer c userId usedSrvs = liftIO (TM.lookupIO userId (userServers c :: TMap UserId (UserServers 'PSMP))) >>= \case Just UserServers {nameSrvs} -> case L.nonEmpty nameSrvs of - Just srvs -> protoServer <$> pickServer srvs + Just srvs -> protoServer . snd <$> getNextServer_ srvs (usedOperatorsHosts srvs usedSrvs) Nothing -> throwE NO_NAME_SERVERS Nothing -> throwE $ INTERNAL "unknown userId - no user servers" diff --git a/src/Simplex/Messaging/Client.hs b/src/Simplex/Messaging/Client.hs index f8f1a4cb9..833a9500e 100644 --- a/src/Simplex/Messaging/Client.hs +++ b/src/Simplex/Messaging/Client.hs @@ -75,6 +75,8 @@ module Simplex.Messaging.Client proxySMPMessage, proxyResolveName, directResolveName, + proxyOwnedNames, + directOwnedNames, forwardSMPTransmission, getSMPQueueInfo, sendProtocolCommand, @@ -154,6 +156,7 @@ import Data.Maybe (catMaybes, fromMaybe) import Data.Text (Text) import qualified Data.Text as T import Data.Time.Clock (UTCTime (..), diffUTCTime, getCurrentTime) +import Data.Word (Word32) import qualified Data.X509 as X import qualified Data.X509.Validation as XV import Network.Socket (HostName, ServiceName) @@ -162,6 +165,8 @@ import Numeric.Natural import qualified Simplex.Messaging.Crypto as C import Simplex.Messaging.Encoding import Simplex.Messaging.Encoding.String +import Simplex.Messaging.Eth.Address (Address) +import Simplex.Messaging.Names.Record (OwnedNames) import Simplex.Messaging.Parsers (defaultJSON, dropPrefix, enumJSON, sumTypeJSON) import Simplex.Messaging.Protocol import Simplex.Messaging.Protocol.Types @@ -1076,6 +1081,25 @@ directResolveName c nm name r -> throwE $ unexpectedResponse r | otherwise = throwE $ PCETransportError TEVersion +-- | Names an address owns, via PFWD: a scan links accounts, so hiding the client IP matters more here than for one name. +proxyOwnedNames :: SMPClient -> NetworkRequestMode -> ProxiedRelay -> Address -> Word32 -> ExceptT SMPClientError IO (Either ProxyClientError OwnedNames) +proxyOwnedNames c nm proxiedRelay addr offset + | prVersion proxiedRelay >= nameOwnedSMPVersion = + proxySMPCommand c nm proxiedRelay Nothing NoEntity (ROWN addr offset) >>= \case + Right (ROWND owned) -> pure $ Right owned + Right r -> throwE $ unexpectedResponse r + Left e -> pure $ Left e + | otherwise = throwE $ PCETransportError TEVersion + +-- | Direct (non-PFWD) owned-names lookup, exposing the client IP. +directOwnedNames :: SMPClient -> NetworkRequestMode -> Address -> Word32 -> ExceptT SMPClientError IO OwnedNames +directOwnedNames c nm addr offset + | thVersion (thParams c) >= nameOwnedSMPVersion = + sendProtocolCommand c nm Nothing NoEntity (Cmd SResolver (ROWN addr offset)) >>= \case + ROWND owned -> pure owned + r -> throwE $ unexpectedResponse r + | otherwise = throwE $ PCETransportError TEVersion + resolvedNameOrNotFound :: SimplexDomain -> NameResponse -> Bool resolvedNameOrNotFound d NameResponse {registration} = case registration of NRRegistered {nameRecord} -> T.toLower (nrName nameRecord) == fullDomainName d diff --git a/src/Simplex/Messaging/Eth/Address.hs b/src/Simplex/Messaging/Eth/Address.hs index f0d9a5ef9..fd95d5e6d 100644 --- a/src/Simplex/Messaging/Eth/Address.hs +++ b/src/Simplex/Messaging/Eth/Address.hs @@ -19,8 +19,10 @@ import Data.Char (isDigit, isHexDigit, isLower, isUpper, toLower) import Data.Word (Word32, Word8) import Simplex.Messaging.Crypto.BIP32 (hardened) import qualified Simplex.Messaging.Crypto.Secp256k1 as S +import Simplex.Messaging.Encoding import Simplex.Messaging.Encoding.String import Simplex.Messaging.Eth.Keccak (keccak256) +import Simplex.Messaging.Util ((<$?>)) -- | A 20-byte Ethereum address. 'Show' renders the EIP-55 checksummed form, as pasted into a block explorer. newtype Address = Address ByteString @@ -48,6 +50,11 @@ instance StrEncoding Address where where letters = BC.filter (not . isDigit) body +-- | The EIP-55 hex, length-prefixed, so a field following it is not swallowed by the hex parser. +instance Encoding Address where + smpEncode = smpEncode . strEncode + smpP = strDecode <$?> smpP + addressSize :: Int addressSize = 20 diff --git a/src/Simplex/Messaging/Names/Record.hs b/src/Simplex/Messaging/Names/Record.hs index 461bec16e..f62ea8ab4 100644 --- a/src/Simplex/Messaging/Names/Record.hs +++ b/src/Simplex/Messaging/Names/Record.hs @@ -13,6 +13,7 @@ module Simplex.Messaging.Names.Record NamePricing (..), USDCents (..), NameReservedReason (..), + OwnedNames (..), ) where @@ -90,6 +91,17 @@ data NameRegistration NRReserved {reservedReason :: NameReservedReason} deriving (Eq, Show) +-- | What the registry holds for an address: the names it owns, each as the 'NameResponse' resolving it gives, and whether the account is in use +data OwnedNames = OwnedNames + { -- | the oldest block any of the reads behind this answer saw, so a lagging registry shows even where no name does + ownLastBlockTs :: Maybe SystemSeconds, + ownNames :: [NameResponse], + ownInUse :: Bool, + -- | the cursor to resume from, absent when the listing is complete + ownNextOffset :: Maybe Int + } + deriving (Eq, Show) + -- | Enough to price the name locally, which the router cannot do behind a hash. data NamePricing = NamePricing { -- | US cents per year, for the lengths the registry prices specially @@ -142,3 +154,5 @@ $(JQ.deriveJSON defaultJSON ''NamePricing) $(JQ.deriveJSON (taggedObjectJSON $ dropPrefix "NR") ''NameRegistration) $(JQ.deriveJSON defaultJSON ''NameResponse) + +$(JQ.deriveJSON defaultJSON {J.fieldLabelModifier = dropPrefix "own"} ''OwnedNames) diff --git a/src/Simplex/Messaging/Protocol.hs b/src/Simplex/Messaging/Protocol.hs index e2cbef3a4..854bd7fae 100644 --- a/src/Simplex/Messaging/Protocol.hs +++ b/src/Simplex/Messaging/Protocol.hs @@ -261,7 +261,7 @@ import qualified Data.Text as T import Data.Text.Encoding (decodeLatin1, encodeUtf8) import Data.Time.Clock.System (SystemTime (..), systemToUTCTime) import Data.Type.Equality -import Data.Word (Word8, Word16) +import Data.Word (Word8, Word16, Word32) import GHC.TypeLits (ErrorMessage (..), TypeError, type (+)) import qualified GHC.TypeLits as TE import qualified GHC.TypeLits as Type @@ -271,6 +271,7 @@ import Simplex.Messaging.Agent.Store.DB (Binary (..), FromField (..), ToField (. import qualified Simplex.Messaging.Crypto as C import Simplex.Messaging.Encoding import Simplex.Messaging.Encoding.String +import Simplex.Messaging.Eth.Address (Address) import Simplex.Messaging.Names.Record import Simplex.Messaging.Parsers import Simplex.Messaging.Protocol.Types @@ -610,6 +611,8 @@ data Command (p :: Party) where RFWD :: EncFwdTransmission -> Command ProxyService -- use CorrId as CbNonce, proxy to relay -- Resolve SimpleX name. RSLV :: NameQuery -> Command Resolver + -- Names an address owns, and whether the account has been used at all. + ROWN :: Address -> Word32 -> Command Resolver deriving instance Show (Command p) @@ -747,6 +750,8 @@ data BrokerMsg where PONG :: BrokerMsg -- What the router knows about a SimpleX name. RNAME :: NameResponse -> BrokerMsg + -- What the router knows about the names an address owns. + ROWND :: OwnedNames -> BrokerMsg deriving (Eq, Show) data RcvMessage = RcvMessage @@ -958,6 +963,7 @@ data CommandTag (p :: Party) where NSUB_ :: CommandTag Notifier NSUBS_ :: CommandTag NotifierService RSLV_ :: CommandTag Resolver + ROWN_ :: CommandTag Resolver data CmdTag = forall p. PartyI p => CT (SParty p) (CommandTag p) @@ -985,6 +991,7 @@ data BrokerMsgTag | ERR_ | PONG_ | RNAME_ + | ROWND_ deriving (Show) class ProtocolMsgTag t where @@ -1022,6 +1029,7 @@ instance PartyI p => Encoding (CommandTag p) where NSUB_ -> "NSUB" NSUBS_ -> "NSUBS" RSLV_ -> "RSLV" + ROWN_ -> "ROWN" smpP = messageTagP instance ProtocolMsgTag CmdTag where @@ -1051,6 +1059,7 @@ instance ProtocolMsgTag CmdTag where "NSUB" -> Just $ CT SNotifier NSUB_ "NSUBS" -> Just $ CT SNotifierService NSUBS_ "RSLV" -> Just $ CT SResolver RSLV_ + "ROWN" -> Just $ CT SResolver ROWN_ _ -> Nothing instance Encoding CmdTag where @@ -1081,6 +1090,7 @@ instance Encoding BrokerMsgTag where ERR_ -> "ERR" PONG_ -> "PONG" RNAME_ -> "RNAME" + ROWND_ -> "ROWND" smpP = messageTagP instance ProtocolMsgTag BrokerMsgTag where @@ -1104,6 +1114,7 @@ instance ProtocolMsgTag BrokerMsgTag where "ERR" -> Just ERR_ "PONG" -> Just PONG_ "RNAME" -> Just RNAME_ + "ROWND" -> Just ROWND_ _ -> Nothing -- | SMP message body format @@ -1846,6 +1857,7 @@ instance PartyI p => ProtocolEncoding SMPVersion ErrorType (Command p) where PFWD fwdV pubKey (EncTransmission s) -> e (PFWD_, ' ', fwdV, pubKey, Tail s) RFWD (EncFwdTransmission s) -> e (RFWD_, ' ', Tail s) RSLV q -> e (RSLV_, ' ', if v >= nameAvailSMPVersion then hashedQuery q else q) + ROWN addr offset -> e (ROWN_, ' ', addr, offset) where e :: Encoding a => a -> ByteString e = smpEncode @@ -1871,6 +1883,7 @@ instance PartyI p => ProtocolEncoding SMPVersion ErrorType (Command p) where PFWD {} -> entityCmd RFWD _ -> noAuthCmd RSLV _ -> noAuthCmd + ROWN _ _ -> noAuthCmd SUB -> serviceCmd NSUB -> serviceCmd -- other client commands must have both signature and queue ID @@ -1953,6 +1966,7 @@ instance ProtocolEncoding SMPVersion ErrorType Cmd where | v >= rcvServiceSMPVersion -> Cmd SNotifierService <$> (NSUBS <$> _smpP <*> smpP) | otherwise -> pure $ Cmd SNotifierService $ NSUBS (-1) mempty CT SResolver RSLV_ -> Cmd SResolver . RSLV <$> _smpP + CT SResolver ROWN_ -> Cmd SResolver <$> (ROWN <$> _smpP <*> smpP) fromProtocolError = fromProtocolError @SMPVersion @ErrorType @BrokerMsg {-# INLINE fromProtocolError #-} @@ -1995,6 +2009,7 @@ instance ProtocolEncoding SMPVersion ErrorType BrokerMsg where | v < clientNoticesSMPVersion -> BLOCKED info {notice = Nothing} _ -> err PONG -> e PONG_ + ROWND owned -> e (ROWND_, ' ', Tail $ LB.toStrict $ J.encode owned) RNAME res | v >= nameAvailSMPVersion -> e (RNAME_, ' ', Tail $ LB.toStrict $ J.encode res) | otherwise -> case registration res of @@ -2046,6 +2061,7 @@ instance ProtocolEncoding SMPVersion ErrorType BrokerMsg where OK_ -> pure OK ERR_ -> ERR <$> _smpP PONG_ -> pure PONG + ROWND_ -> fmap ROWND . J.eitherDecodeStrict . unTail <$?> _smpP RNAME_ | v >= nameAvailSMPVersion -> fmap RNAME . J.eitherDecodeStrict . unTail <$?> _smpP | otherwise -> fmap (RNAME . oldResponse) . J.eitherDecodeStrict . unTail <$?> _smpP @@ -2074,6 +2090,7 @@ instance ProtocolEncoding SMPVersion ErrorType BrokerMsg where RRES _ -> noEntityMsg ALLS -> noEntityMsg RNAME {} -> noEntityMsg + ROWND {} -> noEntityMsg -- other broker responses must have queue ID _ | B.null entId -> Left $ CMD NO_ENTITY diff --git a/src/Simplex/Messaging/Server.hs b/src/Simplex/Messaging/Server.hs index 097cabbbe..55e542851 100644 --- a/src/Simplex/Messaging/Server.hs +++ b/src/Simplex/Messaging/Server.hs @@ -88,6 +88,7 @@ import Data.Time.Clock.System (SystemTime (..), getSystemTime) import Data.Time.Format.ISO8601 (iso8601Show) import Data.Type.Equality import Data.Typeable (cast) +import Data.Word (Word32) import qualified Data.X509 as X import qualified Data.X509.Validation as XV import GHC.Conc.Signal @@ -103,13 +104,14 @@ import Simplex.Messaging.Client.Agent (OwnServer, SMPClientAgent (..), SMPClient import qualified Simplex.Messaging.Crypto as C import Simplex.Messaging.Encoding import Simplex.Messaging.Encoding.String +import Simplex.Messaging.Eth.Address (Address) import Simplex.Messaging.Protocol import Simplex.Messaging.Server.Control import Simplex.Messaging.Server.Env.STM as Env import Simplex.Messaging.Server.Expiration import Simplex.Messaging.Server.MsgStore import Simplex.Messaging.Server.MsgStore.Journal (JournalMsgStore, JournalQueue (..), getJournalQueueMessages) -import Simplex.Messaging.Server.Names (NamesEnv, closeNamesEnv, resolveName) +import Simplex.Messaging.Server.Names (NamesEnv, closeNamesEnv, ownedNames, resolveName) import Simplex.Messaging.Server.MsgStore.STM import Simplex.Messaging.Server.MsgStore.Types import Simplex.Messaging.Server.NtfStore @@ -1276,6 +1278,7 @@ verifyQueueTransmission service thAuth (tAuth, authorized, (corrId, entId, comma vc SProxiedClient _ = VRVerified Nothing vc SProxyService (RFWD _) = VRVerified Nothing vc SResolver (RSLV _) = VRVerified Nothing + vc SResolver (ROWN _ _) = VRVerified Nothing checkRole = case (service, partyClientRole p) of (Just THClientService {serviceRole}, Just role) -> serviceRole == role _ -> True @@ -1506,6 +1509,18 @@ client answered = \case NRRegistered {} -> True _ -> v >= nameAvailSMPVersion + resolverMsg :: VersionSMP -> NamesEnv -> Command Resolver -> M s BrokerMsg + resolverMsg v nenv = \case + RSLV d -> resolveNameMsg v nenv d + ROWN addr offset -> ownedNamesMsg nenv addr offset + ownedNamesMsg :: NamesEnv -> Address -> Word32 -> M s BrokerMsg + ownedNamesMsg nenv addr offset = do + st <- asks (rslvStats . serverStats) + (selector, msg) <- + liftIO (ownedNames nenv addr offset) <&> \case + Right owned -> (rslvSucc, ROWND owned) + Left e -> (rslvResolverErrs, ERR $ NAME e) + incStat (selector st) $> msg transportErr :: TransportError -> ErrorType transportErr = PROXY . BROKER . TRANSPORT mkIncProxyStats :: MonadIO m => ProxyStats -> ProxyStats -> OwnServer -> (ProxyStats -> IORef Int) -> m () @@ -1520,9 +1535,9 @@ client SEND flags msgBody -> response <$> withQueue_ False err (sendMessage flags msgBody) Cmd SIdleClient PING -> pure $ response (corrId, NoEntity, PONG) Cmd SProxyService (RFWD encBlock) -> (response . (corrId, NoEntity,) =<<) <$> processForwardedCommand encBlock - Cmd SResolver (RSLV d) -> rslvNamesEnv >>= \case + Cmd SResolver command -> rslvNamesEnv >>= \case Nothing -> pure $ response (corrId, NoEntity, ERR (NAME NO_RESOLVER)) - Just nenv -> forkCmd serverResolverConcurrency corrId NoEntity (resolveNameMsg (thVersion thParams') nenv d) + Just nenv -> forkCmd serverResolverConcurrency corrId NoEntity (resolverMsg (thVersion thParams') nenv command) Cmd SSenderLink command -> case command of LKEY k -> withQueue $ \q qr -> checkMode QMMessaging qr $ secureQueue_ q k $>> getQueueLink_ q qr LGET -> withQueue $ \q qr -> checkContact qr $ getQueueLink_ q qr @@ -2150,10 +2165,10 @@ client -- rejectOrVerify filters allowed commands, no need to repeat it here. Left r -> pure $ Just r Right t''@(_, (corrId', entId', cmd')) -> case cmd' of - Cmd SResolver (RSLV d) -> lift $ rslvNamesEnv >>= \case + Cmd SResolver command -> lift $ rslvNamesEnv >>= \case Nothing -> pure $ Just (corrId', entId', ERR (NAME NO_RESOLVER)) Just nenv -> forkCmd serverResolverConcurrency corrId NoEntity $ do - msg <- resolveNameMsg (thVersion clntTHParams) nenv d + msg <- resolverMsg (thVersion clntTHParams) nenv command either ERR id <$> runExceptT (encodeResp (corrId', entId', msg)) -- INTERNAL because processCommand never returns Nothing for sender commands; -- `fst` drops the empty message only returned for SUB. @@ -2175,6 +2190,7 @@ client Cmd SSenderLink (LKEY _) -> True Cmd SSenderLink LGET -> True Cmd SResolver (RSLV _) -> True + Cmd SResolver (ROWN _ _) -> True _ -> False verified = \case VRVerified q -> Right (q, t'') diff --git a/src/Simplex/Messaging/Server/Names.hs b/src/Simplex/Messaging/Server/Names.hs index 601519c56..06e59098d 100644 --- a/src/Simplex/Messaging/Server/Names.hs +++ b/src/Simplex/Messaging/Server/Names.hs @@ -12,6 +12,7 @@ module Simplex.Messaging.Server.Names closeNamesEnv, pingEndpoint, resolveName, + ownedNames, ) where @@ -21,7 +22,11 @@ import Data.Bifunctor (first) import Data.Maybe (fromMaybe) import qualified Data.Text as T import Data.Text.Encoding (decodeLatin1) +import Data.Word (Word32) import Simplex.Messaging.Encoding +import Simplex.Messaging.Encoding.String (strEncode) +import Simplex.Messaging.Eth.Address (Address) +import Simplex.Messaging.Names.Record (OwnedNames) import Simplex.Messaging.Protocol (NameErrorType (..), NameQuery, NameResponse) import Simplex.Messaging.Server.Names.HttpResolver ( ResolverEnv, @@ -30,6 +35,7 @@ import Simplex.Messaging.Server.Names.HttpResolver closeResolverEnv, healthHttp, newResolverEnv, + ownedByHttp, resolveHttp, ) import System.Timeout (timeout) @@ -60,9 +66,12 @@ pingEndpoint NamesEnv {resolverEnv, config} = fromMaybe (Left ResolverTimeout) <$> timeout (resolverTimeoutMs config * 1000) (healthHttp resolverEnv) resolveName :: NamesEnv -> NameQuery -> IO (Either NameErrorType NameResponse) -resolveName env q = do - r <- E.try (timeout (resolverTimeoutMs (config env) * 1000) (fetch env q)) - case r of +resolveName env q = resolverCall env $ fetch env q + +-- | Bound the call by the configured timeout, and report a raised exception as an error, leaving async exceptions alone. +resolverCall :: NamesEnv -> IO (Either NameErrorType a) -> IO (Either NameErrorType a) +resolverCall env a = + E.try (timeout (resolverTimeoutMs (config env) * 1000) a) >>= \case Right result -> pure (fromMaybe (Left (RESOLVER "timeout")) result) Left e | Just (_ :: E.SomeAsyncException) <- E.fromException e -> E.throwIO e @@ -74,6 +83,14 @@ fetch :: NamesEnv -> NameQuery -> IO (Either NameErrorType NameResponse) fetch NamesEnv {resolverEnv} q = first mapResolverError <$> resolveHttp resolverEnv (decodeLatin1 $ smpEncode q) +-- | The names an address owns. A resolver without the endpoint answers 404, which is a RESOLVER error, not an empty list. +ownedNames :: NamesEnv -> Address -> Word32 -> IO (Either NameErrorType OwnedNames) +ownedNames env addr offset = resolverCall env $ fetchOwned env addr offset + +fetchOwned :: NamesEnv -> Address -> Word32 -> IO (Either NameErrorType OwnedNames) +fetchOwned NamesEnv {resolverEnv} addr offset = + first mapResolverError <$> ownedByHttp resolverEnv (decodeLatin1 $ strEncode addr) offset + mapResolverError :: ResolverError -> NameErrorType mapResolverError = \case HttpStatusErr code -> RESOLVER ("HTTP " <> T.pack (show code)) diff --git a/src/Simplex/Messaging/Server/Names/HttpResolver.hs b/src/Simplex/Messaging/Server/Names/HttpResolver.hs index 0f272bcf4..faa76d329 100644 --- a/src/Simplex/Messaging/Server/Names/HttpResolver.hs +++ b/src/Simplex/Messaging/Server/Names/HttpResolver.hs @@ -11,6 +11,7 @@ -- GET /v2/resolve/ -> 200 with a NameRegistration JSON document, for -- all three registration shapes; 400 for unknown -- TLDs, 502 for upstream RPC failures +-- GET /v2/owned-by/ -> 200 with an OwnedNames JSON document -- GET /v1/resolve/ -> 200 with a NameRecord, what relays before SMP -- v22 call as /resolve -- GET /health -> 200 when the resolver process is ready @@ -29,6 +30,7 @@ module Simplex.Messaging.Server.Names.HttpResolver newResolverEnv, closeResolverEnv, resolveHttp, + ownedByHttp, healthHttp, ) where @@ -42,6 +44,7 @@ import qualified Data.ByteString.Char8 as B import qualified Data.ByteString.Lazy as BL import Data.Text (Text) import Data.Text.Encoding (encodeUtf8) +import Data.Word (Word32) import Network.HTTP.Client ( HttpException, Manager, @@ -59,7 +62,7 @@ import qualified Network.HTTP.Client as HC import Network.HTTP.Client.TLS (tlsManagerSettings) import qualified Network.HTTP.Types as HT import Network.HTTP.Types.URI (urlEncode) -import Simplex.Messaging.Names.Record (NameResponse) +import Simplex.Messaging.Names.Record (NameResponse, OwnedNames) data RpcAuth = AuthBearer Text | AuthBasic Text Text @@ -117,6 +120,12 @@ resolveHttp env q = (>>= first InvalidJson . J.eitherDecodeStrict . BL.toStrict) <$> httpGet env ("/v2/resolve/" <> B.unpack (urlEncode True (encodeUtf8 q))) +-- | The address is encoded for the same reason the name is: nothing from a client reaches the path raw. +ownedByHttp :: ResolverEnv -> Text -> Word32 -> IO (Either ResolverError OwnedNames) +ownedByHttp env addr offset = + (>>= first InvalidJson . J.eitherDecodeStrict . BL.toStrict) + <$> httpGet env ("/v2/owned-by/" <> B.unpack (urlEncode True (encodeUtf8 addr)) <> "?offset=" <> show offset) + -- | GET /health; success = reachable with status < 400. The body is -- size-capped but NOT decoded — the probe only checks reachability. healthHttp :: ResolverEnv -> IO (Either ResolverError ()) diff --git a/src/Simplex/Messaging/Server/Prometheus.hs b/src/Simplex/Messaging/Server/Prometheus.hs index 85d2624fd..9eb6edde3 100644 --- a/src/Simplex/Messaging/Server/Prometheus.hs +++ b/src/Simplex/Messaging/Server/Prometheus.hs @@ -465,11 +465,11 @@ prometheusMetrics sm rtm ts = in "# Names\n\ \# -----\n\ \\n\ - \# HELP simplex_smp_names_reqs Total RSLV requests forwarded to this server.\n\ + \# HELP simplex_smp_names_reqs Total resolver requests (RSLV and ROWN) forwarded to this server.\n\ \# TYPE simplex_smp_names_reqs counter\n\ \simplex_smp_names_reqs " <> mshow _rslvReqs <> "\n# rslvReqs\n\ \\n\ - \# HELP simplex_smp_names_success NameRecord resolved, or availability answered.\n\ + \# HELP simplex_smp_names_success NameRecord resolved, availability answered, or names owned listed.\n\ \# TYPE simplex_smp_names_success counter\n\ \simplex_smp_names_success " <> mshow _rslvSucc <> "\n# rslvSucc\n\ \\n\ @@ -481,7 +481,7 @@ prometheusMetrics sm rtm ts = \# TYPE simplex_smp_names_resolver_errs counter\n\ \simplex_smp_names_resolver_errs " <> mshow _rslvResolverErrs <> "\n# rslvResolverErrs\n\ \\n\ - \# HELP simplex_smp_names_disabled RSLV requests rejected because no resolver is configured (names role off).\n\ + \# HELP simplex_smp_names_disabled Resolver requests rejected because no resolver is configured (names role off).\n\ \# TYPE simplex_smp_names_disabled counter\n\ \simplex_smp_names_disabled " <> mshow _rslvDisabled <> "\n# rslvDisabled\n\ \\n" diff --git a/src/Simplex/Messaging/Transport.hs b/src/Simplex/Messaging/Transport.hs index d2c30d25a..a3ef423fd 100644 --- a/src/Simplex/Messaging/Transport.hs +++ b/src/Simplex/Messaging/Transport.hs @@ -54,6 +54,7 @@ module Simplex.Messaging.Transport namesSMPVersion, serverInfoSMPVersion, nameAvailSMPVersion, + nameOwnedSMPVersion, simplexMQVersion, smpBlockSize, TransportConfig (..), @@ -177,6 +178,7 @@ smpBlockSize = 16384 -- 20 - public namespaces resolver, RSLV command (6/20/2026) -- 21 - server public information in handshake (7/5/2026) -- 22 - RNAME answers name availability as well as the record (7/25/2026) +-- 23 - ROWN command, the names an address owns (9/22/2026) data SMPVersion @@ -218,6 +220,10 @@ serverInfoSMPVersion = VersionSMP 21 nameAvailSMPVersion :: VersionSMP nameAvailSMPVersion = VersionSMP 22 +-- | ROWN lists the names an address owns; below this a server does not answer it, which is not the same as owning nothing. +nameOwnedSMPVersion :: VersionSMP +nameOwnedSMPVersion = VersionSMP 23 + minClientSMPRelayVersion :: VersionSMP minClientSMPRelayVersion = VersionSMP 14 @@ -225,20 +231,17 @@ minServerSMPRelayVersion :: VersionSMP minServerSMPRelayVersion = VersionSMP 14 currentClientSMPRelayVersion :: VersionSMP -currentClientSMPRelayVersion = VersionSMP 22 +currentClientSMPRelayVersion = VersionSMP 23 currentServerSMPRelayVersion :: VersionSMP -currentServerSMPRelayVersion = VersionSMP 22 +currentServerSMPRelayVersion = VersionSMP 23 -- Max SMP protocol version to be used in e2e encrypted connection between -- client and server, as defined by SMP proxy. Normally set below the current -- version to prevent client version fingerprinting by the destination relays --- when clients upgrade at different times. Pinned to the current version (22) --- for this release because a proxied RSLV only carries availability from --- nameAvailSMPVersion (22), so the one-version anti-fingerprinting buffer does --- not apply yet; it reappears once the current version advances past 22. +-- when clients upgrade at different times. Pinned to the current version (23) because a proxied ROWN needs it, and the buffer returns once current passes 23. proxiedSMPRelayVersion :: VersionSMP -proxiedSMPRelayVersion = VersionSMP 22 +proxiedSMPRelayVersion = VersionSMP 23 -- minimal supported protocol version is 14 supportedClientSMPRelayVRange :: VersionRangeSMP diff --git a/tests/AgentTests/ResolveNameTests.hs b/tests/AgentTests/ResolveNameTests.hs index d8507194e..0a17eae55 100644 --- a/tests/AgentTests/ResolveNameTests.hs +++ b/tests/AgentTests/ResolveNameTests.hs @@ -15,6 +15,7 @@ module AgentTests.ResolveNameTests (resolveNameTests) where import AgentTests.FunctionalAPITests (withAgent) import Control.Monad.Except (runExceptT) import qualified Data.ByteString.Lazy as LB +import Data.IORef (IORef, readIORef) import Data.List (isInfixOf) import Network.HTTP.Types (Status, status200, status404, status502) import NamesResolverServer (memCfg, memCfg2, memProxyCfg, withNames) @@ -22,7 +23,10 @@ import qualified NamesResolverServer as NRS import SMPAgentClient import SMPClient import SMPNamesTests (availableBody, registeredBody, testNameRecord) -import Simplex.Messaging.Agent (resolveSimplexName) +import Data.Text (Text) +import Simplex.Messaging.Agent (ownedSimplexNames, resolveSimplexName) +import Simplex.Messaging.Encoding.String (strDecode) +import Simplex.Messaging.Eth.Address (Address) import Simplex.Messaging.Agent.Client (AgentClient) import Simplex.Messaging.Agent.Env.SQLite (InitialAgentServers (..), ServerCfg, ServerRoles (..), presetServerCfg) import Simplex.Messaging.Agent.Protocol (AgentErrorType (..)) @@ -44,10 +48,14 @@ oneSrv :: ServerCfg 'SMP.PSMP -> InitialAgentServers oneSrv cfg_ = (initAgentServersProxy_ SPMNever SPFProhibit) {smp = [(1, [cfg_])]} withDirectResolver :: (Status, LB.ByteString) -> (AgentClient -> IO a) -> IO a -withDirectResolver (st, body) k = - NRS.withResolverServer (NRS.resolveResp st body) $ \port _ -> +withDirectResolver resp k = withDirectResolverReqs resp $ \c _ -> k c + +-- | As 'withDirectResolver', with the requests the resolver was asked for. +withDirectResolverReqs :: (Status, LB.ByteString) -> (AgentClient -> IORef [[Text]] -> IO a) -> IO a +withDirectResolverReqs (st, body) k = + NRS.withResolverServer (NRS.resolveResp st body) $ \port reqs -> withSmpServerConfigOn (transport @TLS) (withNames port memCfg) testPort $ \_ -> - withAgent 1 agentCfg (oneSrv (nameSrvCfg testSMPServer)) testDB k + withAgent 1 agentCfg (oneSrv (nameSrvCfg testSMPServer)) testDB $ \c -> k c reqs withProxyAndResolver :: (Status, LB.ByteString) -> (AgentClient -> IO a) -> IO a withProxyAndResolver (st, body) k = @@ -87,6 +95,22 @@ resolveNameTests = do it "returns NameRecord" testDirectSuccess describe "name availability" $ it "an unregistered name answers as available" testAvailSuccess + describe "owned names" $ + it "a relay that cannot answer is not the end of the lookup" testOwnedRetriesRelays + +-- | A relay that cannot answer must not end a scan at the account it was asked about. +testOwnedRetriesRelays :: HasCallStack => IO () +testOwnedRetriesRelays = + withDirectResolverReqs (status502, "{}") $ \c reqs -> do + r <- runExceptT $ ownedSimplexNames c NRMInteractive 1 [] testAddr 0 + case r of + Left (SMP _ (SMP.NAME _)) -> pure () + _ -> expectationFailure $ "expected Left (SMP _ (NAME ..)), got: " <> show r + asked <- readIORef reqs + length [q | q@("v2" : "owned-by" : _) <- asked] `shouldBe` 3 + +testAddr :: Address +testAddr = either error id $ strDecode "0x70997970C51812dc3A010C7d01b50e0d17dc79C8" testAvailSuccess :: HasCallStack => IO () testAvailSuccess = diff --git a/tests/NamesResolverServer.hs b/tests/NamesResolverServer.hs index 054d55e40..a21c9df1c 100644 --- a/tests/NamesResolverServer.hs +++ b/tests/NamesResolverServer.hs @@ -47,12 +47,12 @@ withResolverServerDelayed delayMs handler action = do let (st, body) = handler (pathInfo req) send $ responseLBS st [(hContentType, "application/json")] body --- | The resolver API is versioned on its own: v2 answers with NameRegistration --- JSON, which is the only shape the server asks for. +-- | The resolver API is versioned on its own: v2 answers a name with NameRegistration, an address with OwnedNames. resolveResp :: Status -> LB.ByteString -> [Text] -> (Status, LB.ByteString) resolveResp st body = \case ["health"] -> (ok200, "{}") ("v2" : "resolve" : _) -> (st, body) + ("v2" : "owned-by" : _) -> (st, body) _ -> (notFound404, "{}") testNamesConfig :: Int -> NamesConfig diff --git a/tests/RSLVTests.hs b/tests/RSLVTests.hs index 9263b6953..40f7c21ee 100644 --- a/tests/RSLVTests.hs +++ b/tests/RSLVTests.hs @@ -27,6 +27,8 @@ import Simplex.Messaging.Client import qualified Simplex.Messaging.Crypto as C import Simplex.Messaging.Encoding.String (strDecode) import SMPNamesTests (availableBody, registeredBody, reservedBody, resolved, testNameRecord, testPricing) +import Simplex.Messaging.Eth.Address (Address) +import Simplex.Messaging.Names.Record (OwnedNames (..)) import Simplex.Messaging.Protocol ( BrokerMsg (..), Cmd (..), @@ -72,6 +74,16 @@ withProxyAndResolver (st, body) runTest = withSmpServerConfigOn (transport @TLS) memProxyCfg testPort $ \_ -> withSmpServerConfigOn (transport @TLS) (withNames port memCfg2) testPort2 (const runTest) +sendRown :: Transport c => THandleSMP c 'TClient -> B.ByteString -> Address -> IO (Transmission (Either ErrorType BrokerMsg)) +sendRown h@THandle {params} corrId addr = do + let TransmissionForAuth {tToSend} = encodeTransmissionForAuth params (CorrId corrId, NoEntity, Cmd SResolver (ROWN addr 0)) + [Right ()] <- tPut h (Right (Nothing, tToSend) :| []) + r :| _ <- tGetClient h + pure r + +testAddr :: Address +testAddr = either error id $ strDecode "0x70997970C51812dc3A010C7d01b50e0d17dc79C8" + sendRslv :: Transport c => THandleSMP c 'TClient -> B.ByteString -> SimplexDomain -> IO (Transmission (Either ErrorType BrokerMsg)) sendRslv h@THandle {params} corrId d = do let TransmissionForAuth {tToSend} = encodeTransmissionForAuth params (CorrId corrId, NoEntity, Cmd SResolver (RSLV (NQDomain d))) @@ -84,11 +96,14 @@ rslvTests = do describe "RSLV direct (non-forwarded)" $ do it "resolver without the v2 route (404) -> NAME RESOLVER, not NOT_FOUND" testRslvBackendNotFound it "resolver replies 502 -> NAME (RESOLVER ..)" testRslvBackendHttpErr + it "ROWN returns the names an address owns" testRownOwned + it "ROWN on a resolver without the endpoint is a resolver error, not an empty answer" testRownUnsupported it "no names config -> NAME NO_RESOLVER" testRslvDisabled it "refuses to send RSLV on a session below namesSMPVersion" testRslvVersion describe "RSLV forwarded (PFWD)" $ do it "PFWD-wrapped RSLV reaches resolver via proxy (PCEProtocolError (NAME RESOLVER))" testRslvForwarded it "PFWD-wrapped RSLV success returns RNAME (record JSON frames over the proxy)" testRslvForwardedSuccess + it "PFWD-wrapped ROWN reaches the resolver, so a scan need not go direct" testRownForwarded describe "RSLV success path (RNAME response)" $ do it "returns RNAME with NameRecord" testRslvSuccess describe "RSLV availability (RNAME response)" $ do @@ -120,6 +135,36 @@ testRslvBackendHttpErr = (_, _, resp) <- sendRslv h "rs05" (domain "alice.simplex") resp `shouldBe` Right (ERR (NAME (RESOLVER "HTTP 502"))) +-- | The scan reads inUse, so in use with no names must not look like owning nothing. +testRownOwned :: IO () +testRownOwned = + withResolverServer (status200, ownedBody) $ + testSMPClient @TLS $ \h -> do + (corrId, _entId, resp) <- sendRown h "ro01" testAddr + corrId `shouldBe` CorrId "ro01" + case resp of + Right (ROWND owned) -> do + map ownedName (ownNames owned) `shouldBe` ["alice.simplex"] + ownInUse owned `shouldBe` True + r -> expectationFailure $ "unexpected " <> show r + +-- 404 must reach the client as a resolver error; read as "owns nothing" it would end a scan early. +testRownUnsupported :: IO () +testRownUnsupported = + withResolverServer (status404, "{}") $ + testSMPClient @TLS $ \h -> do + (_, _, resp) <- sendRown h "ro02" testAddr + resp `shouldBe` Right (ERR (NAME (RESOLVER "HTTP 404"))) + +-- | Each owned name is the same NameResponse /v2/resolve answers with. +ownedBody :: LB.ByteString +ownedBody = "{\"lastBlockTs\":1813000000,\"names\":[" <> registeredBody testNameRecord <> "],\"inUse\":true,\"nextOffset\":null}" + +ownedName :: NameResponse -> Text +ownedName = \case + NameResponse {registration = NRRegistered {nameRecord}} -> SMP.nrName nameRecord + r -> error $ "expected a registered name, got: " <> show r + testRslvDisabled :: IO () testRslvDisabled = withSmpServerConfigOn (transport @TLS) memCfg testPort $ const $ @@ -142,7 +187,11 @@ testRslvVersion = _ -> expectationFailure $ "expected Left (PCETransportError TEVersion), got: " <> show r forwardedResolveAlice :: IO (Either SMPClientError (Either ProxyClientError SMP.NameResponse)) -forwardedResolveAlice = do +forwardedResolveAlice = forwardedToRelay $ \pc sess -> proxyResolveName pc NRMInteractive sess (domain "alice.simplex") + +-- | Run one proxied resolver command over a PFWD session to the second relay. +forwardedToRelay :: (SMPClient -> ProxiedRelay -> ExceptT SMPClientError IO a) -> IO (Either SMPClientError a) +forwardedToRelay proxiedCmd = do g <- C.newRandom ts <- getCurrentTime let proxyServ = SMPServer testHost testPort testKeyHash @@ -151,7 +200,7 @@ forwardedResolveAlice = do pcE <- getProtocolClient g NRMInteractive (1, proxyServ, Nothing) cfg' [] Nothing ts (\_ -> pure ()) pc <- either (fail . show) pure pcE sess <- runExceptT' (connectSMPProxiedRelay pc NRMInteractive relayServ Nothing) - runExceptT (proxyResolveName pc NRMInteractive sess (domain "alice.simplex")) + runExceptT (proxiedCmd pc sess) testRslvForwarded :: IO () testRslvForwarded = @@ -167,6 +216,14 @@ testRslvForwardedSuccess = Right (Right NameResponse {registration = NRRegistered {nameRecord}}) -> nameRecord `shouldBe` testNameRecord _ -> expectationFailure $ "expected Right (Right NRRegistered), got: " <> show r +-- | Without this a scan falls back to a direct session, handing the relay the address with the IP. +testRownForwarded :: IO () +testRownForwarded = + withProxyAndResolver (status200, ownedBody) $ + forwardedToRelay (\pc sess -> proxyOwnedNames pc NRMInteractive sess testAddr 0) >>= \r -> case r of + Right (Right owned) -> map ownedName (ownNames owned) `shouldBe` ["alice.simplex"] + _ -> expectationFailure $ "expected Right (Right OwnedNames), got: " <> show r + testRslvSuccess :: IO () testRslvSuccess = withResolverServer (status200, registeredBody testNameRecord) $