Repository navigation
feat: a federation chart and its ingress - #110
Merged
Merged
Conversation
ameersohel45
changed the title
The federated half of the catalog crawler on the network role, in both deployment modes. Off by default, so a deployment that has admitted no network behaves as it did. The placeholders are filled by the chart rather than by the init container that renders the keypair: none of them is a credential. An intent that narrows nothing is refused at render, because discovery answers one with no catalogs and the pass would report success having mirrored nothing.
Follows network-adapter#78: the check left the registry's key lookup and became a step, so a mount enforces the agreement whether or not it verifies signatures. Listed after validateSign, where the sender has been proved.
#71] A network recognises its own entry from its configuration, so a stored self value is redundant. federationRole now only records our decision about a peer: source, requester or both.
SchemaRegistry was the only entity whose root set additionalProperties false. The registry merges its own osCreatedAt, osUpdatedAt, osCreatedBy and osUpdatedBy into a record before validating it, so every PUT was rejected over fields the registry itself had added -- a 500 no caller could avoid. Sending a body with none of them in it failed the same way. Measured against a live registry: Participant and ProviderSchema both update cleanly and neither sets it. A capability's name, version and pack URL were permanent for no stated reason.
The charts had no checkCrossNetworkRequester at all, in either mode. A caller whose signature verified was answered whether or not we had agreed to answer them -- the step exists on the adapter and runs in quick-start, but no chart listed it. The provider mount also ended at its capability steps with no sign, so another network received an answer it had nothing to verify. quick-start has signed since e3e7bf2. The three tiers move to /consumer/, /network/ and /provider/ in both modes, as networkofnetwork already had them and quick-start now does. The paths callers see do not change: the gateway rewrites /<action> onto the mount that serves it, which is what keeps another network's discover off the consumer tier -- that tier checks no signature and no agreement, and on / it was the catch-all that answered first. ADR-0002 records the path decision and the rewrite rule that replaces the blanket ban on rewrite annotations.
ADR-0002 records why the tiers moved to /consumer/, /network/ and /provider/, and how the paths callers see stay what ADR-0001 promised: the edge rewrites /discover onto /network/ and the transaction actions onto /provider/, so the mount names are ours and nobody else carries them. NETWORK_BASE_URL goes back to no path. The chart banned snippet, rewrite and regex annotations outright, because one could reach /_internal or /publish. That ban also forbids the mapping this needs. It is replaced by a check on the rewrite TARGET, refusing only /publish and /consumer -- the two mounts that take unsigned calls. Anything else is the deployment's business, so a mount added later needs no chart release. No ingress controller is named: Kong, ingress-nginx and ALB spell a rewrite differently and the annotations come from values. quick-start does the same job in nginx, but on the individual proxy host rather than server_proxy.conf: that file is included in every server block and the upstream differs per host -- in multi mode /select belongs to provider-adapter and /discover to network-adapter, which no shared rule can express. The shared file keeps what is true everywhere, denying every mount by name. Appendix B carries the per-host rule. Verified: the descriptor advertises /discover again, the adapter answers 401 on /network/discover and /provider/select, 404 on /discover, and the chart accepts a rewrite onto /network/ while refusing one onto /publish.
A rewrite is an annotation on the Ingress and holds one target, so the paths sharing an object must share a destination. Single mode serves /discover from the /network/ mount and /select from /provider/, which are two -- and the chart could only ever emit one Ingress. ingress.routes takes a list and emits one Ingress each, with its own annotations and tls. ingress.annotations/hosts/tls still work and still produce exactly one, so nothing already deployed changes. The guard runs per route rather than once: a path that is not a literal /<action>, or a rewrite targeting /publish or /consumer, fails the render wherever it appears. Exactly one route carries the tls block for a hostname. A TLS Secret is namespaced and cert-manager orders per Ingress, so a second block is a second ACME order for a name already covered -- against a limit of five failed authorizations per hostname per hour. Kong terminates on SNI.
The guard refused any rewrite whose value mentioned publish or consumer, which also refused /network/publish -- the mount that serves the publish action BEHIND validateSign and checkCrossNetworkRequester. The word was never the problem; the mount is. It now works out where a request lands -- the rewrite if there is one, else the path unchanged -- and refuses only when the first segment is /publish or /consumer. So /publish rewritten onto /network/publish is allowed, and /publish with no rewrite is not, which is the case that matters: without one it reaches the mount that signs on behalf of its caller and checks nothing. That makes a public /publish safe to expose. The caller has to sign for itself, with a key the registry publishes. Anything that cannot sign reaches the unsigned mount by port-forward, which is the only way it should be reachable.
The charts had no crawler at all, so a Helm-deployed network could answer another network's crawl but never run one of its own. The network role now carries the catalogcrawler block, with the settings an operator flips coming from values.federation and the rest from placeholders the init container already substitutes -- the signing key and key id are the same ones that sign everything else, so what we send is signed as us. Off by default, and off builds no schedule and never dials: a cluster that federates with nobody stays quiet rather than logging a refused connection on an interval. Answering another network's crawl is unaffected either way; that is the /network/ mount's job. No database. dbDsn is required only by staticIndexUrls, networks or publishPipelines, none of which this uses. A crawled catalog is published to this adapter's own /network/publish, the same path a local publish takes, so nothing leaves the process to do it.
Nothing in the charts served .well-known, so a cluster could neither tell another network who it was nor read anyone else's answer. Federation existed only in quick-start. A Job signs the descriptor, JWKS and schema packs from this network's registry entry and writes them into a ConfigMap; a small server mounts that ConfigMap and answers for exactly those three paths. Kong routes to it. The Job mounts the SAME Secret the adapter reads. oanfed refuses to sign with a key the registry does not publish for this network, so sharing it is what guarantees the descriptor names the key that signs traffic -- a mismatch there surfaces much later, as another network failing to verify anything we send. The private key exists only inside the Job; what reaches the ConfigMap is public. Each run publishes the next revision, read from what the ConfigMap already holds. oanfed refuses a revision that does not increase on the one served, and so does every network holding the old one, so the number cannot come from a value in git that a re-run would repeat. Admissions are values, one Job each, so who we agreed to talk to is reviewable rather than a command somebody ran once. They need no key: admission reads the other network's public documents and writes a row in our registry. oanfed's open_registry imported quick-start's setup.py and read .env, which a Job has neither of. It now takes REGISTRY_URL and BECKN_NETWORK_ID from the environment when both are set and falls back to setup.py otherwise, so the same file runs in both places. The server holds nothing secret and serves nothing else: 404 for any other path, and the Ingress template refuses one outside /.well-known/.
The chart's Participant schema was the pre-federation one. role was a single string, so one participant could not serve several layers -- which is what single-adapter mode is -- and there was no federationRole, revision or projectionTtl at all, so `oanfed admit` would have been refused by the registry it writes to. The federation chart could not have worked against it. SchemaRegistry's schemaUrl pattern allowed only raw.githubusercontent, where a crawled catalog carries the published openagrinet.github.io ref in its @context; a pack URL in that form was refused. Both are now the files quick-start runs, which is where the federation work was proven. ProviderSchema was already identical.
adapterKeys pointed each tier at its own key Secret, from when each tier was its own participant. A network has ONE identity: another network admits a network, not three adapters, and the Network Descriptor names one id and one keyId -- so a tier signing with a different key signs with one that descriptor never mentions, and nothing the other side holds can verify it. All three now default to network-adapter-keys. They remain three settings because a deployment that seeded three participants before the modes merged keeps working by pointing each tier at its own Secret, until those identities are retired.
quick-start dropped Keycloak and reaches the registry without a token; the charts still required one. Same stack, two behaviours. authenticationEnabled already existed but did nothing useful: the chart still demanded keycloak.url and an admin client secret, and still emitted every sunbird_sso_* and OAUTH2_* variable, so turning it off meant standing up a Keycloak nothing talked to. Those requirements and that env block are now conditional on it. registry-seed mints a token to write. It now mints one only when the registry checks one, and omits the Authorization header otherwise -- the seed document carries authenticationEnabled so the Job and the registry cannot disagree. keycloak.url and seedUser.passwordSecret are required only while it is true. What kept the registry private was never the token. It has no Ingress and its own namespace, and Sunbird RC uses POST for reads as well as writes, so no method rule would separate them if it ever were routed. Verified: with auth on the chart emits the same 8 Keycloak variables as before; with it off, none, and neither chart asks for a Keycloak value.
…ok [#72] Choosing what an agreement covers meant opening their schema-packs.json in a browser and copying URLs back into PACKS. `make verify` reported a COUNT and the URL to go and read -- the one command whose job is to tell you about them. verify now lists them. admit with no PACKS lists what that direction can cover and refuses, rather than saying only that one is required -- and the list is theirs for `source`, ours for `requester`, because those are different questions. PACKS=all takes every pack they publish TODAY. A snapshot, not a standing agreement: one that silently widened when they added a pack would not be much of an agreement, so taking in a new one is the same command again. Naming a pack the other side does not publish is still refused.
A requester agreement took its candidates from our own SchemaRegistry rows. A row exists the moment a provider is registered; the other network only learns of it when host-descriptor republishes. So we could agree to give a capability that is not in the document they read -- something they have no way to know to ask for. Measured on a live stack: register a provider, do not republish, and the registry holds two packs where the published document holds one. Both directions now read a DOCUMENT rather than a registry: theirs when we read from them, ours when they ask us. Agreeing to give an unpublished pack is refused, and republishing is what makes it available -- which is the same order the other network sees it in. admit takes --out for where host-descriptor wrote .well-known, and refuses with an explanation when it is absent rather than quietly falling back. The test fixture served one .well-known as both our document and the peer's. They are separate directories now, because the two lists are different questions and conflating them is what hid this.
] A requester agreement covers the packs in the document the other network reads. The admit Job mounted only the scripts, so it had no .well-known at all and would have refused every requester admission -- the right refusal for the wrong reason. It mounts the ConfigMap the publish Job writes, at the path it passes as --out. Optional, so a missing document is oanfed naming the command to run rather than a pod stuck Pending with the reason two layers away; publish is an earlier hook weight, so a normal upgrade does both in order.
Comments across this branch ran to ten, twelve, thirty-five lines where two were allowed. The reasoning belongs in a doc, not above a function, and a newcomer should not have to be told how the rest of the system works before a comment makes sense. Nothing about behaviour changes. Both charts lint, every values file renders, the 38 oanfed tests pass, and a live adapter was re-rendered from the trimmed templates and came back with no restarts. What is left over two lines is mostly a worked example -- an ingress route, an admissions entry, a make command -- where the shape is the explanation.
The split put quick-start files here. They belong on the quick-start branch.
The path split left quick-start files here; they belong on the quick-start branch. charts/federation keeps its own copy of oanfed.py.
Replaying onto fix/agmarknet-token-auth dropped four things that branch had added: the 0.8.0 bump, its changelog, Grievance in the single-adapter step list, and a pack URL pattern that allows a dotted ref. The pattern is now both widenings rather than either.
release-0.2.0 is where the networkId one-or-many and discover pagination live; the beckn tag predates both.
nginx-unprivileged keeps its temp paths and its pid under /tmp. The emptyDirs were at /var/cache/nginx and /var/run, where the privileged image keeps them, so with a read-only root nginx could not start: mkdir() "/tmp/proxy_temp" failed (30: Read-only file system).
A secret volume is owned by root and the Job runs as 101, so defaultMode 0400 left the key unreadable: PermissionError on /keys/signingPrivateKey. fsGroup gives the file a group the container is in, and 0440 lets that group read it.
oanfed appends /api/v1, the adapter's own registry.url already carries it, and the comment said to match the adapter -- so the value was copied across and Sunbird answered 404 reading "api" as a schema name.
The consumer routed discover there and nothing else, so a publish arriving at the consumer had no rule and errored before it reached the mount that answers it.
Only services that export OTLP themselves (network adapter, discovery) reached HyperDX; everything else logged to stdout and nothing collected it. A filelog receiver now reads /var/log/pods on each node, k8sattributes names the owning workload, and that becomes service.name. Chart 0.2.0.
…er and /publish routes
ameersohel45
force-pushed
the
feat/73-crawler-config-helm-charts
branch
from
October 30, 2026 11:10
5de31ab to
ebadf8d
Compare
manjudr
approved these changes
Oct 30, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the charts a cluster needs to join a network of networks.
What
charts/federation(new) —files/oanfed.pysigns and publishes the Network Descriptor, JWKS and schema-pack list, and admits another network by writing its key into our registry.job-publish.yamlandjob-admit.yamlrun those two actions;deployment.yamlserves the documents under.well-known/.charts/adapter-service—templates/ingress.yamlrenders one Ingress per rewrite target, keeping mount names off the public path and refusing a rewrite onto/publishor/consumer.values.yamlgains thecrawlerblock with an on/off switch, andingress.allowUnsignedMounts, off by default.charts/registry—_helpers.tpldemands Keycloak settings only whileauthenticationEnabledis true.Participant.jsoncarriesrevisionandfederationRole. Chart to 0.6.1.charts/otel-agent— unrelated to federation: a filelog receiver so every pod's stdout reaches ClickStack. Chart to 0.2.0.Why
Participant.jsonis byte-identical to Merge NetworkofNetwork into development [#72] #109's copy, so the two do not conflict.allowUnsignedMountsis opt-in because exposing/publishlets anyone write a catalog.Testing
helm lintpasses forcharts/federationandcharts/registrywith theirci/lint-values.yaml. Not deployed to a cluster.