Skip to content

feat: publish group documents under the group's own network provider [#190] - #36

Merged
hariharanweb merged 23 commits into
mainfrom
feat/190-group-provider
Oct 20, 2026
Merged

hariharanweb merged 23 commits into
mainfrom
feat/190-group-provider

Conversation

@hariharanweb

@hariharanweb hariharanweb commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

OpenAgriNet/engineering-tracker#190

What

Each catalog published to the network now has a provider per Knowledge Kind and instance. Documents filed under a state (its Group) publish under that Group's own provider. See ADR 0006.

Document Provider id and descriptor code Provider and catalog name Catalog / resource ids
bv / default advisory advisory.bv.provider.oan.dev Provider: Advisories from documents. Catalog: unchanged ("Agricultural advisory from documents") Unchanged: cat-oan-knowledge-provider-advisories
mh advisory advisory.mh.provider.oan.dev Maharashtra Advisories from documents mh-cat-…-advisories, mh-res-…-advisories
mh scheme scheme.mh.provider.oan.dev Maharashtra Schemes from documents mh-cat-…-schemes, mh-res-…-schemes
ka with an override Id from the overrides file, e.g. advisory.ka.com Karnataka Advisories from documents ka-cat-…, ka-res-…
Kind other than advisory/scheme Skipped as before — —

Every state publishes; no state is skipped for missing config.

How

  • Defaults (env, all optional):
    • NETWORK_PROVIDER_ID_TEMPLATE, default {kind}.{instance}.provider.oan.dev.
    • NETWORK_ADVISORY_PROVIDER_NAME and NETWORK_SCHEMES_PROVIDER_NAME, defaults "Advisories from documents" and "Schemes from documents".
    • NETWORK_GROUP_PROVIDER_NAME_TEMPLATE, default {group_name} {provider_name}.
  • Overrides: PROVIDER_CONFIG_OVERRIDE_PATH points to a JSON file of instance → {kind: provider id}, e.g. {"ka": {"advisory": "advisory.ka.com", "scheme": "scheme.ka.com"}}.
    • A kind or instance the file leaves out keeps the template id.
    • When the variable is unset, the bundled empty pipeline/network_group_providers.json is used. There's an example at deploy/network_group_providers.example.json.
  • Boot check: a missing or malformed file, an unknown kind, an empty id, a template with an unknown placeholder, an id template missing {kind} or {instance}, a Group name template missing {group_name}, or a provider id (rendered or overridden) that isn't lowercase letters, digits, - and dot-separated parts stops the API and worker from booting. A document whose instance code renders an invalid id fails its publish, without retry, before anything is sent.
  • Group: taken from the document's instance, not the uploader. PORTAL_INSTANCE and DEFAULT_INSTANCE mean "no Group", and those documents use PORTAL_INSTANCE (bv) in the provider id.
  • Scope: only the Publish to Network payload changes. Chunks and the DEV/PROD indexes are untouched and keep instance/instance_name. senderId/networkId are unchanged.
  • Code: provider resolution is in pipeline/network_provider.py (pure); network_group.py reads env and the overrides file; catalog_builder.py takes the resolved provider.
  • Docs: ADR 0006, CONTEXT.md (Group, Group Provider, Shared Provider), ENV.md, AGENTS.md.

Not in this PR

  • Already-published documents aren't moved.
  • Group search filtering is in #202, which can filter on instance.

Deploy notes

  • Network adapter: it must accept the new provider ids (for bv too) and the Groups' prefixed catalog and resource ids.
  • bv provider changes: the next bv publish re-announces the existing catalogs under advisory.bv / scheme.bv instead of knowledge-provider, with the same catalog ids. Whether the network moves them or keeps the old entry is untested.
  • Overrides: only needed for a state that wants its own id. Mount the file into api and worker and set PROVIDER_CONFIG_OVERRIDE_PATH (compose doesn't do this).

Testing

  • uv run pytest: 563 passed. Tests cover Group resolution, provider naming and overrides, override-file parsing, template validation, boot refusal, the catalog shape, the publish service and the activity.
  • uv run ruff check .: clean. cd ui && npm run build: clean.
  • Local stack: an mh scheme published under the earlier per-Group design and was ACCEPTED by dpg-dev.openagrinet.global. Payloads were rendered in the worker for the current design.
  • Not run: a live publish with the current design.

🤖 Generated with Claude Code

hariharanweb and others added 17 commits October 13, 2026 17:13
Documents filed under a state publish to the network under that Group's
provider (NETWORK_GROUP_PROVIDER_IDS), with group-prefixed catalog and
resource ids. Bharat Vistaar and default keep the shared provider. A Group
with no provider is skipped with a warning shown on the document page; a
malformed map stops the API and worker from booting. Chunks gain a group
field beside instance/instance_name. See ADR 0006.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…[#190]

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Comment thread pipeline/discovery_publish_service.py
Comment thread pipeline/network_group.py Outdated
Comment thread pipeline/network_provider.py Outdated
Comment thread pipeline/document_repository.py Outdated
Comment thread pipeline/network_group.py Outdated
Comment thread pipeline/network_group.py
Comment thread pipeline/discovery_publish_service.py
Comment thread pipeline/catalog_builder.py
hariharanweb and others added 6 commits October 20, 2026 14:02
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@hariharanweb
hariharanweb merged commit 74166ca into main Oct 20, 2026
3 checks passed
@hariharanweb
hariharanweb deleted the feat/190-group-provider branch October 20, 2026 10:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants