Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion charts/registry/files/schemas/SchemaRegistry.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
"name": { "type": "string", "minLength": 1, "maxLength": 200, "pattern": "\\S" },
"version": { "type": "string", "pattern": "^v[0-9]+\\.[0-9]+$" },
"schemaUrl": { "type": "string", "maxLength": 2000,
"pattern": "^https://raw\\.githubusercontent\\.com/OpenAgriNet/network-specs/[A-Za-z0-9_-]+/schema/[A-Za-z0-9]+/v[0-9]+\\.[0-9]+/[A-Za-z0-9_-]+\\.yaml$" },
"pattern": "^https://raw\\.githubusercontent\\.com/OpenAgriNet/network-specs/[A-Za-z0-9._-]+/(schema|api-schemas)/[A-Za-z0-9]+/v[0-9]+\\.[0-9]+/[A-Za-z0-9_-]+\\.yaml$" },
"status": { "$ref": "#/definitions/Status" }
}
},
Expand Down
29 changes: 22 additions & 7 deletions quick-start/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@
#
# One line per variable, and one meaning per name. Where a name carries a
# prefix, the prefix says WHOSE the value is: MAUSAMGRAM_/AGMARKNET_/VISTAAR_/
# POCRA_ are the four upstream entities, CONSUMER_/NETWORK_/PROVIDER_ are the
# three adapters, REGISTRY_/KEYCLOAK_/DISCOVERY_ are the services.
# POCRA_/PMFBY_ are the five upstream entities, CONSUMER_/NETWORK_/PROVIDER_ are
# the three adapters, REGISTRY_/KEYCLOAK_/DISCOVERY_ are the services.
#
# Every credential below is a shipped default, which means it is public. Change
# all of them before the stack is reachable by anyone but you. The adapter
Expand All @@ -24,9 +24,11 @@ DISCOVERY_CONTEXT=../../discovery-service
# subscription. Uncomment only if you have one.
#DISCOVERY_BUILD_IMAGE=dhi.io/golang:1.27-alpine-dev
#DISCOVERY_RUNTIME_IMAGE=dhi.io/static:20260611-musl-alpine
# The two mock upstreams. No capability uses one; `make up` does not start them.
# The mock upstreams. No capability uses one; `make up` does not start them.
MOCKIMD_IMAGE=ghcr.io/nisargabd/oan-mockimd:latest
MOCKAGMARKNET_IMAGE=ghcr.io/nisargabd/oan-mockagmarknet:latest
# Built locally: docker build -t oan-mockpmfby:local mock-server/mockpmfby
MOCKPMFBY_IMAGE=oan-mockpmfby:local

# ---- ports -----------------------------------------------------------------
# All published on 127.0.0.1 except the reverse proxy's 80/443, and the loopback
Expand All @@ -42,6 +44,7 @@ CONSUMER_ADAPTER_PORT=9202
# Only for looking at a mock directly; the adapter reaches it by service name.
MOCKIMD_PORT=9100
MOCKAGMARKNET_PORT=9101
MOCKPMFBY_PORT=9102
# The reverse-proxy profile (nginx-proxy-manager). 80/443 are public, admin is not.
REVERSE_PROXY_HTTP_PORT=80
REVERSE_PROXY_HTTPS_PORT=443
Expand Down Expand Up @@ -149,9 +152,9 @@ CONSUMER_BASE_URL=https://consumer.oan.dev
NETWORK_BASE_URL=https://network.oan.dev
PROVIDER_BASE_URL=https://provider.oan.dev

# ---- the four upstream providers -------------------------------------------
# setup.py creates all four plus a capability binding each -- seven participants
# and four bindings in all. The registry is unreachable from outside the stack,
# ---- the five upstream providers -------------------------------------------
# setup.py creates all five plus a capability binding each -- eight participants
# and five bindings in all. The registry is unreachable from outside the stack,
# so there is no second way to create them.
#
# THE BINDING KEY IS <PARTICIPANT_ID>|<CAPABILITY>. The provider adapter builds
Expand All @@ -160,7 +163,7 @@ PROVIDER_BASE_URL=https://provider.oan.dev
# what keeps the two from disagreeing. Change one, re-run setup.py.
# A mismatch is a 404, either NET_ENTITY_NOT_FOUND or "no active record".
#
# All four are REAL external APIs -- the stack needs egress to each.
# All five are REAL external APIs -- the stack needs egress to each.

# WeatherObservation, from IMD Mausamgram. One ready-made header value, "Basic "
# prefix included, sent verbatim (authScheme header) -- IMD issues a single
Expand Down Expand Up @@ -211,3 +214,15 @@ POCRA_CAPABILITY=openagrinet:AgricultureFacility
POCRA_BASE_URL=https://middleware-bap-client.mahapocra.gov.in
POCRA_PATH=/search
POCRA_MAPPING_URL=https://raw.githubusercontent.com/OpenAgriNet/helmcharts/0.0.1/quick-start/config/mappings/pocra/agriculture-facility.select.yaml

# PMFBYGrievance, from PMFBY's grievance service (KRPH FGMS). authScheme
# tokenHeader: the adapter logs in at TOKEN_URL and sends the token bare in
# Authorization. BASE_URL is the host alone; the paths carry /krphapi/FGMS.
PMFBY_PARTICIPANT_ID=pmfby
PMFBY_CAPABILITY=openagrinet:Grievance
# The directory holding grievance.support.yaml and grievance.status.yaml.
PMFBY_MAPPING_BASE_URL=https://raw.githubusercontent.com/OpenAgriNet/helmcharts/development/quick-start/config/mappings/pmfby
PMFBY_BASE_URL=<the PMFBY host, e.g. https://pmfbydemo.amnex.co.in>
PMFBY_TOKEN_URL=<the PMFBY login endpoint, absolute: https://host/krphapi/FGMS/NICUsersLogin>
PMFBY_ACCESS_UID=<the PMFBY appAccessUID>
PMFBY_ACCESS_PWD=<the PMFBY appAccessPWD>
101 changes: 68 additions & 33 deletions quick-start/bin/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,18 @@
it renders are bind-mounted files, and an adapter started before they exist
leaves a directory in their place.

WHAT IT WRITES. Seven participants, four capability bindings, four schemas:
WHAT IT WRITES. Eight participants, five capability bindings, five schemas:

3 x node one per adapter -- consumer, network, provider -- each with the
public halves of a keypair. The private halves stay in
keys/keys.json and never reach the registry.
4 x upstream the APIs this deployment calls. ALL FOUR are real external
5 x upstream the APIs this deployment calls. ALL FIVE are real external
hosts now, whose base URLs come from .env -- no mock is in
use by any capability. An upstream signs nothing, so it needs
no role and no keys.
4 x binding a ProviderSchema row per capability: which upstream answers
5 x binding a ProviderSchema row per capability: which upstream answers
it, the method and path, timeouts, and the mapping URL.
4 x schema a SchemaRegistry row per capability, saying what the
5 x schema a SchemaRegistry row per capability, saying what the
capability MEANS rather than who serves it: the pack in
network-specs that defines its attributes. One per
capability, so two providers of one capability share a row.
Expand Down Expand Up @@ -125,6 +125,9 @@ def is_placeholder(value):
"VISTAAR_CLIENT_SECRET", "VISTAAR_MAPPING_URL",
"POCRA_PARTICIPANT_ID", "POCRA_CAPABILITY", "POCRA_BASE_URL",
"POCRA_PATH", "POCRA_MAPPING_URL",
"PMFBY_PARTICIPANT_ID", "PMFBY_CAPABILITY", "PMFBY_BASE_URL",
"PMFBY_TOKEN_URL", "PMFBY_ACCESS_UID", "PMFBY_ACCESS_PWD",
"PMFBY_MAPPING_BASE_URL",
]


Expand Down Expand Up @@ -323,7 +326,7 @@ def upstream(participant_id, name, base_url):


def ensure_binding(bearer, participant_id, capability, path, mapping_url,
method="GET"):
method="GET", actions=None):
"""Create a capability binding only when absent.

actions is a list, not a map: the registry treats every nested object as an
Expand All @@ -336,52 +339,72 @@ def ensure_binding(bearer, participant_id, capability, path, mapping_url,
method is a parameter because it is the provider's contract, not a network
convention: Mausamgram and Agmarknet answer a GET whose query the mapping
builds, while the knowledge provider and POCRA take a POST with a JSON body.
The adapter reads it from this row, so nothing about it is compiled in."""
The adapter reads it from this row, so nothing about it is compiled in.

actions, when given, replaces the single select."""
binding = f"{participant_id}|{capability}"
if search("ProviderSchema", {"bindingKey": {"eq": binding}}):
print(f" {binding}: already present")
return
result = post("ProviderSchema", {
"bindingKey": binding, "participantId": participant_id,
"capabilityCode": capability, "status": "active",
"actions": [{"action": "select", "method": method, "path": path,
"mappings": mapping_url,
"timeoutMs": 15000, "retryMax": 2,
"status": "active"}]}, bearer)
"actions": actions or [{"action": "select", "method": method, "path": path,
"mappings": mapping_url,
"timeoutMs": 15000, "retryMax": 2,
"status": "active"}]}, bearer)
print(f" {binding}: {result['params']['status']} "
f"{result['params'].get('errmsg', '')[:160]}")


# The four capability schemas, keyed by the .env variable each capability code
# network-specs keeps resource packs under schema/ and API packs under
# api-schemas/, each folder on its own ref.
RESOURCE_PACKS = "schema"
API_PACKS = "api-schemas"

# The five capability schemas, keyed by the .env variable each capability code
# comes from so the code itself is never written twice.
#
# The display name mirrors the pack's own info.title, minus its "OpenAgriNet -"
# and "Attributes" framing. The version is per capability rather than one
# setting for all four, because a capability can move to v0.2 while the others
# setting for all five, because a capability can move to v0.2 while the others
# stay -- that is the point of versioning the packs separately.
CAPABILITY_SCHEMAS = (
("MAUSAMGRAM_CAPABILITY", "Weather Observation", "v0.1"),
("AGMARKNET_CAPABILITY", "Mandi Price", "v0.1"),
("VISTAAR_CAPABILITY", "Knowledge Advisory", "v0.1"),
("POCRA_CAPABILITY", "Agriculture Facility", "v0.1"),
("MAUSAMGRAM_CAPABILITY", "Weather Observation", "v0.1", RESOURCE_PACKS),
("AGMARKNET_CAPABILITY", "Mandi Price", "v0.1", RESOURCE_PACKS),
("VISTAAR_CAPABILITY", "Knowledge Advisory", "v0.1", RESOURCE_PACKS),
("POCRA_CAPABILITY", "Agriculture Facility", "v0.1", RESOURCE_PACKS),
("PMFBY_CAPABILITY", "Grievance", "v0.1", API_PACKS),
)


def schema_url(capability, version, ref):
def pmfby_actions():
"""PMFBY's support and status. support is never retried: a retry files a
duplicate grievance."""
base = env("PMFBY_MAPPING_BASE_URL")
return [{"action": action, "method": "POST", "path": path,
"mappings": f"{base}/grievance.{action}.yaml",
"timeoutMs": 30000, "retryMax": retries, "status": "active"}
for action, path, retries in (
("support", "/krphapi/FGMS/AddKRPHNCIPGrievenceSupportTicket", 0),
("status", "/krphapi/FGMS/GetGrievenceTicketsStatus", 2))]


def schema_url(capability, version, folder, ref):
"""Build a pack URL from the capability code, which already contains the
pack directory's name: openagrinet:WeatherObservation is served out of
schema/WeatherObservation. Deriving it means the two cannot drift.

The registry pattern-checks the result against exactly this shape, so a
capability whose pack is laid out differently is refused rather than
stored -- which is the behaviour to want, since all eight packs in
network-specs are <Capability>/<version>/attributes.yaml today."""
stored -- which is the behaviour to want, since every pack in
network-specs is <folder>/<Capability>/<version>/attributes.yaml today."""
return ("https://raw.githubusercontent.com/OpenAgriNet/network-specs/"
f"{ref}/schema/{capability.partition(':')[2]}/{version}"
f"{ref}/{folder}/{capability.partition(':')[2]}/{version}"
"/attributes.yaml")


def ensure_capability_schema(bearer, capability, name, version, ref):
def ensure_capability_schema(bearer, capability, name, version, folder, ref):
"""Create a capability's schema row only when absent.

THE REGISTRY NEVER FETCHES schemaUrl -- it only checks its shape. So a row
Expand All @@ -398,7 +421,7 @@ def ensure_capability_schema(bearer, capability, name, version, ref):
return
result = post("SchemaRegistry", {
"capabilityCode": capability, "name": name, "version": version,
"schemaUrl": schema_url(capability, version, ref),
"schemaUrl": schema_url(capability, version, folder, ref),
"status": "active"}, bearer)
print(f" {capability}: {result['params']['status']} "
f"{result['params'].get('errmsg', '')[:160]}")
Expand All @@ -419,7 +442,7 @@ def seed(identities):
wait_for_registry()
bearer = token()

# Seven participants and four capability bindings, all of it from here.
# Eight participants and five capability bindings, all of it from here.
#
# The registry is not reachable from outside this stack -- no published
# port beyond loopback and no proxy host in front of it -- so there is no
Expand All @@ -436,14 +459,14 @@ def seed(identities):
env(f"{role.upper()}_BASE_URL"),
identity["signingPublic"]))

# All four upstreams are real external providers now -- no mock is in use
# All five upstreams are real external providers now -- no mock is in use
# by any capability, so every base URL comes from .env and the stack needs
# egress to all of them.
#
# No default base URL for weather any more: falling back to the mock's
# service name would seed a row that cannot serve the basic auth the
# capability is configured with, and the registry cannot update it after.
print("registry: four upstream providers")
print("registry: five upstream providers")
weather = env("MAUSAMGRAM_PARTICIPANT_ID")
ensure_participant(bearer, weather,
upstream(weather, "IMD Mausamgram NWP",
Expand Down Expand Up @@ -474,11 +497,15 @@ def seed(identities):
upstream(pocra, "POCRA agriculture facilities",
env("POCRA_BASE_URL")))

pmfby = env("PMFBY_PARTICIPANT_ID")
ensure_participant(bearer, pmfby,
upstream(pmfby, "PMFBY grievance portal", env("PMFBY_BASE_URL")))

# And what each of them answers. The binding key is participantId piped to
# capabilityCode, and it has to match what the provider adapter was
# rendered with -- both come from the same .env, which is what keeps them
# from disagreeing.
print("registry: four capability bindings")
print("registry: five capability bindings")
ensure_binding(bearer, weather, env("MAUSAMGRAM_CAPABILITY"),
env("MAUSAMGRAM_PATH", "/get-daily"), env("MAUSAMGRAM_MAPPING_URL"))
ensure_binding(bearer, mandi, env("AGMARKNET_CAPABILITY"),
Expand All @@ -490,6 +517,8 @@ def seed(identities):
ensure_binding(bearer, pocra, env("POCRA_CAPABILITY"),
env("POCRA_PATH"), env("POCRA_MAPPING_URL"),
method="POST")
ensure_binding(bearer, pmfby, env("PMFBY_CAPABILITY"),
None, None, actions=pmfby_actions())

# And what each capability MEANS, which is a different question from who
# serves it. A binding above says mausamgram answers WeatherObservation
Expand All @@ -501,10 +530,11 @@ def seed(identities):
# SEPARATE FROM THE MAPPING URLS ABOVE, which live in helmcharts and
# describe one upstream's wire format. These are network-specs and describe
# the network's own vocabulary, so they move on their own ref.
print("registry: four capability schemas")
ref = env("SCHEMA_PACK_REF", "main")
for var, name, version in CAPABILITY_SCHEMAS:
ensure_capability_schema(bearer, env(var), name, version, ref)
print("registry: five capability schemas")
pack_refs = {RESOURCE_PACKS: env("SCHEMA_PACK_REF", "main"),
API_PACKS: env("API_SCHEMA_PACK_REF", "api-schema-packs-v0.1")}
for var, name, version, folder in CAPABILITY_SCHEMAS:
ensure_capability_schema(bearer, env(var), name, version, folder, pack_refs[folder])


def key_osids(identities):
Expand Down Expand Up @@ -565,6 +595,7 @@ def render(identities):
vistaar_binding = (f"{env('VISTAAR_PARTICIPANT_ID')}"
f"|{env('VISTAAR_CAPABILITY')}")
pocra_binding = f"{env('POCRA_PARTICIPANT_ID')}|{env('POCRA_CAPABILITY')}"
pmfby_binding = f"{env('PMFBY_PARTICIPANT_ID')}|{env('PMFBY_CAPABILITY')}"
for role in ROLES:
identity = identities[role]
template = (ADAPTERS / f"{role}.yaml.tmpl").read_text()
Expand Down Expand Up @@ -592,6 +623,9 @@ def render(identities):
("__AGMARKNET_PARTICIPANT_ID__", env("AGMARKNET_PARTICIPANT_ID")),
("__VISTAAR_PARTICIPANT_ID__", env("VISTAAR_PARTICIPANT_ID")),
("__POCRA_PARTICIPANT_ID__", env("POCRA_PARTICIPANT_ID")),
("__PMFBY_BINDING_KEY__", pmfby_binding),
("__PMFBY_PARTICIPANT_ID__", env("PMFBY_PARTICIPANT_ID")),
("__PMFBY_TOKEN_URL__", env("PMFBY_TOKEN_URL")),
# Telemetry. One switch drives all three signals: with every
# one false the plugin builds no exporter and never dials, so
# a stack running without the observability profile stays
Expand Down Expand Up @@ -629,27 +663,28 @@ def render(identities):
KEYS.write_text(json.dumps(identities, indent=2))
render(identities)
print(f"""
ready. The registry holds seven participants and four capability bindings, and
ready. The registry holds eight participants and five capability bindings, and
the adapter configs are rendered, so nothing further has to be created by hand.

{env('MAUSAMGRAM_PARTICIPANT_ID')}|{env('MAUSAMGRAM_CAPABILITY')}
{env('AGMARKNET_PARTICIPANT_ID')}|{env('AGMARKNET_CAPABILITY')}
{env('VISTAAR_PARTICIPANT_ID')}|{env('VISTAAR_CAPABILITY')}
{env('POCRA_PARTICIPANT_ID')}|{env('POCRA_CAPABILITY')}
{env('PMFBY_PARTICIPANT_ID')}|{env('PMFBY_CAPABILITY')}

Those are the binding keys the provider adapter answers to. They were rendered
into its config from the same .env this seeded the registry from, which is what
keeps the two from disagreeing. A payload naming anything else is answered 404
"this module serves no capability matching the request" -- explicit, but it
names the request rather than the mismatch, so compare it against these four.
names the request rather than the mismatch, so compare it against these five.

The left half of each key is the participant id, and it names the ENTITY that
operates the upstream -- not the capability, which is already the right half,
and not the deployment, which is only a base URL. It is also the string a
catalog publishes as offer.provider.id, so capability-examples/ can be run
against this stack unedited.

All four upstreams are real external providers reached over the internet, so
All five upstreams are real external providers reached over the internet, so
this stack needs egress to each of them and no capability depends on a mock.
Repointing one is an .env edit and a re-run of this, but it takes a NEW
participant id: the registry cannot update a row and its delete is soft and
Expand Down
10 changes: 5 additions & 5 deletions quick-start/bin/stack.sh
Original file line number Diff line number Diff line change
Expand Up @@ -135,15 +135,15 @@ up_setup() {

# Only now do the bind-mounted config files exist.
up_adapters() {
# The mocks are NOT started. All four capabilities point at real external
# upstreams, so starting them would pull two images and run two containers
# nothing calls. They are still defined in docker-compose.yml, so
# mock-pmfby is started with the adapters. The other mocks are NOT: those
# capabilities point at real external upstreams, so starting them would run
# containers nothing calls. They are still defined in docker-compose.yml, so
# `docker compose up -d mock-imd mock-agmarknet` brings them up for anyone
# debugging a mapping without the real credentials -- but that also needs a
# new participant id and base URL in .env plus a setup.py re-run, because
# the registry cannot repoint an existing row.
step 3 "$1" "adapters (provider, network, consumer)"
docker compose up -d provider-adapter network-adapter consumer-adapter
step 3 "$1" "adapters (provider, network, consumer) and mock-pmfby"
docker compose up -d provider-adapter network-adapter consumer-adapter mock-pmfby
}

done_banner() {
Expand Down
Loading
Loading