Skip to content

webapi-openapi: a modular layout for large APIs, current dhall-do-api names, a golden test - #47

Open
mageshb wants to merge 5 commits into
masterfrom
generator-hardening
Open

mageshb wants to merge 5 commits into
masterfrom
generator-hardening

Conversation

@mageshb

@mageshb mageshb commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Generator hardening for zenbridge-connectors (M1): the generator reads a spec the size of Zoho Books (44 files, 881 operations) and writes a package that compiles and seals.

  • Current dhall-do-api names. The registry emits OperationId/mkOperationId (WebApi.Contract hiding (OperationId)) and an ErrorText instance per named error type; regenerating the NetSuite currency slice now matches zenbridge's hand-patched copy.
  • A modular layout, --config FILE (WebApi.OpenAPI.Modular): one package split into public sublibraries by consumer — model-<m> per module of the document's x-zb object; the contract as the main library (App, Routes.<M>, Contract with the Apis list, Contract.<M> with the ApiContract instances — apart because WebApi is ApiContract's superclass); registry with the bridge instances and each module's operations. Hand-written stanzas append from the config; gen/warnings.txt lists what could not be typed.
  • In that layout: names resolve once to PascalCase; untyped or property-less schemas are Opaque (JSON, crossing to Dhall as text) rather than a crash; optional arrays are Maybe and ToJSON omits absent fields; NoFieldSelectors; a route with several captures gets a named path record instead of a tuple dhall-do's bridge cannot carry; JSON is kept of several media types, the lowest of several 2xx; form/multipart bodies and header parameters are left out with a warning (next: GEN-5); array bodies of generated types get a whole-value OverrideType.
  • Legacy layout unchanged, pinned by test/golden/ns-currency (cabal test --project-file=cabal.project.spike webapi-openapi:test:golden, run from webapi-openapi/; GOLDEN_ACCEPT=1 rewrites).

Measured on Zoho Books through zenbridge-connectors' normalizer and overlays: 857 operations, 2,476 data types, a clean -O0 build in 3m27s peaking at 945 MB, and every operation seals under strict mode. dhall-do-api gaps found on the way (worked around here, better fixed there): no bridge instances for tuples, Vector overrides, aeson Value, Day, Scientific. Merge without squash: zenbridge-connectors' gen/PROVENANCE.md names the head sha.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TBKfpQZQ9kXvq9mr6G5qcf

mageshb and others added 5 commits September 19, 2026 01:15
… names, a golden test

The generator could not read a spec the size of Zoho Books (44 files, 881
operations): it stopped on untyped and property-less schemas, several media
types, regex paths and name clashes, emitted everything into two modules,
and its registry no longer compiled against dhall-do-api after the
OperationId rename.

- The registry emits dhall-do-api's current names (OperationId,
  mkOperationId, WebApi.Contract hiding OperationId) and an ErrorText
  instance for every named error type.
- --config FILE selects a modular layout (WebApi.OpenAPI.Modular): one
  package, split into public sublibraries by consumer. model-<m> per module
  of the document's x-zb object; the contract as the main library (App,
  Routes.<M>, Contract with the Apis list, Contract.<M> with the ApiContract
  instances, apart because WebApi is ApiContract's superclass); registry
  with the bridge instances and each module's operations. Hand-written
  stanzas are appended from the config; gen/warnings.txt lists what could
  not be typed.
- In that layout: component names resolve once to PascalCase, avoiding what
  the modules import; an untyped or property-less schema is Opaque (its
  JSON, crossing to Dhall as text); optional arrays are Maybe and ToJSON
  omits absent fields; NoFieldSelectors; a route with several captures gets
  a named path record rather than a tuple the bridge cannot carry; JSON is
  kept of several media types and the lowest of several 2xx responses;
  form, multipart and header parameters are left out with a warning; array
  bodies of generated types get a whole-value OverrideType.
- The legacy single-module layout is unchanged: test/golden/ns-currency pins
  its output (GOLDEN_ACCEPT=1 rewrites it).

Measured on Zoho Books: 857 operations, 2,476 data types, a clean -O0 build
in 3m27s peaking at 945 MB; every operation seals under strict mode.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TBKfpQZQ9kXvq9mr6G5qcf
…ames

In the modular layout an operation's header parameters were left out. They
are a record now, one field per header in snake case (X-Upsert -> x_upsert),
with an explicit ToHeader that sends each under its wire name and leaves an
absent optional header out: webapi's generic ToHeader names a header after
its field, and a header name is not a field name. Zoho Books' upsert
operations (X-Upsert, X-Unique-Identifier-Key/-Value) are the first users:
an idempotent create for EDI writes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TBKfpQZQ9kXvq9mr6G5qcf
…sets

- gen/operations.json: every operation as data, one per line (name, id,
  method, path, module, route, the request and result types, the OAuth
  scopes, tags and x-mcp-group): what a connector's tests, ledger and docs
  read, instead of parsing the generated Haskell.
- connectionParams (config): query parameters the connection sets on every
  request, left out of each operation's own parameters. Zoho Books'
  organization_id is the first: a connection is one organization, so a plan
  never states it, and 638 operations lose their query record entirely.
  The manifest lists, per operation, which of them it took.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TBKfpQZQ9kXvq9mr6G5qcf
…st's components

- envelope (config): a response that is its envelope's fields (Zoho's code,
  message) plus exactly one resource binds as the resource. The binding
  reads the resource out (a response without it fails the step), so its
  fields are what a later step reads and what a class records. Lists (a
  resource beside page_context) and bare acknowledgements keep the envelope.
- classes (config, a connector's reviewed bindings/classes.yaml): each class
  is declared in the registry (recordsClassC at its type, or externalClassC
  when nothing records it), and each operation's bindings carry the result
  fields that record one (cbResultClasses) and the request fields that take
  one (cbRequestClasses: path, path.<capture>, body.<field>, query.<param>).
  A class naming an operation that does not exist fails generation.
- The manifest gains each operation's resource, binding result, and the
  components behind its body and result, for tools that propose classes.
- A leftover debug helper, f, is gone from WebApi.OpenAPI.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TBKfpQZQ9kXvq9mr6G5qcf
For each operation's top-level body and query fields, what the document
declares about a text field becomes a request refinement (requestRefinementC,
composed above the registration it names): maxLength and minLength as length
bounds over Text/length, and an allowed-value list (x-zb-allowed, which
zenbridge-connectors' miner writes from prose, or a query parameter's enum,
which stays Text) as a disjunction of Text/equal. An optional field is held
only when present. A plan that breaks one is refused at check, before the
vendor refuses the call.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TBKfpQZQ9kXvq9mr6G5qcf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant