unhardcoded

An OpenAI-compatible LLM router. Instead of hardcoding a model, a caller sends a policy with each request: the host filters the candidate models to those that qualify, ranks the survivors and picks one (e.g. the cheapest that passes) — over your own provider keys — and falls back automatically when a provider errors. Every run records which models passed, which were rejected and why.

Concretely it's an async FastAPI shim that runs the unhardcoded-engine core and inherits its provider selection, fallback, retry and per-provider auth. The core is vendored as a git submodule under core/; this repo is the host (the I/O, auth, providers, the HTTP service) plus an operator dashboard to compose, test and analyse policies. The policy algebra (Σ_pol) lives in the core.

The model: callers send a policy

A request carries its own Σ_pol policy as data — a policy_ir term in the body. The host admits it (sorts, arity, depth/size bounds), ∧-composes the host's policy_envelope so a caller can only narrow the host's invariants, then runs it: filter the candidate models, score the survivors, pick and fall back. Compose, preview, test live and download policies in the dashboard Builder.

There are no intelligence tiers. When a caller sends no policy, a single declarative default policy (balanced quality/cost) is the fallback — itself a Σ_pol term with an identity, editable like any other.

Layout

core/                  -- git submodule -> genlayerlabs/unhardcoded-engine (the pure Σ_pol core)
llm_router_host.py     -- embeds the core via lupa; sync/async backends; auth resolver
shim.py                -- OpenAI-compatible app: /v1/chat/completions and /v1/responses (+ per-call policy_ir) and /x/* operator endpoints
responses_api.py       -- inbound OpenAI *Responses* API surface for /v1/responses (sibling of the chat-completions surface)
auth_proxy.py          -- ingress + operator dashboard (Analytics · Builder · Activity · Market · Settings)
serve.py               -- entry point: wires the api_kind dispatcher + runs uvicorn
config.live.lua        -- catalog (providers + models), the `default` policy, and the observation fields (incl. OpenRouter benchmark/modality/capability fields)
scripts/refresh_model_meta.py -- the job: writes model_meta.lua (model-level traits pulled from OpenRouter)
codex_auth.py / codex_backend.py -- the ChatGPT-subscription (Codex) provider
metrics.live.lua       -- EMA seed (PLACEHOLDER/fake — see docs/METRICS.md)
docs/METRICS.md        -- the metrics seed: format, codex≈0, regeneration (it's fake)
docs/PROVIDERS.md      -- per-provider auth + the AntSeed node
docs/OPENAI-CODEX.md   -- the ChatGPT-subscription provider (unofficial / ToS-risky)
docs/USAGE_ENDPOINTS.md -- per-key usage stats API endpoints and auth behavior
live_smoke.py          -- drive real providers end-to-end
tests/                 -- the full host unit-test suite
features/              -- BDD user-flow suite (behave); user_flows.json is the spec
SETUP.md               -- agent/human setup runbook (clone -> running -> first call)
scripts/gen-dev-wallet.sh -- generate a local AntSeed dev wallet (testing)

Quickstart

Point a coding agent at SETUP.md ("set up unhardcoded by following SETUP.md") for a guided, copy-pasteable runbook — or do it yourself:

git clone --recursive https://github.com/genlayerlabs/unhardcoded.git && cd unhardcoded
cp .env.example .env.secrets && chmod 600 .env.secrets
# in .env.secrets: set OPENROUTER_API_KEY, and DASHBOARD_NO_AUTH=1 for local dev
docker compose up -d --build
curl -fsS http://127.0.0.1:8080/healthz            # -> {"ok":true,"initialized":true}

Open the dashboard at http://127.0.0.1:8080/dashboard, mint a caller key (POST /dashboard/api/keys {"consumer":"my-app"}, or Consumers → Generate key), then call:

curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H "Authorization: Bearer <key>" -H "Content-Type: application/json" \
  -d '{"model":"","messages":[{"role":"user","content":"Reply: pong"}]}'

Empty model runs the default policy; or send family:/pin:/profile: or a per-call policy_ir/flow_ir (below). Optional providers (Codex, AntSeed) and troubleshooting are in SETUP.md.

Responses-only clients (e.g. the Codex CLI): POST /v1/responses (and POST /{profile}/v1/responses) speak the OpenAI Responses API — translated to the same routing/policy/metering as chat-completions, streaming Responses SSE. Point Codex at the router with base_url=…/v1, wire_api = "responses", and the router key; use a /{profile}/v1/responses base_url to pin a policy by URL (Codex can't put policy_ir in its body).

Develop without Docker (raw shim — no dashboard/auth)

python3 -m venv .venv && . .venv/bin/activate
pip install -r requirements.txt
python serve.py --config config.live.lua --default-profile default --host 127.0.0.1 --port 8080

serve.py is the data plane only (no ingress auth, no dashboard); provider keys come from the process env. The bearer-token contract and the dashboard live in the ingress service (docker compose). (Nix users: a nix-shell with the same packages works too.)

Policies (the model field + policy_ir)

Routing is server-side. The primary path is a per-call policy; the model prefix is sugar for the common cases:

client sends	host does
policy_ir term in the body	run that Σ_pol policy (the primary path)
"" / unprefixed model	the default policy
model = "profile:NAME"	a named profile from the catalog (only default ships)
model = "family:FAMILY"	default, pinned to a model family
model = "pin:PROVIDER/FAMILY"	default, pinned to one (provider, family)

A policy is a Σ_pol term: filter (which models qualify) → score (rank the survivors) → pick (the selector) + request transform + failover. Write it as raw IR, or compose it in the Builder (structured rows ↔ raw term). The vocabulary it filters/scores over includes live fields (price_in/price_out, latency_ms, success_rate, …) and host-declared fields — the OpenRouter benchmarks/modalities/capabilities (bench_intelligence, in_image, cap_tools, …). See the core's core/docs/SIGMA-POL.md for the algebra; config.live.lua declares the host fields and the default policy.

Blessed policy templates

Most callers should not write raw weights. GET /x/policy/templates lists the supported intent-level templates; POST /x/policy/templates/{id} compiles one to an identified policy_ir that can be previewed with POST /x/rank.

curl -s http://127.0.0.1:8080/x/policy/templates/cheapest-family \
  -H "Authorization: Bearer <key>" -H "Content-Type: application/json" \
  -d '{
    "family": "glm-5.2",
    "provider_strategy": "ordered"
  }'

The three templates are:

• cheapest-family — stay in one exact family and minimize expected token cost; provider_strategy: "ordered" instead enforces Codex → AntSeed → Bedrock → OpenRouter, skipping unavailable providers and retaining breaker-open routes only as final fallbacks.
• smart-value — minimize cost among the current top five intelligence models (the shortlist size and price/reliability rails are configurable).
• default — the actual policy used by OpenAI-compatible callers that send no policy: prefer a top-five intelligence model, then enforce Codex → AntSeed → Bedrock → OpenRouter and use a 75% cost / 25% intelligence value score inside each provider. Lower-ranked models remain available when request requirements or an explicit family: model leave no top-five candidate.

The ordered strategy compiles to nested prefer(pred, inner) selectors, not large score weights. Cost still orders candidates inside each provider group. All templates reject unknown prices and default to explicit $5 input / $25 output per-million-token ceilings plus a 0.8 reliability floor; callers can tighten or deliberately raise those rails on the configurable templates.

Dashboard

auth_proxy serves an operator console at /dashboard:

• Analytics — spend, traffic and errors over time, filtered by timeframe, consumer, provider and model.
• Builder — compose a policy over raw + benchmark fields, preview the live ranking, Test call it with a prompt, and download the term to run per call.
• Activity — per-request trace: the policy that ran (copyable), the ordered fallback chain with the error at each step, and the cost paid.
• Market — live price book per model family.
• Settings — consumer keys and provider keys.

Tests

Install the test deps once (a virtualenv keeps them off your system Python):

python3 -m venv .venv && . .venv/bin/activate
pip install -r requirements.txt -r requirements-dev.txt

Unit — boots a real host with mocked provider responses (only the outbound HTTP to upstream providers is mocked):

python -m pytest tests -q

BDD user-flow suite (features/, behave) — drives the live stack end to end the way the dashboard does and asserts the rendered data is correct, including a real headless-browser pass (needs Chrome/Chromium installed; Selenium fetches the matching driver). Free & repeatable: end-to-end chats route to a $0 path. The catalogue of flows it covers is user_flows.json:

behave

(Nix users: nix-shell -p ... with the same packages — plus chromium chromedriver for the browser pass — works as before.)