Skip to content

Fast first-time reports: stop waiting on slow maker recall lookups and NHTSA complaints - #29

Open
keptlive wants to merge 6 commits into
mainfrom
fast-first-report
Open

keptlive wants to merge 6 commits into
mainfrom
fast-first-report

Conversation

@keptlive

Copy link
Copy Markdown
Owner

First-time reports no longer wait for the two slow sources: the maker's VIN recall lookup (Ford: about 9.5-14 s) and NHTSA's complaint list (5-17 s for a common model). Both are now budgeted. If one has not answered in time, the report marks it pending, which means unknown, not zero, and keeps the request running in the background so the cache fills.

Why Ford takes about 9.5 s

Measured from the web server with 4 real requests (1 VIN page, 3 no-VIN pages). Ford's CDN (Akamai) holds every request from our client for about 9-10 s at the edge (server-timing: edge dur=9000-10002). This happens even for a cached 301 and on a reused connection. Ford's origin answers in about 120 ms. Getting past the hold would mean changing the client's identity, which the rules do not allow. The VIN page is also 1.27 MB, with the recall data 81% of the way in. Details: docs/data-sources.md, "When the maker answers slowly".

What changed

  • Budget (lib/reports.mjs): the maker's lookup starts as soon as the decode names the make, beside NHTSA's calls. The report waits up to 2 s from that start (website/REST) or 6 s (MCP decode_vin). Comparisons wait for the full answer. Past the budget, the report returns recall_status: {pending: true, available: false, reason: "lookup_in_progress", open_count: null, completed_count: null, recalls: [], maker, waited_ms, status_url, note}. One lookup per VIN runs at a time, shared by every caller. A background failure is caught and not cached.
  • NHTSA complaints: modelYearComplaints runs one request per make/model/year at a time, caches it for 6 h and allows 30 s. The old 10 s timeout always failed the 2020 Escape's 17 s list, so that report was never cached. Complaints use the same budget, with complaints.pending and status_url: /api/vin/{vin}/complaints. The server-rendered /{vin} page no longer waits up to 8 s for them.
  • REST: ?recall_wait=full waits for the maker (any other value returns 400). GET status_url joins the running lookup. VinClient.decode(vin, {recallWait: 'full'}).
  • MCP: decode_vin has a 6 s budget and an optional recall_wait: "full". A pending status carries next_step: {tool: "vin_recall_status", arguments: {vin}}. vin_recall_status joins the running lookup.
  • Website: shows "Checking with Ford…" in the recall card, the car sheet row and the decode-checks line, and "Loading NHTSA's owner complaints…" in the complaints card. It then asks ?recall_wait=full and /complaints and redraws only the blocks that changed. public/keep-state.js keeps typed text, open forms, focus and caret, the transmission choice and the place on screen. If a sheet is printed while a source is pending, it says "Unknown: Ford had not answered when this sheet was printed" (printing does not wait). If a follow-up fails twice, the block reads as not loaded (unknown).
  • Other waits: related-name recall lookups now run beside the exact lookup, 3 at a time. Safety-rating details run 3 at a time. The Transport Canada generation check takes one round trip instead of two.

Timing (local, Ford's page mocked at 9.5 s)

first report Ford status complete complaints complete
before, upstreams mocked 10.4-11.6 s in report in report
after, upstreams mocked 2.07-2.32 s 9.6-9.8 s in report
before, real NHTSA 11.6-17.5 s in report in report
after, real NHTSA 2.08-2.34 s 9.6-9.8 s 3.3-9.1 s

Checks

  • flock /tmp/vin-mcp-test.lock npm test: 271 node tests + Python tests pass. New tests: tests/fast-first-report.test.mjs, tests/slow-maker.test.mjs (real server, Ford and complaints delayed 2.6 s) and tests/pending-recall-status.test.mjs.
  • Headless Chromium at 390 px, 2020 Escape and 2005 Mustang, follow-ups held then released. Placeholders appear and fill in place. Typed text, <select>, open <details>, focus and caret, and "Mine is a manual" are all kept. The screen position moves 0 px, with and without the browser's own scroll anchoring. Only the status rows and cards change height. There are no page errors. The car sheet prints on one Letter page while pending and after.

Not deployed. After merge, deploy the server files and restart: server.mjs, lib/reports.mjs, lib/recall-status.mjs, lib/nhtsa.mjs, lib/cache.mjs, lib/tire-placards.mjs, lib/mcp/vehicle-tools.mjs, lib/mcp/research-tools.mjs. Static files: public/app.js, car-sheet.js, decode-checks.js, keep-state.js (new), style.css, client.mjs, index.html, research/research.js, llms-full.txt.

🤖 Generated with Claude Code

root and others added 6 commits November 18, 2026 10:19
The maker's VIN recall lookup now starts right after the decode, beside
NHTSA's calls, and a report waits for it at most 2 s (website, REST) or
6 s (MCP decode_vin), counted from its start. Past that, recall_status is
{pending: true, reason: 'lookup_in_progress', status_url} (unknown, never
'no open recalls') and the lookup finishes in the background, filling its
15-minute cache. Concurrent requests share one lookup; a background
failure is contained. ?recall_wait=full (REST) and recall_wait 'full'
(MCP) wait for it; comparisons always wait.

/api/vin/{vin}/recall-status and MCP vin_recall_status resolve the make
from the decode when the WMI alone does not name the maker, so they join
the lookup a report started.

Also: related-name recall lookups start beside the exact one and run
three at a time; safety-rating details three at a time; the Transport
Canada generation check asks all years at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a report arrives with recall_status.pending, the recall card, the
car sheet's recall row and the decode-checks source line say "Checking
with <maker>..." on screen. The page then asks /api/vin/{vin}?recall_wait=full
(the same running lookup) and redraws only what changed: the status
block, the sheet's recall section and footer, and the decode-check line
(or the decode, engine, fuel and test-vehicle blocks if the maker's
record changed a checked value). keep-state.js keeps typed text, open
forms, focus, the transmission choice and the place on screen.

Printed while pending, the sheet says the maker had not answered when
it was printed (never a "checking" placeholder, never "no recalls").
If the follow-up fails twice, the status reads as not loaded (unknown).

Tests: pure-function budget tests with a gated stub maker lookup, a
server test with Ford's mock page delayed 2.6 s (REST, status_url,
recall_wait, MCP 6 s), and the page's pending, follow-up and print text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/data-sources.md: the budget, the pending shape, status_url, the
shared background lookup, recall_wait, the website's in-place update and
print wording, and why Ford's lookup is slow (Akamai holds each request
about 9-10 s). llms-full.txt: recall_status and recall_wait for REST and
MCP, the recall-status endpoint, manufacturer_record "pending", caching.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NHTSA's complaintsByVehicle is the next slowest part of a first report:
measured from the web server on 2026-11-18, a 2020 Ford Escape took
17.3 s, a 2017 Honda CR-V 5.0 s and a 2003 Accord 6.4 s (1.4-2.0 MB).
With the 10 s timeout the Escape's list always failed, so its report
was never cached and every visit waited 10 s again.

- modelYearComplaints: one request per make/model/year at a time, shared
  by reports and /api/vin/{vin}/complaints; successes cached 6 h; the
  request may run 30 s (COMPLAINTS_TIMEOUT_MS).
- The report is assembled without waiting for it. If the list has not
  arrived within the caller's budget (2 s website/REST, 6 s MCP; compare
  waits), complaints is {pending: true, available: false, count: null,
  status_url: /api/vin/{vin}/complaints} - unknown, never zero. The
  complete report is built and cached when the list arrives. The VIN
  page (server-rendered) no longer waits for it either.
- Website: "Loading NHTSA's owner complaints..." then fills the card and
  the car sheet row in place; printed while pending it says the list had
  not arrived. MCP: a note to call decode_vin again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without the browser's scroll anchoring (Safari), the shorter status line
moved the page 19 px after Ford's status arrived; inside the redraw it
is part of the measured correction (0 px in the 390 px check).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant