CodeGen: optional render-and-verify eval mode

[BenchCAD]

What

Adds CodeGen's render-and-verify eval mode — the "with Python tools" setting reported for this task in the model system card. In vanilla Vision2Code the model sees the 4-view target once and writes CadQuery single-shot. Here it is additionally allowed to render its own candidate and visually verify it against the target before submitting, which is the setting frontier models are evaluated under.

Off by default — existing single-shot runs are unchanged.

How it works

pipeline/render_verify.py :: run_record_with_tools(record, …, rounds=N):

1. Round 1 — identical to run_record: build the target prompt, call the model, execute its program to a STEP.
2. Rounds 2..N — render the current best candidate in the same 4 views (same cameras + normalization, via scoring.views.composite_for_step), show the model [target, candidate] with a refine prompt asking it to correct view-by-view mismatches, and keep the revision only if it still executes to a valid STEP. A failed/transient API call submits the best-so-far.
3. Score — raw voxel IoU vs the GT STEP, unchanged from single-shot, so numbers stay directly comparable. The results.jsonl row is the same shape plus a rounds field (attempts actually used).

It reuses the exact renderer, executor and IoU scorer from scoring/, so the only new behavior is the refinement loop.

Integration

• main.py reads render_verify.rounds from the config and dispatches: rounds >= 2 → run_record_with_tools, else the original run_record. The run header prints mode: render-verify (N rounds) / single-shot.
• Enable per-config:

  render_verify:
    rounds: 3      # total attempts including the first single-shot

• configs/prod.yaml and CodeGen/README.md document usage.

Testing

Unit (tests/test_render_verify.py, 3 tests, mocked externals) — pass:

• single executing answer → 1 round, scored;
• a non-executing refinement is rejected, best-so-far kept;
• API failure mid-loop → submits the best so far.

Real end-to-end smoke (gemini-2.5-flash, 1 record, rounds: 2):

mode: render-verify (2 rounds)
[gemini-2.5-flash] 1/1 demo1 ... ok  iou=0.636  (47.9s)

results.jsonl row:

{"record_id":"demo1","model":"gemini-2.5-flash","status":"ok","iou":0.6364,
 "rounds":2,"lat_s":47.9,"err":null,
 "code_path":"…/demo1.py","step_path":"…/demo1.step","png_path":"…/demo1.png"}

rounds: 2 and the emitted demo1_cand.png confirm the verify round actually ran (candidate rendered and fed back), not a single-shot fallback.

Notes

• Cost/latency scale with rounds (N rounds ≈ N model calls); the default config stays single-shot.
• The per-round candidate render (<rid>_cand.png) is left under outputs/ for inspection.

[BenchCAD]

Closing as superseded.

This branch targets CodeGen/, which has since been renamed to Vision2Code/ and moved onto benchcad_core, so it no longer applies to main. The idea it introduced, letting the model render its own candidate and compare it against the target before answering, is carried forward by the sandbox setting in #53. There the model renders through the same renderer as the target, measures the difference in code, and decides for itself when to submit.