Skip to content

verify: Bound what a run builds and what its solvers allocate - #48

Open
SwayamInSync wants to merge 36 commits into
mainfrom
fix/memory-budget
Open

SwayamInSync wants to merge 36 commits into
mainfrom
fix/memory-budget

Conversation

@SwayamInSync

@SwayamInSync SwayamInSync commented Oct 20, 2026 •

Copy link
Copy Markdown
Owner

Nothing bounded the memory of a run. Two constructions grow faster than the program, and the process only stopped when the system killed it:

  • Proof obligations. Each obligation's goal is its condition under a negated copy of every assumption before it, so a module grows with its obligations times the assumptions before them. In suite/llvm_uleb128_decoder_exhaustive.cpp, decode_uleb128_length_4 to _11 have modules of 41 to 248 million expression nodes. On main the file takes about 87 GB and 1655 s; --lower-only alone, with one job, passed 57 GB on one function before I stopped it. --max-query-nodes existed but was 0 by default, and it was checked only once the module had been built.
  • Spec definitions unfolded as deep as their fuel. A spec that applies itself twice doubles with every level. On main, cppverify::reveal_with_fuel(fibo, 18) lowers in 267 s and peaks at 22 GB, and fuel 24 passed 40 GB within 65 s.

Z3 and cvc5 had no memory limit either.

This change bounds each of them. A function that reaches a limit is Unresolved, never a proof or a counterexample, and every function whose proof relies on its contract is Unresolved with callee.contract.

--max-query-nodes: what is built for one function (default 32,000,000 expression nodes; 0 disables it)

  • Obligations, counted before they are built. Their node count, as obligationModuleNodeCount counts it, follows from the lowered statements: four times the goals' nodes (for the goals, their queries, the complete goal, and its query), plus three per obligation, less one. buildPassive computes it before cloning anything and returns ObligationSizeError when it is over the limit. In builds with assertions, the prediction is checked against the built module on every module of the suite.
  • Constructions the program does not bound, counted as they are built. NodeBudget (IR/NodeBudget.h) counts the VExpr and LogicExpr nodes built on its thread, against the limit the driver sets for the function. The spec inliner unfolds nothing more once the budget is spent. Three constructions run inside a budget:
    • the facts the passivizer adds at each spec application (postconditions, definitions unfolded as deep as their fuel); a spent budget leaves PassiveProgram::TooLarge, which makes the build an ObligationSizeError;
    • the definitions SpecAxioms materializes, counted level by level into the module's count;
    • the inlining of specs for BMC.
  • One verdict, never a partial result. The driver reports a size error as the function's verdict, Unresolved with query.size-limit, at the function's declaration. The message says what was too large and what to change:
steps.cpp:1:5: Unresolved: steps [backend=z3] [reason=query.size-limit] (the proof obligations would take at least 8505 expression nodes, more than the limit of 2000 (--max-query-nodes): each check repeats every assumption before it, so split the function or raise the limit)
  • Why 32 million. The largest module in the suite is the ULEB128 case study's, at 22.6 million nodes; it peaks at 9.9 GB, about 440 bytes a node. The next largest, safe_fib.cpp, has 11.3 million (4.1 GB), and every other file stays under 1.2 million. The default keeps the case study with headroom and stops the decoder's functions above 32 million at once.
  • Backends and replay. The limit now applies under every backend, since it bounds construction. The backend-side checks remain for modules that are not built from source (archive replay, modules a backend derives), with the message the obligation module has N expression nodes, more than the limit of M (--max-query-nodes).

--memory-limit=MB: the solvers (default 8192; 0 disables it)

  • Z3: its queries together, enforced by cpp-verify. Z3 runs in the process, so its queries share the limit. Every check's watchdog, which already interrupts a check at its time, polls Z3's allocation every 10 ms while a limit is set and interrupts the check once it is past the limit. The encoder's result is then solver.memory-limit. Z3's own limits are not used:
    • memory_max_size fails whichever allocation passes it, in any context of the process. That includes Z3_mk_context's, which then returns no context, and z3++ dereferences it.
    • The high watermark stops the search where Z3 checks it. On the matching loop below it reported only smt-canceled, and the check freed its memory before returning, so a sample taken after the check could not tell why it stopped.
  • The reason test. A check with an explicit memory reason (memout, max. memory, out of memory, as from a failed allocation) is also classified as solver.memory-limit. Following the existing API-error rule, an encoder that ran out reports that and proves nothing.
  • cvc5: each process, as RLIMIT_DATA. The limit is set with prlimit once the process is spawned; on Windows, ExecuteNoWait's job object sets it. Passing the limit to ExecuteNoWait on POSIX makes it fork rather than spawn, and fork runs Z3's fork handler (scoped_timer::finalize), which waits for the timers of every running check: the fidelity gate's portfolio run hung with four threads in it. When an allocation fails, cvc5 prints (error "std::bad_alloc"), or aborts with std::bad_alloc or its SAT solver's Minisat::OutOfMemoryException; each of these is solver.memory-limit. macOS applies no such limit to mapped memory, so there cvc5 is not bounded.
  • Retry. Under --int-encoding=bitvector, a Z3 query stopped for memory is retried in the integer encoding, as after a timeout.
  • Rejected with Lean. --memory-limit joins verify: Reject the solver options with the Lean backend #41's list of solver options, which are errors with Lean export (error: --memory-limit requires a backend that runs an SMT solver). --max-query-nodes leaves that list: it now bounds what is built under every backend, Lean's included. suite/backend_options.cpp checks both, the first as an error and the second as an accepted --lean-out export.

Results

  • The exhaustive decoder at the defaults (its own --jobs=8 --timeout=30000) finishes in 46 s with a peak of 21 GB, against 87 GB and 1655 s on main. Lengths 4 to 11 are Unresolved with query.size-limit; every other function verifies, as on main.
    • On main, given the memory, lengths 4 to 9 and 11 verified and length 10 timed out. With --max-query-nodes=0 they behave as on main. Bringing them under the default is C3's job (hash-consed Layer 3, where the obligations share their prefixes).
    • The 21 GB is several functions' obligations at once, up to 24.5 million nodes each (length 3). The limit is per function, and each job builds its own function.
  • reveal_with_fuel(fibo, 24) ends in 7 s at 3.9 GB, against more than 40 GB on main.
  • The matching loop ends at 0.14 GB with --memory-limit=100, against 2 GB at its 20 s timeout without a limit.

Docs:

  • The tooling page gets a "Memory limits" section with three checked examples, matching the time limits section: obligations over a small limit, a deep fuel unfolding, and Z3's limit on a matching loop. It also gets rows for both options, how to choose them, and the cvc5 platform note.
  • The book: ch15 gets both flags, and ch16 gets the reasons, in its table and in its paragraph on undecided queries.
  • The limitations table gets the size limit.
  • DESIGN and ARCHITECTURE describe the mechanism, including why Z3's own limits and a forked cvc5 are not used.
  • The release notes get an entry under "Running cpp-verify" and the new reason under "Data formats".
  • The docs checker gets one rule: a function a page says stays unresolved may fail to lower for its size (query.size-limit), since no obligations were built.

Testing:

  • suite/obligation_size_limit.cpp checks:
    • the refusal, its message and location, and its JSON record;
    • the caller demoted with callee.contract, and a sibling that still verifies;
    • the --lower-only error;
    • the exact boundary: at 24,417 nodes steps verifies, at 24,416 it is refused;
    • the facts of reveal_with_fuel(fibo, 40) stopping at the limit;
    • obligations plus spec definitions over the limit (26,167 nodes, exact).
  • suite/solver_memory_limit.cpp checks that the matching loop stops at --memory-limit=100 with solver.memory-limit, in text and JSON, and that the next function verifies within the limit. It also checks that --memory-limit with Lean is an error.
  • suite/cvc5_memory_limit.cpp (Linux, cvc5) checks that a cvc5 process limited to 5 MB reports solver.memory-limit.
  • suite/memory_bounded.test (Linux) runs a 400-check function (45 million nodes predicted) and reveal_with_fuel(fibo, 40) under ulimit -v 3000000. Both end with their verdicts. On main both abort with LLVM ERROR: out of memory within 5 s.
  • On main, every RUN line of the four tests fails.
  • Mutants: each of ten mutants fails at least one of the four tests:
    • no prediction check;
    • the prediction off by one;
    • an inliner that ignores the budget;
    • the passivizer's overrun not recorded;
    • the size error reported as a lowering failure;
    • spec definitions not counted;
    • a watchdog that ignores memory;
    • a memory stop not classified;
    • cvc5's out-of-memory not recognized;
    • cvc5's limit not applied.
  • suite/backend_execution.test now expects the new messages; its --dump-ir=4 run, which builds no module, reports the verdict.

Checked:

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant