Repository navigation
E6: JSON v2 with locations, SARIF, and GitHub annotations - #105
Merged
Merged
Conversation
…peration after its own checks.
…in obligation modules.
…xpression begins.
… calls a removed function.
…nd reference flags, in results.
This was referenced Oct 28, 2026
…S runner can meet.
This was referenced Oct 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
--diagnostics-format=jsonwas the only machine-readable output, and it left out what an editor or a CI system needs (findings F5 and R5 of the 0.2 plan):callee.contract, or unsupported result carried no location at all.sourcebegan at the check's caret and ended at the first column of its last token. Forcv::post(cv::result >= 0)it was columns 23 to 26,>= 0, so a viewer underlined the operator and its right operand, not the expression.cpp-verify(E3 gave the compiler path SARIF through Clang's own writer) and no GitHub annotation.This is plan item E6, stacked on #104 (E3).
JSON Lines:
cppverify.diagnostic/2On
structured_output.cppof the new test,--diagnostics-format=jsonnow prints:function_source. Every record of a function, conversion errors included, names where the function is declared.source. A failure's source is the expression its check checks, from its first character to the column after its last, as SARIF and editors count. The anchor (the caret) stays inobligation.id.counterexampleand each trace step'stextexactly as the text output prints them.frontend, with their-Woption and the notes that follow them asrelated. They come before the verifier's records of their file.cppverify.summary/1record ends the run with the counts of the text summary.--jobs;parallel_functions.testdiffs it.--time-reportaddscppverify.time/1records.SARIF 2.1.0
--diagnostics-format=sarifwrites the same records into a buffer and translates them into one log when the run ends (sarifLoginDriver/StructuredOutput.cpp), so the two formats cannot disagree. The mapping follows Frama-C'smarkdown-reportSARIF generator, which turns every property status into a result:kindlevelfailerrorpostcondition,overflow, ...)openerrorunresolved,interruptedopenwarningbounded-safefailwarningwarningpassnoneverifiedreviewnonetrustedfailorinformationalfrontendEach result also carries:
properties;The run's properties hold the summary. Four logs validate against the official SARIF 2.1.0 schema (
jsonschema,sarif-schema-2.1.0.jsonfrom oasis-tcs/sarif-spec) with no errors: the test's source run, its compiler error,inductive_counterexamples.cpp(12 results with loop notes), and an archive replay.GitHub annotations
--diagnostics-format=githubprints the text, and after each error and warning a workflow command:The file is relative to the working directory (the checkout, in a workflow). The data and properties are escaped as the runner requires.
Ranges, and archive schema v5
ObligationSourcekeeps its anchor (Line,Column), from which stable IDs are built. It gainsBeginLine/BeginColumnfrom E3'sBeginLoc, and its end is now the column after the last character (Lexer::getLocForEndOfToken, through the expansion range in a macro).Verifier::sourceOfcomputes a source once for annotation and for the JSON fallbacks.cppverify.obligation/5archives record the begin, so a replayed record has the same range, and still reads/4to/1: their begin is the anchor and their end stays the first column of the last token.Validation rejects a range whose begin lies after its anchor.
Tests
suite/structured_output.test(new) checks:function_source, the failure's range, counterexample, and step texts;obligation_archive.test:run_backend_fidelity.py, which now skips the summary record) move tocppverify.diagnostic/2: the schema string, exclusive ends, and keys that now sort between adjacent ones.On the base,
structured_output.testfails at its first check: there is nosariforgithubvalue, and the compiler's warning is not a record.Each mutant below, applied alone and rebuilt, is caught by
structured_output.test,obligation_archive.test, orlocated_errors.cpp(23 of 23):function_sourcecounterexampletexttextrelatedoptionopenThe whole Verify suite passes on a release build: in the full run 490 tests passed and the fidelity gate failed, because its helper rejected the summary record; with the helper fixed, the gate passes on its own (1 test unsupported).
Stacked on #104.