From d1f975415e472450dc38c2d8d4e19bd04a3692c8 Mon Sep 17 00:00:00 2001 From: Lucian Popescu Date: Thu, 13 Aug 2026 16:49:10 +0100 Subject: [PATCH 01/14] Add initial draft for rules docs --- docs/src/SUMMARY.md | 9 ++ docs/src/codegen/pointers.md | 5 + docs/src/codegen/temporaries.md | 5 + docs/src/rules/conventions.md | 62 ++++++++ docs/src/rules/format.md | 154 ++++++++++++++++++++ docs/src/rules/ir.md | 116 +++++++++++++++ docs/src/rules/loading.md | 93 ++++++++++++ docs/src/rules/math-test.md | 48 +++++++ docs/src/rules/overview.md | 45 +++++- docs/src/rules/preprocessors.md | 88 ++++++++++++ docs/src/rules/rewriting.md | 85 +++++++++++ docs/src/rules/writing-rules.md | 241 ++++++++++++++++++++++++++++++++ 12 files changed, 948 insertions(+), 3 deletions(-) create mode 100644 docs/src/codegen/pointers.md create mode 100644 docs/src/codegen/temporaries.md create mode 100644 docs/src/rules/conventions.md create mode 100644 docs/src/rules/format.md create mode 100644 docs/src/rules/ir.md create mode 100644 docs/src/rules/loading.md create mode 100644 docs/src/rules/math-test.md create mode 100644 docs/src/rules/preprocessors.md create mode 100644 docs/src/rules/rewriting.md create mode 100644 docs/src/rules/writing-rules.md diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md index 101b03d7..d20cd664 100644 --- a/docs/src/SUMMARY.md +++ b/docs/src/SUMMARY.md @@ -10,7 +10,16 @@ # Translation Rules * [Overview](./rules/overview.md) +* [Rule Format](./rules/format.md) +* [Writing Rules](./rules/writing-rules.md) +* [Conventions](./rules/conventions.md) +* [The Rule Preprocessors](./rules/preprocessors.md) +* [The Rules IR](./rules/ir.md) +* [Loading and Matching](./rules/loading.md) +* [Rule Rewriting](./rules/rewriting.md) # Code Generation * [Overview](./codegen/overview.md) +* [Pointers and References](./codegen/pointers.md) +* [Temporary Materialization](./codegen/temporaries.md) diff --git a/docs/src/codegen/pointers.md b/docs/src/codegen/pointers.md new file mode 100644 index 00000000..6020022f --- /dev/null +++ b/docs/src/codegen/pointers.md @@ -0,0 +1,5 @@ +# Pointers and References + +> TODO: explain how the two models translate C++ pointers and references, in +> particular why the refcount model maps both to `Ptr`, and the role of +> `StrongPtr` for reading through a pointer. diff --git a/docs/src/codegen/temporaries.md b/docs/src/codegen/temporaries.md new file mode 100644 index 00000000..a4a86303 --- /dev/null +++ b/docs/src/codegen/temporaries.md @@ -0,0 +1,5 @@ +# Temporary Materialization + +> TODO: explain how the converter materializes temporaries for expressions +> that need an address but have none (e.g. literals passed where a pointer is +> expected). diff --git a/docs/src/rules/conventions.md b/docs/src/rules/conventions.md new file mode 100644 index 00000000..2d1b70f4 --- /dev/null +++ b/docs/src/rules/conventions.md @@ -0,0 +1,62 @@ +# Conventions + +Most of these conventions are enforced by the preprocessors, and violating +them fails the build; the notes below call out the ones that are not +checked. + +## Naming + +| Element | C++ side | Rust side | +|---|---|---| +| Expression rule | `f1`, `f2`, ... | same name | +| Type rule | `t1`, `t2`, ... via `using`/`typedef` | `fn tN() -> RustType` with no arguments | +| Parameters | free-form (`o`, `it`, `key`, `dst`, `n`, ...) | must be `a0`, `a1`, ... consecutive from 0 | +| Generics | `T1`, `T2`, ... (type and non-type params) | `T1`, `T2`, ... consecutive from 1 | +| Variadic pack | `typename... Args` | trailing `va: &[VaArg]` | +| Locals in Rust bodies | | double-underscore prefix: `__v`, `__fd`, `__e`, ... | + +Notes: + +* Rule numbering is per module, and gaps are currently allowed (e.g. + `rules/map` has no `f4`), though this might change in the future. Names + must be unique across `src.c` and `src.cpp` combined. +* On the C++ side parameter names are free, but the *order* defines the + placeholder indices: the first parameter is `a0` on the Rust side, the + second is `a1`, and so on. The receiver of a method rule is always the + first parameter, hence `a0`. +* Generic parameters are matched positionally between the two sides, so `T1` + in the Rust target means "whatever bound to `T1` in the C++ pattern". +* Locals introduced inside Rust rule bodies use a `__` prefix. This is not + checked by the build, but it is needed: rule bodies are spliced inline into + the generated code, so an unprefixed local could collide with a variable + name from the translated program. + +## Function qualifiers + +* In `tgt_unsafe.rs`, expression rules are `unsafe fn`; type rules (`tN`) are + plain `fn`. +* In `tgt_refcount.rs`, all rules are safe `fn`. The refcount model produces + fully safe Rust, so a refcount rule body must not need `unsafe`. + +The build does not check the qualifiers themselves; only rustc's usual rules +apply when the `rules` crate compiles. In particular, nothing stops an +`unsafe` block inside a refcount rule body from being spliced into the +output, so keeping refcount rules safe is what upholds the model's safety +guarantee. + +## C++ pattern shape + +* An `fN` body must be exactly one `return` statement. The preprocessor + rejects anything else. +* `return` statements are not allowed inside Rust rule bodies; write the + result as a tail expression instead. +* Exercise exactly one construct per rule. If an API has several overloads, + write one rule per overload (including separate rules for `const T &` + versus `T &&` parameters). + +## Type checking + +All `tgt_*.rs` files are compiled as part of the `rules` crate, so a rule body +that does not type-check against `libcc2rs`, `libc`, `nix`, etc. breaks the +build. If a rule needs a new crate dependency, add it to `rules/Cargo.toml` +and teach `rule-preprocessor` to locate its rlib. diff --git a/docs/src/rules/format.md b/docs/src/rules/format.md new file mode 100644 index 00000000..8a858a26 --- /dev/null +++ b/docs/src/rules/format.md @@ -0,0 +1,154 @@ +# Rule Format + +A rule module is a directory under `rules/`, usually named after the header or +library it covers (`rules/unistd`, `rules/vector`, `rules/string`, ...). It +contains: + +* `src.cpp` and/or `src.c`: the C++ (or C) side of each rule. +* `tgt_unsafe.rs`: the Rust targets for the unsafe model. +* `tgt_refcount.rs`: the Rust targets for the reference counting model + (optional, see below). + +A rule is a pair of same-named functions on the two sides. Names determine the +rule kind: + +* `f1`, `f2`, ... are *expression rules*: they map a C++ call, member access, + constructor, or constant to a Rust expression. +* `t1`, `t2`, ... are *type rules*: they map a C++ type to a Rust type. + +## Expression rules + +On the C++ side, an `fN` function must have a body that is exactly one +`return` statement. The returned expression is the pattern: the preprocessor +resolves the *callee* of that expression (the function, method, constructor, +enum constant, or macro being used) and that becomes the rule's matching key. +The function parameters stand for the arguments at the call site. + +```cpp +// rules/unistd/src.cpp +int f4(const char *pathname) { return unlink(pathname); } +``` + +On the Rust side, the same-named function gives the replacement expression. +Parameters must be named `a0`, `a1`, ... and correspond positionally to the +C++ parameters: + +```rust +// rules/unistd/tgt_unsafe.rs +unsafe fn f4(a0: *const libc::c_char) -> i32 { + libc::unlink(a0) +} +``` + +```rust +// rules/unistd/tgt_refcount.rs +fn f4(a0: Ptr) -> i32 { + match nix::unistd::unlink(a0.to_rust_string().as_str()) { + Ok(()) => 0, + Err(__e) => { + libcc2rs::cpp2rust_errno().write(__e as i32); + -1 + } + } +} +``` + +When the converter encounters `unlink(x)` in the input, it emits the rule +body with the translated `x` substituted for `a0`. + +## Type rules + +On the C++ side, a `tN` rule is a type alias (`using` or `typedef`). On the +Rust side, it is a *zero-argument function* whose return type is the mapped +Rust type and whose body is the default initializer for that type: + +```cpp +// rules/vector/src.cpp +template using t1 = std::vector; +``` + +```rust +// rules/vector/tgt_unsafe.rs +fn t1() -> Vec { + Default::default() +} +``` + +## Model layering + +The loader always reads `ir_unsafe.json` first. When translating with the +reference counting model, it then overlays `ir_refcount.json` on top: entries +with the same rule name replace the unsafe ones. + +This means `tgt_refcount.rs` only needs to contain the rules that *differ* +from the unsafe model. For example, the `__builtin_mul_overflow` rule in +`rules/builtin` has a pointer out-parameter (`a2` below), so the two models +translate it differently: in the unsafe model `a2` is a raw `*mut i64` +written through a deref, while in the refcount model it is a `Ptr` +written through `Ptr::write`. The other two arguments are identical in both +models: + +```rust +// rules/builtin/tgt_unsafe.rs +unsafe fn f9(a0: i64, a1: i64, a2: *mut i64) -> bool { + let (val, ovf) = a0.overflowing_mul(a1); + *a2 = val; + ovf +} +``` + +```rust +// rules/builtin/tgt_refcount.rs +fn f9(a0: i64, a1: i64, a2: Ptr) -> bool { + let (val, ovf) = a0.overflowing_mul(a1); + a2.write(val); + ovf +} +``` + +The module's other rules (byte swaps, `__builtin_expect`, ...) translate +identically in both models, so they appear only in `tgt_unsafe.rs` and the +refcount model inherits them. A module where no rule needs a refcount-specific +translation can omit `tgt_refcount.rs` entirely. + +## C and C++ sources + +A module may have both `src.c` and `src.cpp`; both are preprocessed and merged +into one `ir_src.json`. Defining the same rule name in both files is a hard +error, so numbering must not collide. + +This split is necessary because rules match on the exact canonical signature +of the callee, and some libc functions have *different signatures in C and +C++*. For example, C has a single +`char *strchr(const char *, int)`, while C++ replaces it with const-correct +overloads such as `const char *strchr(const char *, int)`. Since the +signatures differ, `rules/cstring` defines one rule per language: + +```cpp +// rules/cstring/src.cpp +const char *f6(const char *a0, int a1) { return strchr(a0, a1); } +``` + +```c +// rules/cstring/src.c +char *f5(const char *a0, int a1) { return (strchr)(a0, a1); } +``` + +The C++ rule matches `strchr` calls in code translated as C++, the C rule +matches them in code translated as C. + +`rules/builtin` uses this to cover both languages: `src.cpp` defines +`f9`/`f10` for the C++ `__builtin_mul_overflow` (returning `bool`) while +`src.c` defines `f12`/`f13` for the C version (returning `int`); their Rust +bodies are identical. + +## The `rules` crate + +The whole `rules/` tree is a single Rust crate. `rules/build.rs` walks the +tree, collects every `tgt_*.rs`, and generates `rules/src/modules.rs` with one +`#[path = ...]` module per file. Building the crate therefore type-checks +every rule body against the crates the rule targets call into, which are +declared as dependencies in `rules/Cargo.toml` (`libcc2rs`, `libc`, `nix`, +...). The +Rust rule preprocessor compiles exactly this crate to resolve types in rule +bodies. `rules/src/` is the only subdirectory that is not a rule module. diff --git a/docs/src/rules/ir.md b/docs/src/rules/ir.md new file mode 100644 index 00000000..29a6933b --- /dev/null +++ b/docs/src/rules/ir.md @@ -0,0 +1,116 @@ +# The Rules IR + +Each rule module compiles to up to three JSON files in +`/rules//`: + +* `ir_src.json`: the C++ side, from + [`cpp-rule-preprocessor`](./preprocessors.md#cpp-rule-preprocessor). +* `ir_unsafe.json`: the Rust side for the unsafe model, from + [`rule-preprocessor`](./preprocessors.md#rule-preprocessor). +* `ir_refcount.json`: the Rust side for the refcount model, also from + `rule-preprocessor` (only if the module has a `tgt_refcount.rs`). + +All three are objects keyed by rule name (`f1`, `t1`, ...), and the loader +joins them by name. + +## Source IR (`ir_src.json`) + +A flat map from rule name to the canonical signature of the C++ construct the +rule matches. For `rules/vector`: + +```json +{ + "t1": "std::vector", + "f3": "_Bool std::vector::empty() const" +} +``` + +This signature string is the lookup key for the whole rule: the converter +prints C++ constructs from the input AST with the same printer and compares +the strings. + +## Target IR (`ir_unsafe.json` / `ir_refcount.json`) + +An expression rule serializes as an `ExprRule` object: the rule's signature +plus its body as a list of *fragments*. For +`unsafe fn f6(a0: &mut Vec) -> *mut T1 { a0.as_mut_ptr() }`: + +```json +"f6": { + "body": [ + { "method_call": { + "receiver": [ { "placeholder": { "arg": 0, "access": "read" } } ], + "body": [ { "text": ".as_mut_ptr()" } ] } } + ], + "generics": { "T1": [] }, + "params": { "a0": { "type": "&mut Vec" } }, + "return_type": { "type": "*mut T1", "is_unsafe_pointer": true } +} +``` + +The fragment kinds are: + +* `text`: literal Rust source, emitted verbatim. +* `placeholder`: a use of one of the rule's `aN` parameters in the body (not + an argument of whatever the body calls); the converter substitutes the + translated call-site argument here. Its fields: + * `arg`: the parameter index N. + * `access`: how the body uses the argument: `read`, `write`, or `move`. + * `is_index_base`: the placeholder is the base of an index expression. +* `generic`: a `TN` slot, replaced with the instantiated Rust type. +* `method_call`: a method call split into `receiver` and `body` fragment + lists, so the code generator can rewrite the pair (see + [Rule Rewriting](./rewriting.md)). +* `va_args`: the expansion point for a variadic tail. + +Every type in the IR (in `params`, `return_type`, and type rules) is a +`TypeInfo` object, the type text plus a set of flags: + +* `is_refcount_pointer`: the type is a `Ptr<...>`. +* `is_unsafe_pointer`: the type is a raw `*mut`/`*const` pointer. +* `derives` (type rules only): the standard traits the mapped type + implements (`Copy`, `Clone`, `Default`, ...). + +An `ExprRule` carries two flags of its own: + +* `multi_statement`: the body has more than one statement and must be + wrapped in a block to stay a single expression. +* `is_extern`: the rule is an extern passthrough declaration and has no + body. + +Fields that are false, empty, or unset are omitted from the JSON. + +A type rule serializes as a `TypeRule` object: its `TypeInfo` plus the +`init` initializer expression, merged into one object: + +```json +"t1": { "type": "Vec", "init": "Default::default()" } +``` + +## In-memory form + +`cpp2rust` mirrors this JSON in C++ structs of the same names, defined in +`cpp2rust/converter/translation_rule.h`. `TranslationRule::Load` reads one +module directory (the `ir_*.json` files described above) and produces two +maps keyed by rule name, one holding `ExprRule`s and one holding `TypeRule`s: + +* An `ExprRule` holds the body + [fragments](#target-ir-ir_unsafejson--ir_refcountjson), the parameter and + return `TypeInfo`s, and the two rule-level flags. The name-keyed JSON maps become + positional vectors: parameter `aN` is entry N of `params`, generic `TN` is + entry N-1 of `generics` (each entry being the bound list). Rules support + at most 9 generic parameters (`kMaxGenerics`). +* A `TypeRule` holds the mapped type's `TypeInfo` and the `initializer` + expression. The same struct also represents the built-in type mappings + (scalars, pointers, ...) that the loader registers directly in code, + without any JSON behind them: for example, `int` maps to `i32`, and + `int *` to `*mut i32` in the unsafe model or `Ptr` in the refcount + model. + +Both also carry `src`, the canonical C++ signature attached from +[`ir_src.json`](#source-ir-ir_srcjson); it is the key the rule is matched +by. + +How the loader finds the IR directory, overlays the refcount model on the +unsafe one, and indexes the loaded rules for matching is covered in +[Loading and Matching](./loading.md). diff --git a/docs/src/rules/loading.md b/docs/src/rules/loading.md new file mode 100644 index 00000000..83e0beee --- /dev/null +++ b/docs/src/rules/loading.md @@ -0,0 +1,93 @@ +# Loading and Matching + +## Finding the rules directory + +`cpp2rust` takes the IR directory via `--rules `. If the flag is omitted +it tries `./rules` and then `/../rules`, accepting the first +candidate that contains (recursively) a subdirectory with `ir_src.json` plus +`ir_unsafe.json` or `ir_refcount.json`. Since the build writes the IR to +`/rules` and the binary lands in `/bin`, the default resolution +picks up the generated IR without any flags. + +## Loading + +Rules are loaded once per process by `Mapper::LoadTranslationRules`: + +1. Built-in type mappings are registered first: scalars, `void *`, `size_t`, + and the per-model pointer forms (`*mut T` for unsafe, `Ptr` for + refcount). +2. Every subdirectory of the rules directory is loaded with + `TranslationRule::Load`, which reads `ir_unsafe.json`, overlays + `ir_refcount.json` when translating with the refcount model, and then + attaches the C++ signature from `ir_src.json` to each rule by name. + +Loading is strict: an `ir_src.json` entry with no matching target rule is a +fatal error (this is what catches mismatched `#if`/`#[cfg]` gating), every +generic declared by a rule must appear in its C++ signature, and two type +rules for the same C++ type are rejected. + +## Matching + +Loaded rules are indexed in two multimaps, one for expression rules and one +for type rules. The multimap key is only a coarse bucket for collecting +candidate rules; whether a candidate actually matches is decided by the +unification step described below. The bucket key is derived from the C++ +signature: + +* For expressions, the qualified function name with the return type, the + parameter list, and all template arguments stripped, so the rule for + `_Bool std::vector::empty() const` lands in the `std::vector::empty` + bucket. +* For types, the text up to the first `<`, so all `std::vector<...>` rules + land in the `std::vector` bucket. + +During translation, the converter prints the construct it encounters with the +same canonical printer used by `cpp-rule-preprocessor`, which is what makes +the two sides comparable: + +* Functions and methods print as + ` ()[ const][ &|&&]`. +* Enum constants and global variables print as their qualified name. +* Integer literals expanded from a macro print as the macro name. + +All rules in the matching bucket are then unified against this string with a +template matcher that binds `T1`...`T9` to the concrete types at the use +site. If several rules match, the one with the longest source signature wins, +so more specific rules take precedence. Type lookups first try the +sugar-preserving spelling (so a rule can match `size_t` as written) and retry +with the desugared type on failure. + +## Application + +When a rule matches, the converter walks its body fragments and emits: + +* `text` fragments verbatim, +* `placeholder` fragments as the translated call-site argument. How the + argument is emitted depends on the placeholder's access and on whether the + argument and the declared parameter type are pointers: + * Read access emits the argument as a plain value, with an implicit + numeric cast when the parameter type asks for one. + * Write access emits the argument as an lvalue. + * Move access wraps the argument in `std::mem::take(&mut ...)`; + temporaries are moved as-is. + * If the rule declares a pointer parameter but the argument is not a + pointer, the converter takes a fresh pointer to it, + [materializing a temporary](../codegen/temporaries.md) when the + argument has no address of its own. For example, + the refcount `std::max` rule declares `Ptr` parameters, since the + C++ side takes `const T1 &` and the refcount model + [translates references as `Ptr`](../codegen/pointers.md), so + `std::max(x1, x2)` on plain locals + substitutes `x1.as_pointer()` and `x2.as_pointer()` for the + placeholders, while `std::max(30, 40)` first materializes `__tmp_0` + and `__tmp_1` values for the literals and points into those. + * If the argument is a pointer but the rule expects a value, the + converter dereferences it. +* `generic` fragments as the Rust mapping of the bound C++ type, +* `va_args` fragments as the converted variadic tail, +* `method_call` fragments as receiver followed by body, possibly rewritten + (see [Rule Rewriting](./rewriting.md)). + +Multi-statement bodies are wrapped in `{ }` so they remain a single +expression. Rules for user-defined C++ types are injected through the same +mechanism at translation time. diff --git a/docs/src/rules/math-test.md b/docs/src/rules/math-test.md new file mode 100644 index 00000000..2fbc5079 --- /dev/null +++ b/docs/src/rules/math-test.md @@ -0,0 +1,48 @@ +# Formula Rendering Test + +This page is a rendering test for hand-translating the inference rules from +`latex/translating.tex` (Fig. 2, the $\mathcal{T}$ function) into the book +via the `mdbook-katex` preprocessor. Only a few representative rules are +shown. + +The macros from `latex/macros.tex` are ported once into +`docs/katex-macros.txt` and are available on every page, so the rule bodies +below stay nearly identical to the paper's source. + +## Axioms + +$$ +\begin{array}{cc} +\text{Int} & \text{String} \\[4pt] +\dfrac{}{\typcomp{\cint} \defeq \rint} +& +\dfrac{}{\typcomp{\cstring} \defeq \rstring} +\end{array} +$$ + +## Rule with premises + +$$ +\begin{array}{c}\text{Unique Pointer}\\[4pt] +\dfrac{ + \typcomp{\typ} = \rtyp + \qquad + \boxTy(\rtyp) = \rtyp' +}{\typcomp{\uniqueptr{\typ}} \defeq \roption{\rtyp'}} +\end{array} +$$ + +## Rule with premises on two lines + +$$ +\begin{array}{c}\text{Ptr - Non-Virtual Class}\\[4pt] +\dfrac + {\begin{array}{c} + \typcomp{\typ} = \rtyp + \qquad + \typ \text{ is not a virtual class} + \\ + \converttoptr(\rtyp) = \rtyp' + \end{array}}{\typcomp{\typ*} \defeq \ptr{\rtyp'}} +\end{array} +$$ diff --git a/docs/src/rules/overview.md b/docs/src/rules/overview.md index dbdacce6..ccf9172e 100644 --- a/docs/src/rules/overview.md +++ b/docs/src/rules/overview.md @@ -2,7 +2,46 @@ Translation rules describe how C++ library APIs are mapped to Rust. Each rule module lives in the `rules/` directory and pairs a C++ source file -(`src.cpp`) with its Rust translation for each model (`tgt_refcount.rs` and -`tgt_unsafe.rs`). +(`src.cpp`) with its Rust translation for each model (`tgt_unsafe.rs` and +`tgt_refcount.rs`). -This part of the book explains how rules work and how to write new ones. +The central idea is that every rule is expressed as ordinary, compilable +C++ and Rust source code, free of anything platform dependent. Both sides are +run through real compilers at build time, so a rule that does not compile +fails the build, and the platform-specific spellings a rule needs to match +against (for example `bool` canonicalizing to `_Bool`) are derived by the +compiler on the host rather than written by hand. The same rule sources work +on every platform `cpp2rust` builds on. + +Rules go through a build-time compilation pipeline before `cpp2rust` can use +them: + +1. You author a rule module: C++ patterns in `src.cpp` and Rust targets in + `tgt_unsafe.rs` / `tgt_refcount.rs`. +2. At build time, two preprocessors compile the module into JSON IR under + `/rules//`: + `cpp-rule-preprocessor` compiles the C++ side into `ir_src.json`, and + `rule-preprocessor` compiles the Rust side into `ir_unsafe.json` and + `ir_refcount.json`. +3. At startup, `cpp2rust` loads the IR files and indexes the rules by the + canonical signature of the C++ construct they match. +4. During translation, the converter looks up rules by signature and splices + their Rust bodies into the output, substituting arguments for placeholders + and rewriting the body when needed (e.g. wrapping method calls in + `with_mut` when the receiver is a `Ptr`). + +The rest of this part covers each stage: + +* [Rule Format](./format.md): the files that make up a rule module and how + the two models are layered. +* [Writing Rules](./writing-rules.md): how to write rules for functions, + methods, operators, types, constants, and variadics. +* [Conventions](./conventions.md): naming and style conventions rule authors + must follow. +* [The Rule Preprocessors](./preprocessors.md): the two build-time tools that + compile rules to IR. +* [The Rules IR](./ir.md): the JSON format the preprocessors emit. +* [Loading and Matching](./loading.md): how `cpp2rust` loads the IR and + matches rules against the input AST. +* [Rule Rewriting](./rewriting.md): how rule bodies are adapted at + application time, in particular the `with_mut` rewrite. diff --git a/docs/src/rules/preprocessors.md b/docs/src/rules/preprocessors.md new file mode 100644 index 00000000..ab4c714c --- /dev/null +++ b/docs/src/rules/preprocessors.md @@ -0,0 +1,88 @@ +# The Rule Preprocessors + +Two build-time tools compile rule modules into the [JSON IR](./ir.md) that +`cpp2rust` loads at runtime. Both write into `/rules//`: + +* `cpp-rule-preprocessor` compiles `src.cpp`/`src.c` into `ir_src.json`. +* `rule-preprocessor` compiles `tgt_unsafe.rs`/`tgt_refcount.rs` into + `ir_unsafe.json`/`ir_refcount.json`. + +The C++ side is keyed by resolved callee signatures; the Rust side by rule +names. The two are joined by rule name when `cpp2rust` loads them. + +## cpp-rule-preprocessor + +A clang LibTooling executable (`cpp2rust/cpp_rule_preprocessor.cpp`) that runs +once per rule directory: + +```bash +cpp-rule-preprocessor --dir rules/string --out /rules/string/ir_src.json +``` + +Extra compiler flags can be passed with repeated `--cxxflags` options. CMake +invokes it for every rule module via the `preprocess-cpp-rules` target. + +For each rule it: + +1. Validates that every `fN` body is exactly one `return` statement. +2. Resolves the *callee* of the returned expression. For non-template rules + this is just the called declaration. For template rules the callee is + unresolved, so the tool instantiates the rule's template parameters with + synthesized dummy types and runs overload resolution to find the function + the rule refers to. +3. Prints the resolved declaration as a canonical signature string: + ` ()[ const][ volatile][ &|&&]`. + For `tN` aliases it prints the underlying type. + +The output is a flat JSON object mapping rule names to these signature +strings. + +Two details of the printer matter for matching. Typedefs that resolve to +builtin types are kept as written instead of being desugared: `size_t` prints +as `size_t`, not `unsigned long`, which is what lets it map to `usize` while +plain `unsigned long` maps to `u64`. And integer literals expanded from a +macro are recorded as the macro *name*, which is how +[constant rules](./writing-rules.md#enum-values-constants-and-macros) like the +`O_CREAT` one match by name. + +## rule-preprocessor + +A Rust binary crate built with the nightly toolchain because it links the +compiler's own libraries (`rustc_driver`, `rustc_middle`, ...). It processes +the whole rules tree in one invocation: + +```bash +CARGO_TARGET_DIR= cargo +nightly run --release \ + --manifest-path rule-preprocessor/Cargo.toml -- /rules [rules-dir] +``` + +CMake first builds the `rules` crate (which also regenerates +`rules/src/modules.rs`) and then runs the preprocessor via the +`preprocess-rust-rules` target. It works in two phases. + +**Phase 1, syntactic.** Each `tgt_*.rs` file is parsed with rust-analyzer's +parser, and functions whose `#[cfg]` does not match the host are dropped. +Every function body is then turned into a list of *fragments*, whose kinds +are described in +[The Rules IR](./ir.md#target-ir-ir_unsafejson--ir_refcountjson). The +fragmentation is mainly concerned with how the rule's arguments are used: +references to parameters and generics become placeholder and generic +fragments, while source text that does not involve an argument is kept +as-is. + +Each placeholder is tagged with an *access*: read, write, or move. Some uses +give the access away syntactically (`&mut a0` is a write); those that do not, +typically method-call receivers and arguments, are left as `unknown` for +phase 2. This phase also applies the two +[preprocessor-side rewrites](./rewriting.md#preprocessor-side-rewrites) that +support rule rewriting. + +**Phase 2, semantic.** The preprocessor compiles the `rules` crate in-process +with `rustc` and walks the typed HIR. This gives it the real signature of +every callee, which resolves the `unknown` accesses: passing to a +`&mut`/`*mut` parameter is a write, to a `&`/`*const` parameter a read, and +to `std::mem::take` a move. For type rules it also records which standard +traits (`Copy`, `Clone`, `Default`, ...) the mapped type implements. A +placeholder still `unknown` after this phase fails the build. + +The result is one `ir_.json` per input file, keyed by rule name. diff --git a/docs/src/rules/rewriting.md b/docs/src/rules/rewriting.md new file mode 100644 index 00000000..48bce900 --- /dev/null +++ b/docs/src/rules/rewriting.md @@ -0,0 +1,85 @@ +# Rule Rewriting + +A rule body is written against idiomatic Rust types: a rule that mutates a +vector declares its parameter as `&mut Vec`. But in the refcount model the +call-site argument is usually a `Ptr>`, and a `Ptr` +[cannot produce a long-lived `&mut`](../codegen/pointers.md). Instead of forcing every rule to handle pointers, the code +generator *rewrites* the rule body at application time. + +## The `with_mut` rewrite + +`libcc2rs` provides + +```rust +impl Ptr { + pub fn with_mut(&self, f: impl FnOnce(&mut T) -> R) -> R { ... } +} +``` + +which checks the pointer, borrows the pointee mutably, and runs the closure +on it (with an immutable sibling `Ptr::with`). The refcount converter uses it +to bridge the gap. The rewrite fires when all three hold: + +1. The rule body fragment is a method call whose receiver contains a + placeholder (the preprocessor splits every method call into receiver and + body fragments precisely to enable this). +2. The receiver placeholder's access is write or move, i.e. the method takes + `&mut self` or the rule mutates the parameter. Read access does not need + the rewrite, since a read can go through a + [`StrongPtr`](../codegen/pointers.md) obtained with `Ptr::upgrade`, or + through a `read()` copy. +3. The call-site argument is a pointer or a reference reached through a + pointer. + +The rule's method call `a0.method(...)` is then emitted as + +```rust +ptr.with_mut(|__v: | __v.method(...)) +``` + +For example, the `push_back` rule is written as an ordinary `&mut` method +call: + +```rust +fn f21(a0: &mut Vec, a1: T1) { ... a0.push(...) } +``` + +Given the C++ input `v.push_back(20);` where `v` is reached through a +`Ptr>`, the generated code is: + +```rust +v.with_mut(|__v: &mut Vec| __v.push(20)); +``` + +When the receiver is a plain local value rather than a pointer, condition 3 +fails and no closure is emitted; the same rule produces a direct call like +`(*v2.borrow_mut()).push(0);`. + +When the pointee is itself a boxed value (`Value`, i.e. +`Rc>`), the closure takes `&mut Value` and an extra borrow is +inserted: + +```rust +ptr.with_mut(|__v: &mut Value>| (*__v.borrow_mut()).push(20)) +``` + +## The read-access counterpart + +For read access the converter does not emit a closure. A pointer receiver +whose rule parameter is a value or `&` type is simply dereferenced +(`p.read()` or `(*p.upgrade().deref())`); conversely, if the rule declares a +`Ptr` parameter but the argument is not a pointer, the converter inserts an +`as_pointer()` cast or +[materializes a temporary](../codegen/temporaries.md). + +## Preprocessor-side rewrites + +Two rewrites in `rule-preprocessor` exist to make the `with_mut` rewrite +possible: + +* A `*` deref in front of a `&mut` parameter is dropped from the body, since + the substituted argument is already an lvalue or pointer expression. +* `std::mem::take(&mut aN)` collapses to a bare placeholder marked as a move, + so the converter can re-express the move against the actual argument (for a + pointer that becomes `std::mem::take(&mut )` on the borrowed + pointee). diff --git a/docs/src/rules/writing-rules.md b/docs/src/rules/writing-rules.md new file mode 100644 index 00000000..1d7300a2 --- /dev/null +++ b/docs/src/rules/writing-rules.md @@ -0,0 +1,241 @@ +# Writing Rules + +This page shows how to write rules for each kind of C++ construct. In every +case the recipe is the same: write an `fN` (or `tN`) function on the C++ side +whose single `return` statement exercises the construct, and a same-named +function on the Rust side giving the translation. + +## Free functions + +```cpp +// rules/stat/src.cpp +int f1(const char *pathname, struct stat *statbuf) { + return stat(pathname, statbuf); +} +``` + +```rust +// rules/stat/tgt_refcount.rs +fn f1(a0: Ptr, a1: Ptr) -> i32 { + match nix::sys::stat::stat(a0.to_rust_string().as_str()) { + Ok(__s) => { + a1.with_mut(|__st| *__st = Stat::from_libc(&__s)); + 0 + } + Err(__e) => { + libcc2rs::cpp2rust_errno().write(__e as i32); + -1 + } + } +} +``` + +Rule bodies may be arbitrarily complex; multi-statement bodies are wrapped in +a block when spliced into the output. + +`return` statements are prohibited in Rust rule bodies (the preprocessor +rejects them); produce the result as a tail expression instead. The body is +not emitted as a function of its own: it is spliced inline into the generated +code as a block expression, so a `return` would not end the rule, it would +return from whatever generated function the rule happens to be expanded in. + +## Methods + +There is no special syntax for member functions: write a free function that +takes the receiver as its first parameter and calls the method on it. On the +Rust side the receiver is `a0`. + +```cpp +// rules/vector/src.cpp +template std::size_t f2(const std::vector &o) { + return o.size(); +} +``` + +```rust +// rules/vector/tgt_unsafe.rs +unsafe fn f2(a0: Vec) -> usize { + a0.len() +} +``` + +Template rules use generic parameters named `T1`, `T2`, ... on both sides, +matched positionally. The rule is written against the open template +`std::vector`, with `T1` left as a placeholder, so a single rule covers +every instantiation: when the input program calls `size()` on, say, a +`std::vector`, the matcher binds `T1 = int`. + +## Constructors + +Constructors are functions returning the type by value, one rule per overload: + +```cpp +// rules/string/src.cpp +std::string f7(const char *s, std::size_t n) { return std::string(s, n); } +std::string f9(std::size_t n, char ch) { return std::string(n, ch); } +``` + +Overloads that differ in value category are distinct rules too: `rules/vector` +has separate rules for `push_back(const T1 &)` and `push_back(T1 &&)`. + +No destructor rules exist so far: the STL and libc APIs covered by the +current rules have not needed any, since their types map to Rust types whose +`Drop` implementations already do the right thing. + +## Operators + +Write operators with explicit `operator` call syntax, in member form +(`x.operator@(...)`) or free form (`operator@(a, b)`): + +```cpp +// rules/map/src.cpp +template +T2 &f1(std::map &o, const T1 &key) { return o.operator[](key); } + +template +bool f11(typename std::map::iterator a, + typename std::map::iterator b) { + return operator!=(a, b); +} +``` + +Post-increment is distinguished from pre-increment by the usual dummy `int` +parameter: `a0.operator++(a1)` versus `it.operator++()`. Member accesses +through iterators are also rules (e.g. `it->first`, `it->second`, `o.second` +in `rules/map` and `rules/pair`). + +## Types + +A type rule has two halves. On the C++ side, declare a type alias named `tN` +for the C++ type being mapped. On the Rust side, write a function with the +same name that takes no arguments: its *return type* is the Rust type that +the C++ type maps to, and its *body* is the default value the generated code +uses when it needs to construct one (e.g. for an uninitialized variable). +Reference and pointer variants of a type each get their own rule: + +```cpp +// rules/iostream/src.cpp +using t1 = std::ostream; +using t2 = std::ostream &; +using t3 = std::ostream *; +``` + +C structs use `typedef` instead of `using`: + +```cpp +// rules/stat/src.cpp +typedef struct stat t1; +``` + +```rust +// rules/stat/tgt_unsafe.rs +fn t1() -> ::libc::stat { unsafe { std::mem::zeroed() } } +``` + +```rust +// rules/stat/tgt_refcount.rs +fn t1() -> libcc2rs::Stat { Default::default() } +``` + +A type rule may map to the sentinel type `libcc2rs::IgnoreRule`, meaning +"this model has no special mapping for the type"; the converter then falls +back to its normal type conversion. This is useful when only one model needs +a custom mapping: `rules/carray` maps multi-dimensional C arrays to nested +boxed slices in the refcount model, while its `tgt_unsafe.rs` targets are +`IgnoreRule` so the unsafe model keeps the default array conversion. + +## Enum values, constants, and macros + +Constants are `fN` functions that take no arguments and return the constant, +one rule per value: + +```cpp +// rules/fcntl/src.cpp +int f3(void) { return O_CREAT; } +int f4(void) { return O_TRUNC; } +``` + +```rust +// rules/fcntl/tgt_unsafe.rs +unsafe fn f3() -> i32 { ::libc::O_CREAT } +``` + +For macros that expand to integer literals, the preprocessor records the +*macro name* rather than the value, so `O_CREAT` in the input matches this +rule by name. Enum constants and global variables (e.g. `std::cout`) are +matched by their qualified name. + +## Variadic functions + +The C++ side uses a template parameter pack rather than a C-style `...` +parameter, out of necessity: a function that takes `...` cannot forward its +variadic arguments to another call, so a rule like + +```cpp +int f1(int a0, int a1, ...) { return fcntl(a0, a1, ...); } +``` + +is not +expressible. A parameter pack can be forwarded (`args...`), which is exactly +what the rule body needs to do. The Rust side takes a trailing parameter that +must be typed `&[VaArg]` and named `va`: + +```cpp +// rules/fcntl/src.cpp +template +int f1(int a0, int a1, Args... args) { + return fcntl(a0, a1, args...); +} +``` + +```rust +// rules/fcntl/tgt_refcount.rs +fn f1(a0: i32, a1: i32, va: &[VaArg]) -> i32 { ... } +``` + +## Passthrough rules + +When a call should be forwarded verbatim to the same-named function in Rust's +`libc` crate, the Rust target can be an `extern` declaration instead of a +body: + +```cpp +// rules/fcntl/src.cpp +template +int f1(int a0, int a1, Args... args) { + return fcntl(a0, a1, args...); +} +``` + +```rust +// rules/fcntl/tgt_unsafe.rs +unsafe extern "C" { + fn f1(a0: i32, a1: i32, ...) -> i32; +} +``` + +The converter then emits a direct `libc::fcntl(...)` call at the call site. + +## Platform-specific rules + +Gate the C++ side with the usual preprocessor conditionals and the Rust side +with `#[cfg(...)]`; the two must agree so that the rule name sets line up: + +```cpp +// rules/socket/src.c +#ifdef __linux__ +int f4(void) { return SOCK_CLOEXEC; } +#endif +``` + +```rust +// rules/socket/tgt_unsafe.rs +#[cfg(target_os = "linux")] +unsafe fn f4() -> i32 { + libc::SOCK_CLOEXEC +} +``` + +The Rust preprocessor evaluates `#[cfg]` attributes against the host target +(only `target_os = linux|macos` and `target_arch = x86_64|x86` are accepted) +and drops non-matching rules. From 62cd7401d7a2454556b65f21b7f51c5dcdc3b9b5 Mon Sep 17 00:00:00 2001 From: Lucian Popescu Date: Thu, 13 Aug 2026 17:33:22 +0100 Subject: [PATCH 02/14] Add section about compat layer --- docs/src/SUMMARY.md | 1 + docs/src/rules/compat.md | 146 ++++++++++++++++++++++++++++++++ docs/src/rules/overview.md | 2 + docs/src/rules/writing-rules.md | 5 ++ 4 files changed, 154 insertions(+) create mode 100644 docs/src/rules/compat.md diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md index d20cd664..38511e53 100644 --- a/docs/src/SUMMARY.md +++ b/docs/src/SUMMARY.md @@ -12,6 +12,7 @@ * [Overview](./rules/overview.md) * [Rule Format](./rules/format.md) * [Writing Rules](./rules/writing-rules.md) +* [Compat Shims](./rules/compat.md) * [Conventions](./rules/conventions.md) * [The Rule Preprocessors](./rules/preprocessors.md) * [The Rules IR](./rules/ir.md) diff --git a/docs/src/rules/compat.md b/docs/src/rules/compat.md new file mode 100644 index 00000000..e456b551 --- /dev/null +++ b/docs/src/rules/compat.md @@ -0,0 +1,146 @@ +# Compat Shims + +Rule matching needs a resolvable callee. The preprocessor keys every +expression rule on the function, method, constructor, constant, or global +that the pattern's `return` expression resolves to, and the only macros it +can record are those that +[expand to an integer literal](./writing-rules.md#enum-values-constants-and-macros), +which match by macro name. Any other macro is invisible to the rule system: +by the time clang has built the AST, the macro is gone and only its expansion +remains. + +That is a problem for a small set of libc APIs that are specified as macros +over platform internals: + +* `errno` is an object-like macro; glibc expands it to + `(*__errno_location())`, macOS to `(*__error())`. +* `assert` expands to a conditional that stringifies the condition and calls + a platform-specific failure handler with file and line arguments. +* `FD_SET`, `FD_CLR`, `FD_ISSET`, and `FD_ZERO` expand to bit manipulation + on the `fd_set` representation, through helpers that differ per platform. +* `ntohl`, `ntohs`, `htonl`, and `htons` expand to byte-swap builtins or to + nothing at all, depending on endianness. + +There is no stable, platform-independent callee here to key a rule on. The +*compat headers* in `cpp2rust/compat/` fix this by rewriting each such macro +into a call to a synthetic, well-known function before matching happens. + +## How the shims work + +`cpp2rust/compat` is injected as a system include directory ahead of the +platform headers in every clang invocation the project makes: both when +`cpp2rust` parses the input program and when `cpp-rule-preprocessor` compiles +rule sources. The shared flag list lives in +`cpp2rust/compat/platform_flags.h` (`getPlatformClangBeginFlags`), and the +directory path is baked in at build time via the `COMPAT_INCLUDE_DIR` +definition. + +A shim header sits at the same relative path as the real header it shadows +(`errno.h`, `sys/select.h`, `arpa/inet.h`, ...), so an ordinary +`#include ` finds the shim first. The header then: + +1. pulls in the real platform header with `#include_next` (a GNU extension; + the shared flags pass `-Wno-gnu-include-next` for it), +2. `#undef`s the macro, +3. declares a `cpp2rust_*` shim function, +4. redefines the macro to call the shim. + +`cpp2rust/compat/errno.h` in full: + +```c +#include_next + +#undef errno + +int *cpp2rust_errno(void); + +#define errno (*cpp2rust_errno()) +``` + +The redefinition keeps `errno` an lvalue by dereferencing the returned +pointer, so both reads and assignments like `errno = 0` still parse; what +the matcher sees in either case is a call to `int *cpp2rust_errno()`. + +Because the input program and the rule sources are compiled with the same +shim headers, both sides canonicalize to the same signature, and an ordinary +[expression rule](./writing-rules.md) matches it: + +```c +// rules/errno/src.c +#include + +int *f1(void) { return cpp2rust_errno(); } +``` + +The shim functions are declared but never defined on the C side. They only +exist so that the callee resolves; translation replaces the call with the +rule body, so no C implementation is ever linked. Whatever the shim is +supposed to *do* is supplied by the Rust targets: + +```rust +// rules/errno/tgt_unsafe.rs +unsafe fn f1() -> *mut i32 { + libcc2rs::cpp2rust_errno_unsafe() +} +``` + +```rust +// rules/errno/tgt_refcount.rs +fn f1() -> Ptr { + libcc2rs::cpp2rust_errno() +} +``` + +In the unsafe model `libcc2rs::cpp2rust_errno_unsafe` wraps the real +platform errno location (`__errno_location` on Linux, `__error` on macOS). +The refcount model instead *virtualizes* errno as a thread-local +`Value` inside `libcc2rs`; this is the same cell that other refcount +rules write when they translate a failing libc call into +`libcc2rs::cpp2rust_errno().write(__e as i32)`. + +A rule pattern may spell either the macro or the shim directly; the two are +identical after expansion. `rules/errno` and `rules/assert` call the shim by +name, while `rules/arpa_inet` and `rules/select` write the macro form: + +```c +// rules/select/src.cpp +void f2(int fd, fd_set *set) { return FD_SET(fd, set); } +``` + +## The current shims + +| Header | Macros | Shim functions | Rules | +|---|---|---|---| +| `assert.h` | `assert` | `cpp2rust_assert_fail(bool)` | `rules/assert` maps it to `assert!(a0)` | +| `errno.h` | `errno` | `cpp2rust_errno()` | `rules/errno`, see above | +| `arpa/inet.h` | `ntohl`, `ntohs`, `htonl`, `htons` | `cpp2rust_ntohl(x)`, ... | `rules/arpa_inet` maps them to `u32::from_be`, `u16::to_be`, ... | +| `sys/select.h` | `FD_SET`, `FD_CLR`, `FD_ISSET`, `FD_ZERO` | `cpp2rust_fd_set(fd, set)`, ... | `rules/select` maps them to `libc::FD_SET(...)` (unsafe) or `CFdSet` methods (refcount) | + +Note how the shim also normalizes the *shape* of the API. C's `assert` is a +macro precisely so it can stringify its condition and capture file and line; +the shim reduces it to a plain `void(bool)` function, and the Rust side +regains the diagnostics by mapping it to the `assert!` macro. + +## Adding a new shim + +To make another macro-based API matchable: + +1. Create the header in `cpp2rust/compat/` at the same relative path as the + platform header that defines the macro. +2. Follow the pattern above: `#include_next` the real header, `#undef` the + macro, declare a `cpp2rust_` function with the macro's effective + signature, and redefine the macro to call it. +3. Write rules for the shim in a `rules/` module as for any other function, + including the corresponding header in `src.c`/`src.cpp`. +4. If a model needs runtime support (as refcount errno does), implement it + in `libcc2rs` and call it from the rule target. + +Keep the shim's signature platform-independent; the whole point is that +both sides of every rule see one canonical declaration on every platform. + +## Related normalization + +The same shared flag list also passes `-D_FORTIFY_SOURCE=0`, which keeps +glibc from substituting fortified variants (`__printf_chk` and friends) for +standard calls. Like the shims, this ensures that calls in the input program +resolve to the standard declarations the rules are written against. diff --git a/docs/src/rules/overview.md b/docs/src/rules/overview.md index ccf9172e..d36aeebf 100644 --- a/docs/src/rules/overview.md +++ b/docs/src/rules/overview.md @@ -36,6 +36,8 @@ The rest of this part covers each stage: the two models are layered. * [Writing Rules](./writing-rules.md): how to write rules for functions, methods, operators, types, constants, and variadics. +* [Compat Shims](./compat.md): how macro-based libc APIs like `errno` and + `FD_SET` are rewritten into matchable function calls. * [Conventions](./conventions.md): naming and style conventions rule authors must follow. * [The Rule Preprocessors](./preprocessors.md): the two build-time tools that diff --git a/docs/src/rules/writing-rules.md b/docs/src/rules/writing-rules.md index 1d7300a2..dbc94078 100644 --- a/docs/src/rules/writing-rules.md +++ b/docs/src/rules/writing-rules.md @@ -165,6 +165,11 @@ For macros that expand to integer literals, the preprocessor records the rule by name. Enum constants and global variables (e.g. `std::cout`) are matched by their qualified name. +Integer-literal macros are the only macros matchable directly. Macros whose +expansions are platform internals with no stable callee, such as `errno` or +`FD_SET`, are first rewritten into calls to synthetic `cpp2rust_*` functions +by the [compat shims](./compat.md); rules then match the shim call. + ## Variadic functions The C++ side uses a template parameter pack rather than a C-style `...` From 5147aa95fce8522ac958d67ce1ae61238aba871b Mon Sep 17 00:00:00 2001 From: Lucian Popescu Date: Thu, 13 Aug 2026 17:57:30 +0100 Subject: [PATCH 03/14] Add more details about rule preprocessors --- docs/src/rules/preprocessors.md | 75 +++++++++++++++++++++++++-------- 1 file changed, 58 insertions(+), 17 deletions(-) diff --git a/docs/src/rules/preprocessors.md b/docs/src/rules/preprocessors.md index ab4c714c..ccd7ffc0 100644 --- a/docs/src/rules/preprocessors.md +++ b/docs/src/rules/preprocessors.md @@ -19,8 +19,17 @@ once per rule directory: cpp-rule-preprocessor --dir rules/string --out /rules/string/ir_src.json ``` -Extra compiler flags can be passed with repeated `--cxxflags` options. CMake -invokes it for every rule module via the `preprocess-cpp-rules` target. +Extra compiler flags can be passed with repeated `--cxxflags` options, +though CMake, which invokes the tool for every rule module via the +`preprocess-cpp-rules` target, passes none. Note that the parent directory +of `--out` must already exist; CMake creates it before each invocation, so +a manual run must do the same. + +Rule sources are always compiled with the fixed flag set from +`cpp2rust/compat/platform_flags.h`, the same one used to parse input +programs (see [Compat Shims](./compat.md)). There is no compilation +database and no `-std=` flag: the language is chosen by clang from the +file extension, and `src.c` is processed before `src.cpp`. For each rule it: @@ -31,17 +40,20 @@ For each rule it: synthesized dummy types and runs overload resolution to find the function the rule refers to. 3. Prints the resolved declaration as a canonical signature string: - ` ()[ const][ volatile][ &|&&]`. - For `tN` aliases it prints the underlying type. + ` ([, ...])[ const][ volatile][ &|&&]`, + where `, ...` appears for C-variadic functions and the trailing + qualifiers only for methods. For `tN` aliases it prints the underlying + type. The output is a flat JSON object mapping rule names to these signature strings. -Two details of the printer matter for matching. Typedefs that resolve to -builtin types are kept as written instead of being desugared: `size_t` prints -as `size_t`, not `unsigned long`, which is what lets it map to `usize` while -plain `unsigned long` maps to `u64`. And integer literals expanded from a -macro are recorded as the macro *name*, which is how +The printer preserves typedef sugar instead of desugaring it: `size_t` +prints as `size_t`, not `unsigned long`, which is what lets it map to +`usize` while plain `unsigned long` maps to `u64` (for `tN` aliases this +preservation is explicit; inside function signatures the spelling survives +through the printing policy). Integer literals expanded from a macro are +recorded as the macro *name*, which is how [constant rules](./writing-rules.md#enum-values-constants-and-macros) like the `O_CREAT` one match by name. @@ -56,9 +68,30 @@ CARGO_TARGET_DIR= cargo +nightly run --release \ --manifest-path rule-preprocessor/Cargo.toml -- /rules [rules-dir] ``` -CMake first builds the `rules` crate (which also regenerates -`rules/src/modules.rs`) and then runs the preprocessor via the -`preprocess-rust-rules` target. It works in two phases. +The environment is load-bearing: + +* `CARGO_TARGET_DIR` must be set (the tool aborts otherwise): the rlibs of + the rule dependencies (`libcc2rs`, `libc`, `nix`, ...) are looked up in + `$CARGO_TARGET_DIR//deps`, which the `cargo run` above + populates. The crate list is hardcoded, so a new dependency in + `rules/Cargo.toml` also needs an entry in + `rule-preprocessor/src/semantic.rs`. + + > [!WARNING] + > Stale rlibs from an earlier build can be picked up silently. Run `ninja + > clean` to fix this. +* The sysroot comes from running `rustc --print=sysroot`, so the `rustc` on + `PATH` must be the same nightly the preprocessor was built with (running + through `cargo +nightly run` guarantees this). +* `rules-dir` is optional and defaults to the relative path `../rules`, + resolved against the *current working directory* of the process. + +CMake drives all of this via the `preprocess-rust-rules` target: it first +builds the `rules` crate with the stable toolchain (which also regenerates +`rules/src/modules.rs`), then runs the preprocessor with +`CARGO_TARGET_DIR=/target_preprocessor`. That initial `cargo build` +of the `rules` crate is what actually gates the build on rule bodies +type-checking (see below). The preprocessor works in two phases. **Phase 1, syntactic.** Each `tgt_*.rs` file is parsed with rust-analyzer's parser, and functions whose `#[cfg]` does not match the host are dropped. @@ -81,8 +114,16 @@ support rule rewriting. with `rustc` and walks the typed HIR. This gives it the real signature of every callee, which resolves the `unknown` accesses: passing to a `&mut`/`*mut` parameter is a write, to a `&`/`*const` parameter a read, and -to `std::mem::take` a move. For type rules it also records which standard -traits (`Copy`, `Clone`, `Default`, ...) the mapped type implements. A -placeholder still `unknown` after this phase fails the build. - -The result is one `ir_.json` per input file, keyed by rule name. +to `std::mem::take` a move. For type rules it also records which of the +nine derivable standard traits (`Copy`, `Clone`, `Debug`, `Default`, +`PartialEq`, `Eq`, `PartialOrd`, `Ord`, `Hash`) the mapped type implements. +A placeholder still `unknown` after this phase fails the build. + +The preprocessor assumes the `rules` crate is buildable, which the earlier +`cargo build` of the crate ensures; errors from the in-process compilation +are therefore only reported as a warning. + +The result is one `ir_.json` per input file, keyed by rule name. The +output file name is derived from the input file name (`tgt_unsafe.rs` → +`ir_unsafe.json`) and the module directory is the direct parent of the +`tgt_*.rs` file. From 9ecced651819291ca6f2dcec213b3740503b1d1d Mon Sep 17 00:00:00 2001 From: Lucian Popescu Date: Thu, 13 Aug 2026 18:00:05 +0100 Subject: [PATCH 04/14] Add more info about rule IR --- docs/src/rules/ir.md | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/docs/src/rules/ir.md b/docs/src/rules/ir.md index 29a6933b..17e88c56 100644 --- a/docs/src/rules/ir.md +++ b/docs/src/rules/ir.md @@ -57,7 +57,8 @@ The fragment kinds are: * `arg`: the parameter index N. * `access`: how the body uses the argument: `read`, `write`, or `move`. * `is_index_base`: the placeholder is the base of an index expression. -* `generic`: a `TN` slot, replaced with the instantiated Rust type. +* `generic`: a `TN` slot, replaced with the instantiated Rust type; + serialized as the 1-based index N. * `method_call`: a method call split into `receiver` and `body` fragment lists, so the code generator can rewrite the pair (see [Rule Rewriting](./rewriting.md)). @@ -71,14 +72,19 @@ Every type in the IR (in `params`, `return_type`, and type rules) is a * `derives` (type rules only): the standard traits the mapped type implements (`Copy`, `Clone`, `Default`, ...). +The two pointer flags are mutually exclusive; the loader rejects a type +with both set. + An `ExprRule` carries two flags of its own: -* `multi_statement`: the body has more than one statement and must be - wrapped in a block to stay a single expression. +* `multi_statement`: the body has more than one statement, or a statement + followed by a tail expression, and must be wrapped in a block to stay a + single expression. * `is_extern`: the rule is an extern passthrough declaration and has no body. -Fields that are false, empty, or unset are omitted from the JSON. +Fields that are false, empty, or unset are omitted from the JSON. A `va` +parameter is never listed in `params`, and a `()` return type is omitted. A type rule serializes as a `TypeRule` object: its `TypeInfo` plus the `init` initializer expression, merged into one object: @@ -87,6 +93,9 @@ A type rule serializes as a `TypeRule` object: its `TypeInfo` plus the "t1": { "type": "Vec", "init": "Default::default()" } ``` +There is no explicit tag distinguishing the two rule kinds: an entry with +`body` is an expression rule, one with `type` and `init` a type rule. + ## In-memory form `cpp2rust` mirrors this JSON in C++ structs of the same names, defined in From 4475b4abc68656aea0f76526d7f3605d817de701 Mon Sep 17 00:00:00 2001 From: Lucian Popescu Date: Thu, 13 Aug 2026 18:03:52 +0100 Subject: [PATCH 05/14] Add more conventions --- docs/src/rules/conventions.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/docs/src/rules/conventions.md b/docs/src/rules/conventions.md index 2d1b70f4..38cb25d3 100644 --- a/docs/src/rules/conventions.md +++ b/docs/src/rules/conventions.md @@ -54,9 +54,18 @@ guarantee. write one rule per overload (including separate rules for `const T &` versus `T &&` parameters). +## Argument accesses + +Every use of an `aN` parameter in a rule body is classified as a read, +write, or move by the +[rule preprocessor](./preprocessors.md#rule-preprocessor). Passing an +argument by value counts as a read, not a move; the only way to record a +move is `std::mem::take(&mut aN)`. + ## Type checking All `tgt_*.rs` files are compiled as part of the `rules` crate, so a rule body that does not type-check against `libcc2rs`, `libc`, `nix`, etc. breaks the build. If a rule needs a new crate dependency, add it to `rules/Cargo.toml` -and teach `rule-preprocessor` to locate its rlib. +and to the hardcoded crate list in `rule-preprocessor/src/semantic.rs` +(see [The Rule Preprocessors](./preprocessors.md#rule-preprocessor)). From 10be55ed8092cefe0b5b370ca8a688c4e8e38b9d Mon Sep 17 00:00:00 2001 From: Lucian Popescu Date: Thu, 13 Aug 2026 18:12:26 +0100 Subject: [PATCH 06/14] Add more info in rule rewriting --- docs/src/codegen/overview.md | 4 ++++ docs/src/rules/rewriting.md | 36 ++++++++++++++++++++++++++---------- 2 files changed, 30 insertions(+), 10 deletions(-) diff --git a/docs/src/codegen/overview.md b/docs/src/codegen/overview.md index fb3916d3..ca97a4f7 100644 --- a/docs/src/codegen/overview.md +++ b/docs/src/codegen/overview.md @@ -2,3 +2,7 @@ This part of the book documents the internals of the code generator: how the clang AST is traversed and how Rust code is emitted. + +TODO: document the converter plugin mechanism +(`cpp2rust/converter/plugins/`), which intercepts constructs ahead of the +translation rules (currently `emplace_back`). diff --git a/docs/src/rules/rewriting.md b/docs/src/rules/rewriting.md index 48bce900..33bba679 100644 --- a/docs/src/rules/rewriting.md +++ b/docs/src/rules/rewriting.md @@ -18,18 +18,20 @@ impl Ptr { which checks the pointer, borrows the pointee mutably, and runs the closure on it (with an immutable sibling `Ptr::with`). The refcount converter uses it -to bridge the gap. The rewrite fires when all three hold: +to bridge the gap; the unsafe converter never rewrites and simply emits +receiver followed by body. The rewrite fires when all three hold: 1. The rule body fragment is a method call whose receiver contains a placeholder (the preprocessor splits every method call into receiver and - body fragments precisely to enable this). + body fragments precisely to enable this). If the receiver contains + several placeholders, the first one is used. 2. The receiver placeholder's access is write or move, i.e. the method takes `&mut self` or the rule mutates the parameter. Read access does not need the rewrite, since a read can go through a [`StrongPtr`](../codegen/pointers.md) obtained with `Ptr::upgrade`, or through a `read()` copy. -3. The call-site argument is a pointer or a reference reached through a - pointer. +3. The call-site argument is a pointer, or an expression of reference type + (which includes an operator call returning a reference). The rule's method call `a0.method(...)` is then emitted as @@ -55,14 +57,25 @@ When the receiver is a plain local value rather than a pointer, condition 3 fails and no closure is emitted; the same rule produces a direct call like `(*v2.borrow_mut()).push(0);`. +The rewrite applies to pointer dereferences (`p->push_back(20)`) and to +reference usages (`r.push_back(20)` with `std::vector &r = *p`); both +are [translated as a `Ptr`](../codegen/pointers.md), and that `Ptr` is what +`with_mut` is called on. + When the pointee is itself a boxed value (`Value`, i.e. `Rc>`), the closure takes `&mut Value` and an extra borrow is -inserted: +inserted. This is the case for nested containers: the refcount model +translates `std::vector>` as `Vec>>` so +that each element has interior mutability of its own, and a `Ptr` to an +inner vector therefore points at a `Value>`, not a `Vec`: ```rust ptr.with_mut(|__v: &mut Value>| (*__v.borrow_mut()).push(20)) ``` +The closure type is built from the C++ argument's type, not the rule's +declared parameter type. + ## The read-access counterpart For read access the converter does not emit a closure. A pointer receiver @@ -75,11 +88,14 @@ whose rule parameter is a value or `&` type is simply dereferenced ## Preprocessor-side rewrites Two rewrites in `rule-preprocessor` exist to make the `with_mut` rewrite -possible: +possible. Both apply only to `&mut` parameters: -* A `*` deref in front of a `&mut` parameter is dropped from the body, since +* A `*` deref in front of the parameter is dropped from the body, since the substituted argument is already an lvalue or pointer expression. -* `std::mem::take(&mut aN)` collapses to a bare placeholder marked as a move, - so the converter can re-express the move against the actual argument (for a +* `std::mem::take(&mut aN)` collapses to a bare placeholder, so the + converter can re-express the move against the actual argument (for a pointer that becomes `std::mem::take(&mut )` on the borrowed - pointee). + pointee). The spelling must be exactly this fully qualified form: + `mem::take` or an imported `take` is not rewritten. The collapsed + placeholder's access is left `unknown` in phase 1; phase 2 resolves the + `std::mem::take` call to a move. From 940915c6c9c53dd291042756d14c1bfb8f32c5f3 Mon Sep 17 00:00:00 2001 From: Lucian Popescu Date: Thu, 13 Aug 2026 18:23:42 +0100 Subject: [PATCH 07/14] Add more info in writing rules --- docs/src/rules/writing-rules.md | 106 ++++++++++++++++++++++++++++++-- 1 file changed, 102 insertions(+), 4 deletions(-) diff --git a/docs/src/rules/writing-rules.md b/docs/src/rules/writing-rules.md index dbc94078..172f25af 100644 --- a/docs/src/rules/writing-rules.md +++ b/docs/src/rules/writing-rules.md @@ -39,6 +39,10 @@ not emitted as a function of its own: it is spliced inline into the generated code as a block expression, so a `return` would not end the rule, it would return from whatever generated function the rule happens to be expanded in. +When the pattern's type cannot be named, the rule uses an `auto` return +type: `rules/iomanip` writes `auto f1(int n) { return std::setw(n); }` +because `std::setw` returns an unspecified type. + ## Methods There is no special syntax for member functions: write a free function that @@ -65,6 +69,28 @@ matched positionally. The rule is written against the open template every instantiation: when the input program calls `size()` on, say, a `std::vector`, the matcher binds `T1 = int`. +## Static member functions + +A static member function is also written with a receiver parameter, which +exists only to name the class. The call site has no receiver argument, so +the Rust side drops it and numbers the remaining parameters from `a0`; here +there are none: + +```cpp +// rules/limits/src.cpp +template T1 f1(std::numeric_limits &a0) { return a0.max(); } +``` + +```rust +// rules/limits/tgt_unsafe.rs +unsafe fn f1() -> T1 { + ::MAX +} +``` + +(`HasMinMax` is a helper trait defined alongside the rules in the same +file.) + ## Constructors Constructors are functions returning the type by value, one rule per overload: @@ -100,9 +126,55 @@ bool f11(typename std::map::iterator a, ``` Post-increment is distinguished from pre-increment by the usual dummy `int` -parameter: `a0.operator++(a1)` versus `it.operator++()`. Member accesses -through iterators are also rules (e.g. `it->first`, `it->second`, `o.second` -in `rules/map` and `rules/pair`). +parameter: `a0.operator++(a1)` versus `it.operator++()`. Conversion +operators use the same explicit syntax: `a0.operator T1 &()` in +`rules/functional` matches the conversion of a `std::reference_wrapper` +back to a reference. Field accesses are rules of their own, matched by the +field: `it->first` and `it->second` through iterators, plain `o.second` on +a pair (`rules/map`, `rules/pair`). + +## Callable arguments + +A rule parameter may be a callable. Function pointers are spelled directly; +for a lambda, whose type cannot be written, the rule declares a file-scope +lambda and takes `decltype(lambda)`: + +```cpp +// rules/algorithm/src.cpp +auto lambda = [](const T2 &a, const T2 &b) { return false; }; +void f6(T1 first, T1 last, decltype(lambda) comp) { + return std::stable_sort(first, last, comp); +} +``` + +```rust +// rules/algorithm/tgt_unsafe.rs +unsafe fn f6(a0: *mut T1, a1: *mut T1, a2: &mut T2) +where + T2: FnMut(&T1, &T1) -> bool, +{ ... } +``` + +`T1` and `T2` are not template parameters here but file-scope helper +structs modelling an iterator and its value type; being named like +generics, they bind as `T1`/`T2` at the use site. The function pointer +version of the comparator is a separate rule (`f7`). + +## Iterators + +There is no iterator abstraction: an iterator type gets a type rule, and +every operation on it its own expression rule (`operator*`, `operator++`, +`operator!=`, ...). What the type maps to is up to the rule: +`std::string::iterator` becomes a plain pointer (`*mut libc::c_char` +unsafe, `Ptr` refcount), while `std::map` iterators become the runtime +types `libcc2rs::UnsafeMapIterator`/`MapIterator`. Dependent iterator +types are named with `typename`: + +```cpp +// rules/map/src.cpp +template +using t2 = typename std::map::const_iterator; +``` ## Types @@ -163,7 +235,9 @@ unsafe fn f3() -> i32 { ::libc::O_CREAT } For macros that expand to integer literals, the preprocessor records the *macro name* rather than the value, so `O_CREAT` in the input matches this rule by name. Enum constants and global variables (e.g. `std::cout`) are -matched by their qualified name. +matched by their qualified name. A global and its address are separate +rules: `rules/iostream` maps both `std::cout` (`f1`) and `&std::cout` +(`f3`). Integer-literal macros are the only macros matchable directly. Macros whose expansions are platform internals with no stable callee, such as `errno` or @@ -198,6 +272,9 @@ int f1(int a0, int a1, Args... args) { fn f1(a0: i32, a1: i32, va: &[VaArg]) -> i32 { ... } ``` +Bodies read the arguments through the va-args API in `libcc2rs` +(`VaArg`, `VaList`, the `VaArgGet` accessors, `format_c`). + ## Passthrough rules When a call should be forwarded verbatim to the same-named function in Rust's @@ -244,3 +321,24 @@ unsafe fn f4() -> i32 { The Rust preprocessor evaluates `#[cfg]` attributes against the host target (only `target_os = linux|macos` and `target_arch = x86_64|x86` are accepted) and drops non-matching rules. + +Mutually exclusive platform branches use `#elif` with disjoint rule +numbers: `rules/errno` defines `f91`–`f135` under `__linux__` and +`f136`–`f153` under `__APPLE__`. Feature-test macros a pattern needs must +come before the includes, as with `#define _GNU_SOURCE` in +`rules/socket/src.c`. + +## Pattern resolution limits + +The preprocessor resolves a template pattern by instantiating its template +parameters with synthesized types +([The Rule Preprocessors](./preprocessors.md#cpp-rule-preprocessor)): + +* A bare `T1` becomes an empty struct, so the pattern cannot use members, + operators, or nested types of `T1`. +* A parameter pack instantiates to the empty pack. +* A non-type parameter is pinned to the value `1`. + +Unqualified callees are looked up in `namespace std` first and in the +global scope only when `std` has no match, so an unqualified name that +exists in both resolves to the `std` one. From edc20386f1d7a97d70f3fc39adc0b9743b7e972d Mon Sep 17 00:00:00 2001 From: Lucian Popescu Date: Thu, 13 Aug 2026 18:34:00 +0100 Subject: [PATCH 08/14] Add The Matching Engine section --- docs/src/SUMMARY.md | 1 + docs/src/rules/loading.md | 38 +++++++++++++++++++------------ docs/src/rules/matching.md | 40 +++++++++++++++++++++++++++++++++ docs/src/rules/overview.md | 2 ++ docs/src/rules/preprocessors.md | 2 +- docs/src/rules/writing-rules.md | 4 ++-- 6 files changed, 70 insertions(+), 17 deletions(-) create mode 100644 docs/src/rules/matching.md diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md index 38511e53..65814ddf 100644 --- a/docs/src/SUMMARY.md +++ b/docs/src/SUMMARY.md @@ -17,6 +17,7 @@ * [The Rule Preprocessors](./rules/preprocessors.md) * [The Rules IR](./rules/ir.md) * [Loading and Matching](./rules/loading.md) +* [The Matching Engine](./rules/matching.md) * [Rule Rewriting](./rules/rewriting.md) # Code Generation diff --git a/docs/src/rules/loading.md b/docs/src/rules/loading.md index 83e0beee..d1556145 100644 --- a/docs/src/rules/loading.md +++ b/docs/src/rules/loading.md @@ -13,9 +13,14 @@ picks up the generated IR without any flags. Rules are loaded once per process by `Mapper::LoadTranslationRules`: -1. Built-in type mappings are registered first: scalars, `void *`, `size_t`, - and the per-model pointer forms (`*mut T` for unsafe, `Ptr` for - refcount). +1. Built-in type mappings are registered first. Every scalar is mapped + with its width taken from the host (`int` maps to `i32`, + `unsigned long` to `u64`), together with its `const` form and its + pointer forms: `*mut`/`*const` in the unsafe model, `Ptr` in the + refcount model, where constness is dropped. `char` maps to + `libc::c_char` in the unsafe model and to `u8` in refcount; + `size_t`/`ssize_t` map to `usize`/`isize`; `void *` maps to + `*mut ::libc::c_void`, or in refcount to `AnyPtr`. 2. Every subdirectory of the rules directory is loaded with `TranslationRule::Load`, which reads `ir_unsafe.json`, overlays `ir_refcount.json` when translating with the refcount model, and then @@ -30,9 +35,9 @@ rules for the same C++ type are rejected. Loaded rules are indexed in two multimaps, one for expression rules and one for type rules. The multimap key is only a coarse bucket for collecting -candidate rules; whether a candidate actually matches is decided by the -unification step described below. The bucket key is derived from the C++ -signature: +candidate rules; whether a candidate actually matches is decided by +[the matching engine](./matching.md). The bucket key is derived from the +C++ signature: * For expressions, the qualified function name with the return type, the parameter list, and all template arguments stripped, so the rule for @@ -50,12 +55,16 @@ the two sides comparable: * Enum constants and global variables print as their qualified name. * Integer literals expanded from a macro print as the macro name. -All rules in the matching bucket are then unified against this string with a -template matcher that binds `T1`...`T9` to the concrete types at the use -site. If several rules match, the one with the longest source signature wins, -so more specific rules take precedence. Type lookups first try the -sugar-preserving spelling (so a rule can match `size_t` as written) and retry -with the desugared type on failure. +All rules in the matching bucket are then unified against this string by +[the matching engine](./matching.md), which binds `T1`...`T9` to the +concrete types at the use site and picks the most specific rule when +several match. Type lookups first try the sugar-preserving spelling (so a +rule can match `size_t` as written) and retry with the desugared type on +failure. + +Running `cpp2rust` with `--verbose` logs every lookup and the rule it +matched, which is the quickest way to see why a rule does or does not +fire. ## Application @@ -81,8 +90,9 @@ When a rule matches, the converter walks its body fragments and emits: substitutes `x1.as_pointer()` and `x2.as_pointer()` for the placeholders, while `std::max(30, 40)` first materializes `__tmp_0` and `__tmp_1` values for the literals and points into those. - * If the argument is a pointer but the rule expects a value, the - converter dereferences it. + * If the receiver argument is a pointer but the rule expects a value, + the converter dereferences it; this only happens for receivers, not + for ordinary arguments. * `generic` fragments as the Rust mapping of the bound C++ type, * `va_args` fragments as the converted variadic tail, * `method_call` fragments as receiver followed by body, possibly rewritten diff --git a/docs/src/rules/matching.md b/docs/src/rules/matching.md new file mode 100644 index 00000000..d31c522e --- /dev/null +++ b/docs/src/rules/matching.md @@ -0,0 +1,40 @@ +# The Matching Engine + +[Loading and Matching](./loading.md) collects the candidate rules for a +construct from a bucket; each candidate's source signature is then +*unified* against the printed construct. The signature is treated as a +template whose `T1`...`T9` slots capture concrete types: +`std::vector::vector()` unifies with `std::vector::vector()` by +binding `T1 = int`. + +Unification works on the two strings: + +* Whitespace differences are ignored. +* A `TN` slot captures up to the next literal text of the pattern, found + at the same `<>`/`()`/`[]` nesting depth. This is how `T1` captures all + of `std::map` in `std::vector>` without + stopping at the inner comma. +* A `TN` that appears again must match its first capture exactly. +* The whole printed string must be consumed; trailing text fails the + match. +* Slots may stay unbound (a pattern can use `T2` without `T1`). + +A rule matches if unification succeeds. When several rules in the bucket +match, the one with the longest source signature wins, so more specific +rules take precedence; between equally long signatures the choice is +unspecified. + +## Bucket keys + +The bucket keys described in +[Loading and Matching](./loading.md#matching) have two special cases: +array types bucket by the text after the first `[` rather than the text +before a `<`, and `operator()` rules are cut at the operator's own +parentheses, so their key ends at `...::operator`. + +## Instantiating the target + +Captures are C++ spellings. Before being substituted into the rule's Rust +fragments, each capture is itself mapped through the type rules, +recursively, so `T1 = std::vector` substitutes as `Vec`. A +captured type with no type rule of its own is an error. diff --git a/docs/src/rules/overview.md b/docs/src/rules/overview.md index d36aeebf..0f74ed68 100644 --- a/docs/src/rules/overview.md +++ b/docs/src/rules/overview.md @@ -45,5 +45,7 @@ The rest of this part covers each stage: * [The Rules IR](./ir.md): the JSON format the preprocessors emit. * [Loading and Matching](./loading.md): how `cpp2rust` loads the IR and matches rules against the input AST. +* [The Matching Engine](./matching.md): how a candidate rule's signature + is unified against the input. * [Rule Rewriting](./rewriting.md): how rule bodies are adapted at application time, in particular the `with_mut` rewrite. diff --git a/docs/src/rules/preprocessors.md b/docs/src/rules/preprocessors.md index ccd7ffc0..7ef6f538 100644 --- a/docs/src/rules/preprocessors.md +++ b/docs/src/rules/preprocessors.md @@ -124,6 +124,6 @@ The preprocessor assumes the `rules` crate is buildable, which the earlier are therefore only reported as a warning. The result is one `ir_.json` per input file, keyed by rule name. The -output file name is derived from the input file name (`tgt_unsafe.rs` → +output file name is derived from the input file name (`tgt_unsafe.rs` becomes `ir_unsafe.json`) and the module directory is the direct parent of the `tgt_*.rs` file. diff --git a/docs/src/rules/writing-rules.md b/docs/src/rules/writing-rules.md index 172f25af..842f43c6 100644 --- a/docs/src/rules/writing-rules.md +++ b/docs/src/rules/writing-rules.md @@ -323,8 +323,8 @@ The Rust preprocessor evaluates `#[cfg]` attributes against the host target and drops non-matching rules. Mutually exclusive platform branches use `#elif` with disjoint rule -numbers: `rules/errno` defines `f91`–`f135` under `__linux__` and -`f136`–`f153` under `__APPLE__`. Feature-test macros a pattern needs must +numbers: `rules/errno` defines `f91` to `f135` under `__linux__` and +`f136` to `f153` under `__APPLE__`. Feature-test macros a pattern needs must come before the includes, as with `#define _GNU_SOURCE` in `rules/socket/src.c`. From 63ff5a74028131e4260e04e9a062455888bc8d86 Mon Sep 17 00:00:00 2001 From: Lucian Popescu Date: Fri, 14 Aug 2026 11:59:46 +0100 Subject: [PATCH 09/14] Final edits --- docs/src/rules/format.md | 2 +- docs/src/rules/ir.md | 23 +++++++++++++++-------- docs/src/rules/loading.md | 8 ++++---- docs/src/rules/overview.md | 25 ++++++++++--------------- docs/src/rules/preprocessors.md | 2 +- 5 files changed, 31 insertions(+), 29 deletions(-) diff --git a/docs/src/rules/format.md b/docs/src/rules/format.md index 8a858a26..f9b6a641 100644 --- a/docs/src/rules/format.md +++ b/docs/src/rules/format.md @@ -70,7 +70,7 @@ template using t1 = std::vector; ```rust // rules/vector/tgt_unsafe.rs fn t1() -> Vec { - Default::default() + Vec::new() } ``` diff --git a/docs/src/rules/ir.md b/docs/src/rules/ir.md index 17e88c56..85234c9a 100644 --- a/docs/src/rules/ir.md +++ b/docs/src/rules/ir.md @@ -64,7 +64,7 @@ The fragment kinds are: [Rule Rewriting](./rewriting.md)). * `va_args`: the expansion point for a variadic tail. -Every type in the IR (in `params`, `return_type`, and type rules) is a +Every type in the Rules IR (in `params`, `return_type`, and type rules) is a `TypeInfo` object, the type text plus a set of flags: * `is_refcount_pointer`: the type is a `Ptr<...>`. @@ -83,7 +83,7 @@ An `ExprRule` carries two flags of its own: * `is_extern`: the rule is an extern passthrough declaration and has no body. -Fields that are false, empty, or unset are omitted from the JSON. A `va` +Fields that are false, empty, or unset are omitted from the Rules IR. A `va` parameter is never listed in `params`, and a `()` return type is omitted. A type rule serializes as a `TypeRule` object: its `TypeInfo` plus the @@ -98,21 +98,28 @@ There is no explicit tag distinguishing the two rule kinds: an entry with ## In-memory form -`cpp2rust` mirrors this JSON in C++ structs of the same names, defined in -`cpp2rust/converter/translation_rule.h`. `TranslationRule::Load` reads one +`cpp2rust` mirrors the Rules IR in C++ structs of the same names, defined in +[`cpp2rust/converter/translation_rule.h`][translation-rule-h]. +`TranslationRule::Load` reads one module directory (the `ir_*.json` files described above) and produces two -maps keyed by rule name, one holding `ExprRule`s and one holding `TypeRule`s: +maps keyed by rule name, one holding [`ExprRule`]s and one holding +[`TypeRule`]s: + +[translation-rule-h]: https://github.com/cpp2rust/cpp2rust/blob/master/cpp2rust/converter/translation_rule.h +[`ExprRule`]: https://github.com/cpp2rust/cpp2rust/blob/master/cpp2rust/converter/translation_rule.h#L71 +[`TypeRule`]: https://github.com/cpp2rust/cpp2rust/blob/master/cpp2rust/converter/translation_rule.h#L84 * An `ExprRule` holds the body [fragments](#target-ir-ir_unsafejson--ir_refcountjson), the parameter and - return `TypeInfo`s, and the two rule-level flags. The name-keyed JSON maps become + return `TypeInfo`s, and the two rule-level flags (`multi_statement` and `is_extern`). The + name-keyed Rules IR maps become positional vectors: parameter `aN` is entry N of `params`, generic `TN` is entry N-1 of `generics` (each entry being the bound list). Rules support at most 9 generic parameters (`kMaxGenerics`). * A `TypeRule` holds the mapped type's `TypeInfo` and the `initializer` expression. The same struct also represents the built-in type mappings (scalars, pointers, ...) that the loader registers directly in code, - without any JSON behind them: for example, `int` maps to `i32`, and + without any Rules IR behind them: for example, `int` maps to `i32`, and `int *` to `*mut i32` in the unsafe model or `Ptr` in the refcount model. @@ -120,6 +127,6 @@ Both also carry `src`, the canonical C++ signature attached from [`ir_src.json`](#source-ir-ir_srcjson); it is the key the rule is matched by. -How the loader finds the IR directory, overlays the refcount model on the +How the loader finds the Rules IR directory, overlays the refcount model on the unsafe one, and indexes the loaded rules for matching is covered in [Loading and Matching](./loading.md). diff --git a/docs/src/rules/loading.md b/docs/src/rules/loading.md index d1556145..2e0cd18b 100644 --- a/docs/src/rules/loading.md +++ b/docs/src/rules/loading.md @@ -2,12 +2,12 @@ ## Finding the rules directory -`cpp2rust` takes the IR directory via `--rules `. If the flag is omitted +`cpp2rust` takes the Rules IR directory via `--rules `. If the flag is omitted it tries `./rules` and then `/../rules`, accepting the first candidate that contains (recursively) a subdirectory with `ir_src.json` plus -`ir_unsafe.json` or `ir_refcount.json`. Since the build writes the IR to +`ir_unsafe.json` or `ir_refcount.json`. Since the build writes the Rules IR to `/rules` and the binary lands in `/bin`, the default resolution -picks up the generated IR without any flags. +picks up the generated Rules IR without any flags. ## Loading @@ -51,7 +51,7 @@ same canonical printer used by `cpp-rule-preprocessor`, which is what makes the two sides comparable: * Functions and methods print as - ` ()[ const][ &|&&]`. + ` ([, ...])[ const][ volatile][ &|&&]`. * Enum constants and global variables print as their qualified name. * Integer literals expanded from a macro print as the macro name. diff --git a/docs/src/rules/overview.md b/docs/src/rules/overview.md index 0f74ed68..dd321667 100644 --- a/docs/src/rules/overview.md +++ b/docs/src/rules/overview.md @@ -5,30 +5,25 @@ Each rule module lives in the `rules/` directory and pairs a C++ source file (`src.cpp`) with its Rust translation for each model (`tgt_unsafe.rs` and `tgt_refcount.rs`). -The central idea is that every rule is expressed as ordinary, compilable -C++ and Rust source code, free of anything platform dependent. Both sides are -run through real compilers at build time, so a rule that does not compile -fails the build, and the platform-specific spellings a rule needs to match -against (for example `bool` canonicalizing to `_Bool`) are derived by the -compiler on the host rather than written by hand. The same rule sources work -on every platform `cpp2rust` builds on. +Every rule is expressed as ordinary, compilable C++ and Rust source code, +free of anything platform dependent. Both sides are run through real +compilers at build time, so a rule that does not compile fails the build, and +the platform-specific spellings (for example `bool` canonicalizing to +`_Bool`) are derived by the compiler on the host rather than written by hand. +The same rule sources work on every platform `cpp2rust` builds on. Rules go through a build-time compilation pipeline before `cpp2rust` can use them: 1. You author a rule module: C++ patterns in `src.cpp` and Rust targets in `tgt_unsafe.rs` / `tgt_refcount.rs`. -2. At build time, two preprocessors compile the module into JSON IR under +2. At build time, two preprocessors compile the module into Rules IR under `/rules//`: `cpp-rule-preprocessor` compiles the C++ side into `ir_src.json`, and `rule-preprocessor` compiles the Rust side into `ir_unsafe.json` and `ir_refcount.json`. -3. At startup, `cpp2rust` loads the IR files and indexes the rules by the +3. At startup, `cpp2rust` loads the Rules IR files and indexes the rules by the canonical signature of the C++ construct they match. -4. During translation, the converter looks up rules by signature and splices - their Rust bodies into the output, substituting arguments for placeholders - and rewriting the body when needed (e.g. wrapping method calls in - `with_mut` when the receiver is a `Ptr`). The rest of this part covers each stage: @@ -41,9 +36,9 @@ The rest of this part covers each stage: * [Conventions](./conventions.md): naming and style conventions rule authors must follow. * [The Rule Preprocessors](./preprocessors.md): the two build-time tools that - compile rules to IR. + compile rules to the Rules IR. * [The Rules IR](./ir.md): the JSON format the preprocessors emit. -* [Loading and Matching](./loading.md): how `cpp2rust` loads the IR and +* [Loading and Matching](./loading.md): how `cpp2rust` loads the Rules IR and matches rules against the input AST. * [The Matching Engine](./matching.md): how a candidate rule's signature is unified against the input. diff --git a/docs/src/rules/preprocessors.md b/docs/src/rules/preprocessors.md index 7ef6f538..ad18ed3c 100644 --- a/docs/src/rules/preprocessors.md +++ b/docs/src/rules/preprocessors.md @@ -1,6 +1,6 @@ # The Rule Preprocessors -Two build-time tools compile rule modules into the [JSON IR](./ir.md) that +Two build-time tools compile rule modules into the [Rules IR](./ir.md) that `cpp2rust` loads at runtime. Both write into `/rules//`: * `cpp-rule-preprocessor` compiles `src.cpp`/`src.c` into `ir_src.json`. From 415138ede01078d7691139a52e38ee99e11fdc6e Mon Sep 17 00:00:00 2001 From: Lucian Popescu Date: Fri, 14 Aug 2026 12:15:54 +0100 Subject: [PATCH 10/14] Delete test file --- docs/src/rules/math-test.md | 48 ------------------------------------- 1 file changed, 48 deletions(-) delete mode 100644 docs/src/rules/math-test.md diff --git a/docs/src/rules/math-test.md b/docs/src/rules/math-test.md deleted file mode 100644 index 2fbc5079..00000000 --- a/docs/src/rules/math-test.md +++ /dev/null @@ -1,48 +0,0 @@ -# Formula Rendering Test - -This page is a rendering test for hand-translating the inference rules from -`latex/translating.tex` (Fig. 2, the $\mathcal{T}$ function) into the book -via the `mdbook-katex` preprocessor. Only a few representative rules are -shown. - -The macros from `latex/macros.tex` are ported once into -`docs/katex-macros.txt` and are available on every page, so the rule bodies -below stay nearly identical to the paper's source. - -## Axioms - -$$ -\begin{array}{cc} -\text{Int} & \text{String} \\[4pt] -\dfrac{}{\typcomp{\cint} \defeq \rint} -& -\dfrac{}{\typcomp{\cstring} \defeq \rstring} -\end{array} -$$ - -## Rule with premises - -$$ -\begin{array}{c}\text{Unique Pointer}\\[4pt] -\dfrac{ - \typcomp{\typ} = \rtyp - \qquad - \boxTy(\rtyp) = \rtyp' -}{\typcomp{\uniqueptr{\typ}} \defeq \roption{\rtyp'}} -\end{array} -$$ - -## Rule with premises on two lines - -$$ -\begin{array}{c}\text{Ptr - Non-Virtual Class}\\[4pt] -\dfrac - {\begin{array}{c} - \typcomp{\typ} = \rtyp - \qquad - \typ \text{ is not a virtual class} - \\ - \converttoptr(\rtyp) = \rtyp' - \end{array}}{\typcomp{\typ*} \defeq \ptr{\rtyp'}} -\end{array} -$$ From b195a255137dd99aed9b7a25bcb4c898decb9277 Mon Sep 17 00:00:00 2001 From: Lucian Popescu Date: Fri, 14 Aug 2026 12:32:43 +0100 Subject: [PATCH 11/14] Format --- docs/src/SUMMARY.md | 34 +++---- docs/src/codegen/overview.md | 6 +- docs/src/codegen/temporaries.md | 4 +- docs/src/project/introduction.md | 20 ++-- docs/src/project/usage.md | 4 +- docs/src/rules/compat.md | 105 ++++++++++---------- docs/src/rules/conventions.md | 85 ++++++++-------- docs/src/rules/format.md | 80 ++++++++------- docs/src/rules/ir.md | 117 +++++++++++----------- docs/src/rules/loading.md | 126 ++++++++++++------------ docs/src/rules/matching.md | 48 +++++---- docs/src/rules/overview.md | 49 +++++---- docs/src/rules/preprocessors.md | 134 ++++++++++++------------- docs/src/rules/rewriting.md | 79 +++++++-------- docs/src/rules/writing-rules.md | 164 +++++++++++++++---------------- 15 files changed, 511 insertions(+), 544 deletions(-) diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md index 65814ddf..b935edd1 100644 --- a/docs/src/SUMMARY.md +++ b/docs/src/SUMMARY.md @@ -2,26 +2,26 @@ # The Project -* [Introduction](./project/introduction.md) -* [Building](./project/building.md) -* [Usage](./project/usage.md) -* [Test Suite](./project/test-suite.md) +- [Introduction](./project/introduction.md) +- [Building](./project/building.md) +- [Usage](./project/usage.md) +- [Test Suite](./project/test-suite.md) # Translation Rules -* [Overview](./rules/overview.md) -* [Rule Format](./rules/format.md) -* [Writing Rules](./rules/writing-rules.md) -* [Compat Shims](./rules/compat.md) -* [Conventions](./rules/conventions.md) -* [The Rule Preprocessors](./rules/preprocessors.md) -* [The Rules IR](./rules/ir.md) -* [Loading and Matching](./rules/loading.md) -* [The Matching Engine](./rules/matching.md) -* [Rule Rewriting](./rules/rewriting.md) +- [Overview](./rules/overview.md) +- [Rule Format](./rules/format.md) +- [Writing Rules](./rules/writing-rules.md) +- [Compat Shims](./rules/compat.md) +- [Conventions](./rules/conventions.md) +- [The Rule Preprocessors](./rules/preprocessors.md) +- [The Rules IR](./rules/ir.md) +- [Loading and Matching](./rules/loading.md) +- [The Matching Engine](./rules/matching.md) +- [Rule Rewriting](./rules/rewriting.md) # Code Generation -* [Overview](./codegen/overview.md) -* [Pointers and References](./codegen/pointers.md) -* [Temporary Materialization](./codegen/temporaries.md) +- [Overview](./codegen/overview.md) +- [Pointers and References](./codegen/pointers.md) +- [Temporary Materialization](./codegen/temporaries.md) diff --git a/docs/src/codegen/overview.md b/docs/src/codegen/overview.md index ca97a4f7..908d56c5 100644 --- a/docs/src/codegen/overview.md +++ b/docs/src/codegen/overview.md @@ -3,6 +3,6 @@ This part of the book documents the internals of the code generator: how the clang AST is traversed and how Rust code is emitted. -TODO: document the converter plugin mechanism -(`cpp2rust/converter/plugins/`), which intercepts constructs ahead of the -translation rules (currently `emplace_back`). +TODO: document the converter plugin mechanism (`cpp2rust/converter/plugins/`), +which intercepts constructs ahead of the translation rules (currently +`emplace_back`). diff --git a/docs/src/codegen/temporaries.md b/docs/src/codegen/temporaries.md index a4a86303..4b2b5520 100644 --- a/docs/src/codegen/temporaries.md +++ b/docs/src/codegen/temporaries.md @@ -1,5 +1,5 @@ # Temporary Materialization -> TODO: explain how the converter materializes temporaries for expressions -> that need an address but have none (e.g. literals passed where a pointer is +> TODO: explain how the converter materializes temporaries for expressions that +> need an address but have none (e.g. literals passed where a pointer is > expected). diff --git a/docs/src/project/introduction.md b/docs/src/project/introduction.md index 404920d7..ff987afb 100644 --- a/docs/src/project/introduction.md +++ b/docs/src/project/introduction.md @@ -9,21 +9,19 @@ published at PLDI 2026. ## Overview -Cpp2Rust first parses the input C++ file(s) with clang and produces an AST. -It then traverses the AST and emits Rust code as strings, inserting -calls to the `libcc2rs` runtime library where needed (e.g., for raw pointer -semantics). +Cpp2Rust first parses the input C++ file(s) with clang and produces an AST. It +then traverses the AST and emits Rust code as strings, inserting calls to the +`libcc2rs` runtime library where needed (e.g., for raw pointer semantics). Finally, the Rust code is pretty-printed using `rustfmt` to a single `.rs` file. -By default the *reference counting model* is used, which produces fully safe -Rust. -A generator of unsafe Rust is also available through the `--model=unsafe` +By default the _reference counting model_ is used, which produces fully safe +Rust. A generator of unsafe Rust is also available through the `--model=unsafe` command line argument for debugging and performance comparisons. ## Runtime library (`libcc2rs`) The generated code relies on a runtime library designed to simplify the -translation process. -C pointers are converted into the `Ptr` type provided by `libcc2rs`. -`Ptr` models C pointer semantics, including null, arithmetic, and aliasing, -while satisfying Rust's borrow checker through checked run-time operations. +translation process. C pointers are converted into the `Ptr` type provided by +`libcc2rs`. `Ptr` models C pointer semantics, including null, arithmetic, and +aliasing, while satisfying Rust's borrow checker through checked run-time +operations. diff --git a/docs/src/project/usage.md b/docs/src/project/usage.md index 64a110cc..deb413d9 100644 --- a/docs/src/project/usage.md +++ b/docs/src/project/usage.md @@ -6,8 +6,8 @@ ./build/cpp2rust/cpp2rust --file=.cpp -o=.rs ``` -By default, the reference counting model is used (fully safe output). -To generate unsafe Rust instead: +By default, the reference counting model is used (fully safe output). To +generate unsafe Rust instead: ```bash ./build/cpp2rust/cpp2rust --file=.cpp -o=.rs --model=unsafe diff --git a/docs/src/rules/compat.md b/docs/src/rules/compat.md index e456b551..5c45492b 100644 --- a/docs/src/rules/compat.md +++ b/docs/src/rules/compat.md @@ -1,28 +1,28 @@ # Compat Shims -Rule matching needs a resolvable callee. The preprocessor keys every -expression rule on the function, method, constructor, constant, or global -that the pattern's `return` expression resolves to, and the only macros it -can record are those that +Rule matching needs a resolvable callee. The preprocessor keys every expression +rule on the function, method, constructor, constant, or global that the +pattern's `return` expression resolves to, and the only macros it can record are +those that [expand to an integer literal](./writing-rules.md#enum-values-constants-and-macros), -which match by macro name. Any other macro is invisible to the rule system: -by the time clang has built the AST, the macro is gone and only its expansion +which match by macro name. Any other macro is invisible to the rule system: by +the time clang has built the AST, the macro is gone and only its expansion remains. -That is a problem for a small set of libc APIs that are specified as macros -over platform internals: +That is a problem for a small set of libc APIs that are specified as macros over +platform internals: -* `errno` is an object-like macro; glibc expands it to - `(*__errno_location())`, macOS to `(*__error())`. -* `assert` expands to a conditional that stringifies the condition and calls - a platform-specific failure handler with file and line arguments. -* `FD_SET`, `FD_CLR`, `FD_ISSET`, and `FD_ZERO` expand to bit manipulation - on the `fd_set` representation, through helpers that differ per platform. -* `ntohl`, `ntohs`, `htonl`, and `htons` expand to byte-swap builtins or to +- `errno` is an object-like macro; glibc expands it to `(*__errno_location())`, + macOS to `(*__error())`. +- `assert` expands to a conditional that stringifies the condition and calls a + platform-specific failure handler with file and line arguments. +- `FD_SET`, `FD_CLR`, `FD_ISSET`, and `FD_ZERO` expand to bit manipulation on + the `fd_set` representation, through helpers that differ per platform. +- `ntohl`, `ntohs`, `htonl`, and `htons` expand to byte-swap builtins or to nothing at all, depending on endianness. There is no stable, platform-independent callee here to key a rule on. The -*compat headers* in `cpp2rust/compat/` fix this by rewriting each such macro +_compat headers_ in `cpp2rust/compat/` fix this by rewriting each such macro into a call to a synthetic, well-known function before matching happens. ## How the shims work @@ -30,17 +30,16 @@ into a call to a synthetic, well-known function before matching happens. `cpp2rust/compat` is injected as a system include directory ahead of the platform headers in every clang invocation the project makes: both when `cpp2rust` parses the input program and when `cpp-rule-preprocessor` compiles -rule sources. The shared flag list lives in -`cpp2rust/compat/platform_flags.h` (`getPlatformClangBeginFlags`), and the -directory path is baked in at build time via the `COMPAT_INCLUDE_DIR` -definition. +rule sources. The shared flag list lives in `cpp2rust/compat/platform_flags.h` +(`getPlatformClangBeginFlags`), and the directory path is baked in at build time +via the `COMPAT_INCLUDE_DIR` definition. A shim header sits at the same relative path as the real header it shadows (`errno.h`, `sys/select.h`, `arpa/inet.h`, ...), so an ordinary `#include ` finds the shim first. The header then: -1. pulls in the real platform header with `#include_next` (a GNU extension; - the shared flags pass `-Wno-gnu-include-next` for it), +1. pulls in the real platform header with `#include_next` (a GNU extension; the + shared flags pass `-Wno-gnu-include-next` for it), 2. `#undef`s the macro, 3. declares a `cpp2rust_*` shim function, 4. redefines the macro to call the shim. @@ -57,12 +56,12 @@ int *cpp2rust_errno(void); #define errno (*cpp2rust_errno()) ``` -The redefinition keeps `errno` an lvalue by dereferencing the returned -pointer, so both reads and assignments like `errno = 0` still parse; what -the matcher sees in either case is a call to `int *cpp2rust_errno()`. +The redefinition keeps `errno` an lvalue by dereferencing the returned pointer, +so both reads and assignments like `errno = 0` still parse; what the matcher +sees in either case is a call to `int *cpp2rust_errno()`. -Because the input program and the rule sources are compiled with the same -shim headers, both sides canonicalize to the same signature, and an ordinary +Because the input program and the rule sources are compiled with the same shim +headers, both sides canonicalize to the same signature, and an ordinary [expression rule](./writing-rules.md) matches it: ```c @@ -72,10 +71,10 @@ shim headers, both sides canonicalize to the same signature, and an ordinary int *f1(void) { return cpp2rust_errno(); } ``` -The shim functions are declared but never defined on the C side. They only -exist so that the callee resolves; translation replaces the call with the -rule body, so no C implementation is ever linked. Whatever the shim is -supposed to *do* is supplied by the Rust targets: +The shim functions are declared but never defined on the C side. They only exist +so that the callee resolves; translation replaces the call with the rule body, +so no C implementation is ever linked. Whatever the shim is supposed to _do_ is +supplied by the Rust targets: ```rust // rules/errno/tgt_unsafe.rs @@ -91,11 +90,11 @@ fn f1() -> Ptr { } ``` -In the unsafe model `libcc2rs::cpp2rust_errno_unsafe` wraps the real -platform errno location (`__errno_location` on Linux, `__error` on macOS). -The refcount model instead *virtualizes* errno as a thread-local -`Value` inside `libcc2rs`; this is the same cell that other refcount -rules write when they translate a failing libc call into +In the unsafe model `libcc2rs::cpp2rust_errno_unsafe` wraps the real platform +errno location (`__errno_location` on Linux, `__error` on macOS). The refcount +model instead _virtualizes_ errno as a thread-local `Value` inside +`libcc2rs`; this is the same cell that other refcount rules write when they +translate a failing libc call into `libcc2rs::cpp2rust_errno().write(__e as i32)`. A rule pattern may spell either the macro or the shim directly; the two are @@ -109,17 +108,17 @@ void f2(int fd, fd_set *set) { return FD_SET(fd, set); } ## The current shims -| Header | Macros | Shim functions | Rules | -|---|---|---|---| -| `assert.h` | `assert` | `cpp2rust_assert_fail(bool)` | `rules/assert` maps it to `assert!(a0)` | -| `errno.h` | `errno` | `cpp2rust_errno()` | `rules/errno`, see above | -| `arpa/inet.h` | `ntohl`, `ntohs`, `htonl`, `htons` | `cpp2rust_ntohl(x)`, ... | `rules/arpa_inet` maps them to `u32::from_be`, `u16::to_be`, ... | +| Header | Macros | Shim functions | Rules | +| -------------- | ----------------------------------------- | ------------------------------- | --------------------------------------------------------------------------------------- | +| `assert.h` | `assert` | `cpp2rust_assert_fail(bool)` | `rules/assert` maps it to `assert!(a0)` | +| `errno.h` | `errno` | `cpp2rust_errno()` | `rules/errno`, see above | +| `arpa/inet.h` | `ntohl`, `ntohs`, `htonl`, `htons` | `cpp2rust_ntohl(x)`, ... | `rules/arpa_inet` maps them to `u32::from_be`, `u16::to_be`, ... | | `sys/select.h` | `FD_SET`, `FD_CLR`, `FD_ISSET`, `FD_ZERO` | `cpp2rust_fd_set(fd, set)`, ... | `rules/select` maps them to `libc::FD_SET(...)` (unsafe) or `CFdSet` methods (refcount) | -Note how the shim also normalizes the *shape* of the API. C's `assert` is a -macro precisely so it can stringify its condition and capture file and line; -the shim reduces it to a plain `void(bool)` function, and the Rust side -regains the diagnostics by mapping it to the `assert!` macro. +Note how the shim also normalizes the _shape_ of the API. C's `assert` is a +macro precisely so it can stringify its condition and capture file and line; the +shim reduces it to a plain `void(bool)` function, and the Rust side regains the +diagnostics by mapping it to the `assert!` macro. ## Adding a new shim @@ -132,15 +131,15 @@ To make another macro-based API matchable: signature, and redefine the macro to call it. 3. Write rules for the shim in a `rules/` module as for any other function, including the corresponding header in `src.c`/`src.cpp`. -4. If a model needs runtime support (as refcount errno does), implement it - in `libcc2rs` and call it from the rule target. +4. If a model needs runtime support (as refcount errno does), implement it in + `libcc2rs` and call it from the rule target. -Keep the shim's signature platform-independent; the whole point is that -both sides of every rule see one canonical declaration on every platform. +Keep the shim's signature platform-independent; the whole point is that both +sides of every rule see one canonical declaration on every platform. ## Related normalization -The same shared flag list also passes `-D_FORTIFY_SOURCE=0`, which keeps -glibc from substituting fortified variants (`__printf_chk` and friends) for -standard calls. Like the shims, this ensures that calls in the input program -resolve to the standard declarations the rules are written against. +The same shared flag list also passes `-D_FORTIFY_SOURCE=0`, which keeps glibc +from substituting fortified variants (`__printf_chk` and friends) for standard +calls. Like the shims, this ensures that calls in the input program resolve to +the standard declarations the rules are written against. diff --git a/docs/src/rules/conventions.md b/docs/src/rules/conventions.md index 38cb25d3..476ed08a 100644 --- a/docs/src/rules/conventions.md +++ b/docs/src/rules/conventions.md @@ -1,71 +1,68 @@ # Conventions -Most of these conventions are enforced by the preprocessors, and violating -them fails the build; the notes below call out the ones that are not -checked. +Most of these conventions are enforced by the preprocessors, and violating them +fails the build; the notes below call out the ones that are not checked. ## Naming -| Element | C++ side | Rust side | -|---|---|---| -| Expression rule | `f1`, `f2`, ... | same name | -| Type rule | `t1`, `t2`, ... via `using`/`typedef` | `fn tN() -> RustType` with no arguments | -| Parameters | free-form (`o`, `it`, `key`, `dst`, `n`, ...) | must be `a0`, `a1`, ... consecutive from 0 | -| Generics | `T1`, `T2`, ... (type and non-type params) | `T1`, `T2`, ... consecutive from 1 | -| Variadic pack | `typename... Args` | trailing `va: &[VaArg]` | -| Locals in Rust bodies | | double-underscore prefix: `__v`, `__fd`, `__e`, ... | +| Element | C++ side | Rust side | +| --------------------- | --------------------------------------------- | --------------------------------------------------- | +| Expression rule | `f1`, `f2`, ... | same name | +| Type rule | `t1`, `t2`, ... via `using`/`typedef` | `fn tN() -> RustType` with no arguments | +| Parameters | free-form (`o`, `it`, `key`, `dst`, `n`, ...) | must be `a0`, `a1`, ... consecutive from 0 | +| Generics | `T1`, `T2`, ... (type and non-type params) | `T1`, `T2`, ... consecutive from 1 | +| Variadic pack | `typename... Args` | trailing `va: &[VaArg]` | +| Locals in Rust bodies | | double-underscore prefix: `__v`, `__fd`, `__e`, ... | Notes: -* Rule numbering is per module, and gaps are currently allowed (e.g. - `rules/map` has no `f4`), though this might change in the future. Names - must be unique across `src.c` and `src.cpp` combined. -* On the C++ side parameter names are free, but the *order* defines the - placeholder indices: the first parameter is `a0` on the Rust side, the - second is `a1`, and so on. The receiver of a method rule is always the - first parameter, hence `a0`. -* Generic parameters are matched positionally between the two sides, so `T1` - in the Rust target means "whatever bound to `T1` in the C++ pattern". -* Locals introduced inside Rust rule bodies use a `__` prefix. This is not +- Rule numbering is per module, and gaps are currently allowed (e.g. `rules/map` + has no `f4`), though this might change in the future. Names must be unique + across `src.c` and `src.cpp` combined. +- On the C++ side parameter names are free, but the _order_ defines the + placeholder indices: the first parameter is `a0` on the Rust side, the second + is `a1`, and so on. The receiver of a method rule is always the first + parameter, hence `a0`. +- Generic parameters are matched positionally between the two sides, so `T1` in + the Rust target means "whatever bound to `T1` in the C++ pattern". +- Locals introduced inside Rust rule bodies use a `__` prefix. This is not checked by the build, but it is needed: rule bodies are spliced inline into - the generated code, so an unprefixed local could collide with a variable - name from the translated program. + the generated code, so an unprefixed local could collide with a variable name + from the translated program. ## Function qualifiers -* In `tgt_unsafe.rs`, expression rules are `unsafe fn`; type rules (`tN`) are +- In `tgt_unsafe.rs`, expression rules are `unsafe fn`; type rules (`tN`) are plain `fn`. -* In `tgt_refcount.rs`, all rules are safe `fn`. The refcount model produces +- In `tgt_refcount.rs`, all rules are safe `fn`. The refcount model produces fully safe Rust, so a refcount rule body must not need `unsafe`. The build does not check the qualifiers themselves; only rustc's usual rules -apply when the `rules` crate compiles. In particular, nothing stops an -`unsafe` block inside a refcount rule body from being spliced into the -output, so keeping refcount rules safe is what upholds the model's safety -guarantee. +apply when the `rules` crate compiles. In particular, nothing stops an `unsafe` +block inside a refcount rule body from being spliced into the output, so keeping +refcount rules safe is what upholds the model's safety guarantee. ## C++ pattern shape -* An `fN` body must be exactly one `return` statement. The preprocessor - rejects anything else. -* `return` statements are not allowed inside Rust rule bodies; write the - result as a tail expression instead. -* Exercise exactly one construct per rule. If an API has several overloads, - write one rule per overload (including separate rules for `const T &` - versus `T &&` parameters). +- An `fN` body must be exactly one `return` statement. The preprocessor rejects + anything else. +- `return` statements are not allowed inside Rust rule bodies; write the result + as a tail expression instead. +- Exercise exactly one construct per rule. If an API has several overloads, + write one rule per overload (including separate rules for `const T &` versus + `T &&` parameters). ## Argument accesses -Every use of an `aN` parameter in a rule body is classified as a read, -write, or move by the -[rule preprocessor](./preprocessors.md#rule-preprocessor). Passing an -argument by value counts as a read, not a move; the only way to record a -move is `std::mem::take(&mut aN)`. +Every use of an `aN` parameter in a rule body is classified as a read, write, or +move by the [rule preprocessor](./preprocessors.md#rule-preprocessor). Passing +an argument by value counts as a read, not a move; the only way to record a move +is `std::mem::take(&mut aN)`. ## Type checking All `tgt_*.rs` files are compiled as part of the `rules` crate, so a rule body that does not type-check against `libcc2rs`, `libc`, `nix`, etc. breaks the -build. If a rule needs a new crate dependency, add it to `rules/Cargo.toml` -and to the hardcoded crate list in `rule-preprocessor/src/semantic.rs` -(see [The Rule Preprocessors](./preprocessors.md#rule-preprocessor)). +build. If a rule needs a new crate dependency, add it to `rules/Cargo.toml` and +to the hardcoded crate list in `rule-preprocessor/src/semantic.rs` (see +[The Rule Preprocessors](./preprocessors.md#rule-preprocessor)). diff --git a/docs/src/rules/format.md b/docs/src/rules/format.md index f9b6a641..199c4481 100644 --- a/docs/src/rules/format.md +++ b/docs/src/rules/format.md @@ -4,25 +4,25 @@ A rule module is a directory under `rules/`, usually named after the header or library it covers (`rules/unistd`, `rules/vector`, `rules/string`, ...). It contains: -* `src.cpp` and/or `src.c`: the C++ (or C) side of each rule. -* `tgt_unsafe.rs`: the Rust targets for the unsafe model. -* `tgt_refcount.rs`: the Rust targets for the reference counting model +- `src.cpp` and/or `src.c`: the C++ (or C) side of each rule. +- `tgt_unsafe.rs`: the Rust targets for the unsafe model. +- `tgt_refcount.rs`: the Rust targets for the reference counting model (optional, see below). A rule is a pair of same-named functions on the two sides. Names determine the rule kind: -* `f1`, `f2`, ... are *expression rules*: they map a C++ call, member access, +- `f1`, `f2`, ... are _expression rules_: they map a C++ call, member access, constructor, or constant to a Rust expression. -* `t1`, `t2`, ... are *type rules*: they map a C++ type to a Rust type. +- `t1`, `t2`, ... are _type rules_: they map a C++ type to a Rust type. ## Expression rules -On the C++ side, an `fN` function must have a body that is exactly one -`return` statement. The returned expression is the pattern: the preprocessor -resolves the *callee* of that expression (the function, method, constructor, -enum constant, or macro being used) and that becomes the rule's matching key. -The function parameters stand for the arguments at the call site. +On the C++ side, an `fN` function must have a body that is exactly one `return` +statement. The returned expression is the pattern: the preprocessor resolves the +_callee_ of that expression (the function, method, constructor, enum constant, +or macro being used) and that becomes the rule's matching key. The function +parameters stand for the arguments at the call site. ```cpp // rules/unistd/src.cpp @@ -30,8 +30,8 @@ int f4(const char *pathname) { return unlink(pathname); } ``` On the Rust side, the same-named function gives the replacement expression. -Parameters must be named `a0`, `a1`, ... and correspond positionally to the -C++ parameters: +Parameters must be named `a0`, `a1`, ... and correspond positionally to the C++ +parameters: ```rust // rules/unistd/tgt_unsafe.rs @@ -53,14 +53,14 @@ fn f4(a0: Ptr) -> i32 { } ``` -When the converter encounters `unlink(x)` in the input, it emits the rule -body with the translated `x` substituted for `a0`. +When the converter encounters `unlink(x)` in the input, it emits the rule body +with the translated `x` substituted for `a0`. ## Type rules -On the C++ side, a `tN` rule is a type alias (`using` or `typedef`). On the -Rust side, it is a *zero-argument function* whose return type is the mapped -Rust type and whose body is the default initializer for that type: +On the C++ side, a `tN` rule is a type alias (`using` or `typedef`). On the Rust +side, it is a _zero-argument function_ whose return type is the mapped Rust type +and whose body is the default initializer for that type: ```cpp // rules/vector/src.cpp @@ -80,13 +80,12 @@ The loader always reads `ir_unsafe.json` first. When translating with the reference counting model, it then overlays `ir_refcount.json` on top: entries with the same rule name replace the unsafe ones. -This means `tgt_refcount.rs` only needs to contain the rules that *differ* -from the unsafe model. For example, the `__builtin_mul_overflow` rule in +This means `tgt_refcount.rs` only needs to contain the rules that _differ_ from +the unsafe model. For example, the `__builtin_mul_overflow` rule in `rules/builtin` has a pointer out-parameter (`a2` below), so the two models -translate it differently: in the unsafe model `a2` is a raw `*mut i64` -written through a deref, while in the refcount model it is a `Ptr` -written through `Ptr::write`. The other two arguments are identical in both -models: +translate it differently: in the unsafe model `a2` is a raw `*mut i64` written +through a deref, while in the refcount model it is a `Ptr` written through +`Ptr::write`. The other two arguments are identical in both models: ```rust // rules/builtin/tgt_unsafe.rs @@ -117,12 +116,12 @@ A module may have both `src.c` and `src.cpp`; both are preprocessed and merged into one `ir_src.json`. Defining the same rule name in both files is a hard error, so numbering must not collide. -This split is necessary because rules match on the exact canonical signature -of the callee, and some libc functions have *different signatures in C and -C++*. For example, C has a single -`char *strchr(const char *, int)`, while C++ replaces it with const-correct -overloads such as `const char *strchr(const char *, int)`. Since the -signatures differ, `rules/cstring` defines one rule per language: +This split is necessary because rules match on the exact canonical signature of +the callee, and some libc functions have _different signatures in C and C++_. +For example, C has a single `char *strchr(const char *, int)`, while C++ +replaces it with const-correct overloads such as +`const char *strchr(const char *, int)`. Since the signatures differ, +`rules/cstring` defines one rule per language: ```cpp // rules/cstring/src.cpp @@ -137,18 +136,17 @@ char *f5(const char *a0, int a1) { return (strchr)(a0, a1); } The C++ rule matches `strchr` calls in code translated as C++, the C rule matches them in code translated as C. -`rules/builtin` uses this to cover both languages: `src.cpp` defines -`f9`/`f10` for the C++ `__builtin_mul_overflow` (returning `bool`) while -`src.c` defines `f12`/`f13` for the C version (returning `int`); their Rust -bodies are identical. +`rules/builtin` uses this to cover both languages: `src.cpp` defines `f9`/`f10` +for the C++ `__builtin_mul_overflow` (returning `bool`) while `src.c` defines +`f12`/`f13` for the C version (returning `int`); their Rust bodies are +identical. ## The `rules` crate -The whole `rules/` tree is a single Rust crate. `rules/build.rs` walks the -tree, collects every `tgt_*.rs`, and generates `rules/src/modules.rs` with one -`#[path = ...]` module per file. Building the crate therefore type-checks -every rule body against the crates the rule targets call into, which are -declared as dependencies in `rules/Cargo.toml` (`libcc2rs`, `libc`, `nix`, -...). The -Rust rule preprocessor compiles exactly this crate to resolve types in rule -bodies. `rules/src/` is the only subdirectory that is not a rule module. +The whole `rules/` tree is a single Rust crate. `rules/build.rs` walks the tree, +collects every `tgt_*.rs`, and generates `rules/src/modules.rs` with one +`#[path = ...]` module per file. Building the crate therefore type-checks every +rule body against the crates the rule targets call into, which are declared as +dependencies in `rules/Cargo.toml` (`libcc2rs`, `libc`, `nix`, ...). The Rust +rule preprocessor compiles exactly this crate to resolve types in rule bodies. +`rules/src/` is the only subdirectory that is not a rule module. diff --git a/docs/src/rules/ir.md b/docs/src/rules/ir.md index 85234c9a..60201d34 100644 --- a/docs/src/rules/ir.md +++ b/docs/src/rules/ir.md @@ -3,15 +3,15 @@ Each rule module compiles to up to three JSON files in `/rules//`: -* `ir_src.json`: the C++ side, from +- `ir_src.json`: the C++ side, from [`cpp-rule-preprocessor`](./preprocessors.md#cpp-rule-preprocessor). -* `ir_unsafe.json`: the Rust side for the unsafe model, from +- `ir_unsafe.json`: the Rust side for the unsafe model, from [`rule-preprocessor`](./preprocessors.md#rule-preprocessor). -* `ir_refcount.json`: the Rust side for the refcount model, also from +- `ir_refcount.json`: the Rust side for the refcount model, also from `rule-preprocessor` (only if the module has a `tgt_refcount.rs`). -All three are objects keyed by rule name (`f1`, `t1`, ...), and the loader -joins them by name. +All three are objects keyed by rule name (`f1`, `t1`, ...), and the loader joins +them by name. ## Source IR (`ir_src.json`) @@ -25,14 +25,14 @@ rule matches. For `rules/vector`: } ``` -This signature string is the lookup key for the whole rule: the converter -prints C++ constructs from the input AST with the same printer and compares -the strings. +This signature string is the lookup key for the whole rule: the converter prints +C++ constructs from the input AST with the same printer and compares the +strings. ## Target IR (`ir_unsafe.json` / `ir_refcount.json`) -An expression rule serializes as an `ExprRule` object: the rule's signature -plus its body as a list of *fragments*. For +An expression rule serializes as an `ExprRule` object: the rule's signature plus +its body as a list of _fragments_. For `unsafe fn f6(a0: &mut Vec) -> *mut T1 { a0.as_mut_ptr() }`: ```json @@ -50,82 +50,81 @@ plus its body as a list of *fragments*. For The fragment kinds are: -* `text`: literal Rust source, emitted verbatim. -* `placeholder`: a use of one of the rule's `aN` parameters in the body (not - an argument of whatever the body calls); the converter substitutes the - translated call-site argument here. Its fields: - * `arg`: the parameter index N. - * `access`: how the body uses the argument: `read`, `write`, or `move`. - * `is_index_base`: the placeholder is the base of an index expression. -* `generic`: a `TN` slot, replaced with the instantiated Rust type; - serialized as the 1-based index N. -* `method_call`: a method call split into `receiver` and `body` fragment - lists, so the code generator can rewrite the pair (see +- `text`: literal Rust source, emitted verbatim. +- `placeholder`: a use of one of the rule's `aN` parameters in the body (not an + argument of whatever the body calls); the converter substitutes the translated + call-site argument here. Its fields: + - `arg`: the parameter index N. + - `access`: how the body uses the argument: `read`, `write`, or `move`. + - `is_index_base`: the placeholder is the base of an index expression. +- `generic`: a `TN` slot, replaced with the instantiated Rust type; serialized + as the 1-based index N. +- `method_call`: a method call split into `receiver` and `body` fragment lists, + so the code generator can rewrite the pair (see [Rule Rewriting](./rewriting.md)). -* `va_args`: the expansion point for a variadic tail. +- `va_args`: the expansion point for a variadic tail. Every type in the Rules IR (in `params`, `return_type`, and type rules) is a `TypeInfo` object, the type text plus a set of flags: -* `is_refcount_pointer`: the type is a `Ptr<...>`. -* `is_unsafe_pointer`: the type is a raw `*mut`/`*const` pointer. -* `derives` (type rules only): the standard traits the mapped type - implements (`Copy`, `Clone`, `Default`, ...). +- `is_refcount_pointer`: the type is a `Ptr<...>`. +- `is_unsafe_pointer`: the type is a raw `*mut`/`*const` pointer. +- `derives` (type rules only): the standard traits the mapped type implements + (`Copy`, `Clone`, `Default`, ...). -The two pointer flags are mutually exclusive; the loader rejects a type -with both set. +The two pointer flags are mutually exclusive; the loader rejects a type with +both set. An `ExprRule` carries two flags of its own: -* `multi_statement`: the body has more than one statement, or a statement - followed by a tail expression, and must be wrapped in a block to stay a - single expression. -* `is_extern`: the rule is an extern passthrough declaration and has no - body. +- `multi_statement`: the body has more than one statement, or a statement + followed by a tail expression, and must be wrapped in a block to stay a single + expression. +- `is_extern`: the rule is an extern passthrough declaration and has no body. Fields that are false, empty, or unset are omitted from the Rules IR. A `va` parameter is never listed in `params`, and a `()` return type is omitted. -A type rule serializes as a `TypeRule` object: its `TypeInfo` plus the -`init` initializer expression, merged into one object: +A type rule serializes as a `TypeRule` object: its `TypeInfo` plus the `init` +initializer expression, merged into one object: ```json "t1": { "type": "Vec", "init": "Default::default()" } ``` -There is no explicit tag distinguishing the two rule kinds: an entry with -`body` is an expression rule, one with `type` and `init` a type rule. +There is no explicit tag distinguishing the two rule kinds: an entry with `body` +is an expression rule, one with `type` and `init` a type rule. ## In-memory form `cpp2rust` mirrors the Rules IR in C++ structs of the same names, defined in [`cpp2rust/converter/translation_rule.h`][translation-rule-h]. -`TranslationRule::Load` reads one -module directory (the `ir_*.json` files described above) and produces two -maps keyed by rule name, one holding [`ExprRule`]s and one holding -[`TypeRule`]s: - -[translation-rule-h]: https://github.com/cpp2rust/cpp2rust/blob/master/cpp2rust/converter/translation_rule.h -[`ExprRule`]: https://github.com/cpp2rust/cpp2rust/blob/master/cpp2rust/converter/translation_rule.h#L71 -[`TypeRule`]: https://github.com/cpp2rust/cpp2rust/blob/master/cpp2rust/converter/translation_rule.h#L84 - -* An `ExprRule` holds the body +`TranslationRule::Load` reads one module directory (the `ir_*.json` files +described above) and produces two maps keyed by rule name, one holding +[`ExprRule`]s and one holding [`TypeRule`]s: + +[translation-rule-h]: + https://github.com/cpp2rust/cpp2rust/blob/master/cpp2rust/converter/translation_rule.h +[`ExprRule`]: + https://github.com/cpp2rust/cpp2rust/blob/master/cpp2rust/converter/translation_rule.h#L71 +[`TypeRule`]: + https://github.com/cpp2rust/cpp2rust/blob/master/cpp2rust/converter/translation_rule.h#L84 + +- An `ExprRule` holds the body [fragments](#target-ir-ir_unsafejson--ir_refcountjson), the parameter and - return `TypeInfo`s, and the two rule-level flags (`multi_statement` and `is_extern`). The - name-keyed Rules IR maps become - positional vectors: parameter `aN` is entry N of `params`, generic `TN` is - entry N-1 of `generics` (each entry being the bound list). Rules support - at most 9 generic parameters (`kMaxGenerics`). -* A `TypeRule` holds the mapped type's `TypeInfo` and the `initializer` + return `TypeInfo`s, and the two rule-level flags (`multi_statement` and + `is_extern`). The name-keyed Rules IR maps become positional vectors: + parameter `aN` is entry N of `params`, generic `TN` is entry N-1 of `generics` + (each entry being the bound list). Rules support at most 9 generic parameters + (`kMaxGenerics`). +- A `TypeRule` holds the mapped type's `TypeInfo` and the `initializer` expression. The same struct also represents the built-in type mappings - (scalars, pointers, ...) that the loader registers directly in code, - without any Rules IR behind them: for example, `int` maps to `i32`, and - `int *` to `*mut i32` in the unsafe model or `Ptr` in the refcount - model. + (scalars, pointers, ...) that the loader registers directly in code, without + any Rules IR behind them: for example, `int` maps to `i32`, and `int *` to + `*mut i32` in the unsafe model or `Ptr` in the refcount model. Both also carry `src`, the canonical C++ signature attached from -[`ir_src.json`](#source-ir-ir_srcjson); it is the key the rule is matched -by. +[`ir_src.json`](#source-ir-ir_srcjson); it is the key the rule is matched by. How the loader finds the Rules IR directory, overlays the refcount model on the unsafe one, and indexes the loaded rules for matching is covered in diff --git a/docs/src/rules/loading.md b/docs/src/rules/loading.md index 2e0cd18b..8865a8af 100644 --- a/docs/src/rules/loading.md +++ b/docs/src/rules/loading.md @@ -2,23 +2,22 @@ ## Finding the rules directory -`cpp2rust` takes the Rules IR directory via `--rules `. If the flag is omitted -it tries `./rules` and then `/../rules`, accepting the first -candidate that contains (recursively) a subdirectory with `ir_src.json` plus -`ir_unsafe.json` or `ir_refcount.json`. Since the build writes the Rules IR to -`/rules` and the binary lands in `/bin`, the default resolution +`cpp2rust` takes the Rules IR directory via `--rules `. If the flag is +omitted it tries `./rules` and then `/../rules`, accepting the +first candidate that contains (recursively) a subdirectory with `ir_src.json` +plus `ir_unsafe.json` or `ir_refcount.json`. Since the build writes the Rules IR +to `/rules` and the binary lands in `/bin`, the default resolution picks up the generated Rules IR without any flags. ## Loading Rules are loaded once per process by `Mapper::LoadTranslationRules`: -1. Built-in type mappings are registered first. Every scalar is mapped - with its width taken from the host (`int` maps to `i32`, - `unsigned long` to `u64`), together with its `const` form and its - pointer forms: `*mut`/`*const` in the unsafe model, `Ptr` in the - refcount model, where constness is dropped. `char` maps to - `libc::c_char` in the unsafe model and to `u8` in refcount; +1. Built-in type mappings are registered first. Every scalar is mapped with its + width taken from the host (`int` maps to `i32`, `unsigned long` to `u64`), + together with its `const` form and its pointer forms: `*mut`/`*const` in the + unsafe model, `Ptr` in the refcount model, where constness is dropped. + `char` maps to `libc::c_char` in the unsafe model and to `u8` in refcount; `size_t`/`ssize_t` map to `usize`/`isize`; `void *` maps to `*mut ::libc::c_void`, or in refcount to `AnyPtr`. 2. Every subdirectory of the rules directory is loaded with @@ -28,76 +27,73 @@ Rules are loaded once per process by `Mapper::LoadTranslationRules`: Loading is strict: an `ir_src.json` entry with no matching target rule is a fatal error (this is what catches mismatched `#if`/`#[cfg]` gating), every -generic declared by a rule must appear in its C++ signature, and two type -rules for the same C++ type are rejected. +generic declared by a rule must appear in its C++ signature, and two type rules +for the same C++ type are rejected. ## Matching -Loaded rules are indexed in two multimaps, one for expression rules and one -for type rules. The multimap key is only a coarse bucket for collecting -candidate rules; whether a candidate actually matches is decided by -[the matching engine](./matching.md). The bucket key is derived from the -C++ signature: +Loaded rules are indexed in two multimaps, one for expression rules and one for +type rules. The multimap key is only a coarse bucket for collecting candidate +rules; whether a candidate actually matches is decided by +[the matching engine](./matching.md). The bucket key is derived from the C++ +signature: -* For expressions, the qualified function name with the return type, the +- For expressions, the qualified function name with the return type, the parameter list, and all template arguments stripped, so the rule for `_Bool std::vector::empty() const` lands in the `std::vector::empty` bucket. -* For types, the text up to the first `<`, so all `std::vector<...>` rules - land in the `std::vector` bucket. +- For types, the text up to the first `<`, so all `std::vector<...>` rules land + in the `std::vector` bucket. During translation, the converter prints the construct it encounters with the -same canonical printer used by `cpp-rule-preprocessor`, which is what makes -the two sides comparable: +same canonical printer used by `cpp-rule-preprocessor`, which is what makes the +two sides comparable: -* Functions and methods print as +- Functions and methods print as ` ([, ...])[ const][ volatile][ &|&&]`. -* Enum constants and global variables print as their qualified name. -* Integer literals expanded from a macro print as the macro name. +- Enum constants and global variables print as their qualified name. +- Integer literals expanded from a macro print as the macro name. All rules in the matching bucket are then unified against this string by -[the matching engine](./matching.md), which binds `T1`...`T9` to the -concrete types at the use site and picks the most specific rule when -several match. Type lookups first try the sugar-preserving spelling (so a -rule can match `size_t` as written) and retry with the desugared type on -failure. +[the matching engine](./matching.md), which binds `T1`...`T9` to the concrete +types at the use site and picks the most specific rule when several match. Type +lookups first try the sugar-preserving spelling (so a rule can match `size_t` as +written) and retry with the desugared type on failure. -Running `cpp2rust` with `--verbose` logs every lookup and the rule it -matched, which is the quickest way to see why a rule does or does not -fire. +Running `cpp2rust` with `--verbose` logs every lookup and the rule it matched, +which is the quickest way to see why a rule does or does not fire. ## Application When a rule matches, the converter walks its body fragments and emits: -* `text` fragments verbatim, -* `placeholder` fragments as the translated call-site argument. How the - argument is emitted depends on the placeholder's access and on whether the - argument and the declared parameter type are pointers: - * Read access emits the argument as a plain value, with an implicit - numeric cast when the parameter type asks for one. - * Write access emits the argument as an lvalue. - * Move access wraps the argument in `std::mem::take(&mut ...)`; - temporaries are moved as-is. - * If the rule declares a pointer parameter but the argument is not a - pointer, the converter takes a fresh pointer to it, - [materializing a temporary](../codegen/temporaries.md) when the - argument has no address of its own. For example, - the refcount `std::max` rule declares `Ptr` parameters, since the - C++ side takes `const T1 &` and the refcount model - [translates references as `Ptr`](../codegen/pointers.md), so - `std::max(x1, x2)` on plain locals - substitutes `x1.as_pointer()` and `x2.as_pointer()` for the - placeholders, while `std::max(30, 40)` first materializes `__tmp_0` - and `__tmp_1` values for the literals and points into those. - * If the receiver argument is a pointer but the rule expects a value, - the converter dereferences it; this only happens for receivers, not - for ordinary arguments. -* `generic` fragments as the Rust mapping of the bound C++ type, -* `va_args` fragments as the converted variadic tail, -* `method_call` fragments as receiver followed by body, possibly rewritten - (see [Rule Rewriting](./rewriting.md)). - -Multi-statement bodies are wrapped in `{ }` so they remain a single -expression. Rules for user-defined C++ types are injected through the same -mechanism at translation time. +- `text` fragments verbatim, +- `placeholder` fragments as the translated call-site argument. How the argument + is emitted depends on the placeholder's access and on whether the argument and + the declared parameter type are pointers: + - Read access emits the argument as a plain value, with an implicit numeric + cast when the parameter type asks for one. + - Write access emits the argument as an lvalue. + - Move access wraps the argument in `std::mem::take(&mut ...)`; temporaries + are moved as-is. + - If the rule declares a pointer parameter but the argument is not a pointer, + the converter takes a fresh pointer to it, + [materializing a temporary](../codegen/temporaries.md) when the argument has + no address of its own. For example, the refcount `std::max` rule declares + `Ptr` parameters, since the C++ side takes `const T1 &` and the refcount + model [translates references as `Ptr`](../codegen/pointers.md), so + `std::max(x1, x2)` on plain locals substitutes `x1.as_pointer()` and + `x2.as_pointer()` for the placeholders, while `std::max(30, 40)` first + materializes `__tmp_0` and `__tmp_1` values for the literals and points into + those. + - If the receiver argument is a pointer but the rule expects a value, the + converter dereferences it; this only happens for receivers, not for ordinary + arguments. +- `generic` fragments as the Rust mapping of the bound C++ type, +- `va_args` fragments as the converted variadic tail, +- `method_call` fragments as receiver followed by body, possibly rewritten (see + [Rule Rewriting](./rewriting.md)). + +Multi-statement bodies are wrapped in `{ }` so they remain a single expression. +Rules for user-defined C++ types are injected through the same mechanism at +translation time. diff --git a/docs/src/rules/matching.md b/docs/src/rules/matching.md index d31c522e..faee2222 100644 --- a/docs/src/rules/matching.md +++ b/docs/src/rules/matching.md @@ -1,40 +1,36 @@ # The Matching Engine [Loading and Matching](./loading.md) collects the candidate rules for a -construct from a bucket; each candidate's source signature is then -*unified* against the printed construct. The signature is treated as a -template whose `T1`...`T9` slots capture concrete types: -`std::vector::vector()` unifies with `std::vector::vector()` by -binding `T1 = int`. +construct from a bucket; each candidate's source signature is then _unified_ +against the printed construct. The signature is treated as a template whose +`T1`...`T9` slots capture concrete types: `std::vector::vector()` unifies +with `std::vector::vector()` by binding `T1 = int`. Unification works on the two strings: -* Whitespace differences are ignored. -* A `TN` slot captures up to the next literal text of the pattern, found - at the same `<>`/`()`/`[]` nesting depth. This is how `T1` captures all - of `std::map` in `std::vector>` without - stopping at the inner comma. -* A `TN` that appears again must match its first capture exactly. -* The whole printed string must be consumed; trailing text fails the - match. -* Slots may stay unbound (a pattern can use `T2` without `T1`). - -A rule matches if unification succeeds. When several rules in the bucket -match, the one with the longest source signature wins, so more specific -rules take precedence; between equally long signatures the choice is -unspecified. +- Whitespace differences are ignored. +- A `TN` slot captures up to the next literal text of the pattern, found at the + same `<>`/`()`/`[]` nesting depth. This is how `T1` captures all of + `std::map` in `std::vector>` without stopping at + the inner comma. +- A `TN` that appears again must match its first capture exactly. +- The whole printed string must be consumed; trailing text fails the match. +- Slots may stay unbound (a pattern can use `T2` without `T1`). + +A rule matches if unification succeeds. When several rules in the bucket match, +the one with the longest source signature wins, so more specific rules take +precedence; between equally long signatures the choice is unspecified. ## Bucket keys -The bucket keys described in -[Loading and Matching](./loading.md#matching) have two special cases: -array types bucket by the text after the first `[` rather than the text -before a `<`, and `operator()` rules are cut at the operator's own +The bucket keys described in [Loading and Matching](./loading.md#matching) have +two special cases: array types bucket by the text after the first `[` rather +than the text before a `<`, and `operator()` rules are cut at the operator's own parentheses, so their key ends at `...::operator`. ## Instantiating the target Captures are C++ spellings. Before being substituted into the rule's Rust -fragments, each capture is itself mapped through the type rules, -recursively, so `T1 = std::vector` substitutes as `Vec`. A -captured type with no type rule of its own is an error. +fragments, each capture is itself mapped through the type rules, recursively, so +`T1 = std::vector` substitutes as `Vec`. A captured type with no type +rule of its own is an error. diff --git a/docs/src/rules/overview.md b/docs/src/rules/overview.md index dd321667..6e3d008e 100644 --- a/docs/src/rules/overview.md +++ b/docs/src/rules/overview.md @@ -1,16 +1,16 @@ # Overview -Translation rules describe how C++ library APIs are mapped to Rust. -Each rule module lives in the `rules/` directory and pairs a C++ source file -(`src.cpp`) with its Rust translation for each model (`tgt_unsafe.rs` and +Translation rules describe how C++ library APIs are mapped to Rust. Each rule +module lives in the `rules/` directory and pairs a C++ source file (`src.cpp`) +with its Rust translation for each model (`tgt_unsafe.rs` and `tgt_refcount.rs`). -Every rule is expressed as ordinary, compilable C++ and Rust source code, -free of anything platform dependent. Both sides are run through real -compilers at build time, so a rule that does not compile fails the build, and -the platform-specific spellings (for example `bool` canonicalizing to -`_Bool`) are derived by the compiler on the host rather than written by hand. -The same rule sources work on every platform `cpp2rust` builds on. +Every rule is expressed as ordinary, compilable C++ and Rust source code, free +of anything platform dependent. Both sides are run through real compilers at +build time, so a rule that does not compile fails the build, and the +platform-specific spellings (for example `bool` canonicalizing to `_Bool`) are +derived by the compiler on the host rather than written by hand. The same rule +sources work on every platform `cpp2rust` builds on. Rules go through a build-time compilation pipeline before `cpp2rust` can use them: @@ -18,29 +18,28 @@ them: 1. You author a rule module: C++ patterns in `src.cpp` and Rust targets in `tgt_unsafe.rs` / `tgt_refcount.rs`. 2. At build time, two preprocessors compile the module into Rules IR under - `/rules//`: - `cpp-rule-preprocessor` compiles the C++ side into `ir_src.json`, and - `rule-preprocessor` compiles the Rust side into `ir_unsafe.json` and - `ir_refcount.json`. + `/rules//`: `cpp-rule-preprocessor` compiles the C++ side into + `ir_src.json`, and `rule-preprocessor` compiles the Rust side into + `ir_unsafe.json` and `ir_refcount.json`. 3. At startup, `cpp2rust` loads the Rules IR files and indexes the rules by the canonical signature of the C++ construct they match. The rest of this part covers each stage: -* [Rule Format](./format.md): the files that make up a rule module and how - the two models are layered. -* [Writing Rules](./writing-rules.md): how to write rules for functions, +- [Rule Format](./format.md): the files that make up a rule module and how the + two models are layered. +- [Writing Rules](./writing-rules.md): how to write rules for functions, methods, operators, types, constants, and variadics. -* [Compat Shims](./compat.md): how macro-based libc APIs like `errno` and +- [Compat Shims](./compat.md): how macro-based libc APIs like `errno` and `FD_SET` are rewritten into matchable function calls. -* [Conventions](./conventions.md): naming and style conventions rule authors +- [Conventions](./conventions.md): naming and style conventions rule authors must follow. -* [The Rule Preprocessors](./preprocessors.md): the two build-time tools that +- [The Rule Preprocessors](./preprocessors.md): the two build-time tools that compile rules to the Rules IR. -* [The Rules IR](./ir.md): the JSON format the preprocessors emit. -* [Loading and Matching](./loading.md): how `cpp2rust` loads the Rules IR and +- [The Rules IR](./ir.md): the JSON format the preprocessors emit. +- [Loading and Matching](./loading.md): how `cpp2rust` loads the Rules IR and matches rules against the input AST. -* [The Matching Engine](./matching.md): how a candidate rule's signature - is unified against the input. -* [Rule Rewriting](./rewriting.md): how rule bodies are adapted at - application time, in particular the `with_mut` rewrite. +- [The Matching Engine](./matching.md): how a candidate rule's signature is + unified against the input. +- [Rule Rewriting](./rewriting.md): how rule bodies are adapted at application + time, in particular the `with_mut` rewrite. diff --git a/docs/src/rules/preprocessors.md b/docs/src/rules/preprocessors.md index ad18ed3c..ffff711b 100644 --- a/docs/src/rules/preprocessors.md +++ b/docs/src/rules/preprocessors.md @@ -3,8 +3,8 @@ Two build-time tools compile rule modules into the [Rules IR](./ir.md) that `cpp2rust` loads at runtime. Both write into `/rules//`: -* `cpp-rule-preprocessor` compiles `src.cpp`/`src.c` into `ir_src.json`. -* `rule-preprocessor` compiles `tgt_unsafe.rs`/`tgt_refcount.rs` into +- `cpp-rule-preprocessor` compiles `src.cpp`/`src.c` into `ir_src.json`. +- `rule-preprocessor` compiles `tgt_unsafe.rs`/`tgt_refcount.rs` into `ir_unsafe.json`/`ir_refcount.json`. The C++ side is keyed by resolved callee signatures; the Rust side by rule @@ -19,49 +19,46 @@ once per rule directory: cpp-rule-preprocessor --dir rules/string --out /rules/string/ir_src.json ``` -Extra compiler flags can be passed with repeated `--cxxflags` options, -though CMake, which invokes the tool for every rule module via the -`preprocess-cpp-rules` target, passes none. Note that the parent directory -of `--out` must already exist; CMake creates it before each invocation, so -a manual run must do the same. +Extra compiler flags can be passed with repeated `--cxxflags` options, though +CMake, which invokes the tool for every rule module via the +`preprocess-cpp-rules` target, passes none. Note that the parent directory of +`--out` must already exist; CMake creates it before each invocation, so a manual +run must do the same. Rule sources are always compiled with the fixed flag set from -`cpp2rust/compat/platform_flags.h`, the same one used to parse input -programs (see [Compat Shims](./compat.md)). There is no compilation -database and no `-std=` flag: the language is chosen by clang from the -file extension, and `src.c` is processed before `src.cpp`. +`cpp2rust/compat/platform_flags.h`, the same one used to parse input programs +(see [Compat Shims](./compat.md)). There is no compilation database and no +`-std=` flag: the language is chosen by clang from the file extension, and +`src.c` is processed before `src.cpp`. For each rule it: 1. Validates that every `fN` body is exactly one `return` statement. -2. Resolves the *callee* of the returned expression. For non-template rules - this is just the called declaration. For template rules the callee is - unresolved, so the tool instantiates the rule's template parameters with - synthesized dummy types and runs overload resolution to find the function - the rule refers to. +2. Resolves the _callee_ of the returned expression. For non-template rules this + is just the called declaration. For template rules the callee is unresolved, + so the tool instantiates the rule's template parameters with synthesized + dummy types and runs overload resolution to find the function the rule refers + to. 3. Prints the resolved declaration as a canonical signature string: ` ([, ...])[ const][ volatile][ &|&&]`, - where `, ...` appears for C-variadic functions and the trailing - qualifiers only for methods. For `tN` aliases it prints the underlying - type. - -The output is a flat JSON object mapping rule names to these signature -strings. - -The printer preserves typedef sugar instead of desugaring it: `size_t` -prints as `size_t`, not `unsigned long`, which is what lets it map to -`usize` while plain `unsigned long` maps to `u64` (for `tN` aliases this -preservation is explicit; inside function signatures the spelling survives -through the printing policy). Integer literals expanded from a macro are -recorded as the macro *name*, which is how -[constant rules](./writing-rules.md#enum-values-constants-and-macros) like the -`O_CREAT` one match by name. + where `, ...` appears for C-variadic functions and the trailing qualifiers + only for methods. For `tN` aliases it prints the underlying type. + +The output is a flat JSON object mapping rule names to these signature strings. + +The printer preserves typedef sugar instead of desugaring it: `size_t` prints as +`size_t`, not `unsigned long`, which is what lets it map to `usize` while plain +`unsigned long` maps to `u64` (for `tN` aliases this preservation is explicit; +inside function signatures the spelling survives through the printing policy). +Integer literals expanded from a macro are recorded as the macro _name_, which +is how [constant rules](./writing-rules.md#enum-values-constants-and-macros) +like the `O_CREAT` one match by name. ## rule-preprocessor A Rust binary crate built with the nightly toolchain because it links the -compiler's own libraries (`rustc_driver`, `rustc_middle`, ...). It processes -the whole rules tree in one invocation: +compiler's own libraries (`rustc_driver`, `rustc_middle`, ...). It processes the +whole rules tree in one invocation: ```bash CARGO_TARGET_DIR= cargo +nightly run --release \ @@ -70,40 +67,37 @@ CARGO_TARGET_DIR= cargo +nightly run --release \ The environment is load-bearing: -* `CARGO_TARGET_DIR` must be set (the tool aborts otherwise): the rlibs of - the rule dependencies (`libcc2rs`, `libc`, `nix`, ...) are looked up in - `$CARGO_TARGET_DIR//deps`, which the `cargo run` above - populates. The crate list is hardcoded, so a new dependency in - `rules/Cargo.toml` also needs an entry in - `rule-preprocessor/src/semantic.rs`. - - > [!WARNING] - > Stale rlibs from an earlier build can be picked up silently. Run `ninja - > clean` to fix this. -* The sysroot comes from running `rustc --print=sysroot`, so the `rustc` on +- `CARGO_TARGET_DIR` must be set (the tool aborts otherwise): the rlibs of the + rule dependencies (`libcc2rs`, `libc`, `nix`, ...) are looked up in + `$CARGO_TARGET_DIR//deps`, which the `cargo run` above populates. The + crate list is hardcoded, so a new dependency in `rules/Cargo.toml` also needs + an entry in `rule-preprocessor/src/semantic.rs`. + + > [!WARNING] Stale rlibs from an earlier build can be picked up silently. Run + > `ninja clean` to fix this. + +- The sysroot comes from running `rustc --print=sysroot`, so the `rustc` on `PATH` must be the same nightly the preprocessor was built with (running through `cargo +nightly run` guarantees this). -* `rules-dir` is optional and defaults to the relative path `../rules`, - resolved against the *current working directory* of the process. +- `rules-dir` is optional and defaults to the relative path `../rules`, resolved + against the _current working directory_ of the process. -CMake drives all of this via the `preprocess-rust-rules` target: it first -builds the `rules` crate with the stable toolchain (which also regenerates +CMake drives all of this via the `preprocess-rust-rules` target: it first builds +the `rules` crate with the stable toolchain (which also regenerates `rules/src/modules.rs`), then runs the preprocessor with -`CARGO_TARGET_DIR=/target_preprocessor`. That initial `cargo build` -of the `rules` crate is what actually gates the build on rule bodies -type-checking (see below). The preprocessor works in two phases. +`CARGO_TARGET_DIR=/target_preprocessor`. That initial `cargo build` of +the `rules` crate is what actually gates the build on rule bodies type-checking +(see below). The preprocessor works in two phases. **Phase 1, syntactic.** Each `tgt_*.rs` file is parsed with rust-analyzer's -parser, and functions whose `#[cfg]` does not match the host are dropped. -Every function body is then turned into a list of *fragments*, whose kinds -are described in -[The Rules IR](./ir.md#target-ir-ir_unsafejson--ir_refcountjson). The -fragmentation is mainly concerned with how the rule's arguments are used: -references to parameters and generics become placeholder and generic -fragments, while source text that does not involve an argument is kept -as-is. - -Each placeholder is tagged with an *access*: read, write, or move. Some uses +parser, and functions whose `#[cfg]` does not match the host are dropped. Every +function body is then turned into a list of _fragments_, whose kinds are +described in [The Rules IR](./ir.md#target-ir-ir_unsafejson--ir_refcountjson). +The fragmentation is mainly concerned with how the rule's arguments are used: +references to parameters and generics become placeholder and generic fragments, +while source text that does not involve an argument is kept as-is. + +Each placeholder is tagged with an _access_: read, write, or move. Some uses give the access away syntactically (`&mut a0` is a write); those that do not, typically method-call receivers and arguments, are left as `unknown` for phase 2. This phase also applies the two @@ -111,17 +105,17 @@ phase 2. This phase also applies the two support rule rewriting. **Phase 2, semantic.** The preprocessor compiles the `rules` crate in-process -with `rustc` and walks the typed HIR. This gives it the real signature of -every callee, which resolves the `unknown` accesses: passing to a -`&mut`/`*mut` parameter is a write, to a `&`/`*const` parameter a read, and -to `std::mem::take` a move. For type rules it also records which of the -nine derivable standard traits (`Copy`, `Clone`, `Debug`, `Default`, -`PartialEq`, `Eq`, `PartialOrd`, `Ord`, `Hash`) the mapped type implements. -A placeholder still `unknown` after this phase fails the build. +with `rustc` and walks the typed HIR. This gives it the real signature of every +callee, which resolves the `unknown` accesses: passing to a `&mut`/`*mut` +parameter is a write, to a `&`/`*const` parameter a read, and to +`std::mem::take` a move. For type rules it also records which of the nine +derivable standard traits (`Copy`, `Clone`, `Debug`, `Default`, `PartialEq`, +`Eq`, `PartialOrd`, `Ord`, `Hash`) the mapped type implements. A placeholder +still `unknown` after this phase fails the build. The preprocessor assumes the `rules` crate is buildable, which the earlier -`cargo build` of the crate ensures; errors from the in-process compilation -are therefore only reported as a warning. +`cargo build` of the crate ensures; errors from the in-process compilation are +therefore only reported as a warning. The result is one `ir_.json` per input file, keyed by rule name. The output file name is derived from the input file name (`tgt_unsafe.rs` becomes diff --git a/docs/src/rules/rewriting.md b/docs/src/rules/rewriting.md index 33bba679..7ce8efda 100644 --- a/docs/src/rules/rewriting.md +++ b/docs/src/rules/rewriting.md @@ -3,8 +3,9 @@ A rule body is written against idiomatic Rust types: a rule that mutates a vector declares its parameter as `&mut Vec`. But in the refcount model the call-site argument is usually a `Ptr>`, and a `Ptr` -[cannot produce a long-lived `&mut`](../codegen/pointers.md). Instead of forcing every rule to handle pointers, the code -generator *rewrites* the rule body at application time. +[cannot produce a long-lived `&mut`](../codegen/pointers.md). Instead of forcing +every rule to handle pointers, the code generator _rewrites_ the rule body at +application time. ## The `with_mut` rewrite @@ -16,20 +17,19 @@ impl Ptr { } ``` -which checks the pointer, borrows the pointee mutably, and runs the closure -on it (with an immutable sibling `Ptr::with`). The refcount converter uses it -to bridge the gap; the unsafe converter never rewrites and simply emits -receiver followed by body. The rewrite fires when all three hold: +which checks the pointer, borrows the pointee mutably, and runs the closure on +it (with an immutable sibling `Ptr::with`). The refcount converter uses it to +bridge the gap; the unsafe converter never rewrites and simply emits receiver +followed by body. The rewrite fires when all three hold: -1. The rule body fragment is a method call whose receiver contains a - placeholder (the preprocessor splits every method call into receiver and - body fragments precisely to enable this). If the receiver contains - several placeholders, the first one is used. +1. The rule body fragment is a method call whose receiver contains a placeholder + (the preprocessor splits every method call into receiver and body fragments + precisely to enable this). If the receiver contains several placeholders, the + first one is used. 2. The receiver placeholder's access is write or move, i.e. the method takes - `&mut self` or the rule mutates the parameter. Read access does not need - the rewrite, since a read can go through a - [`StrongPtr`](../codegen/pointers.md) obtained with `Ptr::upgrade`, or - through a `read()` copy. + `&mut self` or the rule mutates the parameter. Read access does not need the + rewrite, since a read can go through a [`StrongPtr`](../codegen/pointers.md) + obtained with `Ptr::upgrade`, or through a `read()` copy. 3. The call-site argument is a pointer, or an expression of reference type (which includes an operator call returning a reference). @@ -39,8 +39,7 @@ The rule's method call `a0.method(...)` is then emitted as ptr.with_mut(|__v: | __v.method(...)) ``` -For example, the `push_back` rule is written as an ordinary `&mut` method -call: +For example, the `push_back` rule is written as an ordinary `&mut` method call: ```rust fn f21(a0: &mut Vec, a1: T1) { ... a0.push(...) } @@ -58,44 +57,42 @@ fails and no closure is emitted; the same rule produces a direct call like `(*v2.borrow_mut()).push(0);`. The rewrite applies to pointer dereferences (`p->push_back(20)`) and to -reference usages (`r.push_back(20)` with `std::vector &r = *p`); both -are [translated as a `Ptr`](../codegen/pointers.md), and that `Ptr` is what +reference usages (`r.push_back(20)` with `std::vector &r = *p`); both are +[translated as a `Ptr`](../codegen/pointers.md), and that `Ptr` is what `with_mut` is called on. -When the pointee is itself a boxed value (`Value`, i.e. -`Rc>`), the closure takes `&mut Value` and an extra borrow is -inserted. This is the case for nested containers: the refcount model -translates `std::vector>` as `Vec>>` so -that each element has interior mutability of its own, and a `Ptr` to an -inner vector therefore points at a `Value>`, not a `Vec`: +When the pointee is itself a boxed value (`Value`, i.e. `Rc>`), +the closure takes `&mut Value` and an extra borrow is inserted. This is the +case for nested containers: the refcount model translates +`std::vector>` as `Vec>>` so that each element +has interior mutability of its own, and a `Ptr` to an inner vector therefore +points at a `Value>`, not a `Vec`: ```rust ptr.with_mut(|__v: &mut Value>| (*__v.borrow_mut()).push(20)) ``` -The closure type is built from the C++ argument's type, not the rule's -declared parameter type. +The closure type is built from the C++ argument's type, not the rule's declared +parameter type. ## The read-access counterpart -For read access the converter does not emit a closure. A pointer receiver -whose rule parameter is a value or `&` type is simply dereferenced -(`p.read()` or `(*p.upgrade().deref())`); conversely, if the rule declares a -`Ptr` parameter but the argument is not a pointer, the converter inserts an -`as_pointer()` cast or -[materializes a temporary](../codegen/temporaries.md). +For read access the converter does not emit a closure. A pointer receiver whose +rule parameter is a value or `&` type is simply dereferenced (`p.read()` or +`(*p.upgrade().deref())`); conversely, if the rule declares a `Ptr` parameter +but the argument is not a pointer, the converter inserts an `as_pointer()` cast +or [materializes a temporary](../codegen/temporaries.md). ## Preprocessor-side rewrites Two rewrites in `rule-preprocessor` exist to make the `with_mut` rewrite possible. Both apply only to `&mut` parameters: -* A `*` deref in front of the parameter is dropped from the body, since - the substituted argument is already an lvalue or pointer expression. -* `std::mem::take(&mut aN)` collapses to a bare placeholder, so the - converter can re-express the move against the actual argument (for a - pointer that becomes `std::mem::take(&mut )` on the borrowed - pointee). The spelling must be exactly this fully qualified form: - `mem::take` or an imported `take` is not rewritten. The collapsed - placeholder's access is left `unknown` in phase 1; phase 2 resolves the - `std::mem::take` call to a move. +- A `*` deref in front of the parameter is dropped from the body, since the + substituted argument is already an lvalue or pointer expression. +- `std::mem::take(&mut aN)` collapses to a bare placeholder, so the converter + can re-express the move against the actual argument (for a pointer that + becomes `std::mem::take(&mut )` on the borrowed pointee). The spelling + must be exactly this fully qualified form: `mem::take` or an imported `take` + is not rewritten. The collapsed placeholder's access is left `unknown` in + phase 1; phase 2 resolves the `std::mem::take` call to a move. diff --git a/docs/src/rules/writing-rules.md b/docs/src/rules/writing-rules.md index 842f43c6..aa49121c 100644 --- a/docs/src/rules/writing-rules.md +++ b/docs/src/rules/writing-rules.md @@ -1,9 +1,9 @@ # Writing Rules -This page shows how to write rules for each kind of C++ construct. In every -case the recipe is the same: write an `fN` (or `tN`) function on the C++ side -whose single `return` statement exercises the construct, and a same-named -function on the Rust side giving the translation. +This page shows how to write rules for each kind of C++ construct. In every case +the recipe is the same: write an `fN` (or `tN`) function on the C++ side whose +single `return` statement exercises the construct, and a same-named function on +the Rust side giving the translation. ## Free functions @@ -30,18 +30,18 @@ fn f1(a0: Ptr, a1: Ptr) -> i32 { } ``` -Rule bodies may be arbitrarily complex; multi-statement bodies are wrapped in -a block when spliced into the output. +Rule bodies may be arbitrarily complex; multi-statement bodies are wrapped in a +block when spliced into the output. -`return` statements are prohibited in Rust rule bodies (the preprocessor -rejects them); produce the result as a tail expression instead. The body is -not emitted as a function of its own: it is spliced inline into the generated -code as a block expression, so a `return` would not end the rule, it would -return from whatever generated function the rule happens to be expanded in. +`return` statements are prohibited in Rust rule bodies (the preprocessor rejects +them); produce the result as a tail expression instead. The body is not emitted +as a function of its own: it is spliced inline into the generated code as a +block expression, so a `return` would not end the rule, it would return from +whatever generated function the rule happens to be expanded in. -When the pattern's type cannot be named, the rule uses an `auto` return -type: `rules/iomanip` writes `auto f1(int n) { return std::setw(n); }` -because `std::setw` returns an unspecified type. +When the pattern's type cannot be named, the rule uses an `auto` return type: +`rules/iomanip` writes `auto f1(int n) { return std::setw(n); }` because +`std::setw` returns an unspecified type. ## Methods @@ -71,10 +71,9 @@ every instantiation: when the input program calls `size()` on, say, a ## Static member functions -A static member function is also written with a receiver parameter, which -exists only to name the class. The call site has no receiver argument, so -the Rust side drops it and numbers the remaining parameters from `a0`; here -there are none: +A static member function is also written with a receiver parameter, which exists +only to name the class. The call site has no receiver argument, so the Rust side +drops it and numbers the remaining parameters from `a0`; here there are none: ```cpp // rules/limits/src.cpp @@ -88,8 +87,7 @@ unsafe fn f1() -> T1 { } ``` -(`HasMinMax` is a helper trait defined alongside the rules in the same -file.) +(`HasMinMax` is a helper trait defined alongside the rules in the same file.) ## Constructors @@ -104,9 +102,9 @@ std::string f9(std::size_t n, char ch) { return std::string(n, ch); } Overloads that differ in value category are distinct rules too: `rules/vector` has separate rules for `push_back(const T1 &)` and `push_back(T1 &&)`. -No destructor rules exist so far: the STL and libc APIs covered by the -current rules have not needed any, since their types map to Rust types whose -`Drop` implementations already do the right thing. +No destructor rules exist so far: the STL and libc APIs covered by the current +rules have not needed any, since their types map to Rust types whose `Drop` +implementations already do the right thing. ## Operators @@ -126,18 +124,18 @@ bool f11(typename std::map::iterator a, ``` Post-increment is distinguished from pre-increment by the usual dummy `int` -parameter: `a0.operator++(a1)` versus `it.operator++()`. Conversion -operators use the same explicit syntax: `a0.operator T1 &()` in -`rules/functional` matches the conversion of a `std::reference_wrapper` -back to a reference. Field accesses are rules of their own, matched by the -field: `it->first` and `it->second` through iterators, plain `o.second` on -a pair (`rules/map`, `rules/pair`). +parameter: `a0.operator++(a1)` versus `it.operator++()`. Conversion operators +use the same explicit syntax: `a0.operator T1 &()` in `rules/functional` matches +the conversion of a `std::reference_wrapper` back to a reference. Field +accesses are rules of their own, matched by the field: `it->first` and +`it->second` through iterators, plain `o.second` on a pair (`rules/map`, +`rules/pair`). ## Callable arguments -A rule parameter may be a callable. Function pointers are spelled directly; -for a lambda, whose type cannot be written, the rule declares a file-scope -lambda and takes `decltype(lambda)`: +A rule parameter may be a callable. Function pointers are spelled directly; for +a lambda, whose type cannot be written, the rule declares a file-scope lambda +and takes `decltype(lambda)`: ```cpp // rules/algorithm/src.cpp @@ -155,20 +153,20 @@ where { ... } ``` -`T1` and `T2` are not template parameters here but file-scope helper -structs modelling an iterator and its value type; being named like -generics, they bind as `T1`/`T2` at the use site. The function pointer -version of the comparator is a separate rule (`f7`). +`T1` and `T2` are not template parameters here but file-scope helper structs +modelling an iterator and its value type; being named like generics, they bind +as `T1`/`T2` at the use site. The function pointer version of the comparator is +a separate rule (`f7`). ## Iterators -There is no iterator abstraction: an iterator type gets a type rule, and -every operation on it its own expression rule (`operator*`, `operator++`, +There is no iterator abstraction: an iterator type gets a type rule, and every +operation on it its own expression rule (`operator*`, `operator++`, `operator!=`, ...). What the type maps to is up to the rule: -`std::string::iterator` becomes a plain pointer (`*mut libc::c_char` -unsafe, `Ptr` refcount), while `std::map` iterators become the runtime -types `libcc2rs::UnsafeMapIterator`/`MapIterator`. Dependent iterator -types are named with `typename`: +`std::string::iterator` becomes a plain pointer (`*mut libc::c_char` unsafe, +`Ptr` refcount), while `std::map` iterators become the runtime types +`libcc2rs::UnsafeMapIterator`/`MapIterator`. Dependent iterator types are named +with `typename`: ```cpp // rules/map/src.cpp @@ -178,12 +176,12 @@ using t2 = typename std::map::const_iterator; ## Types -A type rule has two halves. On the C++ side, declare a type alias named `tN` -for the C++ type being mapped. On the Rust side, write a function with the -same name that takes no arguments: its *return type* is the Rust type that -the C++ type maps to, and its *body* is the default value the generated code -uses when it needs to construct one (e.g. for an uninitialized variable). -Reference and pointer variants of a type each get their own rule: +A type rule has two halves. On the C++ side, declare a type alias named `tN` for +the C++ type being mapped. On the Rust side, write a function with the same name +that takes no arguments: its _return type_ is the Rust type that the C++ type +maps to, and its _body_ is the default value the generated code uses when it +needs to construct one (e.g. for an uninitialized variable). Reference and +pointer variants of a type each get their own rule: ```cpp // rules/iostream/src.cpp @@ -209,17 +207,17 @@ fn t1() -> ::libc::stat { unsafe { std::mem::zeroed() } } fn t1() -> libcc2rs::Stat { Default::default() } ``` -A type rule may map to the sentinel type `libcc2rs::IgnoreRule`, meaning -"this model has no special mapping for the type"; the converter then falls -back to its normal type conversion. This is useful when only one model needs -a custom mapping: `rules/carray` maps multi-dimensional C arrays to nested -boxed slices in the refcount model, while its `tgt_unsafe.rs` targets are -`IgnoreRule` so the unsafe model keeps the default array conversion. +A type rule may map to the sentinel type `libcc2rs::IgnoreRule`, meaning "this +model has no special mapping for the type"; the converter then falls back to its +normal type conversion. This is useful when only one model needs a custom +mapping: `rules/carray` maps multi-dimensional C arrays to nested boxed slices +in the refcount model, while its `tgt_unsafe.rs` targets are `IgnoreRule` so the +unsafe model keeps the default array conversion. ## Enum values, constants, and macros -Constants are `fN` functions that take no arguments and return the constant, -one rule per value: +Constants are `fN` functions that take no arguments and return the constant, one +rule per value: ```cpp // rules/fcntl/src.cpp @@ -232,17 +230,16 @@ int f4(void) { return O_TRUNC; } unsafe fn f3() -> i32 { ::libc::O_CREAT } ``` -For macros that expand to integer literals, the preprocessor records the -*macro name* rather than the value, so `O_CREAT` in the input matches this -rule by name. Enum constants and global variables (e.g. `std::cout`) are -matched by their qualified name. A global and its address are separate -rules: `rules/iostream` maps both `std::cout` (`f1`) and `&std::cout` -(`f3`). +For macros that expand to integer literals, the preprocessor records the _macro +name_ rather than the value, so `O_CREAT` in the input matches this rule by +name. Enum constants and global variables (e.g. `std::cout`) are matched by +their qualified name. A global and its address are separate rules: +`rules/iostream` maps both `std::cout` (`f1`) and `&std::cout` (`f3`). Integer-literal macros are the only macros matchable directly. Macros whose expansions are platform internals with no stable callee, such as `errno` or -`FD_SET`, are first rewritten into calls to synthetic `cpp2rust_*` functions -by the [compat shims](./compat.md); rules then match the shim call. +`FD_SET`, are first rewritten into calls to synthetic `cpp2rust_*` functions by +the [compat shims](./compat.md); rules then match the shim call. ## Variadic functions @@ -254,10 +251,9 @@ variadic arguments to another call, so a rule like int f1(int a0, int a1, ...) { return fcntl(a0, a1, ...); } ``` -is not -expressible. A parameter pack can be forwarded (`args...`), which is exactly -what the rule body needs to do. The Rust side takes a trailing parameter that -must be typed `&[VaArg]` and named `va`: +is not expressible. A parameter pack can be forwarded (`args...`), which is +exactly what the rule body needs to do. The Rust side takes a trailing parameter +that must be typed `&[VaArg]` and named `va`: ```cpp // rules/fcntl/src.cpp @@ -272,14 +268,13 @@ int f1(int a0, int a1, Args... args) { fn f1(a0: i32, a1: i32, va: &[VaArg]) -> i32 { ... } ``` -Bodies read the arguments through the va-args API in `libcc2rs` -(`VaArg`, `VaList`, the `VaArgGet` accessors, `format_c`). +Bodies read the arguments through the va-args API in `libcc2rs` (`VaArg`, +`VaList`, the `VaArgGet` accessors, `format_c`). ## Passthrough rules When a call should be forwarded verbatim to the same-named function in Rust's -`libc` crate, the Rust target can be an `extern` declaration instead of a -body: +`libc` crate, the Rust target can be an `extern` declaration instead of a body: ```cpp // rules/fcntl/src.cpp @@ -319,14 +314,13 @@ unsafe fn f4() -> i32 { ``` The Rust preprocessor evaluates `#[cfg]` attributes against the host target -(only `target_os = linux|macos` and `target_arch = x86_64|x86` are accepted) -and drops non-matching rules. +(only `target_os = linux|macos` and `target_arch = x86_64|x86` are accepted) and +drops non-matching rules. -Mutually exclusive platform branches use `#elif` with disjoint rule -numbers: `rules/errno` defines `f91` to `f135` under `__linux__` and -`f136` to `f153` under `__APPLE__`. Feature-test macros a pattern needs must -come before the includes, as with `#define _GNU_SOURCE` in -`rules/socket/src.c`. +Mutually exclusive platform branches use `#elif` with disjoint rule numbers: +`rules/errno` defines `f91` to `f135` under `__linux__` and `f136` to `f153` +under `__APPLE__`. Feature-test macros a pattern needs must come before the +includes, as with `#define _GNU_SOURCE` in `rules/socket/src.c`. ## Pattern resolution limits @@ -334,11 +328,11 @@ The preprocessor resolves a template pattern by instantiating its template parameters with synthesized types ([The Rule Preprocessors](./preprocessors.md#cpp-rule-preprocessor)): -* A bare `T1` becomes an empty struct, so the pattern cannot use members, +- A bare `T1` becomes an empty struct, so the pattern cannot use members, operators, or nested types of `T1`. -* A parameter pack instantiates to the empty pack. -* A non-type parameter is pinned to the value `1`. +- A parameter pack instantiates to the empty pack. +- A non-type parameter is pinned to the value `1`. -Unqualified callees are looked up in `namespace std` first and in the -global scope only when `std` has no match, so an unqualified name that -exists in both resolves to the `std` one. +Unqualified callees are looked up in `namespace std` first and in the global +scope only when `std` has no match, so an unqualified name that exists in both +resolves to the `std` one. From b21118a0dc4af3382cdc3d341e37a820670e2d01 Mon Sep 17 00:00:00 2001 From: Lucian Popescu Date: Fri, 14 Aug 2026 12:39:30 +0100 Subject: [PATCH 12/14] Add plugins TODO in separate file --- docs/src/SUMMARY.md | 1 + docs/src/codegen/overview.md | 4 ---- docs/src/codegen/plugins.md | 3 +++ 3 files changed, 4 insertions(+), 4 deletions(-) create mode 100644 docs/src/codegen/plugins.md diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md index b935edd1..e11a8166 100644 --- a/docs/src/SUMMARY.md +++ b/docs/src/SUMMARY.md @@ -25,3 +25,4 @@ - [Overview](./codegen/overview.md) - [Pointers and References](./codegen/pointers.md) - [Temporary Materialization](./codegen/temporaries.md) +- [Translation Plugins](./codegen/plugins.md) diff --git a/docs/src/codegen/overview.md b/docs/src/codegen/overview.md index 908d56c5..fb3916d3 100644 --- a/docs/src/codegen/overview.md +++ b/docs/src/codegen/overview.md @@ -2,7 +2,3 @@ This part of the book documents the internals of the code generator: how the clang AST is traversed and how Rust code is emitted. - -TODO: document the converter plugin mechanism (`cpp2rust/converter/plugins/`), -which intercepts constructs ahead of the translation rules (currently -`emplace_back`). diff --git a/docs/src/codegen/plugins.md b/docs/src/codegen/plugins.md new file mode 100644 index 00000000..dbecbf3b --- /dev/null +++ b/docs/src/codegen/plugins.md @@ -0,0 +1,3 @@ +> TODO: document the converter plugin mechanism (`cpp2rust/converter/plugins/`), +> which intercepts constructs ahead of the translation rules (currently +> `emplace_back`). From 2a513400d45ac403d0bd79ee3603461cb912383b Mon Sep 17 00:00:00 2001 From: Lucian Popescu Date: Fri, 14 Aug 2026 12:45:46 +0100 Subject: [PATCH 13/14] Add markdown formatting check --- .github/workflows/format.yml | 3 +++ .prettierrc | 4 ++++ 2 files changed, 7 insertions(+) create mode 100644 .prettierrc diff --git a/.github/workflows/format.yml b/.github/workflows/format.yml index ea243a85..5c6a93de 100644 --- a/.github/workflows/format.yml +++ b/.github/workflows/format.yml @@ -37,3 +37,6 @@ jobs: - name: Check format run: git diff --exit-code + + - name: Check markdown formatting + run: npx prettier@3.6.2 --check "docs/src/**/*.md" diff --git a/.prettierrc b/.prettierrc new file mode 100644 index 00000000..cfae7e2e --- /dev/null +++ b/.prettierrc @@ -0,0 +1,4 @@ +{ + "proseWrap": "always", + "printWidth": 80 +} From 2bee3ffb9962815e0d21a7b97b366f4da29c2186 Mon Sep 17 00:00:00 2001 From: Lucian Popescu Date: Fri, 14 Aug 2026 15:43:56 +0100 Subject: [PATCH 14/14] Move prettierrc in docs --- .github/workflows/format.yml | 3 ++- .prettierrc => docs/.prettierrc | 0 2 files changed, 2 insertions(+), 1 deletion(-) rename .prettierrc => docs/.prettierrc (100%) diff --git a/.github/workflows/format.yml b/.github/workflows/format.yml index 5c6a93de..361d065e 100644 --- a/.github/workflows/format.yml +++ b/.github/workflows/format.yml @@ -39,4 +39,5 @@ jobs: run: git diff --exit-code - name: Check markdown formatting - run: npx prettier@3.6.2 --check "docs/src/**/*.md" + run: npx prettier@3.6.2 --check "src/**/*.md" + working-directory: docs diff --git a/.prettierrc b/docs/.prettierrc similarity index 100% rename from .prettierrc rename to docs/.prettierrc