Skip to content

Fix Rust codegen matrix return types - #467

Closed
tandede wants to merge 1 commit into
symforce-org:mainfrom
tandede:fix/rust-matrix-return-shape
Closed

Fix Rust codegen matrix return types#467
tandede wants to merge 1 commit into
symforce-org:mainfrom
tandede:fix/rust-matrix-return-shape

Conversation

@tandede

@tandede tandede commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Problem

Rust codegen formats every matrix returned directly from a generated function as nalgebra::SVector<scalar, rows * cols>. This loses the original two-dimensional shape. For example, a function returning Matrix23 is currently generated with an SVector<f64, 6> signature and constructor.

The multi-output path already handles this correctly through format_typename, which selects SVector only when either matrix dimension is one and otherwise selects SMatrix.

Approach

Reuse the same shape-aware formatter in both places used by the single-return path:

  • the generated function return type;
  • the constructor used for the returned value.

This keeps the single-return and multi-output paths consistent without duplicating their vector-versus-matrix condition. Scalar returns and vector-shaped matrices retain their existing behavior.

Regression coverage

The Rust codegen test now includes a non-square 2×3 matrix return. A non-square shape verifies both dimensions explicitly instead of checking only the flattened element count.

Before this change, the generated code used:

nalgebra::SVector<f64, 6>

It now uses:

nalgebra::SMatrix<f64, 2, 3>

The generated crate still includes the existing 3×1 return case, which remains an SVector<f64, 3>.

Validation

  • Rust codegen test with the SymEngine backend
  • Rust codegen test with the SymPy backend
  • Cargo build of both generated test-data crates
  • Ruff 0.9.10 lint and formatting checks
  • git diff --check

Fixes #439

Rust codegen flattened every directly returned matrix into an SVector, even when both dimensions were greater than one. Reuse the existing shape-aware type formatter for both the return signature and constructor so single-output code follows the same rules as output arguments.

Add a non-square Matrix23 regression to both symbolic backends and compile the generated Rust crate. This preserves vector returns while proving that true matrices retain their row and column dimensions.

Signed-off-by: tandede <1090179959@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Rust codegen uses wrong dimensions for return types

2 participants