Skip to content

internal TensorRT support - #56

Merged
liqyn merged 9 commits into
mainfrom
feature/tensorrt
Oct 8, 2026
Merged

liqyn merged 9 commits into
mainfrom
feature/tensorrt

Conversation

@liqyn

@liqyn liqyn commented Sep 25, 2026 •

Copy link
Copy Markdown
Member
  • Add optional support for running with onboard optimized TensorRT models
    • Detection model changed YOLOv7 -> YOLOv8 for ease of conversion
    • Most additional code are wrappers for converting models to TensorRT (with their own pre- & post-processing)
  • Fix segment centroid distance check in merging logic
  • Speedup mask and deprojection computation with GPU parallelization


pcd = o3d.geometry.PointCloud()
pcd.points = o3d.utility.Vector3dVector(inside_mask_points)
pcd.points = o3d.utility.Vector3dVector(ptcld_in_range)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't know how we missed this before, but we seemed to have forgotten to truncate point cloud points beyond depth range...

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Earlier there were two checks - one that checked that a certain percentage of the object was within range (which is what this comment points to) and then later I actually recreate the pointcloud using the max_depth

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess you still have that in the changes, but the refactoring of the ptcld_test meant that this needed to be switched from inside_mask_points to ptcld_in_range, like yuou did.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was true for the depth-generated point clouds, but I think the lidar point cloud path always lacked that step.

@liqyn

liqyn commented Oct 7, 2026 •

Copy link
Copy Markdown
Member Author
  • Added yaw-only refit on force_rm_lc_roll_pitch
  • New params are in corresponding branch in roman_params

New benchmarks:

method 0-60 60-120 120-180 mean all PR AUC PlaceRec AUC ms/reg mapping total wall
MASt3R (GT scale), paper .775 .152 .297 .408 - - - 2950 - -
ROMAN, paper .521 .152 .189 .287 - - .552 28.9 9.6 Hz -
ROMAN-L, paper .723 .370 .432 .508 - - .663 92.9 9.6 Hz -
ROMAN-XL, paper .745 .457 .405 .536 - - .654 213 9.6 Hz -
main stock (no force_rm) .602 .245 .297 .381 .539 .287 .607 4.6 9.8 Hz 78.5 min
main stock L (no force_rm) .757 .429 .486 .557 .700 .365 .711 27.8 9.8 Hz 130.4 min
main stock XL (no force_rm) .777 .490 .541 .603 .726 .314 .702 77.6 9.8 Hz 240.7 min
torch32 best .711 .347 .378 .479 .645 .332 .640 4.9 20.3 Hz 48.9 min
torch32 best L .796 .408 .486 .563 .728 .410 .745 31.6 20.3 Hz 106.8 min
torch32 best XL .815 .531 .568 .638 .764 .351 .740 87.3 20.3 Hz 227.8 min
torch16 best .676 .286 .378 .447 .609 .314 .617 5.1 22.7 Hz 53.2 min*
torch16 best L .779 .388 .459 .542 .711 .398 .744 33.0 22.7 Hz 113.8 min*
torch16 best XL .815 .449 .568 .611 .755 .350 .759 87.9 22.7 Hz 233.0 min*
trt16 best .649 .265 .405 .440 .587 .307 .647 4.8 35.1 Hz 36.5 min
trt16 best L .809 .408 .568 .595 .746 .401 .751 31.5 35.1 Hz 94.9 min
trt16 best XL .820 .531 .622 .658 .773 .340 .760 86.6 35.1 Hz 214.6 min

* torch16 mapped from external SSD over USB; total wall not directly comparable.

@liqyn

liqyn commented Oct 7, 2026

Copy link
Copy Markdown
Member Author

Demo tested; <1.25m ATE and >=12 inliers with/without TensorRT

Comment thread roman/align/object_registration.py Outdated
return solutions

def T_align(self, map1: List[Object], map2: List[Object], correspondences: np.array = None):
def T_align(self, map1: List[Object], map2: List[Object], correspondences: np.array = None, yaw_only: bool = False):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gonna be a bit pedantic and rename this to xyz_yaw_only

Comment thread roman/params/offline_rpgo_params.py Outdated
Comment on lines +28 to +29
lc_t_std: float = 4.0
lc_r_std: float = np.deg2rad(10.0)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's a great idea mass changing a bunch of core algorithm defaults in the code. Smaller things like min_max_extent and max_depth I'm iffy on, but I think that we should keep these two defaults and epsilon the same as the paper reports.

Comment thread README.md Outdated
Comment on lines +98 to +110
### TensorRT

`params/demo_trt` runs the same demo with FastSAM, YOLOv8 and DINOv2 on
TensorRT instead of PyTorch:

```
pip install '.[tensorrt]'
python3 demo/demo.py -p params/demo_trt -o demo_output_trt
```

The engines are exported and compiled on first use and cached in
`$ROMAN_WEIGHTS`.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we make this it's own discussion outside of the demo with a link from the installation instructions

* keep core loop closure covariances and epsilon params same as paper
* reference tensorrt in installation instructions, and give it its own
  section in the readme
* rename variable in T_align from yaw_only to xyz_yaw_only

@mbpeterson70 mbpeterson70 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@liqyn I went ahead and made some minor changes based on my comments above - if those changes look okay to you, I think it's good to be merged in!

@liqyn
liqyn merged commit d3a2834 into main Oct 8, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants