GeoBridge: Evolving Transferable Decoder Topologies for Geometric Vision

Rui Wang¹ · Haoyu Zhang¹ · Ran Cheng^2
1Hangzhou Normal University · ²The Hong Kong Polytechnic University

Evolve once. Reuse decoder topologies across geometric vision systems.

Novel view synthesis Botanical garden · GeoBridgeNet-A3	Open-vocabulary segmentation in 3D Teatime · GeoBridgeNet-A3 + COS3D · “sheep”
Indoor depth estimation Our meeting room · GeoBridgeNet-A3	Outdoor depth estimation KITTI street · GeoBridgeNet-A3

_{GeoBridgeNet-A3 results · Click an animation for the full-quality video · Explore the 3D scenes}

Method · Results · Install · Depth · Novel views · 3D segmentation · Train & search

Overview

GeoBridge is an evolutionary framework for transferable decoder topologies. It searches the internal computation of decoder stages within a fixed multi-scale scaffold, then reuses the selected structures in monocular depth estimation and Gaussian novel view synthesis. The reconstructed scenes also support open-vocabulary 3D segmentation through COS3D.

• Composable topology: residual, sequential, and parallel operators compose configurable primitives while preserving each stage's input and output interfaces.
• Evolution without candidate training: functional trees support stage-aware crossover and hierarchical mutation. NSGA-II balances the gradient sensitivity-alignment proxy (GSAP) with computation and parameter cost.
• Transfer across systems: one source search on TartanAir produces GeoBridgeNet-A1, GeoBridgeNet-A2, and GeoBridgeNet-A3. Each target system retains the selected topology and primitive configurations, and trains its own weights and prediction interface.

Naming and compatibility. GeoBridge is the framework; GeoBridgeNet is the resulting model family. Code and model weights are hosted in the GeoBridge-NAS repositories. Earlier releases used DDN; the ddn Python package and legacy command-line identifiers retain that name for compatibility. In configurations and commands, A1, A2, and A3 denote the corresponding GeoBridgeNet variants.

Main results

Selected main-experiment results from the manuscript. Columns denote the three GeoBridgeNet variants; bold marks the best value for each metric.

Benchmark	Metric	A1	A2	A3
NYU Depth V2	AbsRel ↓	0.0918	0.0895	0.0876
KITTI	AbsRel ↓	0.0582	0.0561	0.0555
RealEstate10K	PSNR ↑	27.02	27.07	27.11
LERF · COS3D	mIoU (%) ↑	51.06	49.77	50.54
LERF · COS3D	mAcc (%) ↑	70.58	67.87	71.32

LERF uses GeoBridgeNet initializers with COS3D refinement; scores are the unweighted mean over four scenes, with mAcc reported as Acc@0.25.

Qualitative comparisons from the manuscript

NYU Depth V2

KITTI

RealEstate10K

COS3D

Model weights

Download the public checkpoints from 🤗 kujimili/GeoBridge-NAS.

python -m pip install -U huggingface_hub
hf download kujimili/GeoBridge-NAS --local-dir ./weights/public

For a single model, add its filename after the repository ID, for example nyu/a3.pth.

Task	Architectures	Checkpoint layout
NYU / KITTI depth	A1, A2, A3	{nyu,kitti}/a{1,2,3}.pth
RealEstate10K novel views	A1, A2, A3	re10k/a{1,2,3}.ckpt
COS3D scene grounding	A1, A2, A3 × four scenes	cos3d/a{1,2,3}/{scene}/

COS3D scenes: Figurines · Teatime · Ramen · Waldo Kitchen. See the scene-weight layout.

Environment

The core environment uses Python 3.10 · PyTorch 2.6.0 · CUDA 12.4. Clone the repository, then run commands from its root.

git clone https://github.com/EMI-Group/GeoBridge-NAS.git
cd GeoBridge-NAS
python -m pip install torch==2.6.0 torchvision==0.21.0 --index-url https://download.pytorch.org/whl/cu124
python -m pip install -r requirements.txt

Depth checkpoints include the student encoder, so evaluation needs no initialization download. Training and search use DINOv2 ViT-S initialization via --dinov2_ckpt; the encoder downloads it when absent. Use separate environments for NVS and COS3D.

Depth evaluation

Choose a genotype from configs/architectures.json and its matching checkpoint. The dataset split files contain relative image/depth paths.

export WEIGHTS="$PWD/weights/public"
export NYU_ROOT=/path/to/nyu
export KITTI_ROOT=/path/to/kitti
ARCH=$(python -c 'import json; print(json.load(open("configs/architectures.json"))["A3"])')

# NYU Depth V2
python scripts/evaluate_depth.py \
  --ckpt "$WEIGHTS/nyu/a3.pth" --arch "$ARCH" \
  --dataset nyu --data_root "$NYU_ROOT" --post_process_eval \
  --filenames_file_eval splits/nyu/nyudepthv2_test_files_with_gt.txt

# KITTI
python scripts/evaluate_depth.py \
  --ckpt "$WEIGHTS/kitti/a3.pth" --arch "$ARCH" \
  --dataset kitti --data_root "$KITTI_ROOT" \
  --gt_root "$KITTI_ROOT/data_depth_annotated" \
  --filenames_file_eval splits/kitti/kitti_eigen_test.txt

For A1/A2, change both the architecture key and checkpoint filename. Parameters load strictly. Add --output metrics.json to save results and protocol; FP32 is the default, and --amp enables CUDA float16 autocast.

Depth evaluation protocol

Setting	NYU Depth V2	KITTI
Input height × width	480 × 640	352 × 1216, KB crop
Depth scale	1000	256
Maximum depth	10 m	80 m
Evaluation crop	No additional spatial crop	Garg crop
Post-processing	Horizontal-flip post-processing	Horizontal-flip post-processing

RealEstate10K

In a separate NVS environment, install the core dependencies above, then add DepthSplat dependencies and evaluate:

python -m pip install -r third_party/depthsplat/requirements.txt
python scripts/run_re10k.py --mode eval --variant A3 \
  --data-root /path/to/re10k \
  --checkpoint "$WEIGHTS/re10k/a3.ckpt" --output-dir outputs/re10k_a3

Use the preprocessed DepthSplat/MVSplat chunk format. Evaluation uses the bundled evaluation_index_re10k.json and a default chunk interval of 10, matching the released evaluation setup. Set --test-chunk-interval 1 to evaluate all available indexed chunks.

Novel-view training and depth pretraining

run_re10k.py --mode train additionally requires --student-checkpoint and --gmflow-checkpoint. Set the training budget with --batch-size, --accumulate and --steps.

The main pretraining pipeline uses scripts/train_re10k_depth.py, scripts/prepare_pseudo_depth.py, scripts/prepare_multiview_depth.py and scripts/pretrain_student.py for monocular pretraining and pseudo-depth preparation.

COS3D

The Gaussian reconstruction system with a GeoBridgeNet decoder initializes the scene, and COS3D refines its RGB and semantic representation. The public bundle includes A1/A2/A3 for all four LERF scenes.

Follow the COS3D guide for its separate environment, scene preparation, RGB refinement, text grounding, rendering and evaluation.

Interactive demos

The project page pairs two novel-view reconstructions and two semantic Gaussian scenes. Rotate, pan and zoom each scene independently; semantic viewers start with Original scene and offer text-query highlighting and object isolation.

To preview the page locally, run from the repository root:

python -m http.server 8000 --directory docs

Open http://localhost:8000 in a browser. The viewer and scene assets are bundled locally. Text controls select saved query predictions; all Gaussians belonging to those predicted objects are retained, with sampled backgrounds for web display.

Full videos: Meeting room · Living room depth · Living room novel views · Garden novel views.

Workflow

Released-model evaluation can start directly from the commands above. To reproduce the pipeline from decoder search, follow this order:

Step	Stage	Entry point
1	Prepare search splits	scripts/prepare_splits.py
2	Cache pretrained teacher features	scripts/cache_features.py
3	Search decoder architectures	scripts/search.py
4	Train the selected depth model	scripts/train_depth.py
5	Evaluate NYU / KITTI	scripts/evaluate_depth.py
6	Transfer to novel view synthesis	scripts/run_re10k.py
7	Refine and ground scene Gaussians	COS3D guide

Training

Retrain a selected depth architecture

For example, retrain A3 on NYU using the variables from depth evaluation:

python scripts/train_depth.py --arch "$ARCH" \
  --dataset nyu --data_root "$NYU_ROOT" --post_process_eval \
  --filenames_file splits/nyu/nyudepthv2_train_files_with_gt_dense.txt \
  --filenames_file_eval splits/nyu/nyudepthv2_test_files_with_gt.txt \
  --dinov2_ckpt /path/to/dinov2_vits14_pretrain.pth \
  --epochs 50 --batch_size 8 --seed 42 --lambda_distill 0 \
  --output_dir outputs/nyu_a3

Use python -m torch.distributed.run --nproc_per_node=N in place of python for distributed training. --batch_size is per GPU; --accum_steps controls gradient accumulation. Live teacher distillation requires --boundary_ckpt, --scale_ckpt and a positive --lambda_distill.

Architecture search

Prepare features and search decoder topologies

python scripts/prepare_splits.py --root /path/to/tartanair
python scripts/cache_features.py --data_root /path/to/tartanair \
  --scale_ckpt /path/to/scale_teacher_best.pth \
  --boundary_ckpt /path/to/depth_anything_v2_vitl.pth \
  --cache_dir /path/to/feature_cache

python scripts/search.py --cache_dir /path/to/feature_cache \
  --dinov2_ckpt /path/to/dinov2_vits14_pretrain.pth \
  --output_dir outputs/search --pop_size 64 --n_generations 100 \
  --grad_batch_size 8 --seed 42

The command above and the search defaults use 64 individuals and 100 generations, matching the population size and generation budget in the manuscript.

Search minimizes GSAP, MACs and decoder parameter count. The selected GeoBridgeNet-A1/A2/A3 tree architectures are stored under A1, A2, and A3 in configs/architectures.json; each downstream task has separately trained weights.

Efficiency measurements

Measure depth and novel-view inference cost

Main-table latency uses batch size 1 and FP32. Compare absolute timings on the paper's GPU model; latency varies across hardware.

python scripts/benchmark_depth.py --resolution all --batch_sizes 1 --scope full \
  --save_json outputs/depth_latency.json

# Run in the NVS environment.
python scripts/benchmark_re10k.py --data-root /path/to/re10k \
  --checkpoint "DDN-A1=$WEIGHTS/re10k/a1.ckpt" \
  --checkpoint "DDN-A2=$WEIGHTS/re10k/a2.ckpt" \
  --checkpoint "DDN-A3=$WEIGHTS/re10k/a3.ckpt" \
  --precision fp32 --output-dir outputs/re10k_latency

The NVS benchmark requires all three checkpoints and uses the legacy labels DDN-A1/A2/A3 for the corresponding GeoBridgeNet variants.

Acknowledgements

Our task-transfer pipelines build on DepthSplat and COS3D / Gaussian Splatting. Upstream code retains its original licenses; see third-party notices.

Demo media: Rui Wang (meeting room), Curtis Adams / Pexels (living room footage), Schöps et al. / ETH3D (novel views, CC BY-NC-SA 4.0), Geiger et al. / KITTI (outdoor depth, CC BY-NC-SA 3.0), and Kerr et al. / LERF (semantic scenes). See media credits for sources, attribution and processing details.

The presentation is inspired by MonST3R, Shape of Motion and HunyuanWorld-Mirror.