Skip to content

fix(detectnet): truncate clustered boxes to MAX_BOXES to avoid broadc… - #599

Open
andrewwhitecdw wants to merge 2 commits into
NVIDIA:caffe-0.17from
andrewwhitecdw:sweep/bugfix-detectnet-box-overflow
Open

fix(detectnet): truncate clustered boxes to MAX_BOXES to avoid broadc…#599
andrewwhitecdw wants to merge 2 commits into
NVIDIA:caffe-0.17from
andrewwhitecdw:sweep/bugfix-detectnet-box-overflow

Conversation

@andrewwhitecdw

Copy link
Copy Markdown

…ast crash

Summary

cluster() in the DetectNet Python layers (ClusterGroundtruth / ClusterDetections) converts grid-format coverage/bbox network output into a fixed-size [batch_size, MAX_BOXES, 5] blob. When an image produces more than MAX_BOXES (50) box proposals, the forward pass dies with:

ValueError: could not broadcast input array from shape (256,4) into shape (50,4)

so any test/val image with >50 ground-truth objects (or >50 clustered detections) aborts the whole test phase instead of just keeping the 50 slots the blob has room for.

Root cause

boxes = np.zeros([batch_size, MAX_BOXES, 5])
...
[r, c] = boxes_cur_image.shape
boxes[i, 0:r, 0:c] = boxes_cur_image

boxes[i] only has MAX_BOXES rows, but r (the number of proposals that survived thresholding / dedup / groupRectangles voting) is unbounded, so the slice assignment fails to broadcast whenever r > MAX_BOXES.

Repro: 16x16 grid, stride 1, all 256 cells covered => 256 ground-truth proposals => ValueError: could not broadcast input array from shape (256,4) into shape (50,4).

Fix

r = min(r, MAX_BOXES)
boxes[i, 0:r, 0:c] = boxes_cur_image[0:r]

Clip the copy to the blob capacity, which is exactly what the "max_bbox_per_image = MAX_BOXES" contract documented in the layer docstrings implies.

Testing

No GPU/caffe build needed: loaded clustering.py standalone with caffe and cv2 stubbed out and drove cluster() with a synthetic batch of 256 covered grid cells (uv run --with numpy python repro.py):

  • pre-fix (stash): ValueError: could not broadcast input array from shape (256,4) into shape (50,4) — crash reproduced.
  • post-fix: passes, output shape (1, 50, 5) with all 50 slots populated.

⚠️ The full pycaffe test suite requires a compiled caffe with GPU support and could not be run locally; please rely on CI for end-to-end validation.

Why existing tests missed it

python/caffe/test/ has no DetectNet clustering coverage, and every shipped DetectNet example happens to produce fewer than 50 boxes per image.

…ast crash

## Summary

`cluster()` in the DetectNet Python layers (`ClusterGroundtruth` /
`ClusterDetections`) converts grid-format coverage/bbox network output into
a fixed-size `[batch_size, MAX_BOXES, 5]` blob. When an image produces more
than `MAX_BOXES` (50) box proposals, the forward pass dies with:

```
ValueError: could not broadcast input array from shape (256,4) into shape (50,4)
```

so any test/val image with >50 ground-truth objects (or >50 clustered
detections) aborts the whole test phase instead of just keeping the 50
slots the blob has room for.

## Root cause

```python
boxes = np.zeros([batch_size, MAX_BOXES, 5])
...
[r, c] = boxes_cur_image.shape
boxes[i, 0:r, 0:c] = boxes_cur_image
```

`boxes[i]` only has `MAX_BOXES` rows, but `r` (the number of proposals that
survived thresholding / dedup / groupRectangles voting) is unbounded, so the
slice assignment fails to broadcast whenever `r > MAX_BOXES`.

Repro: 16x16 grid, stride 1, all 256 cells covered => 256 ground-truth
proposals => `ValueError: could not broadcast input array from shape
(256,4) into shape (50,4)`.

## Fix

```python
r = min(r, MAX_BOXES)
boxes[i, 0:r, 0:c] = boxes_cur_image[0:r]
```

Clip the copy to the blob capacity, which is exactly what the
"max_bbox_per_image = MAX_BOXES" contract documented in the layer
docstrings implies.

## Testing

No GPU/caffe build needed: loaded `clustering.py` standalone with `caffe`
and `cv2` stubbed out and drove `cluster()` with a synthetic batch of 256
covered grid cells (`uv run --with numpy python repro.py`):

- pre-fix (stash): `ValueError: could not broadcast input array from shape
  (256,4) into shape (50,4)` — crash reproduced.
- post-fix: passes, output shape `(1, 50, 5)` with all 50 slots populated.

⚠️ The full pycaffe test suite requires a compiled caffe with GPU support
and could not be run locally; please rely on CI for end-to-end validation.

## Why existing tests missed it

`python/caffe/test/` has no DetectNet clustering coverage, and every
shipped DetectNet example happens to produce fewer than 50 boxes per image.
cluster() assigned an unbounded number of box proposals into a fixed
[batch_size, MAX_BOXES, 5] blob, raising ValueError whenever an image
produced more than 50 boxes.

Red-green:
- with fix: python3 -m pytest python/caffe/test/test_detectnet_cluster_max_boxes.py -v  # 1 passed
- pre-fix (git show HEAD~1:python/caffe/layers/detectnet/clustering.py > python/caffe/layers/detectnet/clustering.py):
  python3 -m pytest python/caffe/test/test_detectnet_cluster_max_boxes.py -v  # 1 failed,
  ValueError: could not broadcast input array from shape (256,4) into shape (50,4)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant