Skip to content

perf: batch PyLovo linking by cell index, not a grid join #213

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jravani merged 2 commits into from
Oct 22, 2026
Merged

perf: batch PyLovo linking by cell index, not a grid join #213

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
eb86806
perf: batch PyLovo linking by cell index, not a grid join
jravani Oct 21, 2026
06636b1
docs: changelog entry for the link batching fix
jravani Oct 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,16 @@
Each release is listed with its breaking changes first. Releases before
v0.8.0 are described in their tag messages.

## Unreleased

Fixes:
- -link-pylovo, and the link stage of an on-request run, no longer
stalls on a database holding regions far apart. Batching built a
grid over the box around all buildings and tested every building
against every cell: for Deggendorf and Koeln in one database that
took 29 minutes before linking began. Each building's cell is now
computed from its coordinates; the batches are unchanged.

## v0.11.4 (2026-10-20)

Features:
Expand Down
29 changes: 6 additions & 23 deletions internal/process/feature_extraction.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -337,38 +337,21 @@ func GetGridBatches(pool *pgxpool.Pool, c2tSchema, lodSchema string, gridSizeM,
unlinkedClause = ""
}

// ST_SquareGrid requires PostGIS >= 3.1.
// Each building goes to the one cell holding a point inside its footprint, so
// a footprint crossing a cell edge is not linked twice by parallel workers.
// DISTINCT ON settles a point lying exactly on an edge. Cells with no
// buildings are excluded.
// Each building goes to the one cell, aligned at the CRS origin, holding a point
// inside its footprint, so parallel workers never link it twice. Cells come from
// coordinates, not an ST_SquareGrid join, whose cost grows with the extent's area.
q := fmt.Sprintf(`
WITH all_buildings AS (
SELECT ab.building_feature_id, ST_Force2D(ab.building_footprint_geom) AS geom
SELECT ab.building_feature_id, ST_PointOnSurface(ST_Force2D(ab.building_footprint_geom)) AS p
FROM %s.%s_building ab
WHERE ab.building_footprint_geom IS NOT NULL
AND ab.object_id IS NOT NULL
%s
%s
),
extent AS (
SELECT ST_Envelope(ST_Collect(geom)) AS bbox
FROM all_buildings
),
grid AS (
SELECT (ST_SquareGrid($1::double precision, bbox)).geom AS cell
FROM extent
WHERE bbox IS NOT NULL
),
assigned AS (
SELECT DISTINCT ON (b.building_feature_id) b.building_feature_id, g.cell
FROM all_buildings b
JOIN grid g ON ST_Intersects(ST_PointOnSurface(b.geom), g.cell)
ORDER BY b.building_feature_id, ST_XMin(g.cell), ST_YMin(g.cell)
)
SELECT array_agg(building_feature_id ORDER BY building_feature_id)
FROM assigned
GROUP BY cell
FROM all_buildings
GROUP BY floor(ST_X(p) / $1::double precision), floor(ST_Y(p) / $1::double precision)
`, c2tSchema, lodSchema, unlinkedClause, limitClause)

rows, err := pool.Query(context.Background(), q, gridSizeM)
Expand Down
Loading