[jravani]

A bbox import passed citydb-tool the whole dataset folder, so with a whole provider on disk every region import read and parsed every file, and adding any tile invalidated every region's import record.

Change

• Before calling citydb-tool, a bbox import (POST /api/v1/runs, -import-data -bbox) reads each model file's declared extent:
  • CityGML: the boundedBy envelope, searched in the first 16 KiB, then up to 1 MiB;
  • CityJSON: metadata.geographicalExtent and referenceSystem, scanned through the file, as metadata may come after CityObjects.
• The bbox is transformed into each file's CRS once (PostGIS), and only intersecting files are passed, through a citydb-tool argument file. A file without a readable extent or a recognised CRS is always passed. A bbox no file reaches runs no import.
• The bbox is recorded against the fingerprint of the selected files alone. A new record replaces only earlier ones of the same bbox and mode. A whole-folder import of the current files still covers any bbox.
• Docs: "Importing one region" in docs/deployment/adding-data.md; CHANGELOG.md.

Tests

• TestReadFileExtent (unit): gzipped CityGML with attributes on the corner elements; CityJSON whose metadata extent comes after a CityObject's own geographicalExtent; files without an extent.

• TestImportCityDBData_SelectsFilesInTheBbox (integration, fake citydb-tool logging the files it is given):

  • region A imports only its tile;
  • a repeat is skipped;
  • a tile added in region B leaves region A's record valid;
  • region B imports its two tiles;
  • a bbox no tile reaches runs no import.
    With every file treated as intersecting it fails: region A: last import got files "a.gml b.gml", want "a.gml".

• go vet ./..., go test ./..., the integration suites (importer, process, onrequest with real citydb-tool, db, api) and mkdocs build --strict pass.

• End to end on all 18,297 Bavarian tiles (142 GB) in one folder:

  Run	Time	Files passed
  THD region	3.2 s, skipped after other regions were added	2 selected
  Plattling, new	8.5 s including import	4 of 18,297
  Plattling, repeat	1.1 s, skipped	4 selected

  3,873 buildings, every object id once.

[codecov]

Codecov Report

❌ Patch coverage is 70.11494% with 52 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/importer/select.go	67.24%	21 Missing and 17 partials ⚠️
internal/importer/extent.go	73.33%	5 Missing and 7 partials ⚠️
internal/importer/citydb.go	84.61%	1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!