envelope-extractor

Extracts per-region 3D building models (CityJSON) from national datasets, cut to live administrative boundaries, ready to feed building-physics pipelines such as city2tabula.

Instead of the fixed national tile grid, it produces a custom cut for exactly the region you ask for -- whether that region spans several tiles or only part of one. The Netherlands is fully implemented; the architecture is built to extend to other countries.

What it does (and deliberately does not do)

For a requested region the pipeline:

1. Resolves the boundary from a live official service (NL: PDOK, with server-side filtering).
2. Fetches only the covering tiles from the national 3D source (NL: 3DBAG tiled CityJSON), downloaded concurrently and cached. It never downloads the whole country.
3. Filters each tile to its in-region buildings first (in parallel), keeping whole buildings (parent Building + BuildingPart children) by representative-point-in-polygon, then merges the shrunk tiles with cjio. Filtering before merging means cjio only handles the kept buildings. Vertices are pruned and re-indexed correctly; geometry is never clipped.
4. Writes one CityJSON file per region, verifies parent-child integrity, and optionally validates the schema with cjval.

It does not strip levels of detail (3DBAG tiles carry LoD0/1.2/1.3/2.2 together; downstream tools select what they need), and does not reproject by default (3DBAG CityJSON is EPSG:7415, whose horizontal component is RD New / EPSG:28992).

Install

cjio must be on the PATH; everything else is a normal dependency.

Conda (recommended). Let conda own the dependency stack, then register the source with pip using --no-deps so pip never re-resolves or shadows the conda builds (geopandas/shapely/GDAL):

conda env create -f infrastructure/environment/data_env.yml
conda activate data_env
pip install -e . --no-deps      # only registers src/, touches no deps

The dev tools (pytest, flake8, mypy) are already in the conda environment, so no extras are needed. Or just run setup.ps1 (PowerShell) / setup.bat (cmd) from the repo root, which does all of the above.

Pure pip (e.g. CI, no conda).

pip install -e ".[dev]"

Verify with cjio --version and pytest -q.

The package version is derived from git tags by setuptools-scm (the generated src/envelope_extractor/_version.py is not committed). Tag a release with git tag vX.Y.Z; a source checkout without a tag falls back to the version in pyproject.toml's [tool.setuptools_scm].

Usage (CLI)

# A whole municipality
python -m envelope_extractor.cli --country netherlands --region Apeldoorn

# A village that needs a parent to disambiguate
python -m envelope_extractor.cli --country nl --level village \
    --region Loenen --parent Apeldoorn

# Several villages of one municipality, in parallel
python -m envelope_extractor.cli --country nl --level village \
    --region Heeten --parent Raalte \
    --sub-regions Loenen Beekbergen --parent-for-subs Apeldoorn \
    --parallel

# Experimental 3DBAG REST API backend instead of tiles
python -m envelope_extractor.cli --country nl --region Apeldoorn \
    --backend api

After pip install, the same is available as the envelope-extractor command. Key flags: --level, --parent, --sub-regions, --parent-for-subs, --backend {tiled,api}, --parallel, --max-workers, --output-dir, --cache-dir, --validate, --purge-cache, --log-level.

Programmatic use:

from envelope_extractor.config import PipelineConfig
from envelope_extractor.region_spec import RegionSpec
from envelope_extractor.registry import get_extractor_cls, load_builtin_countries

load_builtin_countries()
extractor = get_extractor_cls("nl")(config=PipelineConfig())
path = extractor.extract(
    RegionSpec(country="nl", name="Heeten", level="village", parent="Raalte")
)

Drop the resulting file into city2tabula's input folder and run its import as usual.

Attribute extraction (separate analysis path)

attributes_nl.py pulls selected 3DBAG attributes (construction year, floors, greenhouse flag, surface areas, ...) from a region CityJSON into a table for your own analysis -- independent of the city2tabula path:

python -m envelope_extractor.attributes_nl \
    ./data/envelope/heeten.city.json ./data/analysis/heeten.csv

Architecture

The orchestration is country-agnostic. Each country supplies two plugins; the filter/merge/validate stages are shared.

BaseEnvelopeExtractor              # template: resolve -> fetch -> filter -> merge -> validate
├── BoundarySource  (per country)  # region name -> boundary polygon
└── BuildingSource  (per country)  # boundary    -> CityJSON tiles
registry.py                        # country key/alias -> extractor class
region_spec.py                     # RegionSpec: one validated request
cli.py                             # command-line interface

src/envelope_extractor/
├── base.py            # orchestration template method
├── registry.py        # country registration & lookup
├── region_spec.py     # RegionSpec dataclass + slugify
├── cli.py             # CLI
├── config.py          # endpoints, CRS policy, runtime config
├── cityjson_ops.py    # cjio wrappers + pure-Python filter/prune/validate
├── region_select.py   # point-in-polygon building selection
├── logging_setup.py   # central logging
├── attributes_nl.py   # attribute extraction for analysis
├── boundaries/        # BoundarySource ABC + nl_pdok
├── sources/           # BuildingSource ABC + nl_3dbag (tiled + API)
├── envelope_nl.py     # Netherlands extractor (registered)
└── envelope_de.py     # Germany scaffold

Adding a country means implementing those two interfaces and registering the extractor -- see CONTRIBUTING.md.

Configuration notes

• Endpoints and versions live in config.py. 3DBAG tile URLs are read from the WFS tile-index at runtime, so a new 3DBAG version needs no code change.
• Version fallback: if a tile download from the current version fails, the source retries from config.BAG3D_FALLBACK_VERSION.
• CRS: no reprojection by default; PipelineConfig.reproject_to is an escape hatch only.
• Validation: --validate (or PipelineConfig.validate_output) runs cjval if installed.
• Disk footprint: --purge-cache deletes raw tiles after each region is written.

Development

flake8 src/ tests/     # lint (79 cols)
mypy src/              # strict typing
pytest -q              # tests (uses a small real 3DBAG fixture)

CI runs all three on push/PR. See CONTRIBUTING.md and CHANGELOG.md.

Documentation

In-depth guides live in docs/: an overview (what it does, why not QGIS), the architecture & developer guide (where to start, what to change where), multi-country standardisation, performance & I/O, and testing & validation (including how to validate a full municipality and the API backend).

Container

docker build -f infrastructure/container/Dockerfile -t envelope-extractor .
docker run --rm -v "$PWD/data:/app/data" envelope-extractor \
    --country nl --region Apeldoorn