[FEATURE] Add packages for modelling your own data with Overture's schema

.devcontainer/devcontainer.json

@@ -25,7 +25,8 @@
 	"customizations": {
 		"vscode": {
 			"settings": {
-				"python.defaultInterpreterPath": "${containerWorkspaceFolder}/.venv/bin/python"
+				"python.defaultInterpreterPath": "${containerWorkspaceFolder}/.venv/bin/python",
+				"python.languageServer": "None"
 			},
 			"extensions": [
 				"ms-toolsai.jupyter",
@@ -35,7 +36,8 @@
 				"ms-toolsai.vscode-jupyter-slideshow",
 				"ms-python.vscode-pylance",
 				"ms-python.python",
-				"ms-python.debugpy"
+				"ms-python.debugpy",
+				"astral-sh.ty"
 			]
 		}
 	},

README.md

@@ -25,6 +25,15 @@ permalink: /index.html
 
 ---
 
+## Workshop Material
+
+| Name | Description |
+| ---- | ----------- |
+| [my-schema](my-schema/README.md) | A starting point for modelling your own data with Overture's schema framework |
+| [schema-bootstrap](schema-bootstrap/README.md) | Generate a first draft of that model from a data file and the metadata shipped beside it |
+
+---
+
 ## Workshop Setup
 
 ### Local setup (recommended)

examples/bad-depth.yaml

@@ -0,0 +1,10 @@
+id: 5d40bd6c-db14-4492-a29f-5b6f3e7a1e44
+type: Feature
+geometry:
+  type: Polygon
+  coordinates: [[[-111.95, 41.05], [-111.90, 41.05], [-111.90, 41.10], [-111.95, 41.10], [-111.95, 41.05]]]
+properties:
+  theme: base
+  type: bathymetry
+  version: 0
+  depth: -1

examples/bathymetry-example.yaml

@@ -0,0 +1,10 @@
+id: 5d40bd6c-db14-4492-a29f-5b6f3e7a1e44
+type: Feature
+geometry:
+  type: Polygon
+  coordinates: [[[-111.95, 41.05], [-111.90, 41.05], [-111.90, 41.10], [-111.95, 41.10], [-111.95, 41.05]]]
+properties:
+  theme: base
+  type: bathymetry
+  version: 0
+  depth: 10

examples/bathymetry.sql

@@ -0,0 +1,6 @@
+-- Writes examples/bathymetry.parquet: 100 rows of Overture bathymetry.
+-- Run from the repo root: duckdb < examples/bathymetry.sql
+INSTALL spatial; LOAD spatial; SET s3_region='us-west-2';
+COPY (SELECT * FROM read_parquet('s3://overturemaps-us-west-2/release/2026-08-19.0/theme=base/type=bathymetry/*.parquet')
+      LIMIT 100)
+TO 'examples/bathymetry.parquet' (FORMAT PARQUET);

examples/divisions.sql

@@ -0,0 +1,7 @@
+-- Prints 100 Overture division rows as JSON, one per line, geometry as GeoJSON.
+-- duckdb < examples/divisions.sql | overture-schema validate --type division --show-field id -
+INSTALL spatial; LOAD spatial; SET s3_region='us-west-2';
+COPY (SELECT ST_AsGeoJSON(geometry) AS geometry, * EXCLUDE geometry
+      FROM read_parquet('s3://overturemaps-us-west-2/release/2026-08-19.0/theme=divisions/type=division/*.parquet')
+      LIMIT 100)
+TO '/dev/stdout' (FORMAT JSON, ARRAY false);

my-schema/README.md

@@ -0,0 +1,144 @@
+# my-schema
+
+A starting point for modelling your own data with Overture's schema framework.
+Codespaces installs it for you; edit `src/my_schema/models.py` in the editor and
+run the commands below in the terminal.
+
+## Try the example
+
+```console
+overture-schema list-types --tag my_schema
+overture-schema validate my-schema/examples/good.json
+overture-schema validate my-schema/examples/bad.json
+overture-codegen generate --format markdown --tag my_schema --output-dir docs/
+```
+
+`--tag my_schema` selects your models. The tag comes from the tag provider in
+`src/my_schema/tags.py`, which tags every model this package registers.
+
+## Exercise: add a second model
+
+Run these from the repository root. Paths are relative to it.
+
+**1. Write the model.** Append to `my-schema/src/my_schema/models.py`:
+
+```python
+class RailCrossingRating(Feature):
+    """A safety star rating for a level crossing, where a road meets a railway."""
+
+    geometry: Annotated[
+        Geometry,
+        GeometryTypeConstraint(GeometryType.POINT),
+        Field(description="Where the road crosses the railway."),
+    ]
+    stars: StarRating
+```
+
+Then re-export it from `my-schema/src/my_schema/__init__.py`:
+
+```python
+from my_schema.models import RailCrossingRating, RoadSafetyRating
+
+__all__ = ["RailCrossingRating", "RoadSafetyRating"]
+```
+
+**2. Look for it.** The tools can't see it yet:
+
+```console
+$ overture-schema list-types --tag my_schema
+road_safety_rating  feature  my_schema
+```
+
+They find models through entry points, and this one has none.
+
+**3. Register it.** In `my-schema/pyproject.toml`, add a line under the existing one:
+
+```toml
+[project.entry-points."overture.models"]
+road_safety_rating = "my_schema:RoadSafetyRating"
+rail_crossing_rating = "my_schema:RailCrossingRating"
+```
+
+Run `list-types` again: still one model. The tools read entry points from what was
+*installed*, not from `pyproject.toml`, and that record is written only at install
+time. Re-sync:
+
+```console
+$ uv sync --all-packages
+$ overture-schema list-types --tag my_schema
+rail_crossing_rating  feature  my_schema
+road_safety_rating    feature  my_schema
+```
+
+Edits to a model's code never need this step; only a new entry point does.
+
+**4. Validate against the right model.** With two models, name the one you mean:
+
+```console
+$ overture-schema validate --type road_safety_rating my-schema/examples/motorway-without-speed-limit.json
+```
+
+This fails on the motorway rule. Leave out `--type` and the validator picks a model
+for you. Here it picks `RailCrossingRating` and reports the geometry instead.
+
+**5. Tag by mode, and mark a draft.** Replace `my-schema/src/my_schema/tags.py` with:
+
+```python
+"""Tag this package's models so the tools can select them with `--tag my_schema`."""
+
+from collections.abc import Iterable
+
+from pydantic import BaseModel
+
+from overture.schema.system.discovery import ModelKey
+
+# Entry-point name -> mode, and the models still in draft.
+MODE = {"road_safety_rating": "road", "rail_crossing_rating": "rail"}
+DRAFTS = {"rail_crossing_rating"}
+
+
+def my_schema_provider(
+    types: Iterable[type[BaseModel]], key: ModelKey, tags: set[str]
+) -> set[str]:
+    """Tag every model from this package `my_schema`, plus its mode and draft status."""
+    if key.entry_point.startswith("my_schema:"):
+        tags.add("my_schema")
+        tags.add(f"my_schema:mode={MODE[key.name]}")
+        if key.name in DRAFTS:
+            tags.add("my_schema:draft")
+    return tags
+```
+
+This is ordinary code, so no re-sync:
+
+```console
+$ overture-schema list-types --tag my_schema --group-by my_schema:mode
+my_schema:mode=rail (1)
+→ rail_crossing_rating  feature  my_schema  my_schema:draft  my_schema:mode=rail
+
+my_schema:mode=road (1)
+→ road_safety_rating    feature  my_schema  my_schema:mode=road
+
+$ overture-schema list-types --tag my_schema --exclude my_schema:draft
+road_safety_rating  feature  my_schema  my_schema:mode=road
+```
+
+**6. Publish only what's ready.** `--exclude` works on the docs too:
+
+```console
+$ overture-codegen generate --format markdown --tag my_schema --exclude my_schema:draft --output-dir docs/
+```
+
+`docs/my_schema/` has a page for `road_safety_rating` and none for the draft.
+
+## Model your data
+
+1. Write a class in `src/my_schema/models.py` that subclasses `Feature`.
+2. Re-export it from `src/my_schema/__init__.py`.
+3. Register it in `pyproject.toml` under `[project.entry-points."overture.models"]`.
+4. Run `uv sync --all-packages`, then `overture-schema list-types --tag my_schema` to
+   check it appears. (`--all-packages` keeps the workshop's other packages; a bare
+   `uv sync` in this directory removes them.)
+
+Edits to an existing model take effect immediately. Only a new entry point needs
+`uv sync --all-packages`, as in the exercise above.

my-schema/examples/bad.json

@@ -0,0 +1,10 @@
+{
+  "type": "Feature",
+  "id": "rsr-0002",
+  "geometry": {"type": "Point", "coordinates": [-122.68, 45.52]},
+  "properties": {
+    "stars": 7,
+    "road_type": "motorway",
+    "survey": {"assessor": "irap", "rated_by": "me"}
+  }
+}

my-schema/examples/good.json

@@ -0,0 +1,10 @@
+{
+  "type": "Feature",
+  "id": "rsr-0001",
+  "geometry": {"type": "LineString", "coordinates": [[-122.68, 45.52], [-122.67, 45.53]]},
+  "properties": {
+    "stars": 4,
+    "road_type": "arterial",
+    "survey": {"assessor": "iRAP", "surveyed_on": "2026-05-01"}
+  }
+}

my-schema/examples/motorway-without-speed-limit.json

@@ -0,0 +1,9 @@
+{
+  "type": "Feature",
+  "id": "rsr-0003",
+  "geometry": {"type": "LineString", "coordinates": [[-122.68, 45.52], [-122.67, 45.53]]},
+  "properties": {
+    "stars": 2,
+    "road_type": "motorway"
+  }
+}

my-schema/pyproject.toml

@@ -0,0 +1,38 @@
+[project]
+name = "my-schema"
+version = "0.1.0"
+description = "A starting point for modelling your own data with Overture's schema framework"
+requires-python = ">=3.12"
+dependencies = [
+    "overture-schema>=2.0.0",
+    "overture-schema-codegen>=2.0.0",
+]
+
+[build-system]
+requires = ["uv_build>=0.11.32,<0.13"]
+build-backend = "uv_build"
+
+[tool.uv.build-backend]
+module-name = "my_schema"
+
+# The tools find your models through this entry point: one line per model,
+# `name = "module:Class"`. After adding a line, run `uv sync` so it registers.
+[project.entry-points."overture.models"]
+road_safety_rating = "my_schema:RoadSafetyRating"
+
+# Tags the models above with `my_schema`, so `--tag my_schema` selects them.
+[project.entry-points."overture.tag_providers"]
+my_schema = "my_schema.tags:my_schema_provider"
+
+# Example rows appear on each model's generated Markdown page.
+[[examples.RoadSafetyRating]]
+id = "rsr-0001"
+geometry = "LINESTRING (-122.68 45.52, -122.67 45.53)"
+stars = 4
+road_type = "arterial"
+
+[examples.RoadSafetyRating.bbox]
+xmin = -122.68
+xmax = -122.67
+ymin = 45.52
+ymax = 45.53

my-schema/src/my_schema/__init__.py

@@ -0,0 +1,5 @@
+"""Your models. Re-export each one here so its entry point can find it."""
+
+from my_schema.models import RoadSafetyRating
+
+__all__ = ["RoadSafetyRating"]

my-schema/src/my_schema/models.py

@@ -0,0 +1,58 @@
+"""An example model. Copy it, rename it, and make it describe your data."""
+
+from datetime import date
+from typing import Annotated, NewType
+
+from pydantic import BaseModel, Field
+
+from overture.schema.system.doc import DocumentedEnum
+from overture.schema.system.feature import Feature
+from overture.schema.system.geometric import Geometry, GeometryType, GeometryTypeConstraint
+from overture.schema.system.model_constraint import FieldEqCondition, no_extra_fields, require_if
+from overture.schema.system.numeric import uint8, uint16
+
+# A named type carries its description and constraints everywhere it is used.
+StarRating = NewType(
+    "StarRating",
+    Annotated[uint8, Field(ge=1, le=5, description="Star rating from 1 (least safe) to 5 (safest).")],
+)
+
+
+# A coded column: list every legal value and what it means.
+class RoadType(str, DocumentedEnum):
+    """Kind of road that was rated."""
+
+    MOTORWAY = ("motorway", "Divided highway with controlled access.")
+    ARTERIAL = ("arterial", "Major road connecting districts.")
+    LOCAL = ("local", "Street serving the properties along it.")
+
+
+# A struct: a group of related fields.
+@no_extra_fields
+class Survey(BaseModel):
+    """Who rated the road, and when."""
+
+    assessor: str
+    surveyed_on: date | None = None
+
+
+# A rule across fields, written as data so the docs can state it.
+# Avoid @field_validator and @model_validator: they run, but no tool can read them.
+@no_extra_fields
+@require_if(["speed_limit_kph"], FieldEqCondition("road_type", "motorway"))
+class RoadSafetyRating(Feature):
+    """A road-safety star rating for a stretch of road."""
+
+    geometry: Annotated[
+        Geometry,
+        GeometryTypeConstraint(GeometryType.LINE_STRING),
+        Field(description="The rated stretch of road."),
+    ]
+    stars: StarRating
+    road_type: Annotated[RoadType | None, Field(description="Kind of road that was rated.")] = None
+    speed_limit_kph: Annotated[
+        uint16 | None, Field(description="Posted speed limit, in kilometres per hour.")
+    ] = None
+    survey: Annotated[
+        Survey | None, Field(description="The survey this rating came from.")
+    ] = None

my-schema/src/my_schema/tags.py

@@ -0,0 +1,16 @@
+"""Tag this package's models so the tools can select them with `--tag my_schema`."""
+
+from collections.abc import Iterable
+
+from pydantic import BaseModel
+
+from overture.schema.system.discovery import ModelKey
+
+
+def my_schema_provider(
+    types: Iterable[type[BaseModel]], key: ModelKey, tags: set[str]
+) -> set[str]:
+    """Add `my_schema` to every model registered from this package."""
+    if key.entry_point.startswith("my_schema:"):
+        tags.add("my_schema")
+    return tags