Skip to content

[FEATURE] Add packages for modelling your own data with Overture's schema - #62

Open
Seth Fitzsimmons (sethfitz) wants to merge 15 commits into
devcontainer-py312from
schema-bootstrap
Open

[FEATURE] Add packages for modelling your own data with Overture's schema#62
Seth Fitzsimmons (sethfitz) wants to merge 15 commits into
devcontainer-py312from
schema-bootstrap

Conversation

@sethfitz

@sethfitz Seth Fitzsimmons (sethfitz) commented Sep 26, 2026 •
edited
Loading

Copy link
Copy Markdown

What's in this PR?

  • adds my-schema, a template package for modelling your own data with Overture's schema framework: an example Feature model covering a named type, an enum, a struct and a cross-field rule, plus a tag provider so --tag my_schema selects it
  • adds schema-bootstrap, which writes a first draft of such a model from a data file (Shapefile, GeoPackage, GeoJSON or Parquet). Field descriptions and enum domains come from the ISO 19110 or FGDC metadata shipped beside the data; anything it can't source is left as a TODO in the output
  • both are uv workspace members, so uv sync --frozen (locally and in the Codespaces postCreate) installs them editable. schema-bootstrap requires duckdb >= 1.5.5, so the lock moves duckdb from 1.5.2 to 1.5.5
  • lists both under a new "Workshop Material" heading in the README
  • stacked on [CHORE] Rebuild the Codespaces devcontainer on Python 3.12 and uv #60, so its base is devcontainer-py312 until that merges
  • closes [FEATURE] Workshop material for modelling your own data with Overture's schema #63

@sethfitz
Add a template package for modelling your own data
0b41f57 
Workshop attendees at CNG Forum 2026 model their own datasets with
Overture's schema framework. my-schema is the starting point: an example
Feature model covering a named type, an enum, a struct and a cross-field
rule, plus a tag provider so --tag my_schema selects its models.

It is a uv workspace member, so the devcontainer's uv sync installs it
editable: edits to models.py take effect without reinstalling.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@sethfitz
Describe the survey field in the template
0d6c4b4 
A field typed as a struct shows an empty description in the generated
docs unless the field has its own; codegen doesn't fall back to the
struct's docstring yet. The workshop deck shows this row.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@sethfitz
Say to run uv sync from the repository root
7a4f9ed 
my-schema is a workspace member sharing the root .venv. uv sync inside
my-schema/ syncs only that member and removes jupyter, duckdb and the
rest; from the root it registers the new entry point and keeps them.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@sethfitz
Use uv sync --all-packages in the template
6f825f3 
It keeps the workshop's other packages from any directory, where a bare
uv sync inside my-schema/ removes them.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@sethfitz
Add schema-bootstrap for modelling your own data
f1d0744 
schema-bootstrap generates an Overture-convention Pydantic model from a
data file, seeding field descriptions and enum domains from sidecar
metadata. It gives the CNG workshop's schema session a starting point:
point it at your own data and edit the model it writes, rather than
authoring one from scratch.

Adapted from sethfitz/schema-bootstrap at dfc2176: it builds with
uv_build like the workshop's other packages, its README and the
TIGER/Line fixture link the Census source, and the tag-provider test
runs against the real overture-schema it already depends on.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@sethfitz
Install schema-bootstrap as a workspace member
1400cfa 
Like my-schema, schema-bootstrap is a uv workspace member, so the
devcontainer's uv sync puts the schema-bootstrap command on PATH beside
overture-schema. Its own uv.lock goes, since the workspace lock governs
members.

The workspace lock resolves duckdb to 1.5.5, the floor schema-bootstrap
declares (was 1.5.2). Its README's uv sync instructions now name
--all-packages, since a bare uv sync in a member directory removes the
workshop's other packages.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@sethfitz
List the schema packages under Workshop Material
ddd0b82 
Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@sethfitz
Ship the example files the deck's validation slides use
1c0ada2 
Slide 16 validates two bathymetry YAML features, slide 17 converts a
GeoParquet extract with gpq, slide 18 pipes a DuckDB query into the
validator. None of those inputs existed in the repo, so a tester
couldn't run them. bathymetry.parquet (100 rows, 55 KB) is committed so
the slide works without network; bathymetry.sql regenerates it.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@sethfitz
Drop the uv cooldown span a local re-lock recorded
2caf4d5 
cf9ca2c re-locked on a machine whose global uv config sets exclude-newer
= "1 week", so uv.lock recorded exclude-newer-span = "P1W". The workshop
doesn't declare that setting, so the lock disagreed with every
environment but that one (Codespaces, participants' laptops). Re-locked
with XDG_CONFIG_HOME=/nonexistent: only the [options] block changes, no
package versions.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@sethfitz
Add a notebook for the schema slides' hands-on half
a12cbd7 
Slides 20-35 (fields, types, constraints) as runnable cells against
my-schema's RoadSafetyRating, one Try-it exercise per concept that fits
in a cell. Cells meant to fail are marked; %xmode Minimal keeps the
tracebacks to the field and the rule. Executed locally 2026-09-26: every
intended failure fails at the rule it teaches, every other cell passes.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@sethfitz
Install codegen from the stac-table-columns branch
51c8eff 
The deck's STAC slide needs --format stac-table-columns, which is
unreleased (OvertureMaps/schema#724). The branch's own sources pull
overture-schema-system, -common and -cli from the same commit, so all
four come from git until #724 ships. Re-ran every hands-on slide,
schema-bootstrap --report and the notebook on this lock (2026-09-26):
same results as on the 2.0.0 release. Locked without the local uv
cooldown config.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@sethfitz
Walk through adding a second model in my-schema
a3d92e6 
Slides 37-40 describe registering a model, the re-sync an entry point
needs, --type once there are two models, and a tag vocabulary; nothing
let participants do it. Adapted from a tester's walkthrough to the
Codespace (editor, uv sync --all-packages). The tag provider defines the
MODE and DRAFTS that slide 39 uses but never shows. Every command and
output run end to end in a scratch copy of the repo, 2026-09-26.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@sethfitz
Install ty in the devcontainer
4cabbd3 
Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@sethfitz
Make ty the Python language server
2fd44a4 
Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@sethfitz
Forbid extra fields on RoadSafetyRating and Place
5f5272e 
Nested structs already used @no_extra_fields; the feature itself ignored
undeclared keys, so a misspelled top-level field validated and vanished.
The model now rejects it, and the notebook's alias and extra-key cells
show the rejection instead of the silent drop.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@sethfitz