feat(datasets): declare, register and describe the queryable datasets

[aleksdotbar]

Why. Chart builders lack a reusable description of available data, so new query combinations depend on individual metric definitions and warehouse knowledge.

What changed. Adds dataset declarations, schema-snapshot validation, an embedded registry, and admin-only discovery. Each dataset defines its row identity and queryable fields alongside its model, tests, and documentation; shared preparation applies content deduplication for datasets and metric evidence.

Shipped definitions. Declarations are embedded with the service to keep changes release-controlled; runtime editing remains a separate extension. Snapshot validation does not verify a running warehouse's schema.

Coverage. Two datasets: git_commits and git_file_changes, each with declared row identity and dbt uniqueness checks.

Out of scope. Query execution is #3187. Tenant-defined datasets, approval workflows, and access policies remain planned in the design.

Verified. Documentation structure, TOCs, IDs, and links checked. Application checks were not rerun for the documentation update. Stand integration tests and the additional-context image build remain unverified.

[coderabbitai]

Warning

Review limit reached

Next included review available in 50 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

This review ran on the open-source allowance, not this organization's plan, because the pull request author doesn't have an assigned seat. Waiting won't change this — ask an organization admin to assign them a seat, or add seats in Billing if every seat is already assigned, then retry.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 8b2ebe58-5613-43e4-8ed8-0dce12792d77

📥 Commits

Reviewing files that changed from the base of the PR and between 9a344b8 and 285998a.

⛔ Files ignored due to path filters (1)

• src/backend/Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (69)

• .cf-studio/config/artifacts.toml
• .github/workflows/build-images.yml
• .github/workflows/connectors-ddl.yml
• docker-compose.yml
• docs/components/backend/analytics/openapi.json
• docs/domain/datasets/DESIGN.md
• docs/domain/datasets/PRD.md
• docs/domain/query-engine/DESIGN.md
• docs/domain/query-engine/MIGRATION.md
• docs/domain/query-engine/PRD.md
• src/backend/services/analytics/Cargo.toml
• src/backend/services/analytics/Dockerfile
• src/backend/services/analytics/build.rs
• src/backend/services/analytics/src/api/datasets.rs
• src/backend/services/analytics/src/api/error.rs
• src/backend/services/analytics/src/api/mod.rs
• src/backend/services/analytics/src/api/openapi_tests.rs
• src/backend/services/analytics/src/api/query.rs
• src/backend/services/analytics/src/domain/datasets/declaration.rs
• src/backend/services/analytics/src/domain/datasets/describe.rs
• src/backend/services/analytics/src/domain/datasets/mod.rs
• src/backend/services/analytics/src/domain/datasets/validate.rs
• src/backend/services/analytics/src/domain/field_catalog/columns.snapshot.json
• src/backend/services/analytics/src/domain/field_catalog/loader.rs
• src/backend/services/analytics/src/domain/field_catalog/mod.rs
• src/backend/services/analytics/src/domain/field_catalog/model.rs
• src/backend/services/analytics/src/domain/mod.rs
• src/backend/services/analytics/src/domain/query/answer.rs
• src/backend/services/analytics/src/domain/query/compile/aggregates.rs
• src/backend/services/analytics/src/domain/query/compile/filters.rs
• src/backend/services/analytics/src/domain/query/compile/group.rs
• src/backend/services/analytics/src/domain/query/compile/mod.rs
• src/backend/services/analytics/src/domain/query/compile/order.rs
• src/backend/services/analytics/src/domain/query/compile/params.rs
• src/backend/services/analytics/src/domain/query/compile/scan.rs
• src/backend/services/analytics/src/domain/query/compile/time.rs
• src/backend/services/analytics/src/domain/query/contract/dto.rs
• src/backend/services/analytics/src/domain/query/contract/mod.rs
• src/backend/services/analytics/src/domain/query/fixtures.rs
• src/backend/services/analytics/src/domain/query/mod.rs
• src/backend/services/analytics/src/domain/query/plan.rs
• src/backend/services/analytics/src/domain/query/validation.rs
• src/backend/services/analytics/src/domain/query/violation.rs
• src/backend/services/analytics/src/infra/metrics.rs
• src/backend/services/analytics/src/infra/query.rs
• src/ingestion/datasets/git_commits/README.md
• src/ingestion/datasets/git_commits/dataset.yaml
• src/ingestion/datasets/git_commits/git_commits.sql
• src/ingestion/datasets/git_commits/schema.yml
• src/ingestion/datasets/git_file_changes/README.md
• src/ingestion/datasets/git_file_changes/dataset.yaml
• src/ingestion/datasets/git_file_changes/git_file_changes.sql
• src/ingestion/datasets/git_file_changes/schema.yml
• src/ingestion/dbt/.dbtignore
• src/ingestion/dbt/dbt_project.yml
• src/ingestion/dbt/tests/gold/assert_git_commits_unique.sql
• src/ingestion/dbt/tests/gold/assert_git_file_changes_unique.sql
• src/ingestion/gold/git_authored_file_changes.sql
• src/ingestion/gold/git_metric_evidence.sql
• src/ingestion/gold/schema.yml
• src/ingestion/scripts/bootstrap-db/README.md
• src/ingestion/scripts/bootstrap-db/dump-ddl.sh
• src/ingestion/scripts/connectors-ddl/insight.sql
• src/ingestion/tools/seed/insight_seed/generators/git.py
• tests/stand/api/analytics/test_datasets.py
• tests/stand/api/analytics/test_query.py
• tests/stand/api/analytics/test_request_contracts.py
• tests/stand/api/operations.py
• tests/stand/api/schemas/analytics.py

---

_{Comment @coderabbitai help to get the list of available commands.}