diff --git a/docs.json b/docs.json index 5eabb4f..3d80b86 100644 --- a/docs.json +++ b/docs.json @@ -114,6 +114,7 @@ "guides/company-brain", "guides/custom-sparql-engine", "guides/entity-resolution", + "guides/rdf-objections", "guides/multi-wiki" ] }, diff --git a/guides/rdf-objections.mdx b/guides/rdf-objections.mdx new file mode 100644 index 0000000..d6991ac --- /dev/null +++ b/guides/rdf-objections.mdx @@ -0,0 +1,316 @@ +--- +title: "Common objections to RDF" +description: + "Answers to the questions developers coming from SQL or a document store ask + before trusting a world: missing facts, schemaless data, and schema changes." +--- + +Developers who know SQL or a document store tend to raise the same objections +before they put agent context in an RDF graph. This guide answers three of them +with examples you can run against a world. + +## Prerequisites + +- A Wazoo private beta account. Sign up at + [wazoo.dev/beta](https://wazoo.dev/beta). +- A world token (`wzw_`) with the `data:read` and `data:write` scopes, generated + on the [Console tokens page](/console/tokens), exported as `WORLDS_TOKEN`. +- A world ID, exported as `WORLD_ID`. +- `curl` and `jq` for the API calls, and Node.js 22.18 or later, Deno, or Bun to + run the SHACL validator. + +## Load the example data + +The examples use two people. Ethan has an email address on record and Sandra +does not. Save the following as `people.ttl`: + +```turtle people.ttl +@prefix user: . +@prefix schema: . + +user:ethan a schema:Person ; + schema:name "Ethan Davidson" ; + schema:email "ethan@etok.me" . + +user:sandra a schema:Person ; + schema:name "Sandra Ortiz" . +``` + +Import it into your world: + +```bash +jq -n --rawfile data people.ttl '{contentType: "text/turtle", data: $data}' | + curl -s -X POST "https://data.wazoo.dev/worlds/$WORLD_ID/import" \ + -H "Authorization: Bearer $WORLDS_TOKEN" \ + -H "Content-Type: application/json" \ + -d @- +``` + +```json Response +{ "imported": { "quads": 5, "chunks": 0 } } +``` + +Each query below is a `.rq` file sent with the same pattern: + +```bash +jq -n --rawfile query emails.rq '{query: $query}' | + curl -s -X POST "https://data.wazoo.dev/worlds/$WORLD_ID/sparql" \ + -H "Authorization: Bearer $WORLDS_TOKEN" \ + -H "Content-Type: application/json" \ + -d @- +``` + +## Is a missing fact false? + +No. A SQL table makes a closed-world assumption: if a row or column value is +absent, the answer is no. RDF makes an open-world assumption: if a fact is +absent, the answer is unknown. A world records what you asserted and nothing +about what you did not. + +That difference matters for agents. An agent that reads a missing email address +as "has no email" invents a fact. An agent that reads it as "no email on record" +reports what it knows. + +Ask for every person and their email address, if any: + +```sparql emails.rq +PREFIX schema: + +SELECT ?person ?email WHERE { + ?person a schema:Person . + OPTIONAL { ?person schema:email ?email } +} +``` + +```json Response +{ + "head": { "vars": ["person", "email"] }, + "results": { + "bindings": [ + { + "person": { "type": "uri", "value": "https://etok.me/#ethan" }, + "email": { "type": "literal", "value": "ethan@etok.me" } + }, + { "person": { "type": "uri", "value": "https://etok.me/#sandra" } } + ] + } +} +``` + +Sandra's row has no `email` binding. Read it as "not recorded". When your +application needs a definite negative, assert it as its own fact so a query can +tell it apart from a gap. + +## Will a schemaless graph turn into a mess? + +Not if you add constraints once the model settles. A world accepts any +well-formed RDF, so you can start without a schema, the way you start a +JavaScript project without types. When a shape stabilizes, describe it with +[SHACL](https://www.w3.org/TR/shacl/) shapes and validate data against them +before import, the way you add TypeScript types to code that already runs. + +The example data above has no schema. The next section adds one without +rewriting the existing facts. + +Worlds does not validate writes against SHACL shapes yet, so run validation in +your pipeline before data reaches the world. Validation on every write, on by +default with a per-world opt-out, is planned +([worlds-sdk-ts#151](https://github.com/wazootech/worlds-sdk-ts/issues/151)). + +## How does a schema change work? + +In SQL, splitting a `name` column into first and last name means a migration: +alter the table, backfill every row, and deploy readers and writers in step. In +a world, you add the new properties beside the old one and move readers and +writers over at their own pace. + + + + +Assert `schema:givenName` and `schema:familyName` for Ethan. His `schema:name` +stays in place, so existing readers keep working. + +```sparql split-name.rq +PREFIX user: +PREFIX schema: + +INSERT DATA { + user:ethan schema:givenName "Ethan" ; + schema:familyName "Davidson" . +} +``` + +```json Response +{ "ok": true } +``` + +A SPARQL update requires the `data:write` scope. + + + + +Update readers to prefer the new properties and fall back to the old one. The +`CONCAT` is unbound when either part is missing, so `COALESCE` falls back to +`schema:name`. + +```sparql display-names.rq +PREFIX schema: + +SELECT ?person ?displayName WHERE { + ?person a schema:Person . + OPTIONAL { ?person schema:givenName ?given ; schema:familyName ?family } + OPTIONAL { ?person schema:name ?name } + BIND(COALESCE(CONCAT(?given, " ", ?family), ?name) AS ?displayName) +} +``` + +```json Response +{ + "head": { "vars": ["person", "displayName"] }, + "results": { + "bindings": [ + { + "person": { "type": "uri", "value": "https://etok.me/#ethan" }, + "displayName": { "type": "literal", "value": "Ethan Davidson" } + }, + { + "person": { "type": "uri", "value": "https://etok.me/#sandra" }, + "displayName": { "type": "literal", "value": "Sandra Ortiz" } + } + ] + } +} +``` + +Ethan's display name now comes from the new properties and Sandra's from +`schema:name`. Neither record had to change for the query to work. + + + + +Save a shape that requires both new properties on every `schema:Person` as +`shapes.ttl`: + +```turtle shapes.ttl +@prefix sh: . +@prefix schema: . +@prefix shape: . + +shape:Person a sh:NodeShape ; + sh:targetClass schema:Person ; + sh:property [ + sh:path schema:givenName ; + sh:minCount 1 ; + sh:datatype + ] , [ + sh:path schema:familyName ; + sh:minCount 1 ; + sh:datatype + ] . +``` + +A new record written in the old shape fails validation. Save it as +`gregory.ttl`: + +```turtle gregory.ttl +@prefix user: . +@prefix schema: . + +user:gregory a schema:Person ; + schema:name "Gregory Hale" . +``` + +Validate it with +[rdf-validate-shacl](https://github.com/zazuko/rdf-validate-shacl), the SHACL +library behind the [Wiki CLI](/projects/wiki). Save the following as +`validate.ts`: + +```typescript validate.ts +import { readFileSync } from "node:fs"; +import rdf from "@zazuko/env"; +import { Parser } from "n3"; +import SHACLValidator from "rdf-validate-shacl"; + +const load = (path: string) => + rdf.dataset(new Parser().parse(readFileSync(path, "utf8"))); + +const [shapesPath, dataPath] = process.argv.slice(2); +const validator = new SHACLValidator(load(shapesPath), { factory: rdf }); +const report = await validator.validate(load(dataPath)); + +for (const result of report.results) { + console.log( + `${result.focusNode?.value} ${result.path?.value}: ${result.message[0]?.value}`, + ); +} +console.log(report.conforms ? "Conforms" : "Does not conform"); +process.exit(report.conforms ? 0 : 1); +``` + +Install the dependencies and run it with Node.js 22.18 or later, Deno, or Bun: + + + +```bash Node.js +npm init -y +npm pkg set type=module +npm install rdf-validate-shacl@0.6.5 @zazuko/env n3 +node validate.ts shapes.ttl gregory.ttl +``` + +```bash Deno +deno add npm:rdf-validate-shacl@0.6.5 npm:@zazuko/env npm:n3 +deno run --allow-read --allow-env validate.ts shapes.ttl gregory.ttl +``` + +```bash Bun +bun add rdf-validate-shacl@0.6.5 @zazuko/env n3 +bun validate.ts shapes.ttl gregory.ttl +``` + + + +```text Output +https://etok.me/#gregory https://schema.org/givenName: Less than 1 values +https://etok.me/#gregory https://schema.org/familyName: Less than 1 values +Does not conform +``` + +The script exits with status 1, so a CI step or import script stops here. SHACL +is a W3C standard, so any conforming processor, such as +[pySHACL](https://github.com/RDFLib/pySHACL), accepts the same shapes. Rewrite +the record in the new shape: + +```turtle gregory.ttl +@prefix user: . +@prefix schema: . + +user:gregory a schema:Person ; + schema:givenName "Gregory" ; + schema:familyName "Hale" . +``` + +```text Output +Conforms +``` + +Import it with the same request as `people.ttl`. The display-name query returns +"Gregory Hale" from the new properties. + + + + +Records written before the change, like Sandra's, still carry only +`schema:name`. Validate only new data until you backfill them. Once every reader +uses the new properties, you can stop writing `schema:name`. Nothing sets a +deadline for that step. + + + + +## Related pages + +- [Update a world](/worlds/update) covers imports and patches. +- [Query a world](/worlds/query) covers SPARQL against a world. +- [Comparison with relational databases](/comparisons/relational-databases) + compares the two models side by side.