Skip to content
12 changes: 11 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,15 @@ This is useful when you want to:
Character Memory separates planning from persistence. Use `prepare` to build an inspectable `RememberWritePlan`, `validate_plan` to check it against the current graph, and `commit` to persist it. `prepare` and `validate_plan` do not write graph objects, vector entries, retrieval statistics, or raw source data.

```rust
let input = RememberInput::new("caller-provided note or transcript reference");
use character_memory::{Scene, SceneParticipant};

let mut scene = Scene::now();
scene.participants.push(SceneParticipant {
description: Some("a visitor".to_owned()),
..Default::default()
});
scene.setting.words = Some("a quiet room".to_owned());
let input = RememberInput::new("caller-provided note or transcript reference").with_scene(scene);

let plan = memory.prepare(input, PrepareOptions::default()).await?;
let validation = memory.validate_plan(&plan).await?;
Expand All @@ -155,6 +163,8 @@ if validation.iter().all(|candidate| candidate.status == CandidateValidationStat

For callers that want the standard write lifecycle in one call, `remember(RememberInput, RememberOptions)` composes `prepare`, `validate_plan`, and `commit` over the same graph-authoritative machinery.

An episode stores one `Scene` with its time, participants, setting and custom values as supplied. Scene words join its single summary embedding; keys and custom values do not. If the input and episode draft omit the scene, `prepare` fixes the current time once; a caller-built episode must supply its scene. Writes reject participants with neither a key nor nonblank words, and keys that identify no existing or same-plan notion.

The write path is deliberately not an extraction system. Character Memory core does not infer preferences, commitments, corrections, character signals, thread membership, or entity identity from raw text. It does not store raw logs, and `raw_ref` values remain opaque caller-managed provenance pointers. Candidates in a `RememberWritePlan` are not memory until a valid plan is committed.

## Notions and naming beliefs
Expand Down
8 changes: 6 additions & 2 deletions docs/design/database/graph_schema_design.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,11 @@ Hydration reconstructs domain objects and links from the selected RDF named grap

## Experiences, Notions And Threads

Episodes record interaction spans with `summary`, `modality`, optional source and time metadata, `participantEntity` references, salience and retention. Observations record `text` tied to an `episode`, with observation time, an optional `speakerEntity` and an optional raw reference. Dedicated source references keep experience-based provenance queryable.
Episodes record interaction spans with `summary`, `modality`, one `Scene`, optional `endedAt` and `rawRef`, salience and retention. The scene holds the required experience time, participants with any combination of notion key, supplied name and description, a setting key and/or words, and flat string custom values. Missing scene parts stay unspecified. Observations record `text` tied to an `episode`, with an optional `speakerEntity` and raw reference. Preparation defaults observation time to scene time while preserving an explicit observation time. Involved notions, participants, speakers, thread memberships and the episode end time remain independent.

The episode owns the scene. `sceneTime` stores its timestamp losslessly; `sceneParticipants` and `sceneCustomValues` store JSON literals, preserving participant order, duplicates, spelling and custom string values. `settingKey` is a directly queryable literal, separate from `settingWords`. Keys identify application contexts, not notions; sessions may be recorded in custom values. Each participant supplies a key or nonblank name or description; a keyless participant with absent or blank words is rejected. Participant keys must identify existing or same-plan notions. Names, descriptions and setting words are preserved as given and also indexed with the episode summary in its one embedding; they create no notions or links. Setting words follow a `Setting:` label and each participant has one `With:` line joining its nonblank name and description with a comma, in authored order, including when a key is also present. The embedding builder folds whitespace and omits blank word values; the stored scene is unchanged. Keys and custom values are never embedded. Without scene words the episode embedding text is unchanged.

An explicit draft scene replaces the input scene as a whole. If both are absent, preparation fixes the current experience time in the plan; commit never substitutes the episode creation time or a later clock reading. Caller-authored episode candidates require a scene. A derived memory cites its source experiences without copying their scenes, and correction leaves those source scenes intact. The source-object correction guard compares `original_setting_key` with the source episode setting key, including when the target is an observation; it never compares setting words.

A notion stores only its identity, `createdAt` and schema metadata. Its names and descriptions are carried by interpreted memories about it. Multiple notions may share a name, and one notion may have multiple naming beliefs.

Expand All @@ -43,7 +47,7 @@ A `DerivedMemory` stores `derivedType`, `text`, `salienceScore`, `retentionState

An interpreted memory either cites at least one episode or observation, or declares `given_by_application=true`. The latter is persisted as `givenByApplication`, requires at least one notion subject, and excludes experience source references. Application-given beliefs carry application-supplied grounding without inventing an experience. Corrections of such beliefs require an explicit replacement with its grounding declared.

The reference lists have set semantics: IDs are sorted and deduplicated at draft conversion for stable persistence and replay. This applies to episode participant IDs and to the source, thread, subject and predecessor lists on ordinary and replacement interpreted-memory drafts.
The reference lists have set semantics: IDs are sorted and deduplicated at draft conversion for stable persistence and replay. This applies to the source, thread, subject and predecessor lists on ordinary and replacement interpreted-memory drafts. Scene participants retain authored order and repeated values; generated participant links and retrieval-stat edges count each keyed participant once.

### Assertions And Name Lookup

Expand Down
11 changes: 7 additions & 4 deletions docs/design/database/schema_cheat_sheet.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ These are the five emitted payload fields. The service indexes `object_id` and `

| Object type | Surface | Text source | Maximum surfaces |
|---|---|---|---|
| `episode` | `summary` | Episode summary | 1 |
| `episode` | `summary` | Episode summary followed by labelled scene words; keys and custom values excluded | 1 |
| `observation` | `text` | Observation text | 1 |
| `memory_thread` | `summary` | Thread title and summary | 1 |
| `derived_memory` | `derived_text` | Interpreted-memory text | 1 |
Expand Down Expand Up @@ -55,8 +55,11 @@ Predicate names in these tables are suffixes under `urn:cmem:vocab:`.
|---|---|
| `objectId`, `objectType`, `graphUri`, `schemaVersion` | Common identity and schema literals |
| `createdAt`, `updatedAt` | Timestamps on the types that declare them |
| `modality`, `sourceConversationId`, `startedAt`, `endedAt` | Episode source and time metadata |
| `participantEntity` | Episode participant notion |
| `modality`, `endedAt` | Episode modality and optional interval end |
| `sceneTime` | Required experience time on the episode |
| `sceneParticipants` | Lossless JSON array of participants, each with optional key, name and description; a key or nonblank words required |
| `settingKey`, `settingWords` | Independently optional context key and words; the key is directly queryable |
| `sceneCustomValues` | Lossless JSON object of string custom values |
| `summary` | Episode or thread summary |
| `rawRef` | Episode or observation source pointer |
| `episode`, `speakerEntity`, `observedAt` | Observation source, optional speaker and time |
Expand Down Expand Up @@ -91,7 +94,7 @@ Notions carry only common identity/schema literals and `createdAt`. The followin
| `assertionName` | Supplied name spelling |
| `normalizedName` | NFKC, lowercase and whitespace-folded lookup spelling |

Assertion resources use `<memory-uri>:assertion:<zero-padded ordinal>`. Their ordering and repeated values are preserved by the [assertion reader (`shared.rs:389`)](../../../src/adapters/oxigraph/shared.rs#L389). ID lists use set semantics: conversion sorts IDs and removes duplicates. This includes episode participants and ordinary/replacement memory source, thread, subject and predecessor IDs.
Assertion resources use `<memory-uri>:assertion:<zero-padded ordinal>`. Their ordering and repeated values are preserved by the [assertion reader (`shared.rs:389`)](../../../src/adapters/oxigraph/shared.rs#L389). Interpreted-memory source, thread, subject and supersedes ID lists use set semantics: conversion sorts IDs and removes duplicates, including on replacement drafts. Scene participants preserve authored order and duplicates.

Name lookup reads active beliefs with no incoming `Supersedes` link. It can resolve the same normalized name to several notion IDs; see the [name selector (`sparql_selectors.rs:104`)](../../../src/adapters/oxigraph/sparql_selectors.rs#L104).

Expand Down
8 changes: 5 additions & 3 deletions docs/design/database/vector_payload_design.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,19 +18,21 @@ Each emitted point has exactly five payload fields:

The service indexes `object_id` and `object_type`. The remaining fields describe the record; they are not prefilter columns. Readable result content, graph URIs, assertion and grounding data, relationships, lifecycle values, timestamps and raw references are hydrated from Oxigraph.

The [payload writer (`payload.rs:164`)](../../../src/adapters/qdrant/payload.rs#L164) enforces the supported schema marker and emits the five fields. The [candidate reader (`payload.rs:115`)](../../../src/adapters/qdrant/payload.rs#L115) reads `object_id`, `object_type` and `surface`, then combines them with the vector score. It does not read `schema_version`, `embedding_text` or extra payload fields to construct a candidate. Unknown type/surface tokens and malformed IDs fail decoding.
The [payload writer](../../../src/adapters/qdrant/payload.rs) enforces the supported schema marker and emits the five fields. The [candidate reader](../../../src/adapters/qdrant/payload.rs) reads `object_id`, `object_type` and `surface`, then combines them with the vector score. It does not read `schema_version`, `embedding_text` or extra payload fields to construct a candidate. Unknown type/surface tokens and malformed IDs fail decoding.

## Indexed Objects And Surfaces

| Object type | Surface | Embedding text |
|---|---|---|
| `episode` | `summary` | `Episode summary: ` followed by the episode summary |
| `episode` | `summary` | `Episode summary: ` followed by the summary, then labelled scene words when supplied |
| `observation` | `text` | `Observation excerpt: ` followed by observation text |
| `memory_thread` | `summary` | `Thread summary: ` followed by title and summary |
| `derived_memory` | `derived_text` | A category label followed by interpreted-memory text |

Each indexed object has at most one surface. `max_embedding_surfaces` returns zero for `entity` and `memory_link`; neither has an object vector builder. The [embedding builders](../../../src/policy/embedding_surface.rs) define these limits and fold whitespace in the natural-language input. The `query` surface identifies query embeddings rather than stored memory content.

The [episode builder](../../../src/policy/embedding_surface.rs) appends nonblank setting words on a new `Setting:` line, followed by one `With:` line per participant joining its nonblank name and description with a comma, in authored order, even when the participant also has an identity key. It folds whitespace just as it does for the summary, while Oxigraph retains the original scene values. Keys and custom values are excluded. With no scene words the embedding text is byte-identical to the summary-only form. These words share the episode's existing `summary` surface; there is no second vector or scene surface.

The closed surface tokens are `summary`, `text`, `derived_text` and `query`. The domain enums own their persisted spelling and parsing. Graph object kinds still include notions and links even though those kinds have no emitted embedding surface.

## Belief Content And Graph Authority
Expand All @@ -41,7 +43,7 @@ Embedding text uses natural language, for example `User preference: Prefer deter

## Indexing And Currency

Embedding happens before a write takes its turn, so no turn holds a model call; inside the turn, indexing consults incoming graph supersession evidence before writing any interpreted memory's vector. Already-superseded memories are excluded, including when an older plan is replayed. A graph-query failure is reported through the typed vector-indexing repair outcome; it does not permit unverified re-indexing. The [indexing service (`vector_indexing.rs:36`)](../../../src/usecases/vector_indexing.rs#L36) owns this admission check.
Embedding happens before a write takes its turn, so no turn holds a model call; inside the turn, indexing consults incoming graph supersession evidence before writing any interpreted memory's vector. Already-superseded memories are excluded, including when an older plan is replayed. A graph-query failure is reported through the typed vector-indexing repair outcome; it does not permit unverified re-indexing. The [indexing service](../../../src/usecases/vector_indexing.rs) owns this admission check.

Record embeddings with zero norm are rejected before the adapter call, and dimensions must match the configured store. Query embeddings must satisfy the same cosine-search constraints. A successor derives `Supersedes` links from its predecessor list and schedules predecessor-vector deletion. If deletion fails, graph supersession still excludes those predecessors from default retrieval and the write outcome reports the maintenance failure.

Expand Down
28 changes: 17 additions & 11 deletions src/adapters/oxigraph/rdf_mapping.rs
Original file line number Diff line number Diff line change
Expand Up @@ -134,18 +134,31 @@ fn episode_triples(episode: &Episode) -> Vec<RdfTriple> {
enum_value(episode.retention_state),
),
RdfTriple::literal(&subject, vocab::CREATED_AT, timestamp(episode.created_at)),
RdfTriple::literal(&subject, vocab::SCENE_TIME, timestamp(episode.scene.time)),
RdfTriple::literal(
&subject,
vocab::SCENE_PARTICIPANTS,
serde_json::to_string(&episode.scene.participants)
.expect("scene participants contain only strings and UUIDs"),
),
RdfTriple::literal(
&subject,
vocab::SCENE_CUSTOM_VALUES,
serde_json::to_string(&episode.scene.custom_values)
.expect("scene custom values contain only strings"),
),
]);
push_optional_literal(
&mut triples,
&subject,
vocab::SOURCE_CONVERSATION_ID,
episode.source_conversation_id.as_deref(),
vocab::SETTING_KEY,
episode.scene.setting.key.as_deref(),
);
push_optional_literal(
&mut triples,
&subject,
vocab::STARTED_AT,
episode.started_at.map(timestamp),
vocab::SETTING_WORDS,
episode.scene.setting.words.as_deref(),
);
push_optional_literal(
&mut triples,
Expand All @@ -159,13 +172,6 @@ fn episode_triples(episode: &Episode) -> Vec<RdfTriple> {
vocab::RAW_REF,
episode.raw_ref.as_deref(),
);
for entity_id in &episode.participant_entity_ids {
triples.push(RdfTriple::resource(
&subject,
vocab::PARTICIPANT_ENTITY,
graph_uri(ObjectType::Entity, *entity_id),
));
}
triples
}

Expand Down
25 changes: 19 additions & 6 deletions src/adapters/oxigraph/shared.rs
Original file line number Diff line number Diff line change
Expand Up @@ -290,13 +290,26 @@ pub(super) fn memory_object_from_rdf(
id: memory_id_literal(subject, values, super::vocabulary::OBJECT_ID)?,
object_type,
modality: enum_literal(subject, values, super::vocabulary::MODALITY)?,
source_conversation_id: values
.optional_literal(super::vocabulary::SOURCE_CONVERSATION_ID),
started_at: optional_timestamp_literal(values, super::vocabulary::STARTED_AT)?,
scene: crate::domain::Scene {
time: timestamp_literal(subject, values, super::vocabulary::SCENE_TIME)?,
participants: serde_json::from_str(
&values.literal(subject, super::vocabulary::SCENE_PARTICIPANTS)?,
)
.map_err(|error| {
rdf_parse_error(subject, super::vocabulary::SCENE_PARTICIPANTS, error)
})?,
setting: crate::domain::SceneSetting {
key: values.optional_literal(super::vocabulary::SETTING_KEY),
words: values.optional_literal(super::vocabulary::SETTING_WORDS),
},
custom_values: serde_json::from_str(
&values.literal(subject, super::vocabulary::SCENE_CUSTOM_VALUES)?,
)
.map_err(|error| {
rdf_parse_error(subject, super::vocabulary::SCENE_CUSTOM_VALUES, error)
})?,
},
ended_at: optional_timestamp_literal(values, super::vocabulary::ENDED_AT)?,
participant_entity_ids: memory_ids_from_resources(
values.resource_values(super::vocabulary::PARTICIPANT_ENTITY),
)?,
summary: values.literal(subject, super::vocabulary::SUMMARY)?,
raw_ref: values.optional_literal(super::vocabulary::RAW_REF),
salience_score: f32_literal(subject, values, super::vocabulary::SALIENCE_SCORE)?,
Expand Down
Loading
Loading