Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

Character Memory is a Rust library for giving LLM assistants memory that shapes behavior over time.

A recognized key or name brings back where things stand with that person or in that place. A description such as "a man with a deep voice" or "at my desk" brings recent occasions like this to mind, without deciding who the person is or treating the description as a place's identity.

It is built for persistent AI assistants and companions that should remember past interactions, recognize recurring entities, and maintain character continuity across sessions.

Instead of treating memory as a chat log or simple vector search, Character Memory stores episodic memories with temporal and relational structure.
Expand Down Expand Up @@ -48,20 +50,26 @@ Retrieval is graph-authoritative and hybrid:
- **Entity-based retrieval:** includes memories involving the same people, projects, places, or concepts
- **Continuity retrieval:** returns a structured `ContinuityContextPack` rather than a generic ranked list

`RetrievalContext` carries a `Scene` and an optional topic; its time defaults to now. Participant keys and exact names cue notions. The topic searches content surfaces; setting words search episode setting surfaces, and all participants' names and descriptions join into one search of episode participant surfaces. Scene words are indexed separately from the episode summary, with up to three embeddings per episode in the existing write batch. A setting key or custom value recalls current beliefs grounded in experiences with that context; beliefs with several sources can return in any of their recorded places or named contexts. The result returns the present scene, its reference resolutions, and each admitted memory's recorded source scenes even without a trace. Forgotten sources are explicit; forgotten scenes follow `include_suppressed`. Scene differences never exclude a memory or determine who may hear it.
`RetrievalContext` carries a `Scene` and an optional topic; its time defaults to now. The topic recalls what happened or was learned, while descriptions recall occasions with similar surroundings or people. A setting key or custom value recalls current beliefs grounded in experiences with that context; beliefs with several sources can return in any of their recorded places or named contexts. The result returns the present scene, its reference resolutions, and each admitted memory's recorded source scenes even without a trace. Forgotten sources are explicit; forgotten scenes follow `include_suppressed`. Scene differences never exclude a memory or determine who may hear it.

An episode has a content search surface for its summary and up to two separate scene surfaces: the setting words and the participants' words. Scene words are not appended to the content text. All participants' names and descriptions are searched together, as they were recorded, so any similarity score for them belongs to that shared search. See the [vector design](docs/design/database/vector_payload_design.md#indexed-objects-and-surfaces) for the stored forms.

For each recognized participant, `last_interactions` says when the character last met them and how much time has passed. If a name could mean several participants, each has its own answer. No eligible encounter at or before the scene time means never met. These facts remain available even when no memories fit the requested amount. Forgotten encounters count only when `include_suppressed` is enabled.

Use `.with_activity(ActivityRef::Thread(thread_id))` or `.with_activity(ActivityRef::OpenLoop(open_loop_id))` to recall ongoing work without a topic. Thread membership, open-loop sources and linked memories supply candidates within the retrieval limits. The result echoes the activity with `Found` or `Unknown`; finding an activity does not guarantee an admitted memory. With tracing enabled, each section assignment reports its set of `CueKind` values: `Topic`, `Participant`, `Place` and `Activity`.

Thread members are taken most recent first by creation time; an open loop's source episodes are taken most recent first by recorded scene time.

Candidate, graph root and section selection serve each cue kind's floor from its own order. Keyed and named routes take their kind's floor before reminders. Descriptions contribute their most recent recallable occasions at or before the scene time, as many as the kind's floor and at least one. Spare room follows score order at candidate and section caps; only root selection shares spare turns among kinds. A reminder can bring memories resting on its occasion. A person or thread reached only through that reminder is a leaf: it can be included, but recall does not continue through it. When a topic, key or name also reaches the occasion, linked history inherits that route’s score; the occasion itself keeps its best matching score. The pack keeps its final score order.
A description brings the most recent few recallable occasions among its fetched matches, at or before the scene time: as many as its cue kind's reserved amount, called its floor, and at least the latest one. Its other matches contribute nothing; the trace counts those left out by this limit. What was brought then competes by relevance. Keys and names take their kind's reserved room first, so adding a description does not displace what is already known merely to fill that reservation. Turns among cue kinds apply only where the memories to expand from are chosen; everywhere else, spare room goes by score.

An occasion recalled by a description can bring what was observed and concluded there. It does not by itself bring the whole history of a person or thread mentioned there. A topic, key or name can still recall that history on its own strength.

Each cue kind has a floor at the merged candidate cap, the graph root cap and each section cap. The measured defaults reserve one slot each for participant, place, activity and topic. Floors apply per kind, not per person or place: five people share one participant floor. `context.cue_floors` is a calibration knob for measured defaults; applications are not expected to set it. Slots are reserved in successive rounds in that order. Unused room returns to the common pool, and selected memories keep their original order. A zero floor reserves nothing for that kind; a zero cap admits nothing. A single kind also receives its floor from its own queue, then fills remaining room in final-ranked order. Topic-only retrieval keeps its selection. Calibration uses the public companion evaluation repository, a development aid outside the core library.

With tracing enabled, `floor_admissions` identifies the object, stage and cue kind when a reserved or spare turn admitted an object outside that stage's original capped prefix. Earlier-stage evidence remains even when a later stage omits the object.

The trace's `scene_cue_searches` reports the best similarity found for the setting and for the joined participant words, with the references sharing each search; a search with no matches has no score. Each entry identifies its cue kind and counts recallable matches left out by the description's occasion limit in `omitted_count`. There is no minimum similarity: whether a description is too weak to remind the character of anything remains a question for measurement, not a decision made by this score.

A participant present in most experiences brings fewer past encounters to mind, while retaining the latest eligible one within the requested limits. A participant recognized by key or name brings only encounters at or before the scene time. Several remarks or participants in one encounter do not make it count more than once. Familiarity limits recalled encounters, not beliefs about that participant. This applies to every notion, including whichever one the application regards as the character.

Meeting a participant recognized by key or name brings current beliefs about them, even if those beliefs were learned elsewhere. More salient beliefs come first, with newer beliefs breaking ties. Replaced beliefs appear only when `include_superseded` is enabled, and current beliefs get priority. When several participants bring beliefs to mind, they take turns in scene order; a shared belief counts for each participant. The scene time limits recalled encounters, not these beliefs. See the [retrieval design](docs/design/database/graph_schema_design.md#state-and-last-interaction) for selection details.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Plan: where and with whom, given in words, reminds the character of the last times like this, and never crowds out what is being talked about

- status: in-progress
- status: completed
- generated: 2026-09-21
- last_updated: 2026-09-21
- last_updated: 2026-09-23
- work_type: code

## Goal
Expand Down Expand Up @@ -214,7 +214,10 @@ One crate, shared files: sequential, one worker at a time, about 10, 12 and 4 wo

## Progress Log (append-only)

- (none yet)
- 2026-09-23 Task_1 completed at reviewed tip `610b191` ([PR 135](https://github.com/ebigunso/character-memory/pull/135)): content and scene words use separate search surfaces.
- 2026-09-23 Task_2 completed at reviewed tip `63f176f` ([PR 137](https://github.com/ebigunso/character-memory/pull/137)): descriptions contribute recent occasions, score fills spare room, and turns remain at root selection.
- 2026-09-23 Task_3 completed at reviewed tip `4cf9763` ([PR 141](https://github.com/ebigunso/character-memory/pull/141)): per-search best scores and shared references are traced, with consumer and schema documentation.
- 2026-09-23 Completion value audit verdict: COMPLETE against the after-the-fix measurements. Closeout consolidates each search's kind, score, references and omission count in one trace entry, copies only returned vector candidates, and centralizes vector-root construction and the reminder-only predicate; recall behavior is preserved.

## Decision Log (append-only; re-plans and major discoveries)

Expand All @@ -231,6 +234,14 @@ One crate, shared files: sequential, one worker at a time, about 10, 12 and 4 wo
- User approval: ruled by the decider, 2026-09-21.
- Record proposed: none in this slice. One proposed design record with one decision (a description never takes an identity's standing in recall) is drafted only after the fourth-pin measurement.

- 2026-09-23 Decision: accept the completion audit verdict and close the scene-words plan without a similarity bound.
- Trigger / new insight: the public companion evaluation repository, `CharacterMemoryEvals`, provides development evidence outside the core library in [scene reminder calibration, after the fix](https://github.com/ebigunso/character-memory-evals/tree/784c2784260196352d7cc5184e0992260f0b1b0b/docs/evidence/calibration/scene-reminders-2026-09-21). Under its controlled synthetic geometry, default-floor topic targets improve from 3 of 8 to the topic-alone 6 of 8 in all twelve pressure cases spanning identical/reworded descriptions, place/participant/both pressure and both native ID orders. Keyless and scene-only probes bring exactly the latest N occasions at the tested floors.
- Evidence boundary: the orthogonal participant/place shares of 72.7% and 50.0% become structural zeros because that corpus has no stored scene words. Separately, weak stranger and unfamiliar-place queries against populated scene stores contribute one occasion and one exclusive slot at the default floor. The unlived-topic share is 29.4% after the fix, unchanged from the intermediate pin and down from 35.3% at the before pin. These are distinct controls, not one percentage-to-slot comparison. Controlled paraphrase distributions have no separable band and are not a production-language threshold estimate, so no bound ships.
- Plan delta (what changed): mark completed and move this plan to the completed folder; retain the scope and reviewed task results. Consolidate the parallel trace fields while preserving the best-score reader, remove the fetched-pool copy, and use one vector-root constructor and reminder-only predicate. The proposed test-file merge is deferred to phase closeout.
- Tradeoffs considered: retain parallel trace summaries versus one entry per actual search; the latter keeps attribution and omission counts together. A threshold from the controlled geometry was rejected because the measurement cannot choose a production embedder's bound.
- Decider approval: completion accepted through the completion audit and the delegated closeout ruling, 2026-09-22.
- Record proposed: none in this closeout.

## Notes
- Risks: a description brings only as many occasions as its floor value (one at a floor of zero), so in a keyless deployment with default floors the scene's share is small by design, and calibration of the floor values is what widens it; the closed pool for a label repeated for years is a few thousand refs and one bounded read, and if that read measures as costly the worker reports it and builds nothing else; tests that used appended words to make an episode findable by topic fail for a reason unrelated to their intent and are rewritten to say what they mean; embedding cost for a worded scene rises by up to two inputs per episode, not per observation.
- Edge cases, with the expected result: a participant given by key and by description is knowing through the key, and its words still join the participants cue; a memory reached both ways has full standing. The same words as topic and as setting run two scoped searches. A name nobody bears activates nothing as a name, as today, and its words still join the participants cue. A scene with only a time writes exactly what it writes today. A store indexed before this slice has no scene surfaces: every description search reports no best score and brings nothing.
4 changes: 3 additions & 1 deletion docs/design/database/schema_cheat_sheet.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,9 @@ These are the five emitted payload fields. Both adapters index `object_id`, `obj
| `entity` | — | Graph notion identity | 0 |
| `memory_link` | — | Graph relationship | 0 |

The `query` surface represents search input. It is not emitted for a stored memory object. A topic searches content surfaces; scene words search their respective episode surfaces. Exact names also resolve through graph assertions.
The `query` surface represents search input. It is not emitted for a stored memory object. A topic searches content surfaces; scene words search their respective episode surfaces and are not part of the summary text. Exact names also resolve through graph assertions.

Descriptions contribute their most recent recallable occasions from the fetched matches, as many as the kind's floor and at least one; the rest of their matches contribute nothing. Trace-only `scene_cue_searches` lists a best score and the references sharing each search, once for the setting and once for all participant words; no returned match means `null`. Each search entry also carries its `cue_kind` and `omitted_count`, the number of recallable matches excluded by the occasion limit. These are retrieval diagnostics, not stored vector payload fields. See [description recall and trace](vector_payload_design.md#description-recall-and-trace) for score, admission and identity boundaries.

## Graph Resources

Expand Down
12 changes: 11 additions & 1 deletion docs/design/database/vector_payload_design.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,17 @@ The [plural builder](../../../src/policy/embedding_surface.rs) emits the unchang

The closed surface tokens are `summary`, `text`, `derived_text`, `scene_setting`, `scene_participants` and `query`. The domain enums own their persisted spelling and parsing. Graph object kinds still include notions and links even though those kinds have no emitted embedding surface.

Topic searches use only the three content surfaces. Setting words search `scene_setting`; participant words are joined with the same composition as the write surface and searched once on `scene_participants`. Query embeddings may be reused for equal text, but results are cached by text and surface scope. Nearest search, zero-norm scroll and completeness counts share that scope in both adapters. Existing indexes must be rebuilt to add scene surfaces; no migration is provided.
Topic searches use only the three content surfaces. Setting words search `scene_setting`; participant words are joined with the same composition as the write surface and searched once on `scene_participants`. Query embeddings may be reused for equal text, but results are cached by text and surface scope. Nearest search, zero-norm scroll and completeness counts share that scope in both adapters. An index built without scene surfaces returns no description matches until rebuilt; no migration is provided.

## Description Recall And Trace

A key or a known name brings current beliefs about the person or context it identifies. A description brings recent occasions with similar recorded words, without establishing an identity. Each description search contributes only its most recent recallable episodes at or before the retrieval scene time, as many as its cue kind's floor and never fewer than one. Selection uses the fetched search pool, including the fetched tie at the cutoff; an open search boundary limits this claim to the fetched matches. Other matches do not enter the candidate set through that description. Each trace entry in `scene_cue_searches` records in `omitted_count` the number of recallable matches excluded by this occasion limit, once for the setting and once for the joined participants.

The contributed occasions compete by their similarity scores within the existing relevance ranking. Keys, names and current state take their kind's floor before description matches. Spare room follows score order when candidates and context-pack sections are selected; only graph-root selection shares spare room by turns among kinds. An occasion recalled through words can bring its observations and conclusions, but does not by itself open the other occasions of a person or thread reached from it. When the topic or a known reference also reaches it, linked history inherits that source's score; a strong description match cannot lend its score to a weak topic match.

With tracing enabled, [scene search trace emission](../../../src/usecases/retrieve/scene.rs) emits one `scene_cue_searches` entry for each scene search. Each [entry](../../../src/api/types/retrieval.rs) lists its `cue_kind`, the `SceneReference` values sharing that search, its `omitted_count`, and its `best_score`: the highest fetched scene-surface score before graph eligibility and occasion selection, or `null` if no scene-surface match was returned. The setting entry refers to `SettingWords`; the participants entry lists all nonblank names and descriptions that contributed to the single joined search, including names whose identity lookup was unknown. Its score is shared, never a separate score for each person. Keys do not contribute words. An empty scene omits the scene-search field; readers treat an absent field as an empty list. Tracing disabled produces no trace.

This score describes the search, not whether the best match was admitted or whether the description identifies someone. No minimum similarity is applied. These scores support measuring whether a useful boundary for descriptions that remind of nothing exists. That measurement belongs in the public companion evaluation repository, `CharacterMemoryEvals`, whose evaluation tooling is a development aid outside the core library. A decision to apply such a boundary requires that evidence.

## Belief Content And Graph Authority

Expand Down
Loading