Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ Retrieval is graph-authoritative and hybrid:

`RetrievalContext` carries a `Scene` and an optional topic; its time defaults to now. The topic recalls what happened or was learned, while descriptions recall occasions with similar surroundings or people. A setting key or custom value recalls current beliefs grounded in experiences with that context; beliefs with several sources can return in any of their recorded places or named contexts. The result returns the present scene, its reference resolutions, and each admitted memory's recorded source scenes even without a trace. Forgotten sources are explicit; forgotten scenes follow `include_suppressed`. Scene differences never exclude a memory or determine who may hear it.

For an admitted interpreted memory, `memory_scenes[].seconds_since_support` reports whole seconds since its latest supporting experience at or before the reference scene time, even with tracing off. An observation uses its own observed time, falling back to its parent scene time. Suppressed sources count only when `include_suppressed` is enabled; an active observation still counts after its parent episode is forgotten. No eligible support, including application-given beliefs without experiences and future-only support, reports `None` (`null` in JSON). A correction uses its replacement's own sources. This age never changes selection, order or scores.

An episode has a content search surface for its summary and up to two separate scene surfaces: the setting words and the participants' words. Scene words are not appended to the content text. All participants' names and descriptions are searched together, as they were recorded, so any similarity score for them belongs to that shared search. See the [vector design](docs/design/database/vector_payload_design.md#indexed-objects-and-surfaces) for the stored forms.

For each recognized participant, `last_interactions` says when the character last met them and how much time has passed. If a name could mean several participants, each has its own answer. No eligible encounter at or before the scene time means never met. These facts remain available even when no memories fit the requested amount. Forgotten encounters count only when `include_suppressed` is enabled.
Expand Down
6 changes: 6 additions & 0 deletions src/api/types/retrieval.rs
Original file line number Diff line number Diff line change
Expand Up @@ -249,6 +249,12 @@ pub struct MemoryScenes {
pub memory: MemoryObjectRef,
/// Empty means no recorded experience; unavailable sources are explicit entries.
pub sources: Vec<SourceScene>,
/// Whole seconds since this interpreted memory's latest eligible support
/// at or before the retrieval scene time. Observations use their own time, or
/// their parent scene time when absent. None means no such support, or a memory
/// other than an interpreted memory. Suppressed sources count only when
/// include_suppressed is enabled. This fact never changes recall or scores.
pub seconds_since_support: Option<i64>,
}

#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
Expand Down
217 changes: 217 additions & 0 deletions src/memory/retrieval_scene_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1842,6 +1842,223 @@ fn beliefs_context() -> RetrievalContext {
context
}

#[tokio::test]
async fn support_age_reports_experience_without_changing_recall() {
use chrono::{DateTime, Duration, Utc};
use serde_json::json;

let at = |text: &str| text.parse::<DateTime<Utc>>().unwrap();
let reference = at("2026-09-20T10:00:00Z");
let later = at("2026-10-20T10:00:00Z");
let old = at("2025-07-20T10:00:00Z");
let recent = reference - Duration::days(1);
let fallback = reference - Duration::days(3);
let current = reference - Duration::milliseconds(900);
let id = MemoryId::from_u128;
let (memory, queries) = scene_memory().await;
create_notion(&memory, 100, None).await;
let mut plan = RememberWritePlan::new();
for (n, time) in [
(5000, at("2025-06-20T10:00:00Z")),
(6000, reference - Duration::days(2)),
(7000, fallback),
(8000, reference - Duration::days(2)),
(9000, at("2026-12-01T10:00:00Z")),
(10000, reference - Duration::days(1)),
] {
let mut episode = EpisodeDraft::new("An experience behind a belief.");
episode.id = Some(id(n));
episode.scene = Some(Scene::at(time.fixed_offset()));
episode.created_at = Some(later + Duration::days(n as i64));
episode.schema_version = Some(DEFAULT_SCHEMA_VERSION.to_owned());
plan = plan.with_candidate(MemoryCandidate::Episode(EpisodeCandidate::new(
episode,
CandidateProvenance::caller("support age"),
)));
}
for (n, parent, observed_at) in [
(5001, 5000, Some(old)),
(6001, 6000, Some(recent)),
(6002, 6000, Some(recent - Duration::hours(12))),
(7001, 7000, None),
(8001, 8000, Some(at("2026-12-02T10:00:00Z"))),
(10001, 10000, Some(current)),
] {
let mut observation = ObservationDraft::new(id(parent), "Support for a belief.");
observation.id = Some(id(n));
observation.observed_at = observed_at;
observation.created_at = Some(later);
observation.schema_version = Some(DEFAULT_SCHEMA_VERSION.to_owned());
plan = plan.with_candidate(MemoryCandidate::Observation(ObservationCandidate::new(
observation,
CandidateProvenance::caller("support age"),
)));
}
for (n, episodes, observations) in [
(20000, vec![], vec![5001]),
(20001, vec![], vec![6001]),
(20002, vec![], vec![7001]),
(20003, vec![], vec![5001, 6001]),
(20004, vec![5000, 6000], vec![]),
(20005, vec![], vec![8001]),
(20006, vec![9000], vec![]),
(20007, vec![], vec![]),
(20008, vec![], vec![6001, 6002]),
(20009, vec![], vec![10001]),
] {
let mut belief = DerivedMemoryDraft::new(DerivedType::Claim, format!("Belief {n}"));
belief.id = Some(id(n));
belief.entity_ids = vec![id(100)];
belief.given_by_application = episodes.is_empty() && observations.is_empty();
belief.derived_from_episode_ids = episodes.into_iter().map(id).collect();
belief.derived_from_observation_ids = observations.into_iter().map(id).collect();
belief.created_at = Some(later + Duration::days(n as i64));
belief.updated_at = belief.created_at;
belief.schema_version = Some(DEFAULT_SCHEMA_VERSION.to_owned());
plan = plan.with_candidate(MemoryCandidate::DerivedMemory(DerivedMemoryCandidate::new(
belief,
CandidateProvenance::caller("support age"),
)));
}
memory.commit(plan, CommitOptions::default()).await.unwrap();
let mut rows = Vec::new();
let mut mismatches = Vec::new();
for phase in 0..5 {
let targets = match phase {
1 => vec![LifecycleTargetRef::episode(id(6000))],
2 => vec![LifecycleTargetRef::observation(id(6001))],
3 => vec![
LifecycleTargetRef::episode(id(5000)),
LifecycleTargetRef::episode(id(7000)),
],
_ => vec![],
};
for target in targets {
let mut draft = ForgetMemoryDraft::suppress(target, "Retain the supported belief.");
draft.cascade_policy.apply_to_derived_from_target = false;
memory.forget(draft).await.unwrap();
}
if phase == 4 {
let origin = SourceProvenanceReference {
episode_ids: vec![],
observation_ids: vec![id(10001)],
external_refs: vec![],
};
let mut replacement = ReplacementDerivedMemoryDraft::new(
DerivedType::Correction,
"This belief now rests on a different experience.",
);
replacement.id = Some(id(21000));
replacement.entity_ids = vec![id(100)];
replacement.derived_from_observation_ids = vec![id(10001)];
replacement.correction_origin_provenance = origin.clone();
let mut draft = CorrectMemoryDraft::new(
CorrectionTarget::derived_memory(id(20000)),
"Replace the old interpretation.",
)
.with_replacement(replacement);
draft.correction_origin = origin;
memory.correct(draft).await.unwrap();
}
for time in [reference, later] {
for include_suppressed in [false, true] {
let mut present = Scene::at(time.fixed_offset());
present.participants.push(keyed(100));
let mut context = RetrievalContext::default().with_scene(present);
context.candidate_limits.max_graph_roots = 1;
context.graph_limits.max_depth = 1;
context.graph_limits.max_fanout_per_node = 32;
context.graph_limits.timeout_ms = None;
context.graph_limits.allowed_relation_types = vec![RelationType::About];
context.graph_limits.allowed_object_types =
vec![ObjectType::Entity, ObjectType::DerivedMemory];
context.section_limits.derived_memories = 32;
context.lifecycle_policy.include_suppressed = include_suppressed;
let result = memory.retrieve(context.clone()).await.unwrap();
assert!(result.trace.is_none());
assert!(result.pack.relevant_episodes.is_empty());
assert!(result.pack.salient_observations.is_empty());
assert_eq!(result.pack.derived_memories.len(), 10);
let traced = memory.retrieve(context.clone().with_trace()).await.unwrap();
assert_eq!(traced.pack, result.pack);
assert_eq!(traced.memory_scenes, result.memory_scenes);
let trace = traced.trace.as_ref().unwrap();
assert_eq!(
trace
.graph_expansions
.iter()
.filter(|root| root.outcome == GraphExpansionOutcome::Expanded)
.map(|root| root.root.id)
.collect::<Vec<_>>(),
[id(100)]
);
let expected = [
(
if phase == 4 { 21000 } else { 20000 },
Some(if phase == 4 { current } else { old }),
),
(20001, (phase < 2 || include_suppressed).then_some(recent)),
(20002, Some(fallback)),
(
20003,
Some(if phase < 2 || include_suppressed {
recent
} else {
old
}),
),
(
20004,
if phase == 0 || include_suppressed {
Some(reference - Duration::days(2))
} else if phase < 3 {
Some(at("2025-06-20T10:00:00Z"))
} else {
None
},
),
(20005, None),
(20006, None),
(20007, None),
(
20008,
Some(if phase < 2 || include_suppressed {
recent
} else {
recent - Duration::hours(12)
}),
),
(20009, Some(current)),
]
.into_iter()
.map(|(n, support)| {
(
n.to_string(),
support.map(|support| (time - support).num_seconds()),
)
})
.collect::<std::collections::BTreeMap<_, _>>();
for entry in &result.memory_scenes {
let expected_age = expected[&entry.memory.id.as_u128().to_string()];
if entry.seconds_since_support != expected_age {
mismatches.push(format!(
"phase {phase}, reference {time}, include_suppressed {include_suppressed}, memory {:?}: {:?} != {expected_age:?}",
entry.memory, entry.seconds_since_support
));
}
}
rows.push(json!({"phase": phase, "input": context, "outcome": result, "trace": trace, "expected_seconds_since_support": expected}));
}
}
}
assert!(queries.lock().unwrap().is_empty());
println!(
"SUPPORT_AGE_WITNESSES={}",
serde_json::to_string(&rows).unwrap()
);
assert!(mismatches.is_empty(), "{}", mismatches.join("\n"));
}

#[tokio::test]
async fn result_reports_all_source_scenes_beside_recent_episodes_without_trace() {
let (memory, _) = scene_memory().await;
Expand Down
6 changes: 5 additions & 1 deletion src/usecases/retrieve.rs
Original file line number Diff line number Diff line change
Expand Up @@ -531,7 +531,11 @@ where
});

let memory_scenes = self
.memory_scenes(&pack, context.lifecycle_policy.include_suppressed)
.memory_scenes(
&pack,
context.lifecycle_policy.include_suppressed,
context.scene.time.to_utc(),
)
.await?;
Ok(RetrieveOutcome {
scene: context.scene,
Expand Down
46 changes: 45 additions & 1 deletion src/usecases/retrieve/scene.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ use crate::api::types::{
use crate::domain::RetentionState;
use crate::models::vector::CanonicalCandidates;
use crate::ports::graph_authority::GraphObjectQuery;
use chrono::{DateTime, Utc};

pub(super) struct RecallCues {
pub candidates: CanonicalCandidates,
Expand Down Expand Up @@ -351,6 +352,7 @@ where
&self,
pack: &ContinuityContextPack,
include_suppressed: bool,
reference_time: DateTime<Utc>,
) -> Result<Vec<MemoryScenes>, CustomError> {
let mut memories = Vec::new();
let mut objects = HashMap::new();
Expand Down Expand Up @@ -436,6 +438,16 @@ where
Ok(memories
.into_iter()
.map(|(memory, sources)| {
let seconds_since_support = if memory.object_type == ObjectType::DerivedMemory {
sources
.iter()
.filter_map(|source| support_time(*source, &objects, include_suppressed))
.filter(|time| *time <= reference_time)
.max()
.map(|time| (reference_time - time).num_seconds())
} else {
None
};
let mut seen = HashSet::new();
let mut sources = sources
.into_iter()
Expand All @@ -446,12 +458,44 @@ where
let source = source_scene_ref(scene);
(source.object_type.stable_rank(), source.id)
});
MemoryScenes { memory, sources }
MemoryScenes {
memory,
sources,
seconds_since_support,
}
})
.collect())
}
}

fn support_time(
source: MemoryObjectRef,
objects: &HashMap<MemoryObjectRef, MemoryObject>,
include_suppressed: bool,
) -> Option<DateTime<Utc>> {
match objects.get(&source)? {
MemoryObject::Episode(episode)
if include_suppressed || episode.retention_state != RetentionState::Suppressed =>
{
Some(episode.scene.time.to_utc())
}
MemoryObject::Observation(observation)
if include_suppressed || observation.retention_state != RetentionState::Suppressed =>
{
observation.observed_at.or_else(|| {
match objects.get(&MemoryObjectRef::new(
ObjectType::Episode,
observation.episode_id,
))? {
MemoryObject::Episode(episode) => Some(episode.scene.time.to_utc()),
_ => None,
}
})
}
_ => None,
}
}

fn source_scene(
source: MemoryObjectRef,
objects: &HashMap<MemoryObjectRef, MemoryObject>,
Expand Down
Loading