Add cisco:ctr:device:past_ids to the identity-assertion vocabulary (XDR-48383)

[yogsototh]

JIRA https://cisco-sbg.atlassian.net/browse/XDR-48383

Summary

Adds cisco:ctr:device:past_ids to the IdentityAssertion open vocabulary, so
providers can record the device ids a device was previously known by.

Device Insights merges duplicate device records into a single device. Before
XDR-48383 the merged-away id was lost, so "query asset by id" against the old id
failed even though the device still existed. This field publishes the past ids
for that lookup.

Requested by @zterlizz for Device ID Tracking in Assets.

Changes

• src/ctim/schemas/identity_assertion.cljc -- adds cisco:ctr:device:past_ids
  to the assertion set; rewrites the AssertionType :description to document
  the multi-value encoding the producer actually emits (see below).
• src/ctim/examples/identity_assertions.cljc -- identity-assertion-maximal
  gains a cisco:ctr:device:id entry plus a cisco:ctr:device:past_ids entry
  carrying a JSON-encoded array, demonstrating the current-id / past-ids pairing.
• doc/structures/bundle.md, doc/structures/casebook.md -- regenerated with
  lein doc.

Two things worth a reviewer's attention

1. This changes no validation behavior. AssertionType is :open? true,
which flanders compiles to plain s/Str -- the vocabulary set is erased at
compile time. An unlisted assertion name already validated before this change
(s/check on {:name "cisco:ctr:device:past_ids" :value "x"} returned nil
against origin/master). The value of this PR is discoverability and a
published contract, not enforcement.

2. Multi-valued assertions are carried as a JSON-encoded array inside the
single :value string. Assertion is {:name Str, :value Str} -- :value
is a single string. The Device Insights producer encodes any non-string value
as a JSON-encoded array of strings in that one :value, and this is its blanket
rule for every plural-named entry (cisco:ctr:user:emails, :groups,
:phone_numbers, cisco:ctr:device:past_ids); the acceptance test JSON-decodes
the value. The description now documents that shape:

:assertions [{:name "cisco:ctr:device:id" :value "00-00-00-01"}
             {:name "cisco:ctr:device:past_ids" :value "[\"00-00-00-08\",\"00-00-00-09\"]"}]

This matches the 11 plural-named entries already in the vocabulary
(cisco:ctr:user:emails, :phone_numbers, :roles, :groups,
:entitlements, cisco:ctr:device:administrators, the four
cisco:ctr:ad:*_resolved_*, and cisco:ctr:common:ir_attributes), all of which
carry :value Str. Because :value is s/Str, a nested array is invisible to
schema validation -- consumers must JSON-decode it. A consumer that collapses
assertions with (into {} (map (juxt :name :value))) keeps the array string
intact under one key, which is the intended shape.

Alternatives considered and rejected:

• A singular past_id name would be the vocabulary's only singular-repeat
  entry, contradicting 11 plural siblings.
• Repeating the entry once per id -- the convention this PR originally
  documented -- is not what any producer emits (see @msprunck's review thread,
  which verified the producer directly). Documenting it would have marked the
  only producer non-conformant and prescribed a shape no consumer receives.
• Widening Assertion :value to accept arrays, or adding an optional :values
  entry, is forward-incompatible, not merely a contract change: flanders map
  types compile to closed maps, so any consumer validating against an older CTIM
  jar rejects the extra/retyped key as a disallowed-key error.

Testing

• JVM: lein with-profile -user test -- 139 tests, 510 assertions, 0 failures.
• ClojureScript (the path CI actually gates on --
  .github/workflows/build.yml:57 runs lein do clean, compile :all, doo node node once; there is no lein test step in CI):
  lein with-profile -user do clean, doo node node once -- 36 tests, 147
  assertions, 0 failures.
• lein doc regenerates doc/structures/bundle.md and casebook.md with the
  new description.
• Note on doc/json: lein doc on a -SNAPSHOT version also rewrites
  schema_version in 21 doc/json/*.json files (1.3.30 -> 1.3.30-SNAPSHOT);
  that churn is pre-existing and unrelated, so it is excluded here. This
  description edit does not rewrite doc/json on its own, but the earlier claim
  that doc/json's example value is "vocabulary-independent" is not accurate:
  the flanders enum's :default is (-> values sort first) and ->json does
  not null defaults, so doc/json/bundle.json and casebook.json escape churn
  here only because cisco:ctr:ad:host_domain_name still sorts first. A future
  vocabulary addition that sorts earlier (e.g. a cisco:ctr:aa:* name) WILL
  rewrite those files -- the next contributor adding such a name must regenerate.

QA

1. Post a bundle containing an IdentityAssertion whose :assertions include a
   cisco:ctr:device:past_ids entry whose :value is a JSON-encoded array of
   ids (e.g. "[\"00-00-00-08\",\"00-00-00-09\"]"); confirm it round-trips and
   the value is preserved verbatim for consumers to JSON-decode.
2. Confirm cisco:ctr:device:past_ids appears in the AssertionType vocabulary
   in the generated docs (doc/structures/bundle.md, casebook.md).
3. Regression: confirm an assertion using an existing name
   (e.g. cisco:ctr:device:id) still validates unchanged.

Out of scope (pre-existing, filed separately if wanted)

doc/README.md:45,72 link to structures/identity_assertion.md and
json/identity_assertion.json, which have never been generated -- there are no
IdentityAssertion rows in src/ctim/document.clj. Dead since the entity was
added in 2019 (e5f4e62). Not touched here to keep this diff minimal.

Rejected findings

• Cross-reference the multiplicity convention on :assertions :description (L100-101) (msprunck, 2026-08-12 approval) -- Declined. The convention now lives on the exact field it governs (Assertion :value, added in 7cd2fb4) and on AssertionType. :assertions describes the collection at a higher level; duplicating the encoding rule a third time on the parent collection adds drift risk without giving a decoding consumer a new surface to read. The :value cross-reference from the same review was addressed.

🤖 Generated with Claude Code

[yogsototh]

@zterlizz this is the cisco:ctr:device:past_ids field you asked for -- please take a look and comment here if you want anything changed. I can push changes, or you can propose them directly (see the note at the bottom).

Two points that may affect how your side uses it:

The schema cannot hold an array. Assertion is {:name Str, :value Str} -- :value is a single string, and an array value is rejected by validation. So instead of one entry holding an array, the convention is to repeat the entry once per past id:

:assertions [{:name "cisco:ctr:device:past_ids" :value "00-00-00-01"}
             {:name "cisco:ctr:device:past_ids" :value "00-00-00-02"}]

This matches the nine plural entries already in the vocabulary (cisco:ctr:user:emails, :roles, :groups, cisco:ctr:device:administrators, the cisco:ctr:ad:*_resolved_* names), which all carry :value Str. If you need a true array shape instead, say so -- that is a change to Assertion itself and affects every CTIM consumer, so it needs a wider discussion than this PR.

One consumer-side caveat worth flagging: if anything downstream collapses assertions with (into {} (map (juxt :name :value) assertions)), it will silently keep only the last past id. Use group-by :name for multi-valued assertions.

You are likely not blocked today. AssertionType is an open vocabulary (:open? true), which compiles to plain s/Str -- the vocabulary set is erased at compile time. I verified against master that an unlisted assertion name already validates: s/check on {:name "cisco:ctr:device:past_ids" :value "x"} returns nil without this PR. So this publishes the field for discoverability and gives it a documented contract, but it is not a gate you were waiting on. If you were seeing rejections, they are coming from somewhere other than CTIM schema validation and it would be worth chasing separately.

On the push-access point -- you mentioned you could not push to threatgrid/ctim. For future changes you should not need push access: this repo merges PRs from forks, and PR #457 ("XDR-15964 add detection_sources to incident") was authored from a fork by a non-collaborator and merged. Fork, branch, and open a PR should work for you directly.

CODEOWNERS review still applies here, so this needs a maintainer approval before it can merge.

[zterlizz]

This is exactly what I was hoping for. Thank you, Yann!

[msprunck]

The vocabulary entry itself is correct and uncontroversial. One blocking issue: the multiplicity convention documented alongside it describes a shape the producer does not emit. Details inline.

I verified the producer implementation and its integration tests directly; since that source is not public I have kept the specifics out of this review and posted them to XDR-48383 instead. Happy to walk through them internally.

Where the convention lives

Not inlineable (outside the diff), but worth fixing in the same pass: the multiplicity rule is a property of value and of the list, yet it is attached to AssertionType, i.e. the type of the sibling name key. It renders under ## Property name (bundle.md:11120), above the 52-item allowed-values list. A consumer reading ## Property assertions (bundle.md:10913, still "Any known context about the identity attributes.") or the Assertion object table (bundle.md:11108-11110, both Description cells empty) is told nothing. Consider moving it to the :assertions :description at identity_assertion.cljc:97-98, or adding one to :value at L82.

Corrections for the PR description

• "nine plural entries" is 11 — missing cisco:ctr:user:phone_numbers and cisco:ctr:common:ir_attributes. The undercount favours your argument.
• C6's conclusion is right, the reason isn't. doc/json's example value is not vocabulary-independent: flanders' enum sets :default (-> values sort first) and ->json does not null defaults. doc/json escaped churn only because cisco:ctr:ad:host_domain_name still sorts first. A future addition sorting earlier (cisco:ctr:aa:*) will rewrite doc/json/bundle.json and casebook.json. Worth stating so the next contributor regenerates.
• Your dismissal of alternative (c) is stronger than you argued: flanders map types compile to closed maps, so an optional :values entry is rejected as disallowed-key by any consumer validating against an older CTIM jar. Forward-incompatible, not merely a contract change.
• C4 reports a command CI doesn't run. .github/workflows/build.yml:57 gates on lein do clean, compile :all, doo node node once — there is no lein test step. That gate also passes: 36 tests, 147 assertions.

Verified

Independently reproduced: C1 (:open? true makes flanders emit bare s/Str, :values discarded; {:name "totally:made:up"} validated before this change and still does — no validation change in either direction), C2's mechanics, C3, C5 (regenerated docs byte-identical: 678209/678209 chars bundle.md, 696079/696079 casebook.md), C6's conclusion, C7, and C4's 139 tests / 510 assertions. bundle.md and casebook.md are the complete set of generated artifacts carrying this vocabulary; both are updated and nothing is stale.

Non-blocking follow-ups

• doc/README.md:45,72 link to structures/identity_assertion.md and json/identity_assertion.json, which have never been generated — src/ctim/document.clj has no IdentityAssertion row. You declared this out of scope and it is genuinely separable, but it means this field's only published home is inside two ~680KB files. A linked issue would beat an unlinked note.
• Nothing ties example assertion names to the assertion set — a singular past_id typo would pass all 139 JVM tests, all 36 cljs tests and lein doc silently.
• A number of assertion names that appear in practice are absent from this set, and the plural entries are not represented uniformly across every XDR surface. Combined with the encoding issue below, "published contract" is currently aspirational — further argument for describing observed behaviour rather than prescribing new behaviour here.
• Pre-existing: lein release is already broken at HEAD. resources/ctim/version.txt is 1.3.30-SNAPSHOT, committed doc/json says 1.3.30, and :release-tasks runs doc then vcs assert-committed, so the 21-file churn will trip it regardless of this PR.

[yogsototh]

Addressing @msprunck's review

Commit 775b6af addresses all findings. Thanks for verifying the producer directly -- the central finding was correct and changed the design of this PR.

• [correctness, schema :description] Fixed. The prior description prescribed a repeat-per-entry convention that no producer emits. The Device Insights producer emits a single entry whose value is a JSON-encoded array of strings, for every plural name. Adopted your suggested description verbatim.
• [uniformity risk] Addressed. The JSON-encoded-array claim is scoped to the single-value-string case this schema actually enforces (:value is s/Str), with "for example" framing rather than a per-name normative list, so it stays true and does not over-claim for unconfirmed paths.
• ["posture endpoint" wording] Fixed. Replaced with "after duplicate device records were merged into one" -- no undefined term, no producer-internal mechanism in a public repo.
• [L12 permalink] Acknowledged (confirmed still resolving). The rewritten description now names AssetProperty and both entities' value, rather than being scoped to "An Assertion :value".
• [example shape] Fixed. identity-assertion-maximal now shows cisco:ctr:device:id "00-00-00-01" plus a single cisco:ctr:device:past_ids entry carrying ["00-00-00-08","00-00-00-09"] -- current-id/past-ids pairing demonstrated, no cross-fixture id collision.
• [doc regen] Done. lein doc regenerated bundle.md:11120 and casebook.md:5393 with the new description.

PR description corrected too: the multiplicity section now documents the JSON-encoded-array shape (not repeat-per-entry); "nine plural entries" corrected to 11 (added cisco:ctr:user:phone_numbers and cisco:ctr:common:ir_attributes); the alternative-(c) dismissal is now stated as forward-incompatibility (closed maps reject the extra/retyped key against an older jar); the CI note now reflects that .github/workflows/build.yml gates on doo node node once, not lein test; and the doc/json note now explains that those files escape churn only because cisco:ctr:ad:host_domain_name still sorts first.

Verification: 139 JVM tests / 510 assertions and 36 ClojureScript tests / 147 assertions, 0 failures.

[msprunck]

Both blocking items from my previous review are fixed and verified on 775b6af. Approving.

Verified

• identity_assertion.cljc:70-78 — description now documents the encoding the producer actually emits, drops the producer-specific jargon, spells the field `value`, and names AssetProperty alongside IdentityAssertion.
• identity_assertions.cljc:28-29 — fixture is now a single entry with a JSON-encoded array, plus a cisco:ctr:device:id entry, so the example demonstrates the current-id / past-ids pairing and no longer collides with identity-assertion-minimal.
• Docs regenerated: bundle.md:11120, casebook.md:5393.
• PR description updated on all four corrections (11 plural entries, the forward-incompatibility of widening :value / adding :values, the rewritten rejected-alternatives list, and the encoding rationale).

Re-ran the checks myself: lein with-profile -user test → 139 tests, 510 assertions, 0 failures, 0 errors. The command CI actually gates on, lein do clean, compile :all, doo node node once → 36 tests, 147 assertions, 0 failures, 0 errors. lein doc leaves doc/structures/ byte-identical to the committed files, with only the pre-existing 21-file schema_version SNAPSHOT churn appearing — i.e. nothing is missing from this PR.

Retracting one of my earlier comments

My inline comment on the :description asking you to scope the rule per-name, and warning that these names are not represented uniformly across every XDR surface, is withdrawn. Confirmed internally that Device Insights is the only producer and applies this encoding uniformly, so the universal wording is sound as written — no change needed there. Sorry for the detour.

Non-blocking leftovers, none of which should hold this up

• The multiplicity rule still lives on AssertionType, i.e. the type of the sibling name key. :assertions :description (L100-101) still reads only "Any known context about the identity attributes." and :value (L82) has none, so a consumer who navigates to either surface does not see the convention. Worth a follow-up, not a blocker.
• asset_properties.cljc is unchanged. The new text names AssetProperty, but an AssetProperties reader gets no cross-reference back to this vocabulary beyond the existing #L11 permalink.
• Still outstanding from before: doc/README.md:45,72 dead links (no IdentityAssertion row in src/ctim/document.clj); nothing ties example assertion names to the assertion set, so a singular past_id typo would pass every gate silently; a number of names emitted in practice are absent from the set; and lein release is already broken at HEAD (resources/ctim/version.txt is 1.3.30-SNAPSHOT, committed doc/json says 1.3.30, and :release-tasks runs doc then vcs assert-committed).

One observation for a separate cleanup: if there is a single producer, the four cisco:ctr:ad:*_resolved_* entries and cisco:ctr:common:ir_attributes appear to be dead vocabulary that nothing emits.

[yogsototh]

Thanks @msprunck for re-verifying on 775b6af and for re-running the JVM (139 tests, 510 assertions) and cljs (36 tests, 147 assertions) suites plus the lein doc byte-identical check. Noted the retraction on the per-name scoping comment — agreed that universal wording is sound given Device Insights is the sole producer and applies the encoding uniformly.

I've resolved the six inline threads now that each is answered.

The non-blocking leftovers are noted as out-of-scope follow-ups for this single-entry vocabulary addition and can be tracked separately:

• Surfacing the multiplicity convention on :assertions :description (L100-101) and :value (L82) in addition to AssertionType.
• An asset_properties.cljc cross-reference beyond the existing #L11 permalink.
• Pre-existing issues not introduced by this PR: doc/README.md:45,72 dead links; example assertion names not tied to the assertion set; some emitted names absent from the set; the broken lein release at HEAD; and the apparently dead cisco:ctr:ad:*_resolved_* / cisco:ctr:common:ir_attributes vocabulary entries.

[yogsototh]

Thanks @msprunck -- appreciate the re-verification and the retraction on the per-name scoping (agreed: single producer, uniform encoding, so the universal wording holds).

Disposition of the non-blocking leftovers:

Addressed here (7cd2fb4)

• The multiplicity convention was documented only on AssertionType (the type of the sibling :name), and Assertion :value had no description at all. Added a short cross-reference on :value pointing to AssertionType, so a consumer landing on the value surface sees the JSON-encoded-array convention. Doc-only: :value stays f/any-str, no schema-shape or validation change. Regenerated doc/structures/{bundle,casebook}.md; the pre-existing SNAPSHOT schema_version churn in doc/json is excluded, consistent with the rest of this PR.

I left :assertions :description (L100-101) as-is: it describes the collection at a higher level, and the encoding detail now lives on the exact field it governs (:value) plus AssertionType. Duplicating it a third time on the parent collection would add drift risk without adding a surface a decoding consumer actually reads.

Deferred -- out of scope for this PR (pre-existing, not introduced here)

• asset_properties.cljc back-reference: touching a sibling entity's doc to add a cross-reference to this vocabulary is a separate change; the #L11 permalink already exists.
• doc/README.md:45,72 dead links (no IdentityAssertion row in src/ctim/document.clj) -- dead since the entity was added in 2019 (e5f4e62); already called out in this PR's "Out of scope" section.
• No tie between example assertion names and the assertion set (a singular past_id typo would pass silently); names emitted in practice that are absent from the set; lein release broken at HEAD -- all pre-existing infrastructure gaps unrelated to adding past_ids.
• The dead-vocabulary observation (cisco:ctr:ad:*_resolved_*, cisco:ctr:common:ir_attributes) -- a separate cleanup, and removing entries from an open vocabulary is a contract change that warrants its own review.

Happy to file follow-up tickets for any of the deferred items if you'd like them tracked.