Skip to content

Character voice checks with evidence; imported books start as the baseline - #8

Closed
MrPickering wants to merge 3 commits into
agent/weaver-s6-approvalsfrom
agent/weaver-s4-voice
Closed

MrPickering wants to merge 3 commits into
agent/weaver-s6-approvalsfrom
agent/weaver-s4-voice

Conversation

@MrPickering

Copy link
Copy Markdown
Contributor

Stacked on #7. See the commit message for details.

  • Voice: registry, deterministic attribution (never guesses), profiles from approved text, concrete out-of-voice findings with the speaker's own lines as evidence, and 'maybe sounds like' in both directions. Always warnings, never blocking.
  • Import baseline: weaver import commits the author's original as the starting point, so repairs are reviewed against it one chapter at a time.
  • Verification: 48/48, including a hold-out fixture unlike the tuning data, and a baseline test that fails without the feature.

🤖 Generated with Claude Code

MrPickering and others added 3 commits September 25, 2026 12:58
…eline

In multi-POV fiction, AI assistants blur characters. Every dialogue
change is now checked against how that character actually speaks, with
their own lines as evidence. It is a judgement aid, never a verdict.

- world-bible/characters.json registry (contracts/characters.schema.json);
  `weaver characters seed` adds POV characters from an import.
- Deterministic dialogue attribution: speech tags, the single-named-
  character action beat, and multi-paragraph continuation. Anything else
  stays unattributed; speakers are never guessed.
- `weaver voices build`: per-character profiles from APPROVED text (Git
  HEAD): sentence length, contractions, profanity, formal markers,
  exclamation/question share, distinctive words, and typical evidence
  lines. Characters under 8 lines / 80 words get "not enough of their
  lines yet" instead of a verdict. Approve rebuilds profiles; pending
  edits never make them stale.
- `weaver voices check`: out-of-voice findings with concrete numbers
  ("one 17-word sentence; Grell averages 4.2") and "maybe sounds like"
  another character, in both directions. Always warnings, surfaced in
  `changes`, `check` and the PostToolUse hook; never blocking.
- `weaver import` now records the author's original as the approved
  starting point ("Import original manuscript"), so repairs are reviewed
  against it one chapter at a time instead of the whole book sitting
  pending (which deadlocked approval on the very flaws being repaired).

Verified: npm run preflight (48/48). A hold-out test with characters
unlike the tuning fixture (a terse dockhand and a formal magister)
catches swaps in both directions with evidence, leaves in-character and
pronoun-only lines alone, and is now a permanent test. The import-
baseline test fails without the feature.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Session-Id: 04db4d93-9884-44ee-b28f-627e92b33731
Agent: claude-code
On a simulated 17,500-word, 4-POV book, 13 of 15 voice warnings were
false positives on short or situational lines ("Hell," from a dockhand,
"How many?" from a captain). A warning that fires on in-character lines
makes an AI "fix" good prose.

- Rate features are compared only on lines of 12+ words; shorter lines
  can only trip presence-based hard contrasts.
- Absence is never a finding (no profanity, no formal words, no
  questions). Missing contractions count only on long lines from a
  contraction-heavy speaker, and only alongside another signal.
- Questions and exclamations are situational, never reasons on their own.
- Sentence length is directional and relative: LONG needs 3x the
  speaker's average and 15+ words; SHORT only applies to verbose
  speakers across 2+ sentences.
- A finding needs a hard contrast (profanity or formal words from a
  character who never uses them, over 150+ words of evidence) or two
  independent signals.

Test fixture: Lady Veyra sat at 149 evidence words, one under the
hard-contrast minimum, so the precision pass had flipped an assertion to
expect no profanity finding. One more in-character line puts her above
the minimum and the original assertion is restored unchanged.

Verified: npm run preflight (49/49), including new precision tests (7
in-character lines produce nothing; 3 real swaps are caught).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Session-Id: 04db4d93-9884-44ee-b28f-627e92b33731
Agent: claude-code
Profiles are built from the approved manuscript, which contains the very
out-of-voice lines to catch. On a simulated book, a formal scholar's one
planted profane line put his profanity rate above zero, so the "never
swears" hard contrast no longer fired and the flaw was missed.

- Profiles (version 2) carry totals and sorted per-line records (scene,
  paragraph, sha256 of normalised text, feature counts).
- A line that is part of the profile is judged against the profile minus
  itself (and its paragraph's other lines by the same speaker): rates,
  means, z-scores, the 150-word minimum and enough_evidence are all
  recomputed. Evidence lines never include the line being judged.
- Legacy profiles without per-line data still work, with a rebuild note.

Verified: npm run preflight (51/51, new test/voice-loo.test.mjs). On the
simulated 17,500-word book the planted line is now caught ("no profanity
markers across 1162 words of evidence"), sounds like the dockhand, with
no false positives.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Session-Id: 04db4d93-9884-44ee-b28f-627e92b33731
Agent: claude-code
@MrPickering

Copy link
Copy Markdown
Contributor Author

Landed on main via #9 (merge of the full stack).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant