[purpshell]

Summary

polymorfa campaign analytics and campaign view now show campaign A/B test results (polymorfa/polymorfa#406) and weighted message variation (polymorfa/polymorfa#409) in human output. Both features are betas.

• campaign analytics adds the winning criterion, the outcome (winner, inconclusive with its reason, or no winner yet), holdout and reserve counts, and a per-variant table. The table shows label and key, the criterion rate first, then sent, assigned, delivered, read and replied. The winner row is marked (winner).
• campaign view and campaign create show the variant count and outcome of an A/B campaign, and the number of weighted alternatives when messageVariations is set.
• campaign create --help and the README campaign section describe the variants, variantStrategy and messageVariations input fields, their limits and the 403 feature_unavailable response for teams that are not enrolled. create already sends the --input object unchanged, so these fields worked before this PR. There is no campaign update command.

The published SDK pin (0.1.0-dev.20261004024706) types variants and variantStrategy as unknown and has no experiment, experimentOutcome or messageVariations. The CLI reads them as unknown values and narrows them at runtime (src/services/campaign-experiments.ts). A value that does not match the expected shape prints Unreadable in this CLI version; use --json for the full response. and nothing partial. API-supplied labels and keys go through terminalText, so control characters are escaped. --json output is unchanged.

Impact

Impact table

Area	Updated / blocked	Change or named blocker	Paths / companion PRs / evidence
CLI output	Updated	A/B and weighted-variation lines in campaign analytics, view and create human output	src/services/campaign-experiments.ts, src/services/campaigns.ts
CLI help	Updated	campaign create description lists the beta fields and an example	src/commands/campaign/create.ts
CLI README	Updated	New "A/B tests and weighted message variation (beta)" section under Messaging campaigns	README.md
SDK dependency	Blocked	The @polymorfa/sdk prerelease containing polymorfa/sdks#344 (ff51567) is not published; the scheduled publish workflow fails on a missing GitHub App secret. The pin stays at 0.1.0-dev.20261004024706; the code compiles against it and against the SDK main types	package.json, package-lock.json unchanged

No impact table

Area	Why there is no impact	Supporting paths or evidence
API	The CLI consumes the existing analytics and campaign responses; no route, request or error changes	polymorfa/polymorfa#406, #409
SDKs	Types already merged in polymorfa/sdks#344; the CLI defines narrow local types instead of changing the SDK	src/services/campaign-experiments.ts
CLI request path, credentials, exit codes	Only renderers changed; create still passes the JSON body through; the server API key requirement and errors (including 403 feature_unavailable) are unchanged	src/services/campaigns.ts campaignInput
JSON output and MCP tools	--json writes the API data unchanged; the local MCP server exposes no campaign tools	test/commands/campaign-experiments.test.ts JSON case; grep campaign src/mcp returns nothing
Public docs (Mintlify)	The public guides document the API fields (docs/guides/campaigns/ab-test-a-campaign.mdx, send-a-campaign.mdx on polymorfa dev) and do not reference CLI campaign commands; CLI usage lives in this README	git grep "polymorfa campaign" origin/dev -- docs returns nothing
Feature release, enrollment, flags	Enrollment and the beta gate are enforced by the API; the CLI adds no flag, default or client-side gate	polymorfa/polymorfa#406, #409
Admin controls	No new state or staff action; the CLI only displays API data	none
Billing, Usage, Logs, audit logs, Analytics, onboarding, tracking	Display-only change; no metered quantity, log type, analytics event or onboarding step is added	none

Verification

Tested revision: afc9e01 on top of origin/main 43e55a4, with the published SDK pin 0.1.0-dev.20261004024706.

• npm run format:check, npm run lint, npm run typecheck, npm run build and npm pack --dry-run --ignore-scripts pass.
• New test/commands/campaign-experiments.test.ts (13 tests) runs the commands against a local HTTP fixture: promoted, inconclusive, no winner yet, experiment: null, experiment absent, an unreadable shape, control characters in a label, unchanged JSON output, view for promoted, undecided, weighted-variation and broadcast campaigns, and create --help. Campaign, recipient and conversion test files pass: 4 files, 53 tests.
• SDK type compatibility: a scratch check (outside the repo) assigns CampaignExperimentResults and CampaignExperimentOutcome from sdks ff51567 packages/typescript/src/messaging/types.ts to the CLI's local types under tsc --strict; it compiles.
• Full suite: the first local run had 1036 passed and 4 failed. The failures were in cloud-group, project-context and event-stream-process, and the same tests also fail on a clean origin/main checkout on this machine. Later runs timed out widely because the machine load average was about 170 on 15 cores. CI is the authoritative full-suite result.
• Not exercised: a live staging campaign with A/B or weighted variation enabled.

Session provenance and UI evidence

• Agent registration PR: polymorfa/polymorfa#518.
• Sanitized transcript checkpoint and evidence commit: related campaign betas session internal/agent-sessions/sessions/purpshell-t3-claude/campaigns-betas-merge/5e2bb467f1fa750b82bee255e87ee372d1f156c82e8ae302ec53e43e3e63e0f0.md in polymorfa/polymorfa at 3e21bef9c1a8e6ecc48b478c05ab1702e4a52362. This CLI subtask has no separate checkpoint yet.
• Capture coverage and tested revision: afc9e01.
• UI evidence: CLI text output, no graphical UI. Rendered by the built CLI at afc9e01 against a local fixture server (synthetic data, no customer data):

$ polymorfa campaign view campaign_123 --project-slug launch-project
ID	campaign_123
Name	October offer
Status	running
Recipients	5000
Sent	800
Delivered	780
Read	402
Failed	0
Skipped	250
Scheduled	
A/B test	2 variants
A/B outcome	Winner: Long [b], promoted to the reserve

$ polymorfa campaign analytics campaign_123 --project-slug launch-project
Campaign	campaign_123
Recipients	5000
Sent	800
Delivered	780
Read	402
Failed	0
Skipped	250
Responded	61
Response rate	0.07625

A/B criterion	Read rate
A/B outcome	Winner: Long [b], promoted to the reserve
Holdout	250
Reserve	3950

Variant	Read rate	Sent	Assigned	Delivered	Read	Replied
Short [a]	44.3%	400	400	391	177	24
Long [b] (winner)	56.3%	400	400	389	225	37

Other outcomes render as A/B outcome Inconclusive (insufficient_evidence); the reserve was not sent and A/B outcome No winner yet. A weighted-variation campaign adds Message variations 2 weighted alternatives to view.

Rollout and rollback

• Availability: the API features are betas on polymorfa dev (#406, #409) and reach only enrolled teams. This CLI change displays whatever the API returns and grants no access.
• Merge order: this PR can merge independently because it compiles against the current pin. The @polymorfa/sdk pin bump to the SDK types follows in a later PR, after the next SDK prerelease containing Add campaign A/B test and weighted variation types to the TypeScript SDK sdks#344 is published and passes staging acceptance. At that point the local types can be replaced with the SDK exports.
• CLI nightly and stable publication follow the existing gates on accepted SDK releases.
• Rollback: revert this commit. The change is display-only, with no state, config or migration.

---

^{Need help on this PR? Tag @codesmith with what you need. Autofix is disabled.}

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

_{Comment @coderabbitai help to get the list of available commands.}