A team file says who should do what. A benchmark score reports only the final outcome, so it cannot say whether the agents worked together. In OpenCollab's traces they often did not: the lead agent briefed its teammates and then did the work itself, or spent its whole token budget before handing anything off. The declared team collapsed into a single agent. We call this the illusion of collaboration.
Two runs reconstructed from OpenCollab's event stream. (a) With read-only tools, the Adopter (the lead agent) hands the task to the coders and adopts Coder A's commit. (b) In the reference team, the Adopter briefs both coders but keeps working until its budget runs out, so its own code is graded.
- One substrate for every organization. Single agents, Teams, and Workflows run on the same session runtime, with the same model client, context handling, and tool implementations. Changing the organization does not change the rest of the system. Model access, context handling, tool execution, orchestration, and environments have separate extension points, so an experiment can change one component at a time.
- Rules enforced at call time. A message along an undeclared edge is refused, and the refusal is recorded. Each agent's token budget is checked before every model call, and an agent that reaches its ceiling stops with an explicit reason, so a run cut short is never confused with one that finished.
- An event stream instead of prose logs.
--tracewrites one JSONL record per step, stamped with the run, agent, role, event type, and token usage. Scripts can rebuild who did what without parsing chat transcripts. - Isolated workspaces. Teammates can work in their own git worktrees and hand back their diffs.
Comparing two organizations is controlled only if everything else can be held fixed and each run can be checked against what it was declared to be. The paper names seven conditions. Five ask whether a factor can be set explicitly before a run: the model, the tools, the token budget, the context policy, and the topology. Two ask whether a run can be checked afterwards: Realized (the run's record holds the whole declared organization, with every call attributed to its agent) and Compared (every message names its sender and receiver, and every attempt to leave the declaration is refused or flagged). The paper audits ten agent artifacts against them, each read in code at a pinned version except Claude Code, which is read from its documentation:
| Artifact | Model | Tools | Budget | Context | Topology | Realized | Compared |
|---|---|---|---|---|---|---|---|
| AutoGen | ✓ | ✓ | ◐ | ✓ | ✓ | ✗ | ✗ |
| AG2 | ◐ | ◐ | ✗ | ◐ | ◐ | ◐ | ✗ |
| LangGraph | ◐ | ◐ | ✗ | ◐ | ◐ | ✗ | ✗ |
| SWE-agent | ✓ | ✓ | ◐ | ✓ | ✗ | ✓ | ✗ |
| OpenHands | ✓ | ✓ | ◐ | ◐ | ◐ | ◐ | ✗ |
| Claude Code | ✓ | ✓ | ◐ | ✗ | ◐ | ◐ | ✗ |
| Codex CLI | ✓ | ◐ | ✗ | ✗ | ◐ | ◐ | ✗ |
| DeepSeek Harness | ✓ | ✓ | ✗ | ◐ | ◐ | ◐ | ✗ |
| Inspect AI | ✓ | ✓ | ◐ | ◐ | ✓ | ◐ | ◐ |
| HAL | ◐ | ✗ | ✗ | ✗ | ✗ | ◐ | ✗ |
| OpenCollab | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
✓ by the artifact's own settings and records; ◐ only through the researcher's code, or in part; ✗ not met.
OpenCollab's row is read on its Team controller. Its team file sets each role's model, tools, token allowance, and context policy together with the edges. Every model call is admitted against the calling role's remaining allowance before it is sent. Each run's trace opens with the declared organization, every role with its model and tools and every edge, so a declared teammate that never acts still appears. A message along an undeclared edge, or a call outside a role's tool set, is refused and recorded.
A run counts as adherent only if its trace shows that every declared role took part and that delegation, role boundaries, budget sharing, information flow, and context policy all held. Adherence is the share of adherent runs. On SWE-bench Pro, one change to the same team moves it a long way:
| Change to the reference team | Adherence |
|---|---|
| None (Qwen3.8-Flash, open prompt, all tools, star topology) | 47.2% |
| The prompt makes the handoff mandatory | 91.7% |
| The lead agent gets read-only tools | 94.4% |
| Every role runs GPT-5.6-Luna | 97.2% |
Adherence tells you how to read a Team-versus-Single gap. Dividing the Pass@1 difference by Adherence gives the complier average causal effect (CACE), and because Adherence is recorded for every run, the assumption behind that estimate can be examined with data rather than assumed. The paper defines the six audit axes and reports the full ablation over model, tools, budget, context, and topology.