Require product fit before evidence review

README.md

@@ -11,39 +11,47 @@ The website code lives separately in [bradvin/agentfirst.directory-site](https:/
 
 ## Editorial Policy
 
-**Agent First is intentionally opinionated, but agent-first infrastructure is broader than agent-only software.** The directory includes products and protocols that meet at least one of these classifications:
+**Only tools built for agents belong in this directory.** Decide product fit before reviewing evidence or assigning a classification. A human-first app with an API, MCP server, SDK, or embedded AI agent does not become a tool built for agents through integration depth or tool count.
 
-- **`agent-native`** — agents are a core actor, runtime, entity, or participant in the product. Examples include agent orchestration, agent identity and communications, agent-operated interfaces, and human-agent collaboration systems.
-- **`agent-enabling`** — the product materially empowers an agent-first workflow through substantive capabilities such as execution, memory, data access, communication, testing, or safe interaction. Agents do not need to be the product's exclusive or primary users.
-- **`agent-internet-protocol`** — an open protocol or interaction standard lets agents participate in internet communication, commerce, identity, coordination, or other machine-to-machine interaction.
+Agent-oriented infrastructure, protocols, and agent-builder tools can qualify when their operating model is designed for agent operation. They need not serve agents exclusively or have originated as agent-only products. An AI agent serving human customers is not, by itself, a tool built for agents.
 
-Classification explains *how* each listing qualifies; it is not a quality tier.
+### Product-fit gate (before evidence)
+
+Use only the product description and operating model, plus canonical homepage positioning if necessary. Do not inspect evidence arrays, documentation, MCP details, implementation, pricing, licence, or validation results to make or rescue this decision.
+
+Record one bounded semantic decision for each proposed listing:
+
+- **`pass-fit`** — record the intended user, core job, and why the product is built for agents. Only then proceed to evidence, schema, and contradiction review.
+- **`fail-fit`** — record the intended user, core job, and why it is not built for agents. Stop with nonacceptance; do not review evidence, enrich metadata, or commit listing changes.
+- **`unclear-fit`** — record the intended user, core job, and what prevents a built-for-agents decision. Hold for product-positioning clarification; do not review evidence or proceed with metadata or listing commits.
+
+A classification cannot bypass this gate. Category guidance suggesting that internal autonomous agents alone establish eligibility is superseded by this rule. Historical reassessments and public corrections require separate owner approval; this policy update does not authorize changes to existing listings.
+
+### Classifications (after `pass-fit`)
+
+- **`agent-native`** — the product is built for agent operation, with agents as core actors, runtimes, entities, or participants; for example, agent orchestration, identity, communication, or agent-operated interfaces.
+- **`agent-enabling`** — infrastructure or builder tooling designed for agent operation materially enables execution, memory, data access, communication, testing, or safe interaction. General-purpose usefulness or a secondary API/MCP integration cannot rescue a human-first product.
+- **`agent-internet-protocol`** — an open protocol or interaction standard designed for agent participation in communication, commerce, identity, coordination, or other machine-to-machine interaction.
+
+Classification explains *how* an eligible listing qualifies; it is not a quality tier.
 
 ## Inclusion Test
 
-A listing must pass all of these checks:
+After `pass-fit`, a listing must pass all of these checks:
 
-1. **Identify one classification.** Which of the three definitions does the product satisfy?
+1. **Identify one classification.** Which of the three definitions does the eligible product satisfy?
 2. **Name the concrete agent outcome.** What can an agent or agent builder materially do because this product or protocol exists?
 3. **Verify the claim from first-party evidence.** Product documentation, source repositories, protocol specifications, or first-party pages must describe the relevant capability.
-4. **Check that the capability is substantive.** The qualifying value must come from the product or protocol itself, not a thin wrapper or an integration badge.
-5. **Keep the directory defensible.** The listing and its classification must remain accurate without stretching generic technical compatibility into an agent-first claim.
+4. **Check that the capability is substantive.** The qualifying value must come from the product or protocol itself, not an integration badge.
+5. **Check schema and contradictions.** Validate the submission and resolve conflicting claims without treating a schema pass as proof of product fit.
 
 ## What Does Not Qualify
 
-A listing should be rejected when its only support is:
-
-- generic technical compatibility with software an agent could call;
-- a thin MCP, API, or command wrapper over an otherwise unrelated product;
-- vague or unsupported marketing language about agents;
-- a general-purpose primitive with no material agent-first workflow or agent-internet role;
-- third-party claims that cannot be confirmed from an authoritative first-party source.
-
-General-purpose products can qualify as `agent-enabling` when their documented capabilities materially empower agent-first workflows. Conversely, adding an agent integration does not automatically qualify a product.
+Reject products not built for agents, including human-first apps with secondary REST, MCP, or SDK access, however extensive. Internal AI agents completing a human customer's task do not establish product fit. Generic compatibility, integration depth, tool count, and unsupported agent marketing cannot establish eligibility.
 
-For coding-agent orchestrators, the product must own a material lifecycle across agent runs rather than merely invoke an agent. Evidence should show lifecycle controls such as task assignment and isolation, retained or shared state or validation evidence, and a separate gate that determines completion, review, or landing. Merely launching one agent, supplying prompts, wrapping an API or MCP server, advertising agent compatibility, or automating an ordinary developer task does not satisfy this boundary.
+For coding-agent orchestrators, the product must own a material lifecycle across agent runs rather than merely invoke an agent. After product fit passes, evidence should show lifecycle controls such as task assignment and isolation, retained or shared state or validation evidence, and a separate gate that determines completion, review, or landing. Merely launching one agent, supplying prompts, wrapping an API or MCP server, advertising agent compatibility, or automating an ordinary developer task does not satisfy this boundary.
 
-When evidence is ambiguous, do not invent a rationale: request stronger first-party evidence or reject the listing.
+When evidence is ambiguous after `pass-fit`, do not invent a rationale: request stronger first-party evidence or reject the listing.
 
 ## Structure
 
@@ -249,7 +257,7 @@ One broad homepage link should not be used to support unrelated claims. The cont
 
 ## How To Add A Tool
 
-1. Fork this repo or create a branch.
+1. Record `pass-fit` using the product-fit gate above before researching evidence or creating listing changes; otherwise stop. Fork this repo or create a branch only for a passing candidate.
 2. Pick the closest existing category in `categories/`.
 3. Add a new file in `tools/<slug>.md`.
 4. Fill in every required frontmatter field, including a non-empty `evidenceSources` array.

docs/tool-review-guide.md

@@ -1,5 +1,19 @@
 # Tool review guide
 
+## Product fit comes first
+
+Before reviewing any evidence, record `pass-fit`, `fail-fit`, or `unclear-fit`, with the intended user, core job, and why the product is or is not built for agents. Use only its product description and operating model, plus canonical homepage positioning if necessary. Do not inspect evidence arrays, docs, MCP details, implementation, pricing, licence, or validation at this stage.
+
+- `pass-fit`: only now perform full first-party evidence, schema, and contradiction review. Passing fit is necessary, not acceptance.
+- `fail-fit`: stop with nonacceptance. Do not review evidence, enrich metadata, or commit listing changes.
+- `unclear-fit`: hold for product-positioning clarification. Do not review evidence or proceed with metadata or listing commits.
+
+A human-first app with REST, MCP, or SDK access is not rescued by integration depth or tool count. An AI agent serving humans is not, by itself, a tool built for agents. Agent-oriented infrastructure, protocols, and agent-builder tools may pass when designed for agent operation; exclusive agent use or agent-only origins are not required. `agent-enabling` and category guidance about internal autonomous agents cannot bypass this gate.
+
+Historical reassessments and public corrections require separate owner approval. This guidance authorizes no existing-listing changes.
+
+## Review after `pass-fit`
+
 Tool authors may suggest `seoTitle`, `seoDescription`, and `agentSummary` in tool frontmatter. All three are optional, including for existing listings. The editorial reviewer owns the final public wording and may revise, omit, or remove a suggestion before approval.
 
 Review each field against the tool's cited first-party evidence and the existing profile. Keep the title specific to the tool and its agent-facing use (usually about 50–60 characters). Make the description a concise, factual reason to inspect the profile (usually about 140–160 characters). Make the agent summary one short plain-text paragraph about what the tool enables an agent to do (roughly 40–70 words). These are editorial drafting targets, not hard validation limits or automatic truncation rules. Avoid unsupported superlatives, promotional claims, repeated keywords, markdown, and HTML. A valid value must be a non-empty string after surrounding whitespace is trimmed.

test/editorial-policy.test.mjs

@@ -0,0 +1,29 @@
+import assert from "node:assert/strict";
+import { readFileSync } from "node:fs";
+import { test } from "node:test";
+
+for (const path of ["README.md", "docs/tool-review-guide.md"]) {
+  const source = readFileSync(new URL(`../${path}`, import.meta.url), "utf8");
+
+  test(`${path}: product fit precedes evidence and classification`, () => {
+    assert.match(source, /before reviewing evidence|Before reviewing any evidence/);
+    assert.match(source, /intended user, core job/);
+    assert.match(source, /product description and operating model/);
+    assert.match(source, /canonical homepage positioning if necessary/);
+    assert.match(source, /Do not inspect evidence arrays, (?:documentation|docs), MCP details, implementation, pricing, licence, or validation/);
+    assert.match(source, /pass-fit[^\n]*(?:evidence, schema, and contradiction|first-party evidence, schema, and contradiction) review/);
+    assert.match(source, /fail-fit[^\n]*(?:Stop|stop) with nonacceptance[^\n]*(?:do not|Do not) review evidence, enrich metadata, or commit listing changes/);
+    assert.match(source, /unclear-fit[^\n]*(?:Hold|hold) for product-positioning clarification[^\n]*(?:do not|Do not) review evidence or proceed with metadata or listing commits/);
+  });
+
+  test(`${path}: integrations and internal agents cannot bypass fit`, () => {
+    assert.match(source, /human-first app/);
+    assert.match(source, /integration depth or tool count/);
+    assert.match(source, /An AI agent serving human(?:s| customers) is not, by itself, a tool built for agents/);
+    assert.match(source, /designed for agent operation/);
+    assert.match(source, /agent-only (?:products|origins)/);
+    assert.match(source, /internal autonomous agents/);
+    assert.match(source, /Historical reassessments and public corrections require separate owner approval/);
+    assert.doesNotMatch(source, /General-purpose products can qualify|Agents do not need to be the product's exclusive or primary users/);
+  });
+}