watermark-agent

APOLLO13 is delivered as an MVP with generation tracking across the backend API and frontend Editor page.

APOLLO13 is the user-facing identity of the system. The project is framed as:

• one visible research collaboration agent, APOLLO13
• several internal specialist agents for writing, trace audit, planning, editor revision, paper audit, editing-trace analysis, and reporting
• model/provider choice treated as an implementation detail rather than the headline architecture

Multi-Agent Implementation Stack

The multi-agent implementation is wired in code through a small orchestration stack:

• LangGraph (backend/app/core/workflow/pipelines.py) defines the workflow graph with explicit nodes such as generate, audit, supervisor, editor_revise, human_edit, paper_audit, editing_trace_analysis, report, and final_artifact_audit
• ResearchState (backend/app/core/workflow/states.py) is the shared state object passed between graph nodes; it carries the draft text, audit outputs, provenance fields, and agent_trace
• each node delegates to a specialist agent class such as GenerationAgent, AuditAgent, SupervisorAgent, EditorAgent, PaperAuditAgent, EditingTraceAnalysisAgent, or ReportingAgent
• each agent inherits from BaseAgent and emits an AgentAction, which is appended into agent_trace for the frontend APOLLO13 panel
• LangChain Core is used as the lightweight model I/O layer: LLMRequest and LLMResponse standardize prompts and responses across provider adapters
• LLMRouter maps agent roles to providers so the orchestration layer stays role-centric rather than provider-centric

In short:

• LangGraph handles control flow and state transitions
• agent classes handle role-specific logic
• provider adapters handle model-specific execution
• the UI reads agent_trace, supervisor_plan, paper_audit_summary, and editing_trace_summary as the transparent record of orchestration

Dependency layout

• backend/requirements.txt: backend runtime (FastAPI app boot)
• backend/requirements-dev.txt: tests/lint/type-check tools
• backend/requirements-llm.txt: optional provider SDKs (OpenAI/Bedrock/Vertex AI)
• frontend/package.json + frontend/package-lock.json: frontend runtime/dev dependencies

Local run (without Docker)

Backend

python3 -m venv .venv
source .venv/bin/activate
pip install -r backend/requirements.txt
cp backend/.env.example backend/.env
uvicorn app.main:app --app-dir backend --reload --host 0.0.0.0 --port 8000

Run that command from the repository root (watermark-agent/).

If you are already inside backend/, use:

uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

Example command used in the development environment:

PYTHON_BIN=/absolute/path/to/python
"$PYTHON_BIN" -m uvicorn app.main:app --app-dir backend --reload --host 0.0.0.0 --port 8000

Frontend

Use a current Node.js runtime that satisfies Vite 7, such as Node 20.19+ or 22.12+.

cd frontend
npm ci
npm run dev

Example command used in the development environment:

cd frontend
npm run dev -- --host 0.0.0.0 --port 5173

During npm run dev, the frontend uses the Vite dev-server proxy and calls /api by default. If your backend is not reachable at http://127.0.0.1:8000 from the machine running Vite, set VITE_DEV_PROXY_TARGET.

If you need the browser to call an explicit backend origin directly, set VITE_API_BASE_URL.

For a built-app check instead of live development, use:

npm run build
npm run preview

Generated traces are stored in SQLite (DATABASE_URL, default ./data/db/app.db).

Provider settings

Architecture note:

• roles and providers are separate concepts

• writer, audit, editor, and supervisor are agent-role slots

• provider settings decide which model executes each role

• the multi-agent structure comes from role separation in the workflow, not from provider diversity

• Primary role settings: WRITER_PROVIDER, AUDIT_PROVIDER, EDITOR_PROVIDER, SUPERVISOR_PROVIDER

• Supported writer providers: mock, llama_vllm, azure_openai, bedrock_claude

• Supported audit providers: heuristic, vertex_ai, bedrock_claude

• GENERATION_PROVIDER remains as a legacy writer fallback only

• AUDIT_PROVIDER defaults to heuristic when unset

• AUDIT_PROVIDER=heuristic runs deterministic metadata/rule-based checks only

• AUDIT_PROVIDER=vertex_ai internally combines heuristic checks with Vertex Gemini findings

• AUDIT_PROVIDER=bedrock_claude internally combines heuristic checks with Bedrock Claude findings

pip install -r backend/requirements-llm.txt

Then set in backend/.env:

Optional script/runtime paths:

PYTHON_BIN=/absolute/path/to/python
VLLM_BIN=/absolute/path/to/vllm

Minimal bootable config:

WRITER_PROVIDER=mock
AUDIT_PROVIDER=heuristic

Writer role examples: Llama via vLLM:

WRITER_PROVIDER=llama_vllm
VLLM_BASE_URL=http://localhost:8001/v1
LLAMA_MODEL_NAME=/absolute/path/to/llama3.1-8b

Azure OpenAI:

WRITER_PROVIDER=azure_openai
AZURE_API_KEY=...
AZURE_ENDPOINT=https://<your-resource>.openai.azure.com
AZURE_OPENAI_API_VERSION=2024-12-01-preview
AZURE_OPENAI_DEPLOYMENT=<azure-deployment-name>

Bedrock Claude:

WRITER_PROVIDER=bedrock_claude
AWS_REGION=ap-northeast-2
AWS_SECRET_ACCESS_KEY=...
AWS_ACCESS_KEY_ID=...
BEDROCK_MODEL_ID=..

Editor role examples: Azure OpenAI:

EDITOR_PROVIDER=azure_openai
AZURE_API_KEY=...
AZURE_ENDPOINT=https://<your-resource>.openai.azure.com
AZURE_OPENAI_API_VERSION=2024-12-01-preview
AZURE_OPENAI_DEPLOYMENT=<azure-deployment-name>

Llama via vLLM:

EDITOR_PROVIDER=llama_vllm
VLLM_BASE_URL=http://localhost:8001/v1
LLAMA_MODEL_NAME=/absolute/path/to/llama3.1-8b

Provider smoke tests:

python infra/scripts/smoke_test_providers.py

Audit provider modes:

# default / offline-friendly mode
AUDIT_PROVIDER=heuristic

Vertex AI audit:

AUDIT_PROVIDER=vertex_ai
GCP_PROJECT_ID=<your-gcp-project-id>
GCP_LOCATION=global
VERTEX_MODEL=gemini-3.1-pro-preview

Bedrock audit:

AUDIT_PROVIDER=bedrock_claude
AWS_REGION=ap-northeast-2
AWS_SECRET_ACCESS_KEY=...
AWS_ACCESS_KEY_ID=...
BEDROCK_MODEL_ID=...

Authentication options:

# Option 1: Application Default Credentials (local/browser flow)
gcloud auth application-default login

# Option 2: Service account JSON
export GOOGLE_APPLICATION_CREDENTIALS=/absolute/path/to/service-account.json

Legacy fallback example:

GENERATION_PROVIDER=mock

Active role status:

• WRITER_PROVIDER is active in the main workflow
• AUDIT_PROVIDER is active in the main workflow
• SUPERVISOR_PROVIDER is active with LLM reasoning plus rule fallback
• EDITOR_PROVIDER is active in the main workflow and falls back to passthrough when unavailable
• paper audit, editing-trace analysis, and reporting are deterministic Python agents in the main workflow

Quick Check

Repository validation can be started with:

bash infra/scripts/check_local.sh

Optional provider smoke tests can be added when the relevant credentials or local services are ready:

bash infra/scripts/check_local.sh --providers all

To run only a subset, pass the provider names explicitly, for example --providers vllm vertex or --providers azure bedrock.

If you also want the offline quantitative metrics in the same pass, add --eval:

bash infra/scripts/check_local.sh --eval

What this script does:

• runs backend/tests with the project Python environment when available
• optionally runs infra/scripts/smoke_test_providers.py as a wrapper step for live provider checks
• optionally runs evaluation/run_eval.py

Data Folder

The project persists runtime data to SQLite.

Docker Image

• run_agent.sh: single entrypoint — supports CLI pipeline mode and web server mode
• src/main.py: reads input JSON and writes output JSON
• Dockerfile: multi-stage build (Node.js frontend + Python backend)

Build from the repository root:

docker build -t apollo13-agent .

The build compiles the React frontend automatically (no manual npm run build needed).

Web demo mode

docker run --rm -p 8000:8000 \
  -v "/path/to/gcp-service-account.json:/app/gcp-key.json:ro" \
  -e AZURE_API_KEY="<your-key>" \
  -e AZURE_ENDPOINT="https://<your-resource>.openai.azure.com" \
  -e AZURE_OPENAI_API_VERSION="2024-12-01-preview" \
  -e AZURE_OPENAI_DEPLOYMENT="<deployment-name>" \
  -e AWS_REGION="ap-northeast-2" \
  -e AWS_ACCESS_KEY_ID="<your-key-id>" \
  -e AWS_SECRET_ACCESS_KEY="<your-secret>" \
  -e BEDROCK_MODEL_ID="arn:aws:bedrock:ap-northeast-2:<account-id>:inference-profile/global.anthropic.claude-sonnet-4-6" \
  -e GCP_PROJECT_ID="<your-project>" \
  -e GCP_LOCATION="global" \
  -e VERTEX_MODEL="gemini-3.1-pro-preview" \
  -e GOOGLE_APPLICATION_CREDENTIALS="/app/gcp-key.json" \
  apollo13-agent --web

Open http://localhost:8000 in a browser. The React UI and FastAPI backend are served from the same container on port 8000.

Notes:

• the image includes backend/requirements-llm.txt required for real providers

• API keys must be injected at docker run time

• without any credentials the image falls back to mock providers automatically

• data/db/app.db: the database created when you run the backend from the repository root

• backend/data/db/app.db: the database created when you run the backend from inside backend/

• data/artifacts/: reserved for saved output artifacts

Two SQLite files can appear:

• DATABASE_URL defaults to sqlite:///./data/db/app.db
• that path is relative to the current working directory
• running from different directories can therefore create different local databases

For one stable database location, set DATABASE_URL to an absolute SQLite path in backend/.env.

CLI mode

docker run --rm --gpus all \
  -v "$(pwd)/input_data:/app/data" \
  -v "$(pwd)/output_data:/app/results" \
  -v "/path/to/gcp-service-account.json:/app/gcp-key.json:ro" \
  -e AZURE_API_KEY="<your-key>" \
  -e AZURE_ENDPOINT="https://<your-resource>.openai.azure.com" \
  -e AZURE_OPENAI_DEPLOYMENT="<deployment-name>" \
  -e AWS_REGION="ap-northeast-2" \
  -e AWS_ACCESS_KEY_ID="<your-key-id>" \
  -e AWS_SECRET_ACCESS_KEY="<your-secret>" \
  -e GCP_PROJECT_ID="<your-project>" \
  -e GCP_LOCATION="global" \
  -e GOOGLE_APPLICATION_CREDENTIALS="/app/gcp-key.json" \
  -e VLLM_BASE_URL="http://<vllm-host>:8001/v1" \
  -e LLAMA_MODEL_NAME="meta-llama/Llama-3-8b-hf" \
  apollo13-agent \
  /app/data/problem_set.json /app/results/solution.json

Providers basically assigned as:

• editor → Azure OpenAI GPT (AZURE_API_KEY + AZURE_ENDPOINT)
• supervisor → AWS Bedrock Claude (AWS_REGION + AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY)
• audit → GCP Vertex AI Gemini (GCP_PROJECT_ID + GCP_LOCATION + GOOGLE_APPLICATION_CREDENTIALS)
• writer / watermark → vLLM / LLaMA (VLLM_BASE_URL); falls back to Azure if not set

Any missing credentials cause that role to fall back to mock automatically.

Docker Compose (alternative multi-service setup)

docker compose -f docker-compose.prod.yml up --build

• Backend API: http://localhost:8000
• Frontend preview: http://localhost:4173

Note: for a single-container web demo, prefer the --web mode above. Docker Compose is provided for development and multi-service deployments.

Evaluation Folder

evaluation/ is for offline benchmark-style checks, not the main product workflow.

Available metrics:

• watermark_persistence: watermark detection rate, mean z-score, and signal drop after edits
• audit_accuracy: audit-to-heuristic agreement proxy using precision, recall, and F1
• report_quality: attribution coverage, diff coverage, similarity, and contribution percentages

Useful commands:

Show evaluation module status:

PYTHON_BIN=/absolute/path/to/python
"$PYTHON_BIN" evaluation/run_eval.py --status

Show starter dataset status:

PYTHON_BIN=/absolute/path/to/python
"$PYTHON_BIN" evaluation/run_eval.py --dataset-status

Known limitation:

• the default evaluation/run_eval.py full DB-backed metric run may fail if the evaluation loaders lag behind the latest backend model schema

Run evaluation-related checks as part of the one-shot local check with:

bash infra/scripts/check_local.sh --eval

Docs Folder

The main docs are:

• docs/architecture.md: system scope, workflow flow, and persistence model
• docs/demo_script.md: step-by-step demo flow for the UI and APIs
• docs/technical_report.md: report-oriented summary of the implementation, validation path, and known limitations
• docs/internal_agents.md: internal agent definitions, active role status, and provider-role interpretation