A modular, automated sentinel for a Metaculus tournament/project (default 32916, "Current AI Tournament").
The bot reads configuration only from environment variables (GitHub Secrets in Actions):
-
METACULUS_TOKEN(preferred) -
METACULUS_API_KEY(fallback alias) -
EXA_API_KEY -
OPENROUTER_API_KEY -
TOURNAMENT_ID(project/tournament identifier; accepts numeric ID like32916or slug likespring-aib-2026)
TOURNAMENT_ID should be the Metaculus project/tournament identifier from the tournament page URL (slug) or from API payload IDs (numeric).
Secrets are never hardcoded.
.github/workflows/{ci.yml,scheduler.yml}
src/
main.py
config/{settings.py,constants.py,timezone.py,logging.yaml}
metaculus/{client.py,schemas.py,selection.py,windows.py,state.py}
research/{exa_client.py,retrieval.py,source_ranker.py,evidence.py}
llm/{openrouter_client.py,roles.py,structured.py,prompts/*.md}
forecasting/{baselines.py,features.py,ensemble.py,validators.py,stats/*}
execution/{runner.py,submitter.py,risk.py,dedupe.py}
storage/{csv_logger.py,jsonl_logger.py,report.py,git_commit.py}
data/
tests/
Every run (30-minute schedule):
- Load tournament/questions for
TOURNAMENT_ID(default32916) - Determine open-window status (tournament and each question)
- Select eligible questions
- Retrieve evidence (Exa)
- Run multi-role LLM pipeline (OpenRouter)
- Compute baseline + type-aware statistical forecast
- Ensemble + validation
- Submit when secrets exist and the market is open
- Log outputs to
data/runs.csv,data/forecasts.csv,data/forecasts.jsonl, anddata/latest_summary.md
Window checks use America/New_York logic with UTC+US timestamp logging. If a market is not open, the bot skips submission and logs market closed/not open with one of:
CLOSED_WINDOWLOCKEDRESOLVEDNOT_YET_OPEN
If STRICT_OPEN_WINDOW behavior is needed, it can be added as a code change.
export METACULUS_TOKEN=...
export EXA_API_KEY=...
export OPENROUTER_API_KEY=...
python -m src.mainci.yml:ruff+pytestscheduler.yml: runs every 30 minutes (*/30 * * * *), uploads artifacts, and commitsdata/updates.