"Our model scores 99.7% on SWE-Bench."
"Can we see the methodology?"
"Trust me bro."
A Trust Me Bro Benchmark is any AI evaluation where:
- A company posts impressive benchmark scores
- The methodology, prompts, or evaluation code are undisclosed
- Independent reproduction is... not possible
- The response to scrutiny is essentially: "Trust me bro"
Fireship calls them TMBBs. This phenomenon was named after the repeated pattern of AI labs claiming state-of-the-art results on benchmarks like SWE-Bench, TerminalBench, and WebArena — only for researchers to discover the scores were achieved through exploitation, cherry-picking, or outright gaming.
Contribute to this website here
- Leaderboard with the Trust Me Bro Intelligence Index, Benchmaxx Gap, Days to Ship and other fake metrics, styled after Artificial Analysis (not affiliated)
- 15 evaluations inspired by real meme benchmarks (BullshitBench, the pelican, Brainfuck tic-tac-toe, lava lamps, the strawberry...)
- Incident database of claims that outran their evidence, sourced from public posts
- In the Wild: the real, verbatim posts that inspired it all
- Chart Crime Studio: truncate a y-axis and watch Tufte's Lie Factor climb
Every score is fabricated. The quoted posts are real and link to their originals.
Static Astro site built with bun, deployed to GitHub Pages on every push to main.
bun install
bun dev # http://localhost:4321
bun run build # outputs to dist/
bun run check # typecheck- Tweets are fetched at build time and rendered with the Magic UI tweet card (cached in
node_modules/.astro/tweets) - Every page gets its own share image, rendered with satori at build time (
src/lib/og.tsx,src/pages/og/) - Favicons, app icons and the web manifest are generated from one SVG (
src/lib/icons.ts) - Navigation uses Speculation Rules prerendering and cross-document view transitions
- Production builds load Cloudflare Web Analytics (cookieless, no consent banner needed)
- New posts and news are found and verified with the
tmb-scoutagent skill (.claude/skills/tmb-scout)
Data lives in src/data/ (models, benchmarks, tweets, incidents). Add a meme benchmark or a tweet there and open a PR. bun run build checks every cross-reference first (src/lib/content.ts) and lists the broken ones.