Skip to content

About

Parody of artificialanalysis.ai meant to be about satyrical benchmarks and models benchmaxxing their scores

Resources

Stars

6 stars

Watchers

0 watching

Forks

Repository files navigation

🤝 trustmebrobenchmarks.com

The definitive leaderboard for AI benchmarks that ask you to just trust them.

"Our model scores 99.7% on SWE-Bench."
"Can we see the methodology?"
"Trust me bro."

trust me bro benchmark reproducibility


What is a "Trust Me Bro" Benchmark?

A Trust Me Bro Benchmark is any AI evaluation where:

  1. A company posts impressive benchmark scores
  2. The methodology, prompts, or evaluation code are undisclosed
  3. Independent reproduction is... not possible
  4. The response to scrutiny is essentially: "Trust me bro"

Fireship calls them TMBBs. This phenomenon was named after the repeated pattern of AI labs claiming state-of-the-art results on benchmarks like SWE-Bench, TerminalBench, and WebArena — only for researchers to discover the scores were achieved through exploitation, cherry-picking, or outright gaming.

Contribute to this website here

What's on the site

  • Leaderboard with the Trust Me Bro Intelligence Index, Benchmaxx Gap, Days to Ship and other fake metrics, styled after Artificial Analysis (not affiliated)
  • 15 evaluations inspired by real meme benchmarks (BullshitBench, the pelican, Brainfuck tic-tac-toe, lava lamps, the strawberry...)
  • Incident database of claims that outran their evidence, sourced from public posts
  • In the Wild: the real, verbatim posts that inspired it all
  • Chart Crime Studio: truncate a y-axis and watch Tufte's Lie Factor climb

Every score is fabricated. The quoted posts are real and link to their originals.

Development

Static Astro site built with bun, deployed to GitHub Pages on every push to main.

bun install
bun dev          # http://localhost:4321
bun run build    # outputs to dist/
bun run check    # typecheck
  • Tweets are fetched at build time and rendered with the Magic UI tweet card (cached in node_modules/.astro/tweets)
  • Every page gets its own share image, rendered with satori at build time (src/lib/og.tsx, src/pages/og/)
  • Favicons, app icons and the web manifest are generated from one SVG (src/lib/icons.ts)
  • Navigation uses Speculation Rules prerendering and cross-document view transitions
  • Production builds load Cloudflare Web Analytics (cookieless, no consent banner needed)
  • New posts and news are found and verified with the tmb-scout agent skill (.claude/skills/tmb-scout)

Data lives in src/data/ (models, benchmarks, tweets, incidents). Add a meme benchmark or a tweet there and open a PR. bun run build checks every cross-reference first (src/lib/content.ts) and lists the broken ones.

About

Parody of artificialanalysis.ai meant to be about satyrical benchmarks and models benchmaxxing their scores

Resources

Stars

6 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages