Planeje e interprete testes A/B com z-test, Bonferroni e motor de decisão em 4 estados com direção do efeito.
Plan and interpret A/B tests with z-test, Bonferroni and a 4-state decision engine with signed effect direction.
PT-BR · English · Live Demo · Stack · Architecture · Quick Start · Author
Stats lab notice: frequentist helpers for two-proportion experiments. Decision states (Melhora / Regressão / Efeito Fraco / Inconclusivo) are pedagogical decision-support — not automatic product rollout authority.
O StatLab Experiments calcula tamanho amostral (Cohen's h + poder), analisa duas proporções (z-test) e classifica o resultado em Melhora / Regressão / Efeito Fraco / Inconclusivo — com direção do efeito assinada (diff = pB − pA), correção de Bonferroni e IC de Newcombe no nível ajustado.
Times leem A/B só pelo p-valor, param cedo e tratam qualquer “significativo” como vitória operacional — sem poder, tamanho de efeito nem relevância prática.
PMs, analistas e data scientists que precisam de um cockpit simples de planejamento + interpretação frequentista.
- Planejamento de tamanho amostral (baseline, MDE absoluto, poder e alpha)
- Análise de duas proporções (z-test) via API Flask + SciPy/statsmodels
- Motor de decisão em 4 estados com direção do efeito (pB − pA vs MPE)
- IC de Newcombe coerente com Bonferroni (a UI mostra o nível real, ex.: IC 98,33%)
- Contratos honestos: p-valor null em casos degenerados, uplift null com baseline zero, 400 estruturado para entradas inválidas
- Cenários/chips de demonstração na UI Next.js
start.batpara subir o lab no Windows
- Foco em proporções / A/B binário — não é suite completa de experimentação
- Não substitui design de experimento, SRM checks nem sequential testing avançado
- Decisão do motor é apoio — rollout continua humano
StatLab Experiments plans sample size (Cohen's h + power), analyzes two proportions (z-test) and classifies outcomes as Melhora / Regressão / Efeito Fraco / Inconclusivo — with signed effect direction (diff = pB − pA), Bonferroni correction and a Newcombe CI at the adjusted level.
Teams read A/B tests from p-values alone, stop early and treat any “significant” result as an operational win — without power, effect size or practical relevance.
PMs, analysts and data scientists who need a simple plan + interpret frequentist cockpit.
- Sample-size planning (baseline, absolute MDE, power and alpha)
- Two-proportion analysis (z-test) via Flask API + SciPy/statsmodels
- 4-state decision engine with signed effect direction (pB − pA vs MPE)
- Newcombe CI coherent with Bonferroni (UI shows the real level, e.g. 98.33%)
- Honest contracts: null p-value on degenerate counts, null uplift on zero baseline, structured 400s for invalid input
- Demo scenarios/chips in the Next.js UI
- Windows
start.batfor local lab bring-up
- Focused on binary A/B proportions — not a full experimentation suite
- Does not replace experiment design, SRM checks or advanced sequential testing
- Engine output is support — humans still own rollout
| Surface | URL |
|---|---|
| Public lab | https://statlab-ab.vercel.app |
| GitHub | see Repo badge above |
How to try: plan a sample size → analyze a scenario → compare Melhora vs Regressão / Weak Effect / Inconclusive → read why the state was chosen (direction, adjusted alpha, real CI level).
 Plan sample size |
 Analyze — Melhora |
 Analyze — Regressão |
 Weak effect |
| Layer | Technology |
|---|---|
| Web | Next.js 16, React, TypeScript, Tailwind, Recharts |
| API | Flask (WSGI on Vercel), NumPy, SciPy, statsmodels |
frontend/ Next.js UI
api-server/ Flask handlers (`api/index.py`) + tests
docs/ demo script + screenshots
start.bat local launcher.\start.batManual:
# API
cd api-server
pip install -r requirements.txt
# run as documented / Vercel-style handler locally
# Web
cd frontend
npm install
npm run dev- 4-state engine with signed direction — a significantly worse B is Regressão, never a win (the historical
abs(diff)bug is locked out by golden test G3) - Newcombe CI at the Bonferroni-adjusted level — inference and interval are family-wise coherent; the UI shows the real level (e.g. 98.33%)
- Honest edge contracts — no fabricated p-values or uplifts; structured 400s for invalid input
- Flask on Vercel for a slim stats endpoint next to Next.js
- Teach Bonferroni / power / direction instead of p-value-only screenshots
Statistical methodology and assumptions: docs/STATISTICAL_METHOD.md.
- More educational scenarios and guardrails (SRM messaging)
- Exportable analysis memo
- Broader test families beyond two proportions
Felipe Alirio Baruja — data / product / full-stack portfolio.
- Portfolio: https://barujafe.vercel.app/
- GitHub: https://github.com/BarujaFe1
- LinkedIn: https://www.linkedin.com/in/barujafe/
MIT — see LICENSE.