Skip to content

docs: eight bits a component, on the roadmap - #58

Merged
tanvincible merged 1 commit into
mainfrom
docs/roadmap-int8
Oct 12, 2026
Merged

tanvincible merged 1 commit into
mainfrom
docs/roadmap-int8

Conversation

@tanvincible

Copy link
Copy Markdown
Owner

Puts eight-bit vectors on the roadmap. The lab's follow-up run validated them with the plan, predictions and decision rule committed before any results.

  • Scheme: LVQ-8. Each vector is stored relative to a center, the mean of the first vectors added, in its own 256-step range.
  • Result: within 0.01 recall@10 of half precision on all seven text configurations, at ef 64 and 128, with nothing re-ranked. That holds with the center from the first 1,000 vectors (worst 0.005) and from 1,000 on one topic (worst 0.006).
  • Checked by skeptics:
    • Recomputed independently, the verdict is the same.
    • No pairing of builds pushes a loss past 0.0063.
    • 40 topic seeds on openai3-384 and on bge-384: worst 0.0082 with every vector ranked, the bge worst.
    • One vector is too few to center on (within 0.002 of the bar); ten were enough.
  • Not measured: the kernel and its speed, the file format, and the vectors added before there is a center.

The plan, the results and the skeptic pass are in bench/lab/quant-plan.md on lab/x86, from runs 38090885820 and 38131118757.

@tanvincible
tanvincible merged commit 41c93bd into main Oct 12, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant