Repository navigation
Close the three t1 A/B defects: derived shares, ladder monotonicity, funded-startup search - #2
Merged
Conversation
… 95.25 → 95.37) The real-model A/B on eval/fixtures/t1-energy-audit (eval/results/t1_ab_report.md) found two arithmetic defects and one research miss that the skills should have prevented. Each fix is a short reasoned paragraph in the skill's existing register. market-sizing: a derived share (the "62% pre-2000" built from NAHB buckets, one of them mislabelled by a decade; the correct figure is ~72%) is recomputed in the artifact from the cited buckets, with the boundaries quoted as the source labels them and the sum written out; the closing self-check re-adds any such share from the quoted boundaries. pricing-wtp: the value ladder is checked for increasing price and increasing value before it is written (the generated ladder ran $0, $29, $79, $60), and LTV:CAC is arithmetic over the file's own fields, stated once in ltv_cac_ratio with its formula beside it (the generated file reported 3.6 while its inline math gave 3.8); the closing self-check covers both. competitive-landscape: one search per landscape goes to funded startups in the space the idea brief's solution hypothesis actually describes, plus two query templates, because both pack runs missed HomeBoost, a funded direct competitor one TechCrunch search away. Benchmark: python3 eval/fable_prompt_bench.py primary 95.25 → 95.37, zero integrity failures, boilerplate_dups 0. Frontmatter, contracts/, eval/, and LICENSE untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The real-model A/B on
eval/fixtures/t1-energy-audit(eval/results/t1_ab_report.md) found two arithmetic defects and one research miss the skills should have prevented. Each fix is a short reasoned paragraph in the skill's existing Fable 5.1 register (no CAPS, no checklists, no step scripts); frontmatter,contracts/,eval/, andLICENSEare untouched.value_ladderran $0, $29, $79, $60, andltv_cac_ratiowas 3.6 while its own inline math gave 3.8. The ladder is checked for increasing price and increasing value before it is written (a rung that breaks the order is usually a different product wearing a tier label), and LTV:CAC is arithmetic over the file's own fields, stated once inltv_cac_ratiowith its formula beside it; the closing self-check covers both.Benchmark
python3 eval/fable_prompt_bench.py -v: primary 95.25 → 95.37, zero integrity failures,boilerplate_dups0. Per-skill: competitive-landscape 93.7 → 94.6, market-sizing 96.6 → 96.7, pricing-wtp 92.2 → 93.0.🤖 Generated with Claude Code