Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
7a11663
fix: remove Cyrillic-to-Cyrillic conversion and limit Latin targets t…
Hantok Oct 12, 2026
4e6f411
feat: add vnc to English allow list
Hantok Oct 12, 2026
4d019f1
fix: classify Apple keyboard layouts by their primary language only
Hantok Oct 13, 2026
4b928be
fix: correct earlier words of a typing run once a later word confirms…
Hantok Oct 13, 2026
b847164
docs: add layout reliability plans and execution tracking
Hantok Oct 18, 2026
5916e6d
feat: add benchmark layout tables, seed corpus, and validation tooling
Hantok Oct 18, 2026
f2f4602
feat: implement benchmark runner, SwitchFix replay adapter, and pipel…
Hantok Oct 18, 2026
34a77cd
ci: add LayoutBenchmark self-check command and GitHub Actions workflow
Hantok Oct 18, 2026
33a9f54
feat: record and freeze SwitchFix baseline v1
Hantok Oct 18, 2026
2061ee9
feat: add personal exceptions policy, validation, and storage
Hantok Oct 18, 2026
06569c1
feat: add trigger provenance, detector exception barriers, and config…
Hantok Oct 18, 2026
bbd1d8e
feat: wire personal exceptions to PreferencesManager, AppDelegate, an…
Hantok Oct 19, 2026
0d83769
fix: remediate personal exceptions findings from verification
Hantok Oct 19, 2026
5daf653
fix: make the benchmark replay measure what the app does
Hantok Oct 19, 2026
c22be93
feat: add benchmark corpus v2 without leaks and with hard negatives
Hantok Oct 19, 2026
39c8612
chore: record baseline v2 and gate CI against it
Hantok Oct 19, 2026
77561e5
fix: keep layout corrections valid and benchmark failures visible
Hantok Oct 20, 2026
68873cc
fix: confirm emitted corrections and verify benchmark inputs
Hantok Oct 20, 2026
49ca29a
Prepare v0.0.12 so users can discover and install the reliability imp…
Hantok Oct 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions .github/workflows/layout-benchmark.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
name: Layout Reliability Benchmark

on:
push:
branches:
- master
- develop
- "feat/**"
pull_request:
branches:
- master
- develop

permissions:
contents: read

jobs:
benchmark:
name: Layout Reliability Check (macos-15)
runs-on: macos-15
timeout-minutes: 20

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Show toolchain environment
run: |
swift --version
sw_vers
uname -a

- name: Run TestRunner
run: swift run -c release TestRunner

- name: Run InputPipelineTestRunner
# Hosted runners pass the Accessibility-dependent TextCorrector test (release CI runs
# the same runner); every failure here is a real failure.
run: swift run -c release InputPipelineTestRunner

- name: Run LayoutBenchmark self-check
run: swift run -c release LayoutBenchmark self-check

- name: Validate corpora and frozen layout tables
run: |
swift run -c release LayoutBenchmark validate --corpus Benchmarks/layout-corpus/v2 --tables Benchmarks/layout-tables/v1 --strict
# v1 is frozen with known label conflicts; it must still validate, warnings allowed.
swift run -c release LayoutBenchmark validate --corpus Benchmarks/layout-corpus/v1 --tables Benchmarks/layout-tables/v1

- name: Development smoke run gated against baseline v2
run: |
mkdir -p Benchmarks/results/ci
swift run -c release LayoutBenchmark run \
--adapter switchfix \
--split development \
--profile boundary-layout-only \
--corpus Benchmarks/layout-corpus/v2 \
--baseline Benchmarks/baselines/v2/corpus-v2/baseline.json \
--output Benchmarks/results/ci

- name: Upload benchmark artifacts
if: always()
uses: actions/upload-artifact@v4
with:
name: benchmark-results-ci
path: Benchmarks/results/ci/
if-no-files-found: ignore
5 changes: 4 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,9 @@ jobs:
contents: write

steps:
- name: Checkout release notes
uses: actions/checkout@v4

- name: Download all release artifacts
uses: actions/download-artifact@v4
with:
Expand All @@ -112,4 +115,4 @@ jobs:
with:
files: |
release-assets/*.zip
generate_release_notes: true
body_path: docs/releases/${{ github.ref_name }}.md
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,4 @@ temp.txt
*.key
brag-output/
brag-output-*/
Benchmarks/results/
11 changes: 11 additions & 0 deletions Benchmarks/baselines/v1/SUPERSEDED.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Baseline v1 is superseded

Do not compare against this baseline. Use `Benchmarks/baselines/v2/corpus-v2/baseline.json`.

Verification on 2026-10-19 found that the harness that recorded v1 did not measure the app's behavior:

- It configured the detector with the `.standard` Ukrainian variant for every trial. All trials use `com.apple.keylayout.Ukrainian`, whose frozen table (key 1 = и, key 11 = і) the app classifies `.legacy`. The four EN → UK misses (for example `добрий день` → `добрій день`) come from this setting alone. Rerunning the same harness with `.legacy` gave 40/40. They were not caused by static mapping, as `baseline.md` states.
- It fed every key to the state machine as a letter, while the app treats `/`, `!`, `%`, `(` and other hard punctuation as word boundaries. This hid a real false positive: `/usr/local/bin` on ABC becomes `/гик/local/bin`.
- Its scorer dropped missing results, ignored error status and checkpoints, and scored layout-only switches as text damage. Its runner counts and analysis text were hardcoded.

The files here are kept unchanged as the historical record. `Benchmarks/baselines/v2/corpus-v1` is the corrected measurement on the same corpus (40/40 recall, 1/160 text-damage false positives).
Loading
Loading