ReaLackey

Your AI lackey inside REAPER. ReaLackey is a native REAPER extension (written in Rust) that puts a capable AI assistant right in your DAW. Ask it about your project, have it explain what's going on, or just tell it what you want done — add an EQ, write a MIDI bassline, balance your levels, tidy up markers — and it does the work, every change wrapped in a labelled undo block you can revert.

It was built accessibility-first, for producers who work with a screen reader (NVDA + OSARA) — but it's just as useful with your eyes open. When a plugin's GUI can't be read by a screen reader, ReaLackey can look at it for you and operate it.

One process, no servers, no copy-paste. It talks to the model over the network and drives REAPER directly through its API.

---

What it can do

ReaLackey drives REAPER through ~100 tools. It reads the project to answer questions accurately (rather than guessing), and makes changes on your behalf — each one confirmed and undoable. Highlights:

• Tracks, FX & routing — list and read tracks, FX and their parameters; add, remove, bypass and configure FX (including a track's input and the master's monitoring FX chains); read and load FX presets; manage sends/receives; set volume/pan, arm/mute/solo.
• MIDI & composition — read a take's notes (with neighbouring context), write and delete notes, create MIDI items.
• Automation — create envelopes (FX-parameter, track volume/pan/mute, and send/receive) and write, edit, or clear their points.
• Arrangement & editing — markers and regions, the tempo/time-signature map, take stretch markers, render settings; item/take/track properties, edge trimming, grouping, and copy/move/delete/duplicate.
• Transport & session — play/stop/record, move the edit cursor, set/read the time selection (loop/edit range), change the playback rate and ruler unit, toggle metronome/snap/ripple.
• Actions & shortcuts — search REAPER's action list, run any action by name or id (a fallback for anything without a dedicated tool), and read or edit an action's keyboard shortcuts.
• Listen & measure — pure-Rust DSP analysis of a take or track: loudness (integrated LUFS, LRA, true-peak), peak/RMS, clipping and a spectral profile — for the raw source or the processed post-FX signal — plus over-time analysis: level envelopes, silence detection, transient onsets, and tracking a single frequency across a passage. On audio-capable models it can even listen to a rendered clip to judge tone.
• See inaccessible GUIs — for custom-drawn plugin interfaces a screen reader can't parse, ReaLackey takes a screenshot, reasons about it, and (with your one-time consent) clicks/drags/types directly in the plugin window. It always prefers the undoable parameter API and only falls back to pixel input for controls that have no automatable parameter (e.g. a Kontakt patch switch). It can also snapshot the Video window to see the processed video frame.
• Per-project memory — a scratchpad stored in the .rpp so it remembers decisions, TODOs and progress across sessions, plus read/append access to the project and per-track notes.

Bring your own model

ReaLackey speaks to Claude natively and to everything else through the OpenAI-compatible API, so you can pick whatever suits your budget and privacy:

Claude (Anthropic) · OpenAI · Google Gemini · Groq · OpenRouter · DeepSeek · xAI (Grok) · Ollama (local) · LM Studio (local) · or any custom OpenAI-compatible endpoint.

Manage accounts in Extensions → ReaLackey → Providers: add/edit/delete, pick a default, fetch the model list from the provider, and set per-provider options (model, max tokens, tool-step limit, vision). Vision and audio tools are offered only when the selected model actually supports them. Running a local model (Ollama/LM Studio) means no rate limits and no cost.

A provider can hold several API keys in priority order (add/reorder them in its settings). The top key is used until it hits a quota or auth error, then ReaLackey automatically fails over to the next — announced in the chat — so a conversation keeps going when one key runs out (handy for e.g. multiple Gemini free-tier keys).

Accessibility

ReaLackey is designed to be driven entirely by keyboard and screen reader:

• Final answers are spoken through OSARA; the chat pane is navigable by headings, links open in your browser, and the status line carries a role="status" so you can query it on demand.
• Alt+1 … Alt+0 jump to a message and focus it (read out by the screen reader); press the same combo again quickly to copy that message — your request or the reply — to the clipboard.
• When OSARA is running, the assistant is told it's talking to a blind user and avoids "look for the cog icon" style directions — it reads controls out or operates them itself.
• Every destructive action asks first, via a native, screen-reader-accessible Yes/No box.

Install

1. Download the plug-in for your platform from the latest release — reaper_realackey.dll on Windows, or reaper_realackey.dylib on macOS (experimental — it builds, but hasn't been validated in a live host yet). Or build it yourself (below).
2. Copy it into REAPER's UserPlugins folder. (Find it via Options → Show REAPER resource path.) The filename must start with reaper_.
   • macOS: the released .dylib is a universal binary (Apple Silicon + Intel), so it loads whether REAPER runs natively, on an Intel Mac, or under Rosetta. It's Developer-ID-signed and Apple-notarized, so it should load directly. If Gatekeeper still blocks it (e.g. no network on first load, or a dylib you built yourself), clear the quarantine flag once, in Terminal: xattr -dr com.apple.quarantine "<REAPER resource path>/UserPlugins/reaper_realackey.dylib"
3. Restart REAPER — ReaLackey loads silently (no console window, and no screen-reader chatter over REAPER's own launch feedback).
4. Extensions → ReaLackey → Providers → add a provider and paste your API key (stored securely in your OS credential store, never in a file).
5. Extensions → ReaLackey → Open window — type a message, press Send, and the reply streams into the chat.

Local word-timing refinement (optional)

Cut-by-text lives and dies by how precisely each word's start and end are known. Some transcription endpoints return excellent timings; plain Whisper endpoints are often 50–200 ms off — the difference between a clean cut and a clipped consonant. ReaLackey can close that gap on your own machine with a local forced-alignment model (no audio leaves your computer for this step):

• Per provider, off by default. In the provider settings (Transcription tab), enable "Refine word timings locally" on the account you use for cut-by-text. When you save with the checkbox on and the files aren't installed yet, ReaLackey offers a one-time download (~318 MB: the alignment model plus the ONNX Runtime library) into <resource path>/ReaLackey/models.
• It costs CPU time. Alignment runs after each transcription; on an older CPU expect up to a third of the clip's length in extra processing (a 2017 quad-core aligns a 5-minute clip in ~110 s; modern machines are faster). The progress dialog shows the stage, and Cancel works throughout.
• Have a graphics card? (Windows) Also check "Use the graphics card for refinement": alignment then runs via DirectML on any DirectX-12 GPU — NVIDIA, AMD, or Intel, no driver toolkits to install — and is dramatically faster (measured 7–8× on a 2016 GTX 1060: a 4½-minute clip aligns in ~14 s). The GPU lane uses a different model variant, so the one-time download is ~670 MB instead of ~350 MB; if the card can't serve, it quietly falls back to CPU inference.
• Limited data plan? Every release also ships realackey-<version>-with-models-<platform>.zip with the model already bundled: unzip and merge its UserPlugins/ and ReaLackey/ folders into your REAPER resource path — nothing is downloaded then.
• The model (Meta AI's MMS forced aligner, ONNX conversion by onnx-community) is licensed CC-BY-NC 4.0 — non-commercial use. ONNX Runtime is MIT.
• macOS: Apple Silicon only (ONNX Runtime no longer ships Intel-mac builds); transcription simply runs without refinement elsewhere.

Configuration

• Config is portable. Your provider list lives under REAPER's resource path (…/ReaLackey/providers.json), so a portable REAPER install carries it along.
• API keys live in the OS credential store (Windows Credential Manager / macOS Keychain / Linux Secret Service) — never in plain text.
• Environment overrides (all optional):
  • RAAI_CONFIRM=off — disable the change-confirmation prompt.
  • RAAI_MAX_TURNS=N — override the per-provider agentic tool-step limit.
  • RAAI_MODEL — set the default Claude model.
  • RAAI_MEDIA_KEEP=N — how many of the most recent screenshots / audio / video captures stay live in the conversation before older ones are dropped to save tokens (default 2).
  • RAAI_PROMPT_CACHE=off — disable Anthropic prompt caching of the tools+prompt prefix.
  • RAAI_PROGRESSIVE_TOOLS=on — send only a small core tool set plus a load_tools loader and let the model pull in the rest on demand; cuts per-request tokens (good for rate-limited free-tier keys).
  • RAAI_VIDEO_SETTLE_MS=N — delay in ms after seeking before grabbing each frame in capture_video_clip (default 250; raise for heavy video-FX chains).

Safety model

• Mutations are confirmed and undoable. Every change is shown for approval and wrapped in a labelled AI: … undo block, so you and the assistant can both revert it.
• Uploads are consent-gated. Sending a screenshot or an audio clip to the cloud provider always asks first, every time.
• Pixel control is opt-in. Synthetic clicks/drags into plugin GUIs (which REAPER can't undo) require a one-time per-session approval; "close the window" is a hard kill switch that disarms it.

Build from source

Windows (the primary, tested target):

cargo build            # -> target/debug/reaper_realackey.dll
cargo test
cargo build --release  # optimized

You'll need the Rust x86_64-pc-windows-msvc toolchain, the Visual Studio Build Tools (MSVC linker + the Windows SDK's rc.exe), and libclang (bundled with Visual Studio — .cargo/config.toml points bindgen at it; adjust if your VS install differs). reaper-rs is pulled from git and pinned to one rev.

macOS (via SWELL): additionally needs a WDL checkout at vendor/WDL (git clone https://github.com/justinfrankel/WDL vendor/WDL) and PHP on PATH (for swell_resgen.php). A plain cargo build only makes a single-arch, lib-prefixed dylib for the host; to build the universal binary REAPER wants (reaper_realackey.dylib — Apple Silicon + Intel, ad-hoc signed so it loads), run the helper:

./build-macos.sh       # -> target/release/reaper_realackey.dylib

It just wraps two cargo build --target … runs, lipo, and an ad-hoc codesign (set LIBCLANG_PATH yourself if it isn't found). Copy the result into REAPER's UserPlugins. Releases are additionally Developer-ID signed + notarized — see docs/macos-notarization.md.

Linux (via SWELL): the same WDL checkout + PHP on PATH; compile-checked in CI on every push.

Platform status

Platform	Build	Runtime
Windows	✅ CI	✅ used daily
macOS	✅ CI (compiles + links)	⏳ not yet validated in a host
Linux	🚧 SWELL path present	⏳ not yet validated

The chat pane is an embedded HTML view — WebView2 on Windows, WKWebView on macOS — falling back to a native edit control on Linux. Screen capture and synthetic input have Windows (GDI / SendInput) and macOS (Core Graphics / CGEvent) backends. The macOS backends and webview compile and link in CI but have not yet been exercised in a live REAPER host.

Development

ReaLackey is developed with Claude Code, Anthropic's agentic coding tool, so a large share of the code is written by Claude. That does not mean it ships unchecked: the developer driving Claude Code is a working software engineer who reads, tests, and vets every change before it is committed — nothing lands unreviewed. The AI is a fast collaborator here, not an unsupervised author.

Contributing

Issues and PRs welcome. The codebase is English-only. Changelog entries go under ## [Unreleased] in CHANGELOG.md using Keep a Changelog style; releases roll that section into a version automatically (see .github/workflows/release.yml).

License

MIT OR Apache-2.0.