From 50e6b447c18357e5f6c0343b60ddd066ca3af740 Mon Sep 17 00:00:00 2001 From: David Rowland Date: Thu, 15 Oct 2026 11:19:53 +0100 Subject: [PATCH] Changelist: complete the 2.0.0 notes and fold in the unreleased 1.0.5 entries --- CHANGELIST.md | 45 ++++++++++++++++++++++++++++++--------------- 1 file changed, 30 insertions(+), 15 deletions(-) diff --git a/CHANGELIST.md b/CHANGELIST.md index 97d9d8a..47f6fb0 100644 --- a/CHANGELIST.md +++ b/CHANGELIST.md @@ -1,33 +1,48 @@ # pluginval Change List ### 2.0.0 -- Updated JUCE to 9.0.4 -- Fixed the crash handler so a backtrace is shown again when a plugin crashes, and made it async-signal-safe [#144] -- Added `--junit-xml ` and `--json-results ` to both `pluginval validate` and `pluginval test` to write machine-readable reports for CI (also settable as `JUNIT_XML` / `JSON_RESULTS`). A report that can't be written fails the run. See `docs/Adding pluginval to CI.md` + +**Upgrading from 1.x:** the command line now uses subcommands (`pluginval validate [options] `; `pluginval ` still works) and is stricter: unrecognised options and invalid values are errors instead of being ignored, and `--vst3validator` has been removed because the VST3 validator is now built in. CI scripts that download `releases/latest` will get 2.0.0 automatically, so check the **Breaking** items below. + +#### New - Added acceptance testing: `pluginval test ` renders a plugin with a known state, input (audio/MIDI) and optional playhead, then compares the output against a checked-in reference WAV, exiting `0` on a match and `1` on a mismatch (with a diff WAV). Use `--record-missing` to record new references. See `docs/Acceptance testing.md` - Added `--plugin ` to `pluginval test` to run a config against a given plugin instead of the one it names (the config's `plugin` can then be omitted) +- Added `--junit-xml ` and `--json-results ` to both `pluginval validate` and `pluginval test` to write machine-readable reports for CI (also settable as `JUNIT_XML` / `JSON_RESULTS`). A report that can't be written fails the run. See `docs/Adding pluginval to CI.md` - Added two GitHub Actions for use from other repositories: `Tracktion/pluginval/validate` (validate a list of plugins) and `Tracktion/pluginval/acceptance` (run every acceptance config in a directory against a built plugin). See `.github/actions/README.md` +- Added real-time safety checking on macOS, using [rtcheck](https://github.com/Tracktion/rtcheck): `--rtcheck enabled` checks the plugin's process calls during the tests for real-time-unsafe behaviour, and `--rtcheck relaxed` skips the first process call to allow for initialisation. Off by default [#156] +- The Steinberg VST3 validator is now built into pluginval and runs automatically for VST3 plugins at strictness level 5 and above, so it no longer needs downloading and passing in separately [#162] +- Added `--config ` to load settings from JSON (repeatable; later files win per key) +- Added a "Discrete parameter state restoration" test (strictness 8) which checks a discrete parameter has the same value after restoring a state, whatever value it held beforehand. This fails for parameters that keep the raw normalised value from the host instead of mapping it to a step (e.g. `juce::AudioParameterBool`) +- Added drag-and-drop of plug-in files onto the main window, with two drop zones to either validate the plug-in or add it to the plugin list [#170] +- Enabled the editor stress and extreme (real-time allocation / oversized block) tests that were present in the source tree but had not been compiled into the build + +#### Changes +- Updated JUCE to 9.0.4 - Restructured the command line into subcommands: `pluginval validate [options] ` (the default), `pluginval run-tests` and `pluginval strictness-help [level]`. The plugin path is now a positional argument of `validate` - **Deprecated:** the flat flags `--validate `, `--run-tests` and `--strictness-help [level]` still work as aliases (with a one-line notice) but will be removed in a future version. `pluginval ` remains a silent shorthand for `validate` - Replaced the hand-rolled command-line parser with CLI11 and a single JSON-based settings pipeline -- Added `--config ` to load settings from JSON (repeatable; later files win per key) - Settings precedence is now (lowest to highest): defaults, environment variables, `--config`, command-line options - Environment variable support is now automatic for every option (including `DISABLED_TESTS`) -- **Breaking:** unrecognised options now produce an error instead of being ignored -- **Breaking:** invalid option values now error instead of silently falling back (e.g. `--rtcheck banana`, `--strictness-level abc`) -- **Breaking:** falsey flag environment variables now mean off (e.g. `SKIP_GUI_TESTS=0` no longer enables skipping) - `--help` output is now generated by CLI11 -- Added a "Discrete parameter state restoration" test (strictness 8) which checks a discrete parameter has the same value after restoring a state, whatever value it held beforehand. This fails for parameters that keep the raw normalised value from the host instead of mapping it to a step (e.g. `juce::AudioParameterBool`) -- Fixed false failures in the "Plugin state restoration" test where the first parameter was compared against the last value sent by a previous test rather than the plugin's own value - Below strictness level 8, two-step (on/off) parameters are now only sent 0 or 1 when tests randomise parameter values. Arbitrary values are still sent to them at level 8 and above, and to all other parameters at every level +- Added static linking to the Windows runtime so it should run on more Windows systems (particularly non-dev machines) +- Made `PluginInfoTest` run on the message thread as the functions it calls aren't thread-safe + +#### Breaking +- Removed `--vst3validator `: the VST3 validator is now built in (see above), and passing the option is an error [#162] +- Unrecognised options now produce an error instead of being ignored +- Invalid option values now error instead of silently falling back (e.g. `--rtcheck banana`, `--strictness-level abc`) +- Falsey flag environment variables now mean off (e.g. `SKIP_GUI_TESTS=0` no longer enables skipping) + +#### Fixes +- Fixed the crash handler so a backtrace is shown again when a plugin crashes, and made it async-signal-safe [#144] +- Fixed false failures in the "Plugin state restoration" test where the first parameter was compared against the last value sent by a previous test rather than the plugin's own value - Fixed the "Plugin state restoration" test's exact binary state comparison never running. It was documented as running at strictness level 8 and above but checked the test's own level instead of the run's +- Fixed races between a timeout and the writing of the log and report files, which could leave a report that disagreed with the exit code [#198] +- Fixed garbled non-ASCII characters in output from the validation child process when a character was split across reads [#198] +- Fixed `--output-filename` without `--output-dir` not being resolved against the working directory [#198] +- pluginval's `operator delete` replacement no longer throws (which ended the process), and its `operator new` throws `std::bad_alloc` instead of returning null [#198] - Fixed the macOS app failing to start on any machine other than the one that built it ("Library not loaded: @rpath/librtcheck.dylib"); this affected the `v2.0.0-beta1` and `v2.0.0-beta2` releases -- Enabled the editor stress and extreme (real-time allocation / oversized block) tests that were present in the source tree but had not been compiled into the build - -### 1.0.5 -- Added drag-and-drop of plug-in files onto the main window, with two drop zones to either validate the plug-in or add it to the plugin list [#170] -- Added static linking to the Windows runtime so it should run on more Windows systems (particularly non-dev machines) -- Made `PluginInfoTest` run on the message thread as the functions it calls aren't thread-safe ### 1.0.4 - Limit auval's stress test to 20 seconds (vs 600) [#135]