Align dependency families in the version catalog and clear security advisories - #466
tonygermano wants to merge 6 commits into
Conversation
tonygermano
added
the
4.6.x CVE Fixes
21885ce to
343e1f5
Compare
343e1f5 to
8485722
Compare
Every catalog entry carried its own inline version, so single-artifact bumps (mostly Renovate security PRs) left families split across versions: jackson-core, netty-codec and derby were each bumped alone, and log4j-core was once bumped without log4j-api, breaking the build. Move each family whose versions currently agree onto a shared [versions] ref so a bump moves the whole family. No version changes: the resolved dependency trees for every project and classpath are identical before and after. The artifacts that have already drifted keep their inline versions, with a comment, until they are realigned. jackson-core's comment records why the Jackson family cannot simply be realigned: databind 2.16 and later remove APIs that swagger-core 2.0.10 and the AWS SDK 2.15.28 regions module still call, so the family stays split until those callers are upgraded. Signed-off-by: Tony Germano <tony@germano.name>
Aliases were bare artifactIds, so accessors like libs.utils, libs.auth, libs.annotations and libs.policy did not say which project they came from, netty-nio-client read as part of the io.netty family, and apache-jsp-v8-5-70 implied a second version of apache-jsp when it is a different artifact (org.mortbay.jasper). - Prefix AWS SDK modules with awssdk- (and rename the version ref to match); prefix the Jasper, hk2-external, GlassFish, JAX-WS and pdfbox-graphics2d entries with their project. - Sort [libraries] and each bundle alphabetically, and write bundles one entry per line so dependency changes diff and merge cleanly. Only the catalog changes; build scripts reference bundles, which keep their names. Every alias and bundle expands to the same module:version set as before. Bundle order sets classpath order, so the order changed; the only resource whose providing jar changed is the Eclipse about.html shared by jetty apache-jsp and ecj. Signed-off-by: Tony Germano <tony@germano.name>
netty-codec was bumped alone to 4.1.136.Final for a security fix, leaving the other catalog modules on 4.1.119.Final, and the classified netty-transport-native-epoll jar was pinned inline in server/build.gradle at 4.1.119.Final, outside the catalog. Netty supports only matching versions across its modules. 4.1.137 rather than codec's 4.1.136: 4.1.119 has open advisories in codec, codec-http, codec-http2, handler and native-epoll, and 4.1.136 still has GHSA-8c42-7qj2-3j46 (codec-http), CVE-2026-75595 and CVE-2026-75596 (handler). 4.1.137 is the first release with none in any shipped module; 4.1.138 carries no further security fixes. native-epoll now has a catalog entry on the netty ref, and server/build.gradle selects its linux-x86_64 classifier with variantOf(), so it can no longer drift from the family. No new runtime dependencies: the 4.1.137 modules depend only on each other. Every io.netty member reference on the project classpaths resolves as before; the only unresolved ones are netty-nio-client's references to the epoll channel classes, which live in netty-transport-classes-epoll and have never shipped (the native-epoll jar carries only the native library), unchanged by this commit. Checksums regenerated with a cold cache per CONTRIBUTING.md. Signed-off-by: Tony Germano <tony@germano.name>
v4.6.0 shipped derby and derbytools 10.10.2.0. Since then derby was bumped alone to 10.11.1.1, leaving derbytools (ij, dblook) a release behind the engine it operates on. Put both on a shared derby version ref. 10.12.1.1 rather than derby's 10.11.1.1: 10.11.1.1 is still affected by CVE-2015-1832 (XXE in the embedded engine's XML parsing), which is fixed in 10.12.1.1. The two remaining advisories are not reachable here: CVE-2018-1313 needs the network server (derbynet is not shipped) and CVE-2022-46337 needs LDAP authentication (not used). On Maven Central only 10.17.1.0 clears the latter, and it requires a newer Java than the Java 17 toolchain. Compatibility with v4.6.0 (10.10.2.0): 10.12 targets Java 6+ and is still a single jar with no dependencies, so no new artifacts or placements. None of the 13 incompatibilities in the 10.11 and 10.12 release notes affects the engine's own SQL (server/dbconf/derby, donkey's derby.xml, the delta scripts): - stricter CASE result typing (DERBY-6566, DERBY-2002): every CASE branch in the engine's SQL has a known type - no default on identity columns (DERBY-6545): the one ALTER COLUMN SET DEFAULT targets a non-identity column - DROP fails with dependent triggers (DERBY-2041): no triggers - LOG10/COSH/SINH/TANH results (DERBY-6447): not used - SQL authorization privileges (DERBY-6429, DERBY-6434): not enabled - Java 5 support, serialized DataSources (DERBY-6213, DERBY-6128): not applicable - XML external entities, SecurityManager permission, UPDATE ... SET identity = DEFAULT (DERBY-6807, DERBY-6648, DERBY-6414): not used, no policy installed, only turns an error into success - LOB hash joins now spill to disk past 1 MB (DERBY-6096): performance only Identity columns switch to sequence generators (DERBY-6542) only on a hard upgrade. database.url has no upgrade=true, so existing databases are soft-upgraded and keep their data dictionary version. Verified with the real server schema: a database created by 10.10.2.0, then used alternately by 10.12.1.1 and 10.10.2.0 (identity inserts, the donkey statistics CASE update, the alert and message-search CASE queries), works in both directions with the dictionary left at 10.10 and no identity gaps, so rolling back to v4.6.0 stays possible. User-written SQL against Derby (for example in Database Reader/Writer channels) that relies on the old behaviour, such as a CAST(NULL AS <wrong type>) in a CASE or XML external entity expansion, would now fail. Checksums regenerated with a cold cache per CONTRIBUTING.md. Signed-off-by: Tony Germano <tony@germano.name>
The catalog carried two versions of each module, with the version encoded in the alias: - junit 4.8.1 (command-test, donkey-test) and junit-v4-13-1 (server-test). 4.8.1 is affected by CVE-2020-15250; align on 4.13.1, which server tests already use and which has no known advisories. JUnit 4.13 no longer embeds Hamcrest (4.8.1 did) and its runner classes need it at runtime, so command-test and donkey-test gain the hamcrest 2.2 entry server-test already uses. Test classpaths only; nothing shipped changes. - javax-annotation-api 1.3.2 (client, server) and javax-annotation-api-v1-3 (command). Align on 1.3.2, so cli-lib now ships javax.annotation-api-1.3.2.jar like client-lib and server-lib. All versions involved were already in verification-metadata.xml; a cold-cache regeneration produced no changes. The donkey (23) and command (6) tests pass on the new classpath. Signed-off-by: Tony Germano <tony@germano.name>
The OWASP dependency-check plugin version was pinned inline in the root build.gradle, the one version in the build outside the catalog. Declare it under a new [plugins] section and apply it with alias(), so it is updated alongside everything else. Same plugin and version; a cold-cache verification-metadata regeneration produced no changes. The catalog header said versions match the vendored jars byte for byte, which stopped being true with the first security bump. Describe what the file is now: non-transitive resolution means every Maven Central artifact is listed explicitly, the versions originated from the vendored-jar SHA-1 sweep but have since moved on, and checksums and distribution placement are enforced by verification-metadata.xml and vendored-layout.json. Signed-off-by: Tony Germano <tony@germano.name>
8485722 to
de1a8c0
Compare
pacmano1
left a comment
pacmano1
left a comment
There was a problem hiding this comment.
Existing Derby installs can't upgrade to this build. The engine never starts, and afterwards 4.6.0 can't open the database either. Current main fails the same way (Derby 10.11.1.1, from the Renovate bump in ac47cc8), so this predates the PR, but the compatibility notes here say existing installs upgrade fine.
Installs don't get the soft upgrade described here. ServerMigrator.migrateConfiguration runs Migrate3_0_0.updateConfiguration on every boot, which appends ;upgrade=true to a Derby database.url before the engine connects and saves it to mirth.properties. Deleting it from the file doesn't help: it comes back and the boot fails the same way. Only an explicit ;upgrade=false stays. So every existing install gets a hard upgrade, and that fails in DD_Version.doFullUpgrade:
ERROR XJ040: Failed to start database 'appdata/mirthdb' ...
Caused by: ERROR XSDA8: Exception during restore of a serializable or SQLData object of class
Caused by: java.io.InvalidClassException: org.apache.derby.iapi.sql.execute.ExecRowBuilder; local class incompatible: stream classdesc serialVersionUID = -1078823466492523202, local class serialVersionUID = 9151849461018459842
Derby is deserializing compiled stored statements that 10.10 left in SYS.SYSSTATEMENTS. In a database made with the steps below, only getTables and getColumns were compiled, and the engine calls getTables on every startup (DatabaseUtil.tableExists), so any install that has run will have it. The engine retries startup every 10 seconds indefinitely. After that, 10.10 refuses the database (XSLAN: ... upgraded by version 10.12), and 10.12 opens it only without the upgrade flag. The data is intact.
With ;upgrade=false this build starts on the same database, shows the old messages and processes new ones, which matches your result.
To reproduce (Temurin 17.0.20.1, Linux):
- Install the 4.6.0 release (
sh oie_unix_4_6_0.sh -q -dir /opt/oie), start it, deploy a channel, send a few messages, stop it.database.urlnow ends in;upgrade=true. - Copy
appdata/andconf/mirth.propertiesfrom that install intoserver/setupfrom a build of this branch or of main. That's the state an in-place upgrade leaves, since the installer keepsconf/andappdata/. - Start it with
./oieserver.
@pacmano1 do you recommend downgrading derby and undoing #378 ? |
4 participants
Summary
gradle/libs.versions.tomlgave every library its own inline version, so single-artifact bumps (mostly Renovate security PRs) left library families split across versions:netty-codec4.1.136 with the restof Netty on 4.1.119,
derbyahead ofderbytools, andjackson-core2.18.8 with the rest of Jackson on 2.14.3. Resolution is non-transitive, so a mismatch like this shows up as a runtime linkage error, nota build failure. It has already happened once:
log4j-corewas bumped withoutlog4j-apiand broke the build.This PR moves families onto shared
[versions]refs so a bump moves the whole family, realigns Netty and Derby, removes duplicate module versions, and tidies the catalog. Jackson is not realigned; seebelow. Please review commit by commit. Each commit is self-contained and builds on its own, and the commits that change shipped jars are separate from the ones that only restructure the file.
Commits
[versions]refs); document the Jackson blockerawssdk-*,jasper-*,hk2-*, …), sorted file, one bundle entry per lineserver/build.gradlederby+derbytoolsaligned on 10.12.1.1 (v4.6.0 shipped 10.10.2.0)cli-libonly (annotation-api 1.3 → 1.3.2); JUnit is test-only[plugins]; catalog header rewrittenVersion choices
The rule used throughout: align each family on a version already in use, and go further only when a newer version clears known advisories (checked against OSV). Every target also meets Renovate's 14-day
minimumReleaseAge.any shipped module.
derbyhas since been bumped alone to 10.11.1.1. Both of those are affected by CVE-2015-1832 (XXE in the embedded engine), which 10.12.1.1 fixes.CVE-2018-1313 (network server, not shipped) and CVE-2022-46337 (LDAP authenticator, not used) remain. On Maven Central only 10.17.1.0 fixes the latter, and it needs a newer Java than our Java 17 toolchain.
Why Jackson is not realigned
An earlier revision aligned Jackson on 2.18.10, which clears all known databind advisories, and it failed integration tests with
NoSuchMethodError: BeanDescription.findJsonValueMethod(). databind 2.16 andlater remove APIs that swagger-core 2.0.10 (
findJsonValueMethod,findPropertiesToIgnore) and the AWS SDK 2.15.28regionsmodule (PASCAL_CASE_TO_CAMEL_CASE) still call. The last databind release withall three is 2.15.4, which has more open advisories than today's 2.14.3. So Jackson stays exactly as it ships today (core 2.18.8, the rest 2.14.3). The catalog now documents the blocker next to
jackson-coreand the
jacksonref. Realigning Jackson needs swagger-core and the AWS SDK upgraded first, in a separate PR.Compatibility checks
oie/mainwith this branch. As a positive control, the check reproduces the Jackson 2.18.10 failure above. Result: no new unresolved references.lz4-java, which has never shipped, so it's unresolved before and after.derbytools10.10.2.0 called 12 Derby internals that don't exist inderby10.11.1.1, and mockito neededorg.junit.rules.TestRule, which JUnit 4.8.1 lacks.database.urlhas noupgrade=true). I tested it with the real server schema: a database created by 10.10.2.0, then used alternately by 10.12.1.1 and 10.10.2.0 (identity inserts, the donkey statisticsCASEupdate, the alert andmessage-search
CASEqueries), keeps its data dictionary at 10.10 and works in both directions, so rolling back to v4.6.0 works. None of the 13 incompatibilities in the 10.11 and 10.12 release notes affectsthe engine's own SQL. User-written SQL against Derby that relies on the old behaviour (a
CAST(NULL AS …)of the wrong type in aCASE, or XML external entities) would now fail; worth a release-note line.about.html, a licensenotice in both jetty's
apache-jspandecj. Launcher manifestClass-Pathentries are built from fixed lists of module names, so the new order doesn't affect them.Testing
./gradlew build dist -PdisableSigning=true -Pcoverage=truepasses.verification-metadata.xmlwas regenerated with a coldGRADLE_USER_HOMEfor each commit that changes versions, as described in CONTRIBUTING.md. New entries are only the new versions and their parent POMs.server/setupstages only the new Netty and Derby jar versions, and the same Jackson jars asoie/main.Notes for reviewers
donkey/setupandcommand/build/cli-libare filled byCopytasks and keep old jar versions until./gradlew clean. The shipped distribution isn't affected:the server's
syncDonkeyLibsexcludesdatabase/**.netty-transport-native-epollcontains only the native library. The Java classes it needs (netty-transport-classes-epoll) have never shipped, so the AWS clientuses NIO. This PR only aligns the jar's version; removing it would be a separate change.
--write-verification-metadataonly adds entries, as the Renovate PRs have done. Pruning unused entries would be a separate cleanup.🤖 Generated with Claude Code