Repository navigation
feat: generate process test coverage reports in the BPMN examples - #110
Merged
Merged
Conversation
Add the CIB seven community extension Process Test Coverage to the process tests of cibseven-embedded, cibseven-process-engine-api, cibseven-remote and fullstack-cibseven-embedded. Spring tests load it through a test auto-configuration, the remote example's standalone engine through the JUnit 5 extension. Gradle writes the report to build/reports/process-test-coverage.
Add fluxnova-bpm-testing-process-coverage, which registers itself through Spring auto-configuration. Gradle writes the report to build/reports/process-test-coverage.
…uild directory Camunda Process Test already writes a coverage report, but to target/ in the Gradle variants as well. Point it to build/reports/process-test-coverage and document the report.
…classpath of the Maven app
…the coverage extension
emaarco
marked this pull request as ready for review
November 5, 2026 18:44
emaarco
enabled auto-merge (squash)
November 5, 2026 18:44
emaarco
added a commit
that referenced
this pull request
Nov 7, 2026
…gateway (#111) ## Why #110 writes one process test coverage report per test class, but there was no overview across classes and nothing failed when a path went untested. Turning on a 100 % minimum surfaced two gaps that held in every example: the 7-day signature reminder was never tested, and in cancel-bike-order the condition of the gateway "Cancellation possible?" sat on the "No" flow, so a cancellable order went through "Clarify return with supplier" and "Book costs" (ADR-0014 of the Temporal example already noted that the model reads as inverted). ## What - **Combined report (CIB seven):** the four CIB seven examples apply the extension's report aggregator. Gradle loads it from the buildscript classpath, because no plugin marker is published, and runs `aggregateProcessTestCoverage` after `test`. Maven runs the `aggregate` goal in the `test` phase. The result is `all/report.html` in the report folder. - **Coverage gate, 100 % per process test class (CIB seven, Fluxnova):** - The minimum is set in one place, as the extension's system property on the test task or in Surefire: `org.cibseven.community.process_test_coverage.ASSERT_AT_LEAST`, or `process_test_coverage.ASSERT_AT_LEAST` for Fluxnova. - The Spring examples get a test-only `ProcessTestCoverageContextCustomizerFactory`, registered in `META-INF/spring.factories`. It builds the coverage properties with the extension's builder only for classes ending in `ProcessTest`. Other classes that run the process, such as `BikeUnavailableTransactionTest`, are still reported but not gated. The extension's own inclusion patterns cannot be used for this, because an excluded class that starts a process fails with "Coverage state must be initialized". - `cibseven-remote` uses the JUnit 5 extension, whose builder reads the property itself. - Running single test methods covers less, so the minimum can be lowered for that run with `-PprocessTestCoverageMinimum=0` (Gradle) or `-Dprocess-test-coverage.minimum=0` (Maven). - There is no gate for Camunda 8, Operaton and Temporal. Camunda Process Test has no minimum option, Operaton has no working coverage tool (see #110) and Temporal has no model. - **Cancellation gateway fix (all 12 examples):** - The "No" flow now carries `${!cancellationPossible}` (Camunda 8: `=not(cancellationPossible)`). The models were edited with bpmn-cli, so each diff is one line, and the generated process APIs were regenerated. - The Temporal workflow and ADR-0014 follow the same rule: possible reverses the order right away, not possible waits for the return clarification and books the costs. - **Demo dealer:** an order of the custom build `BIKE-CUSTOM` cannot be cancelled. Every other order still can. The rule is encoded in the order id prefix `ORDER-CUSTOM-`, so the adapter keeps no state. - **Process tests:** - The abort test now covers the "not possible" path and checks that costs are booked once. - The abort after an accepted alternative covers the "possible" path, with no clarify-return task and no costs. - A new test covers the signature reminder: one reminder, then the deadline rejects the application. - **Bruno:** - `03-abort` and the demo data `07-withdrawn-return-open` order `BIKE-CUSTOM`, so they keep the clarify-return path. - The new scenario `11-abort-order-cancelled` shows the direct cancellation. - In Temporal, `03-abort` also checks that the application is still WITHDRAWN after a few seconds. - **fullstack-cibseven-embedded:** - "Custom Build" is seeded into the bike catalogue, and the Playwright abort spec orders it, so the UI still reaches the clarify-return path. - The withdrawal note (`withdrawRequestedNote`, en/de) now says the Tasklist step is only needed when the dealer cannot cancel. - **Docs:** each example's AGENTS.md describes the combined report and the gate, the review-process agents list `BIKE-CUSTOM` as intended, and `drift.yml` records why the Fluxnova factory and the Temporal check differ. ## Verification - **Clean full builds after the last change, all passing:** `./gradlew clean build` / `./mvnw -B clean verify` in cibseven-embedded, cibseven-process-engine-api and cibseven-remote (both variants), fullstack-cibseven-embedded and fluxnova-embedded (both variants). Each `BikeLeasingProcessTest` reaches 100 %. The CIB seven report folders contain `all/report.html`. - **Gate checked:** without the new reminder test, `BikeLeasingProcessTest` fails with "matches if the coverage ratio is at least 1.0" in cibseven-embedded (93 %), cibseven-remote (94.6 %) and fluxnova-embedded (94.4 %), in both variants. `BikeUnavailableTransactionTest` (46 %) passes ungated. - **Single-method runs:** a run of one `BikeLeasingProcessTest` method fails the gate, and passes with `-PprocessTestCoverageMinimum=0`. `./mvnw test -Dtest=…#… -Dprocess-test-coverage.minimum=0` passes and writes `all/`. - **Other examples:** `./gradlew build` / `./mvnw -B verify` passed in operaton-embedded, operaton-process-engine-api, operaton-remote, temporal-native and the three Zeebe examples. These builds ran before the last whitespace and Bruno-only edits in those examples. The Zeebe builds ran one at a time, because Camunda Process Test containers time out under podman load. - **Repository checks:** `npm run drift`, `npm run generate:check`, `scripts/check-owner.sh`, `npm test` and `npm --prefix examples/fullstack-cibseven-embedded/frontend run verify` all pass. `bpmn-cli validate` reports every edited model valid, with no warnings. - **Not run:** the Bruno collections and the Playwright e2e suite, which need a running stack. ## Out of scope / Follow-ups - The UI copy of `withdrawRequestedNote` (en/de) and the visible "Custom Build" bike in the fullstack catalogue are user-facing; please confirm the wording and the choice. - The extension's Gradle aggregator task is not compatible with the configuration cache (it holds a `Project`); the cache is off in these builds. - The aggregators read every `report.json` under the report folder, so reports of renamed or deleted test classes stay in `all/` until `clean`. - The flipped branch in the Temporal workflow is not guarded with `Workflow.getVersion`; the dev server keeps no state, and on a real cluster a parked cancel workflow would simply complete without booking costs. - An upstream issue for the CIB seven extension: excluding a test class that starts a process crashes instead of skipping it. Refs Miragon/cibseven-developer-training#249
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The process tests check that each path through the bike-leasing model works, but nothing shows which parts of the model those tests never reach. The CIB seven developer training will cover Process Test Coverage in its testing chapter. The blueprints should produce the report without extra setup, for CIB seven and for every other engine where a working tool exists.
What
Each engine needed a different tool:
cibseven-embedded,cibseven-process-engine-api,fullstack-cibseven-embedded,cibseven-remoteorg.cibseven.community.process_test_coverage2.2.0fluxnova-embeddedorg.finos.fluxnova.bpm.test:fluxnova-bpm-testing-process-coverage2.0.0zeebe-native,zeebe-process-engine-api,zeebe-vanillabpoperaton-embedded,operaton-process-engine-api,operaton-remoteio.github.ronalders:operaton-process-test-coverage1.0.0temporal-nativespring-test-platform-7artifact is a test dependency, and a file in the test resources (META-INF/spring/…AutoConfiguration.imports) loadsProcessEngineCoverageConfiguration. Fluxnova's extension registers itself the same way. This keeps the process test classes identical across the cibseven, fluxnova and operaton embedded examples, which the drift check requires.cibseven-remote: this test runs a standalone in-memory engine, so it now uses thejunit5-platform-7extension with aProcessCoverageInMemProcessEngineConfiguration. The models come from a class-level@Deploymentinstead of a manual deployment. The engine is now shared by the whole test class instead of rebuilt per test;@Deploymentcleans up after each test.kotlin-reflectis now a test dependency incibseven-embedded,cibseven-remoteandfluxnova-embedded. The CIB seven and Fluxnova extensions need it but don't declare it, and without it Spring skips their test execution listener without any error. The tests then fail withCoverage state must be initialized.cibseven-process-engine-apialready gets it at compile scope throughprocess-engine-api, so it declares nothing: a test-scoped declaration would narrow it and the packaged app would fail to start. The Kotlin variants already have it on the classpath.fluxnova-engineandfluxnova-engine-spring3.0.1 directly, and Maven's nearest-wins rule would put those into the packaged app instead of the starter's 3.1.1. The managed dependency excludes both, so the app keeps 3.1.1 like the Gradle variant.target/tobuild/reports/process-test-coverage/, next to the other reports. Maven keeps the default (target/process-test-coverage/, ortarget/coverage-report/for Camunda 8).AGENTS.mdsays where its report is written. No minimum coverage is enforced; that is a separate decision.Verification
./gradlew buildpassed in every touched Gradle variant: the four CIB seven examples,fluxnova-embeddedand the three Zeebe examples. Each build wrote a report tobuild/reports/process-test-coverage/../mvnw verifypassed incibseven-embedded,cibseven-process-engine-api,cibseven-remoteandfluxnova-embedded(Java/Maven). Each build wrote a report undertarget/.npm run drift,npm run generate:checkandnpm testpass.drift.lock.ymlnow locks the new auto-configuration files.tooling/starter/export.sh cibseven-remote java-mavenkeeps only the Maven report path in the exportedAGENTS.md.Out of scope / Follow-ups
Operaton: there is no official extension. The only one is a single-maintainer port (
io.github.ronalders, 1.0.0), built against Operaton 1.1.2. With Operaton 2.2.0 and our stack it fails in two ways:kotlin-logging-jvm7, butbpmn-to-code-testingbrings in 8, and the listener then throwsNoSuchMethodError;CompensationEventCoverageHandlerthrows an NPE (parameter payloadis null) on every compensation path.The Operaton examples stay unchanged until a compatible release exists.
Upstream: the CIB seven extension (2.2.0) and the Fluxnova extension (2.0.0) should declare
kotlin-reflect. Worth an upstream issue.Later: a minimum-coverage assertion as a quality gate, and uploading the reports as CI artifacts.
Refs https://github.com/Miragon/cibseven-developer-training/issues/249