Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
91788ce
test(flagd): adopt the OpenFeature Provider TCK
aepfli Sep 11, 2026
113f8a3
test(flagd): declare the numeric-coercion gap as a defect, not a choice
aepfli Sep 11, 2026
e7df677
test(flagd): record what flagd-testbed does not serve, and why that i…
aepfli Sep 11, 2026
f868242
test(flagd): withhold @reinitialization, and settle @stale by evidence
aepfli Sep 11, 2026
2908c7a
test(flagd): withhold @large-integers explicitly
aepfli Sep 11, 2026
156b73b
docs(flagd): say what the new tags gate here, and what the testbed st…
aepfli Sep 12, 2026
f9ada04
test(flagd): declare @disabled-flags, on the evidence of both resolvers
aepfli Sep 12, 2026
110d98e
test(flagd): follow the provider-tck -> tck rename
aepfli Sep 12, 2026
bc9375c
test(flagd): give the in-process resolver a deadline its sync can meet
aepfli Sep 12, 2026
9a4c159
test(flagd): declare numeric coercion and let its failure show
aepfli Sep 12, 2026
9cda368
docs(flagd): say that the TCK suites are Docker-gated and hand-run
aepfli Sep 12, 2026
f33a7a1
test(flagd): keep the TCK suites out of CI, which the e2e profile was…
aepfli Sep 12, 2026
58bae70
docs(flagd): point at Appendix F for the CI-exclusion reasoning
aepfli Sep 12, 2026
d746672
test(flagd): record that both resolvers report the standard reasons
aepfli Sep 13, 2026
c52d0f3
test(flagd): stop withholding a capability nobody in Java can hold
aepfli Sep 13, 2026
8e0cdd7
docs(flagd): cite the declaring rule this adoption argued out in long…
aepfli Sep 13, 2026
168faf9
test(flagd): run the conformance suites from a step of their own
aepfli Sep 13, 2026
6b6a8be
test(flagd): give the conformance adoption a package of its own
aepfli Sep 13, 2026
266c809
docs(flagd): cut the conformance section to what is this adoption's
aepfli Sep 14, 2026
f59c5b6
test(flagd): one Compose file for both adoptions, and cut the prose a…
aepfli Sep 14, 2026
4389dd9
test(flagd): follow @disabled-flags' corrected gating question
aepfli Sep 14, 2026
3f41f9a
test(flagd): record the intermittent testbed failure observed here too
aepfli Sep 14, 2026
23a5a2d
test(tck): run the conformance suite against flagd-testbed v3.10.1
aepfli Sep 14, 2026
b193a88
test(flagd): declare @string-typing, on a run rather than on the default
aepfli Sep 15, 2026
aff72ac
test(flagd): declare @fully-typed-values too, on a run in both modes
aepfli Sep 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions providers/flagd/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -358,3 +358,41 @@ FlagdOptions options = FlagdOptions.builder()
.resolverType(Config.Resolver.IN_PROCESS)
.build();
```

## Provider conformance (TCK)

This provider adopts the [OpenFeature Provider TCK](../../tools/tck/README.md), once per resolver:
`RpcTest` and `InProcessTest`, both over the shared `AbstractResolverTest` in
`src/test/java/.../flagd/tck/`. **Read that class before changing either.** It records, against
measured behaviour rather than assumption, which capabilities are declared, which are withheld and
why, and every known deviation — that reasoning is the most valuable thing about this adoption and it
lives next to the declaration rather than here.

```bash
# once, if tools/tck is not in your local repository yet
mvn -pl tools/tck -am -DskipTests install

mvn -Ptck -pl providers/flagd test # both resolvers
mvn -Ptck -pl providers/flagd -Dtest=InProcessTest test # one
```

Do not add `-am` to the run itself; the TCK README says why.

**`-Pe2e` does not run these suites — it is the profile that keeps them out.** Worth stating plainly,
because `mvn -Pe2e -pl providers/flagd test` reads as if it ran everything and instead runs the legacy
`Run*Test` suites in silence: 788 tests, no scenario tally, and no mention of either conformance
suite. This module excludes `**/e2e/*.java,**/tck/*.java` by default; the `e2e` profile drops only the
first, and the `tck` profile only the second. It used to clear the property outright, which — since
`ci.yml`'s `main` job activates `e2e` on every push, on a runner that has a Docker daemon — ran these
suites in CI, where they are expected to fail, and turned every unrelated pull request red.

**No CI job runs them**, so a maintainer runs them by hand before merging a change to the provider's
resolution, event or lifecycle behaviour, and quotes the result in the pull request. A scheduled or
path-filtered workflow was considered and declined: a suite whose red is diagnosed by whoever happens
to read the notification is worse than one whose red is diagnosed by the person who caused it.

Both suites are **expected to fail**, identically: 65 scenarios each, 59 passing, 2 skipped, 4
failing. Three of the four failures come from flags the pinned `flagd-testbed` image does not serve
(open-feature/flagd-testbed#392) and one is the real numeric-coercion defect, declared and left
visible rather than skipped (open-feature/flagd#1996). `AbstractResolverTest` enumerates them by name.
Anything else is a regression.
90 changes: 86 additions & 4 deletions providers/flagd/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,21 @@
<version>0.14.2</version> <!--x-release-please-version -->

<properties>
<!-- exclusion expression for e2e tests -->
<testExclusions>**/e2e/*.java</testExclusions>
<!--
Exclusion expression for the two Docker-dependent test packages: the legacy e2e suites
over the test-harness submodule, and the Provider TCK adoption in tck. Two directories
rather than one because they answer different questions and are run by different
profiles; see the `e2e` and `tck` profiles below, which each drop one of the two.
-->
<testExclusions>**/e2e/*.java,**/tck/*.java</testExclusions>
<io.grpc.version>1.82.0</io.grpc.version>
<!-- caution - updating this will break compatibility with older protobuf-java versions -->
<protobuf-java.min.version>3.25.6</protobuf-java.min.version>
<com.vmlens.version>1.2.28</com.vmlens.version>
<!-- Transitive flagd-core version -->
<flagd-core.version>[2.0.0,3.0.0)</flagd-core.version>
<!-- Match any tck version locally; CI resolves it from the reactor -->
<tck.version>[0.1.0,)</tck.version>
</properties>

<name>flagd</name>
Expand Down Expand Up @@ -98,6 +105,17 @@
<version>5.14.3</version>
<scope>test</scope>
</dependency>
<!--
OpenFeature Provider TCK. Brings its own Gherkin, step definitions and Compose
lifecycle; the three classes under src/test/java/.../flagd/tck are the whole
adoption. Version range so a local reactor build matches whatever is checked out.
-->
<dependency>
<groupId>dev.openfeature.contrib.tools</groupId>
<artifactId>tck</artifactId>
<version>${tck.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers</artifactId>
Expand Down Expand Up @@ -250,8 +268,24 @@
<!-- this profile handles running the flagd e2e tests -->
<id>e2e</id>
<properties>
<!-- run the e2e tests by clearing the exclusions -->
<testExclusions/>
<!--
Run the legacy flagd e2e suites (Run*Test, over the test-harness submodule) by
dropping their directory from the exclusions and keeping tck's, so the Provider
TCK suites stay out.

Clearing the property outright would run the conformance suites in CI, because
ci.yml's `main` job activates this profile (it passes `e2e` alongside the matrix
profile) and a GitHub runner does have a Docker daemon. That is not what we
want: those two suites are expected to fail on flags the pinned flagd-testbed
image does not serve plus one recorded provider defect, so running them in the
default build would make every unrelated pull request red. They are run by hand
before merge instead. See providers/flagd/README.md.

This used to read `**/e2e/*TckTest.java`, a filename pattern, because both kinds
of suite shared a directory. They no longer do, and a profile naming a directory
cannot be defeated by a class someone names badly.
-->
<testExclusions>**/tck/*.java</testExclusions>
</properties>
<build>
<plugins>
Expand Down Expand Up @@ -300,6 +334,54 @@
</plugins>
</build>
</profile>
<profile>
<!--
This profile runs the Provider TCK conformance suites, and nothing else:

mvn -Ptck -pl providers/flagd test

Do not add `-am`: it pulls tools/tck and tools/flagd-core into the reactor and
runs their own suites first, which puts two kinds of failure back on one signal.
A one-off `mvn -pl tools/tck -am -DskipTests install` is what it was there for.

It is a step of its own rather than a corner of the `e2e` profile above because
of what a red build says. `-Pe2e` red means the provider's own end-to-end suites
regressed; `-Ptck` red means conformance failed — and a conformance run carries
failures by design, wherever AbstractResolverTest declares a knownDeviation.
Sharing one signal between "you broke something" and "this is the known state"
ends with somebody silencing the informative half. See Appendix F, "Running the
suite in CI".

Nothing activates this profile in CI, deliberately, for the reason the `e2e`
profile's comment gives: these suites are expected to fail today. They are run by
hand before merge. See providers/flagd/README.md.
-->
<id>tck</id>
<properties>
<!--
Drop tck from the exclusions, the mirror image of what the `e2e` profile does,
and keep e2e excluded. Both halves are needed: the include below alone leaves
the default exclusion in force and runs nothing, and dropping the exclusion
alone runs the module's unit tests alongside the conformance suites.
-->
<testExclusions>**/e2e/*.java</testExclusions>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<includes>
<!-- The whole package: RpcTest and InProcessTest, plus the
abstract parent they share, which Surefire does not run. -->
<include>**/tck/*.java</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>

</project>
Loading
Loading