V9.0.0 - Modernize CI workflow, update project structure, and enhance API coverage

.github/workflows/bump-api-enumerations.yml

@@ -0,0 +1,48 @@
+name: Bump api-enumerations submodule
+
+on:
+  schedule:
+    # 03:00 UTC on the 1st of every month.
+    - cron: "0 3 1 * *"
+  workflow_dispatch: {}
+
+jobs:
+  bump:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          submodules: recursive
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update api-enumerations submodule
+        id: update
+        run: |
+          cd external/api-enumerations
+          git fetch origin
+          before=$(git rev-parse HEAD)
+          git checkout origin/master
+          after=$(git rev-parse HEAD)
+          cd ../..
+          if [ "$before" = "$after" ]; then
+            echo "changed=false" >> "$GITHUB_OUTPUT"
+          else
+            echo "changed=true" >> "$GITHUB_OUTPUT"
+          fi
+
+      - name: Create pull request
+        if: steps.update.outputs.changed == 'true'
+        uses: peter-evans/create-pull-request@v6
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+          commit-message: "Bump api-enumerations submodule"
+          title: "Bump api-enumerations submodule"
+          body: |
+            Automated monthly update of the `external/api-enumerations` submodule
+            to the latest upstream `master` commit.
+
+            Review the upstream diff before merging — new/changed enum
+            descriptions will flow into the generated value types (plan `04`)
+            the next time the package is built and released.
+          branch: chore/bump-api-enumerations
+          delete-branch: true

.github/workflows/continuous-integration-workflow.yml

@@ -1,21 +1,73 @@
 name: Continuous Integration Workflow
 
-on: [push, pull_request]
+on:
+  push:
+  pull_request:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: "NuGet package version (SemVer, e.g. 9.1.0-pre.1)"
+        required: true
+        type: string
+      publish_package:
+        description: "Publish packages and create a release"
+        required: true
+        default: true
+        type: boolean
+      prerelease:
+        description: "Mark the GitHub release as prerelease"
+        required: true
+        default: true
+        type: boolean
 
 jobs:
   build:
     runs-on: ubuntu-latest
     env:
-      VERSION: 8.0.${{ github.run_number }}
+      VERSION: ${{ github.ref == 'refs/heads/master' && 'v9.0.0' || (github.event_name == 'workflow_dispatch' && format('v{0}', github.event.inputs.version) || format('v9.0.0-pre{0}', github.run_number)) }}
+      PACKAGE_VERSION: ${{ github.ref == 'refs/heads/master' && '9.0.0' || (github.event_name == 'workflow_dispatch' && github.event.inputs.version || format('9.0.0-pre{0}', github.run_number)) }}
+      PUBLISH_PACKAGE: ${{ (github.event_name == 'workflow_dispatch' && github.event.inputs.publish_package == 'true') || github.ref == 'refs/heads/master' || github.ref == 'refs/heads/prerelease' }}
+      IS_PRERELEASE: ${{ (github.event_name == 'workflow_dispatch' && github.event.inputs.prerelease == 'true') || github.ref != 'refs/heads/master' }}
       DOCKER_BUILDKIT: 1
       BUILDKIT_PROGRESS: plain
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v4
+      with:
+        submodules: recursive
     - name: Setup .NET
-      uses: actions/setup-dotnet@v1
+      uses: actions/setup-dotnet@v4
+      with:
+        global-json-file: global.json
     - name: Docker Build NuGet packages
+      env:
+        COMPANIES_HOUSE_API_KEY: ${{ secrets.COMPANIES_HOUSE_API_KEY }}
+      run: |
+        docker build --build-arg NUGET_PACKAGE_VERSION=${{ env.PACKAGE_VERSION }} --secret id=companies_house_api_key,env=COMPANIES_HOUSE_API_KEY -f ./Dockerfile --output ./ .
+    - name: Validate NuGet package metadata
       run: |
-        docker build --build-arg NUGET_PACKAGE_VERSION=${{ env.VERSION }} --build-arg COMPANIES_HOUSE_API_KEY=${{ secrets.COMPANIES_HOUSE_API_KEY }} -f ./Dockerfile --output ./ .
+        mapfile -t packages < <(find ./artifacts -maxdepth 1 -type f -name '*.nupkg' ! -name '*.snupkg' | sort)
+        if [ "${#packages[@]}" -eq 0 ]; then
+          echo "No .nupkg files were produced."
+          exit 1
+        fi
+
+        for package in "${packages[@]}"; do
+          if ! unzip -Z1 "$package" | grep -qx "README.md"; then
+            echo "README.md missing from package: $package"
+            exit 1
+          fi
+
+          nuspec_path="$(unzip -Z1 "$package" | grep -E '\.nuspec$' | head -n 1)"
+          if [ -z "$nuspec_path" ]; then
+            echo "No .nuspec file found in package: $package"
+            exit 1
+          fi
+
+          if ! unzip -p "$package" "$nuspec_path" | grep -q '<readme>README.md</readme>'; then
+            echo "NuSpec readme metadata missing from package: $package"
+            exit 1
+          fi
+        done
     - name: Publish Unit Test Results
       uses: dorny/test-reporter@v1
       if: always()
@@ -26,19 +78,40 @@ jobs:
         fail-on-error: true
         fail-on-empty: true
     - name: NuGet.Org push
-      if: github.ref == 'refs/heads/master'
+      if: ${{ env.PUBLISH_PACKAGE }}
+      run: |
+        dotnet nuget push ./artifacts/*.nupkg --source NuGet.org --api-key ${{ secrets.NUGET_API_KEY }} --skip-duplicate
+    - name: Generate release notes with NuGet links
+      if: ${{ env.PUBLISH_PACKAGE }}
       run: |
-        dotnet nuget push ./artifacts/*.nupkg --source NuGet.org --api-key ${{ secrets.NUGET_API_KEY }}
+        printf '%s\n' \
+          '## NuGet Packages' \
+          '' \
+          'This release includes the following NuGet packages:' \
+          '' \
+          "- [CompaniesHouse](https://www.nuget.org/packages/CompaniesHouse/${{ env.PACKAGE_VERSION }}) - Core .NET client for Companies House API" \
+          "- [CompaniesHouse.Extensions.Microsoft.DependencyInjection](https://www.nuget.org/packages/CompaniesHouse.Extensions.Microsoft.DependencyInjection/${{ env.PACKAGE_VERSION }}) - DI helpers for ASP.NET Core / generic-host apps" \
+          '' \
+          'Install via:' \
+          '```' \
+          'dotnet add package CompaniesHouse' \
+          'dotnet add package CompaniesHouse.Extensions.Microsoft.DependencyInjection' \
+          '```' \
+          '' \
+          'See the [README](https://github.com/kevbite/CompaniesHouse.NET#readme) for usage instructions.' \
+          > release_notes.md
     - name: Create Release
       id: create_release
-      if: github.ref == 'refs/heads/master'
-      uses: actions/create-release@v1
+      if: ${{ env.PUBLISH_PACKAGE }}
+      uses: softprops/action-gh-release@v2
       env:
         GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
       with:
         tag_name: ${{ env.VERSION }}
-        release_name: Release ${{ env.VERSION }}
-        body: |
-          Release ${{ env.VERSION }}
+        name: Release ${{ env.VERSION }}
+        body_path: release_notes.md
+        files: |
+          ./artifacts/*.nupkg
+          ./artifacts/*.snupkg
         draft: false
-        prerelease: false
+        prerelease: ${{ env.IS_PRERELEASE == 'true' }}

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "external/api-enumerations"]
+	path = external/api-enumerations
+	url = https://github.com/companieshouse/api-enumerations

.plans/README.md

@@ -0,0 +1,44 @@
+# .plans
+
+The work breakdown for the **CompaniesHouse.NET v-next** major rewrite (on the
+`prerelease` branch).
+
+## How this folder works
+
+- **`outstanding/`** — plans not yet completed. Each file is a self-contained,
+  refinable unit of work. Numeric prefixes suggest ordering (lower first).
+- **`completed/`** — plans that have been fully delivered and verified. When you
+  finish a plan, **move its file here** in the same change.
+
+Plans are **living documents**. Refine tasks, record decisions, and capture
+open questions as you learn. A plan is "done" only when its acceptance criteria
+are met and its code is merged to `prerelease`.
+
+## Plan index (outstanding)
+
+| # | Plan | Theme |
+|---|------|-------|
+| 00 | `00-foundation-solution-and-build.md` | `.slnx`, central packages, multi-target net8/9/10, drop Newtonsoft, CI |
+| 01 | `01-core-client-architecture.md` | `CompaniesHouseClient` entry point, sub-client pattern, `System.Text.Json`, response/error model |
+| 02 | `02-di-extensions-ioptions.md` | Modern DI with `IOptions<>` / `AddOptions` / config binding |
+| 03 | `03-string-backed-value-types.md` | Replace all enums with string-backed `readonly record struct`s |
+| 04 | `04-enum-source-generator.md` | Roslyn generator that emits the value types |
+| 05 | `05-api-enumerations-submodule.md` | `api-enumerations` git submodule + local "extra" lists |
+| 06 | `06-endpoint-search.md` | All 7 search endpoints (incl. advanced search) — **start here for endpoints** |
+| 07 | `07-endpoint-company-profile.md` | Company profile |
+| 08 | `08-endpoint-officers.md` | Officer list + get appointment |
+| 09 | `09-endpoint-catalogue-remaining.md` | Every other endpoint, to be split into its own plan when picked up |
+| 10 | `10-testing-strategy.md` | Unit / scenario / integration / generator tests |
+| 11 | `11-docs-samples-migration.md` | README, samples, v-old → v-next migration guide |
+| 99 | `99-recurring-issues-backlog.md` | Historical pain points the design must eliminate |
+
+## Suggested execution order
+
+1. **Foundation** (`00`) — get the solution building on modern targets first.
+2. **Core + DI + enums** (`01`, `02`, `03`, `04`, `05`) — the plumbing every
+   endpoint depends on. `03`/`04`/`05` can proceed in parallel with `01`.
+3. **Endpoints, one at a time** (`06` → `07` → `08` → `09`), starting with
+   search. Each endpoint should be shippable on its own.
+4. **Testing and docs** (`10`, `11`) run continuously alongside the endpoints.
+
+Keep `99` open as a checklist to validate the design against real-world bugs.

.plans/completed/00-foundation-solution-and-build.md

@@ -0,0 +1,158 @@
+# 00 — Foundation: solution, packaging & build
+
+**Status:** complete
+**Depends on:** nothing (do this first)
+**Blocks:** everything
+
+## Goal
+
+Get the repository building on a modern, consistent foundation so every
+subsequent plan lands on solid ground: modern target frameworks, central
+package management, `.slnx` solution, and an updated CI pipeline. No behaviour
+changes to the client itself — this is pure infrastructure.
+
+## Why
+
+The current projects target `netstandard1.1;netstandard2.0;net45`, pin package
+versions per-project, use a classic `.sln`, and depend on `Newtonsoft.Json`.
+For a clean-slate major version we want the most modern setup possible.
+
+## Scope
+
+### Target frameworks
+- Multi-target the shippable libraries to **`net8.0;net9.0;net10.0`**.
+  - `src/CompaniesHouse`
+  - `src/CompaniesHouse.Extensions.Microsoft.DependencyInjection`
+- Remove `netstandard*` / `net45` targets and the
+  `Microsoft.NETFramework.ReferenceAssemblies` and `Microsoft.Net.Http`
+  package references.
+- Tests target `net8.0;net9.0;net10.0` (or just `net10.0` if multi-targeting
+  tests is not worth the run time — decide and note it).
+- The source generator project (plan `04`) targets **`netstandard2.0`** — this
+  is a hard Roslyn requirement and is the one exception to the "no netstandard"
+  rule.
+
+### Central Package Management (CPM)
+- Add a root **`Directory.Packages.props`** with
+  `<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>` and a
+  `<PackageVersion .../>` for every dependency used anywhere in the repo.
+- Strip `Version="..."` from every `<PackageReference>` in every `.csproj`.
+- Consolidate versions so all projects share one version per package
+  (previously `Microsoft.Extensions.*` was pinned to `3.1.9`).
+
+### Directory.Build.props / .targets
+- Enable **`<Nullable>enable</Nullable>`** (already have `ImplicitUsings`,
+  `LangVersion latest`, `TreatWarningsAsErrors`, `EnforceCodeStyleInBuild`).
+- Refresh `<Copyright>` (currently hard-coded to 2020) — use a year-agnostic or
+  current value.
+- Keep `IncludeSymbols` + `snupkg`; add `<PublishRepositoryUrl>`,
+  `<EmbedUntrackedSources>`, `<ContinuousIntegrationBuild>` and
+  **deterministic builds** for good source-link/NuGet hygiene.
+- Add `<GenerateDocumentationFile>true</GenerateDocumentationFile>` on the
+  shipped libraries so public XML docs are packaged.
+
+### Solution format
+- Convert `CompaniesHouse.sln` to **`CompaniesHouse.slnx`** (the new XML
+  solution format). Verify `dotnet build CompaniesHouse.slnx` works with the
+  installed SDK (repo has 10.x and 11.x preview SDKs available). Delete the old
+  `.sln` once the `.slnx` is proven, or keep both briefly if tooling needs it —
+  decide and note.
+- Add an `.slnx` entry for the future source-generator project.
+
+### CI workflow
+- Update `.github/workflows/continuous-integration-workflow.yml`:
+  - Ensure the SDK it installs can build `net10.0` (+ `.slnx`); pin via
+    `global.json` if needed.
+  - Recursively checkout submodules (needed once plan `05` lands):
+    `actions/checkout` with `submodules: recursive`.
+  - Bump the `VERSION` scheme to the new major (the prerelease tag currently
+    produces `9.0.0-preN` — align with the chosen next major).
+  - Modernise action versions (`checkout@v2`/`setup-dotnet@v1` are old).
+
+## Tasks
+
+- [x] Add `Directory.Packages.props` and migrate all `PackageReference`s.
+- [x] Retarget both library projects to `net8.0;net9.0;net10.0`.
+- [x] Remove framework-reference/`Microsoft.Net.Http` packages.
+- [x] Enable nullable + doc generation + deterministic build in `Directory.Build.props`.
+- [x] Convert solution to `.slnx`; add all existing projects.
+- [x] Update CI (SDK, submodules, versioning, action versions).
+- [x] `dotnet build -c Release` and `dotnet test -c Release` are green.
+- [x] Migrate test stack from NUnit/FluentAssertions to xUnit/Shouldly (scope
+      addition requested mid-execution — FluentAssertions' license changed to a
+      paid tier from v8; NUnit swapped along with it). See "Test stack
+      migration" below.
+
+## Design decisions
+
+- **CPM over per-project versions** — single place to bump, no drift.
+- **Drop `netstandard`** — the new major only supports in-support .NET; this
+  is an intentional breaking change and is fine for a new major.
+
+## Open questions
+
+- Do we keep `net8.0` (LTS) as the floor, or go `net9.0`+ only? (Assumption:
+  keep `net8.0` for the widest supported reach; revisit if a dependency forces
+  it.)
+Keep net8.0 for the time being.
+
+- Should tests multi-target or run once on `net10.0`? (Assumption: run on
+  `net10.0` only for speed; multi-target the libraries only.)
+Just target the latest version of `net10.0`
+
+## Acceptance criteria
+
+- Solution builds and tests pass from a clean checkout with only the .NET SDK
+  installed.
+- No `Newtonsoft.Json`, `netstandard`, or `net45` remain in any shipped
+  project (Newtonsoft removal itself is finished in plan `01`).
+- All package versions resolve from `Directory.Packages.props`.
+
+## Test stack migration (Shouldly + xUnit)
+
+Mid-execution the user asked to drop FluentAssertions (license changed to a
+paid tier from v8) in favour of **Shouldly**, and to swap **NUnit for xUnit**
+at the same time. This expanded plan `00`'s scope to a full test-framework
+port across all four test projects (~66 files). Completed:
+
+- `Directory.Packages.props`: removed `NUnit`, `NUnit3TestAdapter`,
+  `FluentAssertions`; added `xunit` (2.9.2), `xunit.runner.visualstudio`
+  (2.8.2), `Shouldly` (4.2.1).
+- All NUnit attributes converted to xUnit: `[TestFixture]` removed,
+  `[Test]` → `[Fact]`/`[Theory]`, `[TestCase]` → `[InlineData]`,
+  `[TestCaseSource]` → `[MemberData]`, `[SetUp]`/`[TearDown]` → constructor
+  or `IAsyncLifetime`.
+- All FluentAssertions/`NUnit` classic assertions converted to Shouldly
+  (`.Should().Be(x)` → `.ShouldBe(x)`, etc).
+- **Enum-equivalency redesign**: `CompaniesHouse.Tests` had bespoke
+  FluentAssertions `IEquivalencyStep` classes (`ComparingEnumWith`,
+  `ComparingArrayEnumWith`) registered via a `[SetUpFixture]` (`Initializer`)
+  to bridge test-fixture raw wire strings against real deserialized C# enum
+  properties during `BeEquivalentTo` comparisons. Shouldly has no equivalent
+  extensibility point. Replaced with a single dependency-free helper,
+  `tests/CompaniesHouse.Tests/EquivalencyAssertionExtensions.cs`, exposing
+  `actual.ShouldBeEquivalentTo(expected, params string[] excludingPropertyNames)`.
+  It recursively walks public properties and bridges enum ↔ raw wire string
+  automatically via each enum member's `[EnumMember(Value=...)]` attribute (no
+  per-enum registration needed — a strict improvement over the old
+  `MapProviders` dictionaries, which had to be hand-maintained in parallel
+  with the enums). The old `ComparingEnumWith.cs`, `ComparingArrayEnumWith.cs`
+  and `Initializer.cs` were deleted. `EnumerationMappings.cs`/`MapProviders/*`
+  were kept — they're still used to enumerate wire-string values for
+  parameterized `TestCaseSource`/`MemberData` test data.
+- Result: `CompaniesHouse.slnx` builds with 0 errors; 594/597 tests pass. The
+  3 failures (`OfficersTestsInvalid`, `PersonsWithSignificantControlTestsInValid`,
+  `CompanyFilingHistoryTestsInvalid` in `CompaniesHouse.IntegrationTests`) are
+  pre-existing/unrelated to this migration — the live Companies House API now
+  returns `200` with an empty result set for malformed company numbers instead
+  of `404`, so the "invalid number ⇒ null data" assumption in these three
+  tests is stale against current API behaviour. Not fixed here (out of scope
+  for infrastructure plan `00`); worth a follow-up ticket.
+- `AGENTS.md` and `.plans/outstanding/10-testing-strategy.md` updated to
+  document xUnit + Shouldly as the standing test-stack convention.
+
+## References
+
+- Issue #188 (System.Text.Json), #199/#191 (move to GitHub Actions — already
+  done, keep modern).
+- `.slnx` format: current .NET SDK solution tooling.