ci: parse and analyse the documentation's PHP examples - #69
Open
DannyvdSluijs wants to merge 3 commits into
Open
ci: parse and analyse the documentation's PHP examples#69DannyvdSluijs wants to merge 3 commits into
DannyvdSluijs wants to merge 3 commits into
Conversation
The package this site documents lives in a different repository, so nothing in the build compiles the examples. A snippet that is not valid PHP renders perfectly and ships broken, which is how the drift reported in #63 went unnoticed for as long as it did. Adds a script that pulls every ```php fence out of _docs/ and resources/includes/ and runs php -l over it, reporting failures against the markdown file and the line the fence starts on rather than the temporary file the linter saw. The workflow runs it on pull requests and on pushes to main. Deliberately a parse check and no more. Running the examples would mean installing json-mapper/json-mapper here and pinning a version, which is the coupling this repo has so far avoided; the linter needs only a PHP binary and no Composer install, so the job takes seconds. The README gains the conventions the examples follow, since they are now partly enforced: fully-qualified names with no import block, examples defining the classes they map onto, and version annotations only for APIs added during 2.x. Refs #63
The parse check added in the previous commit catches only invalid PHP, which is the smallest part of the problem. Every defect reported in #63 was valid PHP: JsonMapperBuilder::create() parses perfectly and does not exist, and mapToClass('{ "name": ... }', User::class) parses perfectly and is handed a string where the signature wants a \stdClass. Adds a second pass that runs PHPStan against json-mapper/json-mapper, now a dev dependency so the examples are checked against the API they document. Verified by re-introducing five of the original defects: each was reported on the right page and the exact line, while the parse pass stayed green throughout. Mechanics worth knowing: - The extractor keeps every line at its markdown line number, blanking whatever is not code, so PHPStan's line numbers need no translation. - Examples are extracted per page, since a page often defines a class in one fence and maps onto it in the next, and each page gets its own namespace so the User that several pages define does not collide. - A fence declaring its own namespace becomes its own analysed file. Classes the examples borrow from the reader's application, or from companion packages that cannot install alongside Hyde, are stubbed. Where a real package does coexist it is required instead: json-mapper/laravel-package and monolog/monolog are installed so those pages are checked for real. Four examples changed to name their placeholders in full — \App\YourExtended JsonMapper, \App\Models\License, \JsonMapper\SymfonyBundle\JsonMapperBundle — which matches the site's fully-qualified convention. The final callback example dropped a Cache::put() line that referenced an undefined $seconds and stood for nothing in particular. Still nothing executes the examples, so a well-typed snippet that throws at run time gets through; (new JsonMapperFactory())->create() with no middleware is the one #63 defect this cannot see. Refs #63
The docs index claimed "These pages describe JsonMapper 2.25.1" as prose, which is a number nobody will remember to change. Now that the examples are analysed against an installed json-mapper/json-mapper, the version the site documents and the version its examples are checked against are the same fact, so it should be read rather than written. App\Support\DocumentedVersion resolves it from Composer\InstalledVersions, and config/docs.php shows it once at the foot of the documentation sidebar, beside the existing links. Bumping the dependency now updates the site. The deploy installs with --no-dev and JsonMapper is a dev dependency, so InstalledVersions does not know it at build time. Verified that it throws OutOfBoundsException there, and the resolver falls back to composer.lock, which ships regardless. A --no-dev build was run end to end to confirm the sidebar still renders the version. Refs #63
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #63 / #68 — the preventative half of that issue, for this repo.
Why
The package these pages document lives in
JsonMapper/JsonMapper, so nothing in the build compiles the
examples. A snippet that is not valid PHP renders perfectly and ships broken. That is how the drift in
#63 survived: the Setup page called
JsonMapperBuilder::create(), a method that does not exist, and thelanding page passed a JSON string to
mapToClass(), whose first parameter is a\stdClass. Both lookedfine on the page.
What this adds
.github/scripts/lint-code-examples.phppulls every```phpfence out of_docs/andresources/includes/and runsphp -lover each one, then.github/workflows/lint-examples.ymlruns it on pull requests and pushes tomain.Failures are reported against the markdown file and the line the fence starts on, not the temporary
file the linter actually saw:
Blocks are linted individually, because that is how a reader meets them. A fence may or may not open
with its own
<?php; the script normalises to exactly one and tracks whether that shifted the code, sothe reported line stays correct either way. Both shapes were tested by deliberately breaking a fence of
each kind and confirming the reported line matched the break.
Scope
It is a parse check and nothing more — it will not tell you an example is wrong, only that it is not
valid PHP. Actually running the examples means installing
json-mapper/json-mapperhere and pinning aversion, which is the coupling this repo has so far avoided. The linter needs only a PHP binary and no
Composer install, so the job takes seconds.
That leaves the stronger check — executing the examples against a real package — as a candidate for the
package's own repository, where the dependency already exists.
Also
The README gains a Code examples section, since the conventions are now partly enforced: full class
names with no import block (and the
namespace-declaring exception), examples defining the classes theymap onto rather than pointing at the package's test fixtures, and "Available since" only for APIs added
during 2.x.
Run it locally with:
$PHP .github/scripts/lint-code-examples.phpCurrently:
All 31 PHP code blocks in 24 files parse cleanly.🤖 Generated with Claude Code
Update: a second pass that catches more than parse errors
A parse check alone is too weak, and #63 proves it — every defect in that issue was valid PHP.
JsonMapperBuilder::create()parses perfectly and does not exist.So this PR now runs two passes. The second installs
json-mapper/json-mapperas a dev dependency andruns PHPStan against it, checking the examples against the API they actually document.
I verified it by re-introducing five of the original defects. The parse pass stayed green; the analysis
pass caught all five, on the right page and the exact line:
Six of the seven defect shapes from #63 are now caught. The seventh —
(new JsonMapperFactory())->create()with no middleware — is valid, well-typed, and throws only at runtime, so no static analyser can see it. That would need the examples executed, which is better placed
in the package's own repository where the dependency already exists.
How the line numbers work
The extractor keeps every line at its markdown line number, blanking out whatever is not code, so
PHPStan's "line 48" is line 48 of the page. No line map, no translation.
Examples are extracted per page, because pages routinely define a class in one fence and map onto it in
the next, and each page gets its own namespace so the
Userthat several pages define does notcollide. A fence declaring its own
namespacebecomes its own analysed file.Placeholders
Classes the examples borrow from the reader's application, or from companion packages that conflict
with Hyde's dependencies, are declared in
.github/stubs/placeholders.php. Where a real package doescoexist it is required instead of stubbed —
json-mapper/laravel-packageandmonolog/monologareinstalled, so the Laravel and debugging pages are checked against genuine classes.
Four examples now name their placeholders in full (
\App\Models\License,\JsonMapper\SymfonyBundle\JsonMapperBundle, and so on), which matches the site's fully-qualifiedconvention. The final-callback example dropped a
Cache::put()line that referenced an undefined$secondsand stood for nothing in particular.Job runs in 23s.
Update: the documented version is now read, not written
_docs/index.mdused to claim "These pages describe JsonMapper 2.25.1" in prose — a numbernobody would remember to change. Now that the examples are analysed against an installed
json-mapper/json-mapper, the version the site documents and the version its examples are checkedagainst are the same fact, so it is read from Composer instead.
App\Support\DocumentedVersionresolves it, andconfig/docs.phprenders it once, at the foot ofthe documentation sidebar next to the existing links:
Bumping the dependency now updates the site. No version number is hardcoded in any page.
The
--no-devcatch.Composer\InstalledVersionsonly knows about installed packages, and thedeploy workflow runs
composer install --no-devwhile JsonMapper is a dev dependency — so at buildtime it is absent and
getPrettyVersion()throwsOutOfBoundsException. The resolver falls back tocomposer.lock, which ships either way and pins the same version.Verified rather than assumed: both branches were exercised directly, and a full
--no-devbuild wasrun end to end, which still rendered
Documenting JsonMapper 2.25.1in the sidebar.A
dev-branch alias resolves to no version at all rather than printing something meaningless, inwhich case the sidebar simply omits the line.