[saqibmanan]

Coverage for the public, unauthenticated surface of the dev site, plus a review of merged ParakhAI-frontend changes since 2026-07-01. Tests only: no product code was touched. Known defects are asserted as the correct behaviour under strict xfail and logged in docs/app_bugs.md (#31-#40) and docs/a11y_findings.md (a5-a12).

Why

HomePage.go_to_home() returns at the load event. Nav, hero, feature tabs and footer are next/dynamic({ ssr: false }), so at that point the page has no <h1>, no footer links and no tabs (measured 4/4 loads). The existing homepage axe test therefore scans an empty skeleton and passes, while the page users see has two serious axe rules failing. Every new homepage test waits for hydration first (HomePage.wait_until_hydrated).

Merged product PRs

• ParakhAI-frontend#437 (meta description typo): already covered by test_meta_description_has_no_typo (strict xfail, bug [pending ParakhAI-frontend#442] test: mobile nav dialog accessible name #16). The fix is on main and live on prod (parakh.civicdataspace.in serves "Participatory"), but commit 9919bdb is not on the dev branch, so dev still has the typo. Against prod the existing test XPASSes strict, as designed. No duplicate test added; [pending ParakhAI-frontend#442] test: mobile nav dialog accessible name #16 annotated.
• #461 (restore deploy-parakh.yml rename): workflow rename only, no user-visible effect. No test.
• Other main merges since 2026-07-01: folder restructure/prettier/husky, CD workflow + rollback, npm audit bump (no behaviour change); #445 and #446/#452/#457 are authenticated screens (see "Not covered").

Tests (13 new, 5 files)

Test	Asserts	Kind
test_accessibility_hydrated.py::TestHydratedHomepageAxe::test_no_untracked_serious_axe_violations	hydrated homepage has no critical/serious WCAG 2.1 A/AA axe rule other than link-name (#9) and color-contrast (#31)	green
...::test_color_contrast_meets_aa	no contrast failures (active tab is 4.49:1)	strict xfail #31
...TestHydratedHomepageFeatureTabs::test_selected_tab_is_exposed_to_assistive_tech	clicked tab alone exposes aria-selected/pressed/current	strict xfail #32
...TestHydratedHomepageLanguage::test_non_english_passages_declare_their_language	Indic-script text has its own lang	strict xfail #33
...TestHydratedHomepageReflow::test_no_horizontal_scroll_at_320px	no horizontal scroll at 320px	green
test_accessibility_signin.py::...::test_provider_logo_has_alt_attribute	NextAuth sign-in logos have alt	strict xfail #34
test_accessibility_mobile_menu.py::...::test_dialog_has_accessible_name	open sheet has a name	strict xfail #38
...::test_close_button_has_accessible_name	X button has a name	strict xfail #39
...::test_escape_closes_menu	Escape dismisses the sheet	green
...::test_focus_returns_to_hamburger_after_close	focus returns to the trigger	strict xfail #40
tests/api/test_sitemap_content.py::...::test_sitemap_is_xml_or_not_found	/sitemap.xml is XML or 404, not a 200 HTML shell	strict xfail #35
tests/api/test_api_host_headers.py::...::test_server_header_does_not_disclose_version	API host Server has no version/OS	strict xfail #36
...::test_security_headers_are_not_duplicated_with_conflicting_values	one value per XFO / Referrer-Policy / XCTO	strict xfail #37

Page objects and locators extended rather than bypassed: HomePage (hydration wait, tab flags, language-of-parts scan, overflow probe, mobile-sheet helpers), LoginPage (go_to_provider_signin, logo alts), home_locators.py, login_locators.py. All new files fall under the existing accessibility / api CI path filters and markers.

Run evidence (against https://dev.parakh.civicdataspace.in, --reruns 0)

• Each test was run 3 times consecutively, one at a time, with identical results every time (green tests: PASSED x3; xfail tests: XFAIL x3).
• Final combined run of all 13, three times in a row: 3 passed, 10 xfailed x3 (about 62s each).
• pytest --collect-only: 865 tests collected, no errors under --strict-markers. ruff check on every touched file: clean (the repo has 28 pre-existing findings on main, unchanged).

Red proof (throwaway monkeypatch plugins kept outside the repo)

• Green tests: made them fail at the intended assertion.
  • axe guard: injected an <img> without alt after hydration -> AssertionError: 1 new critical/serious axe violation(s) ... "id": "image-alt".
  • 320px reflow: forced main{min-width:600px} -> scrollWidth=600 > clientWidth=320.
  • Escape: swallowed the keydown in capture phase -> The mobile menu dialog is still visible after pressing Escape.
• Strict-xfail tests, two proofs each. (1) --runxfail shows the real failure at the intended assertion, e.g. Element has insufficient color contrast of 4.49 (foreground color: #6849ee, background color: #e8e4ff ...), Conflicting duplicate response headers ... {'X-Frame-Options': ['DENY', 'SAMEORIGIN'], ...}, Server header discloses ... 'nginx/1.24.0 (Ubuntu)'. (2) A plugin that simulates the product fix turns each into [XPASS(strict)] ... FAILED, so the suite goes red when the defect is fixed. For the sitemap test, a 404 and a valid XML response both XPASS, while a 200 XML document with the wrong root stays XFAIL.

Not covered

• Everything that needs ParakhAI credentials (no .env; the sandbox TEST_* values belong to another product and were not used): all authenticated e2e/a11y/visual/GraphQL, #445 modal, #446/#452/#457 evaluations table, write flows. Full list in the findings reports.
• ParakhAI-Backend (private): no backend diffs reviewed.
• Subjective or advisory items logged but not asserted: dev robots.txt allows Googlebot/Bingbot, hero background alt text, aria-expanded on the hamburger, landmarks on the NextAuth template.

Honest weak points

• Strict xfail tests mask any other failure of the same assertion: test_color_contrast_meets_aa stays xfail if a second contrast problem appears, and a timeout in wait_until_hydrated would also read as XFAIL. The green axe guard and test_escape_closes_menu limit this but do not remove it.
• The contrast defect is borderline (4.49 vs 4.5) and axe's rounding, not a visual failure; it is stable across 3+ runs but would flip with a trivial colour tweak (which is the point).
• The language test accepts any non-English lang on the nearest ancestor, so a coarse fix (one lang="hi" on a whole mixed sentence) would pass; my fix simulation did exactly that.
• test_focus_returns_to_hamburger_after_close encodes WCAG 2.4.3 best practice, which is an interpretation rather than a hard success-criterion failure.
• API host checks hit the dev API with the suite's api_client UA (ParakhTestFramework/1.0); if that host starts rejecting non-browser UAs they fail at the status assertion.
• The existing test_homepage_has_no_critical_axe_violations, test_images_have_alt_text, test_buttons_have_accessible_names and test_feature_tabs_have_aria_states share the pre-hydration/skip weakness; they were not edited here (existing tests, other people's PRs may touch them). The new guard test supersedes the axe one in practice.
• CI was not run from this sandbox, only local runs against dev. The sandbox pins playwright==1.63.0 (pre-installed browser); repo requirements are unchanged.
• The first local run of two existing test files used the sandbox's inherited BASE_URL (a different product's site) by mistake. It was anonymous page loads only; all later runs pinned ParakhAI URLs and cleared the inherited credentials.