end-to-end-tests

End-to-end tests for the Open edX Platform, written in Playwright and TypeScript.

The suite is deployment-agnostic: most providers should be able to point it at their own Open edX installation through configuration alone, without editing test code.

See ADR-0002 for the core principles guiding development, and docs/findings.md for the defects this suite has surfaced upstream — the reason behind every test.fixme() and workaround in the tree.

Prerequisites

• Node.js 24 (see .nvmrc). With nvm:

  nvm install
  nvm use

• npm 11+ (ships with Node 24).

• A reachable Open edX installation to test against (see Configuration).

• The Open edX Demo Course imported on the target for the course-completion specs.** Those specs need real content to work through, and importing content differs per installation. So the course is a prerequisite of the environment rather than something the suite creates. Import one with whatever mechanism your installation uses — on Tutor:

  tutor local do importdemocourse

  then point COURSE_KEY at the imported key (the demo course's key differs between installs, e.g. course-v1:OpenedX+DemoX+DemoCourse). Leave COURSE_KEY unset and the course-completion specs skip cleanly; set it to a course that doesn't exist on the server and pre-test checks will fail at startup.

Setup

# 1. Install dependencies
npm install

# 2. Install Playwright browser binaries (first run only)
npm run install:browsers

# 3. Create your local configuration
cp .env.example .env
# then edit .env to point at your installation

Configuration

All installation-specific values are read from environment variables and validated at load time — a missing or malformed value fails fast with a clear message instead of a confusing test failure. Every variable is documented in .env.example.

Configuration is only required by what actually drives an installation. Test collection, --list and the node-only unit project report an invalid environment as a warning and carry on, so a fresh clone with no .env can still run the unit tests and the quality gates; browser projects fail on it fatally.

The essentials:

Variable	Required	Description
LMS_BASE_URL	✅	LMS origin, e.g. http://local.openedx.io
APPS_BASE_URL	✅	MFE host origin, e.g. http://apps.local.openedx.io
CMS_BASE_URL	—	Studio origin; required when studio is in CAPABILITIES
ADMIN_USERNAME / ADMIN_PASSWORD	—	Admin/staff account (set both or neither); also grants author
ORG	—	Organization short code, e.g. OpenedX
COURSE_KEY	—	Course the course-completion specs work through; unset ⇒ they skip
CAPABILITIES	—	Comma-separated capabilities enabled on your install
ALLOW_CROSS_SITE_ORIGINS	—	Escape hatch for non-same-site deployments
ACCOUNT_BACKEND	—	How new accounts clear email activation (see below)
CUSTOM_ACCOUNT_BACKEND_PLUGINS	—	Comma-separated paths of custom account backends
MAIL_PROVIDER	—	Mailbox the suite reads e-mail from; required with email-inbox
CUSTOM_MAIL_PROVIDER_PLUGINS	—	Comma-separated paths of mailbox provider plugins (src/mail/)
RUN_ID_SUFFIX	—	Up to 3 lowercase letters/digits appended to the run id (CI shards)
CI_PROFILE	—	The CI profile a job runs (.ci/profiles.json); set by CI

Where values come from. Configuration is read from process.env, with values from a local .env file layered in underneath. Real environment variables take precedence - a variable already set in your shell or CI environment is not overridden by .env. In practice: use .env for local development, and set environment variables directly in CI (no .env needed there). A .env value only applies when that variable is not already present in the environment.

Capabilities. Coverage that depends on what an installation runs is gated on an explicit declaration: a spec tagged @teams runs only where CAPABILITIES names teams.

• Default-on: stock surfaces and stock settings are on unless turned off with a - prefix (CAPABILITIES=-mfe-authn), so a missing declaration never silently drops coverage an install has.
• Required partners: analytics-in-context, analytics-pii and analytics-staff-video-counts describe parts of an Aspects deployment and need analytics declared too; analytics-in-context-cards needs analytics-in-context.
• Mutually exclusive pairs: two implementations or configurations of one surface. Declaring both halves fails validation.
• Declared but missing: a declared capability the target lacks fails its specs rather than skipping.

docs/capabilities.md describes every capability: what it means, which specs it gates, which Open edX releases are known to support it, and where CI turns it on. Sign-in and sign-out coverage is not gated — it runs through the account backend's own UI flows, whatever those are.

Origin requirements. All origins (LMS, Studio, MFEs) must share one scheme (all http:// or all https://) and one registrable parent domain (e.g. *.local.openedx.io), so a single sign-in covers every sub-domain. Do not use localhost or IP addresses - they have no sub-domain structure for shared cookies. Over HTTP, the target must serve SameSite=Lax, non-Secure cookies.

Account creation & email activation

Installations differ in how a newly-registered account becomes able to sign in — chiefly around email activation. Rather than assume one path, the suite obtains accounts through a user-choosable backend selected by ACCOUNT_BACKEND, so the same specs run against very different targets (see issue #10 and src/accounts/README.md).

ACCOUNT_BACKEND	Use when…	Behaviour
automatic	"Automatic login on" — the default (incl. Tutor/sandbox)	Generates a throwaway @example.com identity; no email needed. The reusable session is taken from the one registration itself creates.
manual	Target enforces email activation and can't be reconfigured	Interactive: prompts you for an email to register with, then for the activation link/token.
openinbox	You want activation email automated, unattended	Example plugin (plugins/openinbox.plugin.ts): registers with a disposable openinbox.io inbox and visits the activation link it receives.
plugin name	Your install has its own auth/mailbox flow	Loaded from CUSTOM_ACCOUNT_BACKEND_PLUGINS; see src/accounts/README.md.

The automatic backend works against the default without any email setup: registration auto-authenticates the account, so the captured (reusable) session and the registration spec pass even on installs that leave accounts inactive until activation. Specs that drive a separate UI sign-in (login, logout) still need an install where the account can actually log in — i.e. SKIP_EMAIL_VALIDATION = True, or the manual backend to activate the account first.

A 3rd-party mailbox API is covered by the openinbox example plugin rather than a built-in backend; a plugin for any other mailbox service follows the same contract. Plugins are currently local-only: the CI workflows expose no input for CUSTOM_ACCOUNT_BACKEND_PLUGINS.

Registration rate limit

Open edX caps account registrations per IP with REGISTRATION_RATELIMIT, whose default is 60/7d (60 per 7 days). Because each run creates several accounts, frequent runs exhaust it and registration then fails with HTTP 403 forbidden-request — the suite reports this explicitly. For any target you run against repeatedly, raise the limit to a short, self-resetting window (the platform's own test settings use per-minute values). With Tutor, override the LMS setting and restart:

# a Tutor plugin patch on "openedx-common-settings"
REGISTRATION_RATELIMIT = "100/m"

tutor local restart lms   # or: tutor dev restart lms

If you have already tripped the default limit, either apply the override above and restart, or wait for the 7-day window to reset.

Password reset rate limit

The password-reset request is throttled twice over, per IP and per email address:

Setting	Default	Scope
PASSWORD_RESET_IP_RATE	1/m	one request per minute per IP
PASSWORD_RESET_EMAIL_RATE	2/h	two requests per hour per email

The per-IP limit is the one that bites: one request per minute covers every worker on the machine, so a second reset request within the same minute — a re-run, a Playwright retry, or another spec — is rejected. POST /account/password then answers HTTP 403 with:

{
  "success": false,
  "value": "Your previous request is in progress, please try again in a few moments."
}

Note that message is misleading: nothing is in progress, the request was throttled. The authn MFE renders it as a generic error alert, so the symptom is a password-reset spec that passes alone and fails when re-run within the minute.

Only the reset-request test posts (an invalid address is rejected in the browser without a request), so a single run is fine; back-to-back runs and CI retries are not. Raise the limit on any target you run against repeatedly:

# a Tutor plugin patch on "openedx-common-settings"
PASSWORD_RESET_IP_RATE = "100/m"
PASSWORD_RESET_EMAIL_RATE = "100/m"

tutor local restart lms   # or: tutor dev restart lms

Running with the manual backend

Set ACCOUNT_BACKEND=manual and run with a single worker so the interactive prompts don't interleave:

ACCOUNT_BACKEND=manual npx playwright test --project=smoke --workers=1 \
  -g "signs in with valid credentials"

For each account the suite provisions, it prompts on your terminal to:

1. enter an email to register with — use an inbox you can read; plus-addressing (you+e2e1@example.com) lets one inbox serve several runs; and
2. paste the activation link or token from the resulting email (either the full .../activate/<key> URL or just the key).

The prompt reads your controlling terminal directly (via /dev/tty) and disables the test timeout while it waits, so an interactive run won't time out. Each provisioned account needs a unique email, so specs that provision their own account (login, logout, and the setup sign-in) prompt once each.

Getting the activation key without email (Tutor). If you administer the target and would rather not wait on (or configure) email, read the pending registrations' activation keys straight from the database and paste the one matching the email you registered with:

tutor local run lms ./manage.py lms shell -c "from common.djangoapps.student.models import Registration; rs = [ (r.user.email, r.activation_key) for r in Registration.objects.select_related('user').all()]; print(rs);"

Use tutor dev run on a dev stack. The prompt accepts either the bare key or the full .../activate/<key> link.

Running tests

npm test                 # run everything
npm run test:smoke       # critical-path (@smoke) tier only
npm run test:regression  # broader (@regression) tier
npm run test:ui          # interactive Playwright UI mode
npm run test:headed      # headed browser
npm run test:debug       # step-through debugger
npm run report           # open the last HTML report

Tests are organized into Playwright projects:

• unit — pure logic tests (e.g. config validation); no browser or target needed.
• setup — signs in once per role and writes reusable auth state to .auth/.
• smoke / regression — anonymous browser tests tagged @smoke / @regression.
• lms-learner — authenticated tests (@authenticated); depends on setup. The project loads the captured learner session; specs that change course state use the courseLearner fixture instead, which provisions a fresh learner per test (one registration each — see the rate limit below).
• studio-author — Studio tests (@author, the tests/studio/ tree); depends on setup. Each worker provisions an author of its own (see "Studio coverage") whose session is valid on Studio as well as the LMS. Runs only when the studio capability is declared.

Responsive coverage runs at the sizes in src/config/viewports.ts (phone 375, tablet 768, small desktop 1024, desktop 1280, each on one side of a breakpoint the Open edX frontends switch on) inside the same projects; there is no separate mobile project.

On a Tutor main target, two toggles that plugins set only in FEATURES must also be set as flat settings for their coverage to run (the TUTOR-001 flattening): ENABLE_COURSE_DISCOVERY for catalog-search, and ENABLE_EDXNOTES for notes. The CI workflow's settings patch sets both.

Run a single project or filter by tag:

npx playwright test --project=unit        # just the unit tests
npx playwright test --project=smoke
npx playwright test --grep @smoke

The unit project needs no configuration or browsers, so you can run it right after npm install:

npx playwright test --project=unit

Studio coverage

Studio specs run when CAPABILITIES includes studio and CMS_BASE_URL is set (declaring one without the other fails configuration). Leave studio undeclared on an LMS-only target and the whole tests/studio/ tree skips.

The author role. The setup project provisions a fresh account, gives it a Studio session (Studio keeps its own session behind a silent OAuth handshake with the LMS — no second sign-in, no credentials; an install that fronts Studio with its own IdP replaces this through the account backend's signInStudio), and grants it course-creator status. The studio-author project then provisions one such author per worker (workerAuthor) and runs that worker's tests as it: the platform's PREVENT_CONCURRENT_LOGINS (on by default) ends a user's other sessions on every sign-in, and the browser specs sign the author in through the UI per test, so a single shared author would have each worker logging the others out mid-test. The admin is the one account that stays shared, so anything that signs in as the admin (the course-creator grant, the superuser-in-a-browser fixtures) runs under a cross-worker lock and reuses the setup session where it can — which also keeps admin sign-ins clear of the platform's per-account login rate limit (LOGISTRATION_PER_EMAIL_RATELIMIT_RATE, default 30/5m; exceeded, sign-in answers 400 "Too many failed login attempts"). Repeated local runs within five minutes can still reach it; wait it out or raise the setting. On a default install (ENABLE_CREATOR_GROUP on) the only grant path is the Studio Django admin, so the default account backend signs in as ADMIN_USERNAME / ADMIN_PASSWORD (a superuser) to approve the request. Without an admin account the author role — and every Studio spec — skips, unless the install grants every user (then no admin is needed). Installs that gate course creation differently implement grantCourseCreator in an account backend plugin (src/accounts/README.md).

Courses the suite creates. There is no API to delete a course, so the suite keeps the count low: the settings specs share one course per worker (authoredCourse), the authoring-to-learner round-trip specs share two more per worker (contentCourse and futureCourse, with authoringCourse giving a per-test course only where a spec needs isolation), the instructor-dashboard certificate specs share one more (certificateCourse, set up so certificates can be issued), and each builds a uniquely named section inside its course rather than a new course. Only the specs whose subject is course creation make their own. Content libraries accumulate the same way: DELETE /api/libraries/v2/<lib>/ answers 500 for any library that ever held a unit, subsection or section (LIB-001, filed as edx-platform#39117), so the library specs seed one small library per test (seededLibrary, plus an empty authoringLibrary where a spec mutates), attempt the delete, and rely on run-unique slugs (e2e-<run id>-…) to keep runs apart; every list they select a library from is filtered by that slug or title first. Every suite course is numbered E2E<run id><slot> under ORG (or E2E when ORG is unset). On a persistent target, purge them with the CMS management command — it prompts, so pipe yes into it:

yes | tutor local exec cms ./manage.py cms delete_course <course key>

then ./manage.py cms reindex_course --all --setup if deleted courses still show in catalog search.

There is no equivalent for v2 libraries today. The CMS offers export_content_library, import_content_library and migrate_course_legacy_library_blocks_to_item_bank, but nothing that deletes a v2 library, and LIB-001 blocks the per-library API for any library that held a container — so the libraries the suite seeds stay on a persistent target. The legacy half is different: ./manage.py cms delete_v1_libraries (run it with --help for its arguments) covers the library-v1: libraries the legacyLibrary fixture creates.

CI's Tutor installs are ephemeral, so nothing accumulates there.

Quality gates

npm run check         # typecheck + lint + format:check (run before pushing)

npm run typecheck     # tsc --noEmit (strict; catches what Playwright's transpile won't)
npm run lint          # ESLint
npm run lint:fix      # ESLint with autofix
npm run format        # Prettier write
npm run format:check  # Prettier check

CI workflows

Browser specs need a running Open edX installation, so they run in dedicated GitHub Actions workflows rather than the fast PR gate (ci.yml, which runs static checks and the browser-free unit project as required, blocking checks). Both workflows share the same run-suite composite action, which runs the suite (or one shard of it) and uploads a Playwright blob report, and the report-suite action, which merges blob reports (merge.config.ts) into the HTML report and suite reports, writes the job summary and publishes BTR results. They differ only in how the target installation is provisioned.

ci.yml additionally runs run_tests_tutor.yml twice every PR and push to main: once against an ephemeral Tutor main, and once against the last named release environment (currently verawood). Each runs the full suite under the default and extended CI profiles (see CI profiles and shards below), so a PR's result covers everything the scheduled runs do.

run_tests_tutor.yml — ephemeral Tutor installation

Stands up a fresh Tutor "local" Open edX install on the runner (installing the named release's Tutor/plugin versions, launching it, importing the demo course, and creating an admin user), then runs the suite against it. Everything is provisioned from scratch, so this is a heavier, self-contained job — nothing to configure beforehand beyond triggering it.

Triggers:

• workflow_dispatch — run on demand with:

  Input	Description
  openedx_release	Named release (main, verawood, ulmo, teak, sumac, redwood); drives the Tutor/plugin version and capabilities. Default verawood.
  test_ref	Git ref of this repo to test. Defaults to the branch the workflow runs from.
  domains	Space-separated domains to run (e.g. lms studio). Empty = all.
  features	Space-separated tag filter (e.g. @smoke @discussions). Empty = all.
  exclude_features	Space-separated tags to exclude (mapped to --grep-invert, e.g. @unit). Empty = exclude nothing.
  capabilities	Override the release's default capabilities (comma-separated); each profile's delta still applies. Empty = use the release default from .ci/openedx-releases.json.
  profiles	Space-separated CI profiles to run (keys of .ci/profiles.json, see below), e.g. default extended. Default default.
  btr_sheet_url	Override: publish BTR results to this Google Sheet instead of the release's BTR_SHEET_URL_<RELEASE> variable (see BTR results sheets).

• schedule — automatically at 09:00 UTC (5am US Eastern in daylight time), Mondays and Fridays, always against this repo's main branch with the main Open edX release (domains/features/capabilities are dispatch-only and don't apply to scheduled runs, so scheduled runs cover the full suite with main's default capabilities).

• workflow_call — called by ci.yml's docker-main and docker-last-release jobs (see above); same inputs as workflow_dispatch.

The Tutor install also runs a Mailpit catcher (a local-docker-compose-services patch, with SMTP_HOST=mailpit and RUN_SMTP=false), so the platform's mail lands where the mailpit provider reads it and email-inbox is declared for main and verawood. Tutor's own relay delivers straight to recipients' mail servers on port 25, which GitHub-hosted runners block. To run the e-mail specs against a local Tutor, do the same there (the tutor-contrib-mailpit plugin targets tutor dev only, so copy its service into a local-docker-compose-services patch) and set MAIL_PROVIDER=mailpit, CUSTOM_MAIL_PROVIDER_PLUGINS=./plugins/mailpit.plugin.ts and MAILPIT_BASE_URL=http://localhost:8025.

CI profiles and shards. .ci/profiles.json defines the named configurations a release can run under. Each profile lists the Tutor plugin files that configure the install (.ci/tutor/e2e_base.py holds the settings every profile starts from), a capability delta applied to the release's list, and a shard count. A plan job expands the selected profiles into one run job per shard. Each provisions its own Tutor install, so shards never share users, courses or site settings, and runs its slice of the suite with --shard. A merge job then combines every shard's blob report into the release's single playwright-report-<release> and suite-reports-<release> artifacts. scripts/ci-profiles.mts reads the file, and the unit project validates it. Each shard's run id carries a RUN_ID_SUFFIX (profile code + shard number), so data created by different shards never collides.

Three profiles exist, and ci.yml runs all three for main and verawood:

Profile	Runs	Installation
default	the whole suite, in 4 shards	.ci/tutor/e2e_base.py
extended	only the cases default skips (select: "delta")	plus .ci/tutor/e2e_extended.py (AuthZ migration left to an operator, a SUPPORT_URL), .ci/seed/extended.sh (a second-organization course, an intro video on the demo course) and, where the release has the plugin, tutor-contrib-codejail for Python-graded problems
aspects	only the analytics cases (select: "delta"), on main and verawood	plus tutor-contrib-aspects (a line per release in the profile: 6.x on main, frontend-base 2; 5.1+ on verawood, frontend-base 1) and .ci/tutor/e2e_aspects.py (learner PII and in-context metrics on, Superset reachable from the LMS container), on its own prebuilt image variant

A select: "delta" profile runs only the tests tagged with a capability it declares and default does not. The merge then reports those tests from it and everything else from default. rbac-global is not in extended: turning the authz flag on for the whole site locks every unmigrated course's team out of Studio, which the other cases need. It would need a profile of its own.

A profile can declare an image variant (images): build_tutor_main_images.yml builds it nightly for each release the profile runs on, with the profile's Tutor plugins and extensions enabled, and pushes it under its own tags and BuildKit caches (<image>:<release>-<variant>). The aspects variant is the Open edX, MFE, Aspects and Superset images. A variant job pulls it; if the pull fails it stops (a variant cannot be rebuilt on the runner, where the profile's plugins are enabled only after launch), so run the build workflow and retry.

The MySQL state after migrations is cached per release/Tutor-version so most runs skip the ~20-minute migration step; delete the tutor-mysql-* cache from the Actions UI if it ever needs a clean rebuild.

Per-release configuration. .ci/openedx-releases.json is the single source of truth for what each named release needs: the Tutor version constraint (replacing what used to be a hardcoded shell case statement) and the default CAPABILITIES to declare for that release, since feature/MFE availability can differ across platform versions. Adding or removing a supported release is just an edit to that file (plus the matching openedx_release dropdown option in the workflow — ci.yml's "Verify openedx_release choices match .ci/openedx-releases.json" step fails the build if the two ever drift apart).

run_tests_external.yml — arbitrary, already-running installation

Runs the suite against any installation you already have running — a Tutor dev stack, a staging server, or a provider's own environment — instead of provisioning one. This is how providers point the suite at their own installation without editing any code.

Credentials are sourced from a GitHub Environment (Settings → Environments) rather than hard-coded, so different targets (and their approval/protection rules) stay isolated from each other:

Input	Description
environment (required)	Name of the GitHub Environment to source ADMIN_USERNAME/ADMIN_PASSWORD secrets from.
test_ref	Git ref of this repo to test. Defaults to the branch the workflow runs from.
lms_base_url (required)	LMS origin, e.g. https://courses.example.com.
apps_base_url (required)	MFE host origin, e.g. https://apps.example.com.
cms_base_url	Studio origin. Leave empty to skip Studio specs.
org	Organization short code.
course_key	Default course for course-completion specs.
capabilities	Comma-separated capabilities enabled on the target.
account_backend	automatic (default; works on the default install, see above) or manual (interactive — not usable in CI).
allow_cross_site_origins	Set when LMS/Studio/MFE origins are not same-site.
mail_provider	none (default) or openinbox: where the target's notification e-mail is read. openinbox needs an OPENINBOX_API_KEY Environment secret; add email-inbox to capabilities.
domains / features	Same filters as above.
exclude_features	Space-separated tags to exclude (mapped to --grep-invert, e.g. @unit). Empty = exclude nothing.
openedx_release	Release the target runs (a key of .ci/openedx-releases.json). Enables publishing to that release's BTR results sheet. Empty = no publish.
btr_sheet_url	Override sheet URL for this run; requires openedx_release.

To run it against your own installation: create a GitHub Environment (e.g. staging) with ADMIN_USERNAME/ADMIN_PASSWORD secrets (and any required approval rule), then trigger the workflow with that environment name and your target's base URLs.

BTR results sheets

Every run writes test-results/btr-run.json: one row per BTR Release Test Plan case exercised (spec, result, notes, timing) plus the run's metadata. Scheduled and manually dispatched runs can publish that file to a Google Sheet, one spreadsheet per Open edX release, kept separate from the manually maintained BTR sheet. Each publish overwrites the sheet's Latest tab (for an unfiltered run of the default branch), appends a row to the Runs index, and adds a timestamped tab that is kept for comparison over time. PR and push runs never publish.

To enable it for a release, in the repository (or a fork, or a provider's deployment):

1. Create a Google Cloud service account, download a JSON key, and store the key's text as the BTR_SHEET_CREDENTIALS secret (repository-wide, or on the GitHub Environment an external target uses).
2. Create an empty Google Sheet per release and share each one with the service account's client_email as an Editor.
3. Set a repository variable BTR_SHEET_URL_<RELEASE> per sheet, with the release upper-cased: BTR_SHEET_URL_MAIN, BTR_SHEET_URL_VERAWOOD, … An Environment-scoped variable of the same name overrides the repository one for run_tests_external.yml runs.

The credentials always come from that secret — no workflow or action input passes them in. A repository without it (a fork, say) still runs the suite: the publish step warns and skips, leaving the run's result unchanged.

The first publish bootstraps the sheet (tabs, headers, title). A sheet remembers its release and refuses runs for another one, so a mis-set variable cannot mix releases. ci.yml warns (without failing) about releases that have no variable. Details and the local re-publish command: src/reporting/README.md.

Consuming the suite from another CI system

The two workflows above are how this repository runs the suite. A deployment pipeline that owns an Open edX installation should not reimplement them - it should reuse the same runnable unit and supply only the target.

• GitHub Actions: use the run-suite composite action.
• GitLab CI: include ci/gitlab/run-suite.yml and extend the hidden .openedx-e2e job it defines. That file's header comment is the reference for the variables it accepts, and ci/gitlab/README.md has worked examples - gating on a deploy, waiting for the target to serve, serialising runs - plus what to check on the installation under test.

Both own the same thing - the image, the install, the Playwright command line, and the report artifacts - so the consuming pipeline is left with the target's origins, its readiness check, and its credentials. Their inputs correspond: domains / features / exclude_features / select_grep / shard on the action are E2E_DOMAINS / E2E_FEATURES / E2E_EXCLUDE_FEATURES / E2E_SELECT_GREP / E2E_SHARD in the template, and adding one to either generally means adding it to both. Inputs that only set an environment variable (mail_provider, profile, run_id_suffix, …) need nothing in the template: a consuming job sets them as variables:.

include:
  - remote: 'https://raw.githubusercontent.com/openedx/end-to-end-tests/<ref>/ci/gitlab/run-suite.yml'

e2e:
  extends: .openedx-e2e
  stage: test
  variables:
    E2E_SUITE_REF: <commit-sha> # required; keep equal to the include ref
    LMS_BASE_URL: https://courses.example.com
    APPS_BASE_URL: https://apps.example.com

The suite itself is configured by its ordinary environment variables, set as variables: on that job - so .env.example stays the single reference for what those are, and a new one works in GitLab the day it works locally. Put secrets in masked CI variables rather than variables: values, which are committed in plain text.

Set PLAYWRIGHT_JUNIT_OUTPUT_FILE in any CI system to have the suite append a JUnit reporter to the ones playwright.config.ts declares. Do not pass --reporter=junit: that replaces the whole list and silently disables the BTR-coverage and accessibility reporters.

Project structure

tests/                 # specs, grouped by platform domain (lms/, studio/)
  lms/auth/            # authn + account MFEs: registration, login, logout, session,
                       #   password reset, profile
  lms/catalog/         # catalog MFE: discovery, enrollment
  lms/course-home/     # learning MFE: outline, progress, unit/course completion
  lms/courseware/      # learning MFE: outline sidebar
  lms/dashboard/       # learner dashboard
  lms/instructor/      # instructor dashboard MFE: course info, enrollments, grading,
                       #   date extensions, data downloads, certificates
  lms/landing.spec.ts  # proof-of-life smoke test
  studio/              # authoring MFE + Studio APIs (bootstrap: author session, course factory)
  conventions/         # suite-wide rules enforced as tests (no displayed text)
  config/ api/ auth/ accounts/ a11y/ reporting/   # unit tests (@unit) per module
  auth.setup.ts        # auth setup project
  global-setup.ts      # clears .auth/, preloads account backends
src/
  config/              # typed, validated environment configuration
    selectors/         # structural anchors per surface, one module each
  api/                 # typed API clients + data factories
  accounts/            # user-choosable account backends (+ plugin loader)
  auth/                # provider-swappable authentication contract
  pages/{lms,studio}/  # page objects, one per surface, in the spec's domain folder
  steps/               # reusable multi-page business flows
  fixtures/            # composition root (config + pages + api + auth + skips)
    assets/            # bundled media served in place of course-hosted sources
  reporting/           # BTR test_id annotations, coverage + run-detail + a11y reporters
  a11y/                # @axe-core/playwright WCAG 2.2 AA gate
plugins/               # example account-backend plugin (openinbox)
.ci/                   # CI configuration: releases, profiles, Tutor plugins, seeds
docs/
  decisions/           # ADRs
  capabilities.md      # every capability: meaning, releases, CI declarations
  findings.md          # defects the suite has surfaced

The layers have a strict dependency direction — config → api → pages → accounts → {auth, steps} → fixtures → tests — with each layer's responsibility described in its own README.md under src/. See ARCHITECTURE.md (layer diagram and responsibilities) and CONVENTIONS.md (locator priority, tagging, test-data rules) for the full mechanics.

Contributing

Start with DEVELOPER.md, the condensed guide to writing a well-formed spec, with links into the detailed documents.

• Write TypeScript under strict settings; npm run check must pass.
• Keep configuration centralized in src/config/ — never read process.env directly in specs.
• Never commit a real .env or captured auth state (both are gitignored).
• When a change works around an upstream defect, or skips a case because of one, record it in docs/findings.md and quote its ID in the test.fixme() reason or the comment. Add the entry in the pull request that needs it, so a reviewer can see what the branch worked around and why.