Define launch user testing and accessibility validation plan

[nonprofittechy]

Illinois partner feedback

Illinois asked what testing is planned and specifically named:

• end users;
• court clerks;
• Court Guides at Illinois Court Help;
• ADA/accessibility.

Scope

Create a launch-readiness test plan covering at least:

• Start a new case;
• File into an existing case;
• drafts/resume;
• payment and fee-waiver paths;
• Review and Submit;
• submitted filing/status/refile paths;
• desktop and mobile/small viewport behavior;
• keyboard-only and screen-reader use;
• accessible errors, focus, navigation, labels, instructions, and document-related interactions.

Record which accessibility standard/conformance target is being used and how defects will be triaged before launch.

Related

• Change all buttons that change the URL to links #86 — navigation controls should use appropriate link semantics
• Remove toasts - they are likely inaccessible #121 — inaccessible toast/error behavior
• Confused by the different chrome on screen when go from opening/landing dashboard to login #127 — inconsistent login/application chrome
• Flatten PDFs and DOCX on upload #113 — preserve accessibility during document conversion where feasible

[nonprofittechy]

An extensive testing plan outside of our own internal staff was not contemplated in our funding for this project. My guess is that we are not able to manage an extensive testing plan with court staff outside of our own resources, although there is a planned TIG evaluation of the project whose scope will include LITEFile, correct @VTskier? I would be pretty concerned about how engaging with court staff now, outside of our current very clear boundaries, would derail the timeline for the project. Just running those sessions and triaging feedback (and managing expectations for feedback given that we can't act on for various reasons) is something in my experience that will take a couple of months that we probably don't have.

We have a large suite of unit tests, and we have consistent process in place to catch regressions and defects, much of it with automated testing, including LLM-powered "random walks" through the tool.

We are incorporating automated accessibility audits, as well as LLM-mediated accessibility audits. Our staff also has now developed a few years of experience with WCAG remediation. Accessibility is a top level priority for us.

We will discuss usability testing since we have training and access to students to help with that, but if we want test subjects outside of Massachusetts, we would appreciate help locating them.

Also pinging @samglover in case you have more thoughts on this.

[nonprofittechy]

@mnewsted — tagging for visibility on this Illinois feedback item.

[VTskier]

I can help get a half-dozen general public testers from Vermont, when needed. I'm used to running virtual user testing of our guided interviews and I've been collecting names of interested folks.

When appropriate, I bet I can get the two staff members at the court's access and resource center, too.

The TIG evaluation is an evaluation of our project, and I'm betting they will want to see who LITEfile was tested with.

[nonprofittechy]

I can help get a half-dozen general public testers from Vermont, when needed. I'm used to running virtual user testing of our guided interviews and I've been collecting names of interested folks.

When appropriate, I bet I can get the two staff members at the court's access and resource center, too.

Great! It might be good to try to schedule those pretty soon (end of September maybe?) so we have time to incorporate any feedback they have. I think it would be useful as long as they understand the tool isn't done yet.