Operationalize Usability Research: Zebra batch analysis & synthesis

[brunerae]

After completing usability testing sessions, we'll spend time analyzing the data and then pulling it together to make sense of it and gain useful insights.

[jaclinec]

Here is the zebra test analysis document 🔒 where detailed observations and notes are captured, plus some first level findings summaries in each section.

I've distilled the data into a draft of the Zebra test findings report. 🔒 It's getting close to finished but I'm still editing and enhancing it.

[jaclinec]

I'm pretty much finished writing up the findings report and have started creating the zebra test findings slide deck to present to the team (still a WIP).

[jaclinec]

The findings report and slide deck are completed and a meeting has been scheduled for me to present the findings to the entire team for April 30th. It's possible I may further tweak the report and slide deck but this issue can be marked as complete.

Next, we'll create some issues from the findings so they are documented in Github and we can better take action on solutions.

[annepetersen]

You could consider drafting the wiki summary and issues before presenting your findings 💡

[thisisdano]

Can we get a link to the summary slides and wiki findings in this issue before closing?

[jaclinec]

@thisisdano the public wiki has not been published yet as its in review (#5704), but it'll be done in the next sprint.

Here's the Zebra findings slide deck and here's a summary of findings adapted from the zebra findings written report:

Summary

In March through April 2024, we conducted usability testing with eight participants with various disabilities (including visual impairments, motor impairments, ADHD, and anxiety and depression). Participants used a mix of assistive technology, including screen readers and screen magnification software, and we tested on desktop and mobile devices in a variety of browsers.

Our research assessed the usability and accessibility of several U.S. Web Design System (USWDS) components:

• Banner
• Site alert
• Memorable date
• Time picker
• Radio button

We also tested disabled states and error handling in our prototype — a fictional form for reserving a special activity at a fictional national park.

We found that most components performed well overall for all participant groups. We identified the most opportunities for improvements with the memorable date and time picker components as well as with error handling.

For the memorable date component, we heard that participants value the ability to avoid and fix errors easily, so participants preferred the drop-down option over the text input for the memorable date “month” field. However, we also noticed evidence that multiple input types (e.g., dropdowns and text inputs) could be problematic. We found unhelpful and verbose instruction announcements for both the memorable date and time picker components. Finally, we observed issues with error handling that have wider implications for the guidance USWDS offers web teams.

[thisisdano]

Thank you!