Overview
The Problem
Microsoft's Accessibility Insights tool was difficult for novice users to understand.
The Goal
Improving learnability and efficiency of novice users
We conducted a learnability study with novice users, that simulated real usage frequency. Evaluated designs with experts, novices, and disabled individuals.
Redesigned interfaces helped novice users report accessibility failures 55% faster
Role
UX Researcher (Qualitative)
UX Designer
UX Writer
Team
Product Manager from Microsoft
UX Researchers
UX Designer
Duration
6 months
What makes this project special?
We didn't just design for novice users, we also designed for and tested with disabled individuals. We wanted to ensure the product itself was accessible and empowered disabled developers in their workflows.
Introduced a data analytics section to indicate progress
Improved button copy and hierarchy
Introduced sorting and filtering options for ease of use
Broke down the entire product flow in Figjam
Critiqued every screen and flow
Conducted a competitive study of other product's flows
Conducted a deep dive into the research areas of interest
Collated data and information needs and chose appropriate methods
The idea was to understand how novice users' learned the platforms functionality over time and improve their learning curve.
3 tasks based on 3 common novice user behaviours
Ran 3 trials per participant with a gap of 1-2 days (average gap for early usage)
Measured time on task, number of errors
For the third task, we asked users to summarise the test results
The hypothesis was that users would use the overview page
Interestingly, most users started summarising using the Notes app, Microsoft Word, Google Sheets, etc
Conducted visual analysis of user's notes from Task 3
Plotted a curve of time on task vs number of errors
Users were generally able to perform all tasks by the third trial
Interestingly, the time on task for the third task increased as users got familiar with the tool
We uncovered over 13 problems hampering learnability
As the product manager, I analysed the effort for each and prioritised three areas to work on
I ideated predictable design patterns specific to novice user behaviour based on the visual analysis of user's notes
We conducted design critique sessions with UX experts
I converted the sketches to mid-fidelity wireframes and conducted some early user testing
After three iterations, we had enough to begin the design phase
We conducted user testing with 3 visually impaired / legally blind users
Interesting insight - Screen readers do not explain graphs well, ARIA labels if not descriptive are usually useless for graphs
Based on users vocabulary and notes, I created UX writing guidelines
Clarified vocabulary and added guidelines on graph descriptions for accessibility
For onboarding, users’ average time on task went down by 93%
This increase in efficiency is also supported by qualitative data in which users expressed that the information provided about each test was comprehensive and clear.
For the task where users were asked to report failures from the overview page, the time on task went down by 55%
