U.S. Patent and Trademark Office - PE2E
Problem
Patent examiners spent the majority of their time gathering information from disjointed sources—resulting in confusion, mistakes, and a rapidly growing backlog. The USPTO aimed to solve this problem with the release of PE2E but they had no way to benchmark and measure the experience.
Objective
Measure the usability and efficiency of the PE2E experience
Process
Define metrics to measure the usability and efficiency of PE2E:
System Usability Scale (SUS)
Single Ease Question (SEQ)
Time-on-task
Completion Rate
Errors
Error Impact
Error Frequency
Error Severity = Impact x Frequency
Pass Rate
Fail Rate
Set up process for a bi-monthly usability testing program for PE2E
Collaborate with PMs and designers on tasks to test
Set up Morae software and testing lab with metrics, hot keys, tasks, surveys
Collaborate with Office of Personnel Management to recruit patent examiners from 400 member beta group
Collate information into test plan and write the Usability Testing Manual
Report findings
Gather, analyze, and write findings from each test into a bi-monthly report
Collate a video highlights reel for especially impactful user insights
Recommend and prioritize improvements based off of testing measurements and qualitative observation
Present report to USPTO executives, PE2E leaders, and consultants
Outcomes
Based off of report: work with PMs and designers on improving PE2E with requirements generation, use cases, and wireframes
Provide quantitative benchmark for PMs and designers to exceed in prioritized, follow up work
Sample Artifacts
Usability Testing Manual - I used Morae as the main source of information during the usability testing sessions. Since the lab and tests needed to be set up in a consistent manner I documented the step by step process to set up the lab, Morae, and develop questions. The result was a faster set up of the lab and testing.
Test Plan - Test plans for the federal government are known to be incredibly detailed in their documentation. I much prefer a leaner process that is focused on early and often results. However, there are some benefits to writing a detailed test plan that sets expectations across the board about what type of information would be delivered and how.
Test Report - In each test report I would describe the issues examiners would encounter. Later, I worked with designers, developers, and product owners on how to more effectively design the experience.
Sample UI screen