U.S. Patent and Trademark Office - PE2E

Problem

Patent examiners spent the majority of their time gathering information from disjointed sources—resulting in confusion, mistakes, and a rapidly growing backlog. The USPTO aimed to solve this problem with the release of PE2E but they had no way to benchmark and measure the experience.

Objective

  • Measure the usability and efficiency of the PE2E experience

Process

  • Define metrics to measure the usability and efficiency of PE2E:

    • System Usability Scale (SUS)

    • Single Ease Question (SEQ)

    • Time-on-task

    • Completion Rate

    • Errors

    • Error Impact

    • Error Frequency

    • Error Severity = Impact x Frequency

    • Pass Rate

    • Fail Rate

  • Set up process for a bi-monthly usability testing program for PE2E

    • Collaborate with PMs and designers on tasks to test

    • Set up Morae software and testing lab with metrics, hot keys, tasks, surveys

    • Collaborate with Office of Personnel Management to recruit patent examiners from 400 member beta group

    • Collate information into test plan and write the Usability Testing Manual

  • Report findings

    • Gather, analyze, and write findings from each test into a bi-monthly report

    • Collate a video highlights reel for especially impactful user insights

    • Recommend and prioritize improvements based off of testing measurements and qualitative observation

    • Present report to USPTO executives, PE2E leaders, and consultants

  • Outcomes

    • Based off of report: work with PMs and designers on improving PE2E with requirements generation, use cases, and wireframes

    • Provide quantitative benchmark for PMs and designers to exceed in prioritized, follow up work

Sample Artifacts

Usability Testing Manual - I used Morae as the main source of information during the usability testing sessions. Since the lab and tests needed to be set up in a consistent manner I documented the step by step process to set up the lab, Morae, and d…

Usability Testing Manual - I used Morae as the main source of information during the usability testing sessions. Since the lab and tests needed to be set up in a consistent manner I documented the step by step process to set up the lab, Morae, and develop questions. The result was a faster set up of the lab and testing.

Test Plan - Test plans for the federal government are known to be incredibly detailed in their documentation. I much prefer a leaner process that is focused on early and often results. However, there are some benefits to writing a detailed test plan…

Test Plan - Test plans for the federal government are known to be incredibly detailed in their documentation. I much prefer a leaner process that is focused on early and often results. However, there are some benefits to writing a detailed test plan that sets expectations across the board about what type of information would be delivered and how.

Test Report - In each test report I would describe the issues examiners would encounter. Later, I worked with designers, developers, and product owners on how to more effectively design the experience.

Test Report - In each test report I would describe the issues examiners would encounter. Later, I worked with designers, developers, and product owners on how to more effectively design the experience.

Sample UI screen

Sample UI screen