Usability testing of hardware during a pandemic

PatientSecure’s hardware, the palm scanners, were banned in hospitals during the pandemic. Our product team pivoted and addressed the need of the hour by partnering with a vendor (Keyo) to introduce touchless palm scanners to the healthcare industry.

In order to vet the usability of the touchless device in a hospital setting, I spearheaded a 7 week long in-person usability study which I designed, planned, recruited for and conducted during COVID-19.

The analysis from the study –

Informed the direction of the new hardware within the product ecosystem

Influenced patient-facing marketing collaterals and registrar facing training materials

Introduced a key feature enhancement that significantly improved the patient exprience

Solidified cross-functional stakeholder’s trust in the benefits of timely user research

Overview:

Our R&D team had recognized a technically viable vendor (Keyo) that built touchless palm scanners.

With this new hardware, patients would only have to touch the palm scanner once to enroll (this was unavoidable due to technology limitations). At every consecutive visit, they would have to simply “hover” their hand about an inch above the scanner to quickly and securely check-in for their appointment.

The Challenge:

  1. The context of use of Keyo’s existing customers was wildly different from ours.
  2. While the palm scanner was being branded as “contactless” by Keyo, this was not entirely true. Patients would still have to touch the scanner for a
    one-time enrollment.
  3. Patients would now have to use two different gestures for different interactions with the product.
  4. Registrars would have to communicate the two different gestures to patients based on their enrollment status and the type of scanner attached.
The challenge

The process:

DETERMINING THE GOAL OF THE STUDY

Mapping research questions

I kicked off the study by conducting a quick whiteboard session with the stakeholders. This helped determine our main research questions, which I synthesized into two buckets – Perception and Usability.

The business goals dictated the focus of the study³ the patient’s experience with the hardware.

DESIGNING THE STUDY

In order to test the usability of this hardware, remote testing was out of question.

COVID-19 posed an interesting challenge while designing the usability sessions. We were forced to consider many factors that we would otherwise take for granted like –

Length of the session

We didn t want to expose ourselves or others for long, so each session could not be longer than 15 minutes.

Recruitment

We only recruited from within the people we knew well - colleagues, friends, neighbors, extended family; people who could trust us.

Getting to the session

We didn’t want anybody taking public transport, so for about 2 weeks I would drive to people s houses after work to conduct sessions in their backyards/driveway.

Location/Setting

All sessions had to be conducted outdoors - we had sessions in the office parking lot, my backyard, my manager's frontyard, participant's driveway.

COVID-19 protocols

There was at least a 6’ distance between the moderator and the participant. I maintained a very strict cleaning protocol and always traveled with a sanitization kit to every session.

Empathetic messaging

Messaging to participants included a disclaimer to make them feel very comfortable if they wished to revoke their participation without the need to offer an explanation.

RECRUITING

We accepted the constraints that Covid-19 introduced on recruiting participants for our patient persona and managed our expectations for recruitment goals accordingly. We categorized our goals as “must have” and “nice to have” to help us remember our priorities –

Must have:

  • Participants who are unfamiliar with the product

Nice to have:

  • A healthy age distribution
  • A healthy gender distribution
  • Varied technical proficiency
  • Not from a security systems background

Having conducted the first round of sessions with Imprivata employees, I noticed the presence of a strong underlying bias and made the case to the stakeholders to classify “not from a security systems background” as a “must have” goal for recruitment.

CONDUCTING SESSIONS: THE USABILITY TEST METHOD

The sessions were designed to mimic the check-in experience at a hospital clinic. The moderator would play the role of the registrar and guide the participant (who would play the role of the patient) to enroll and authenticate with the new palm scanner.

To ensure consistency across all sessions, I used the same prompts and guidance language while guiding participants to enroll and then authenticate their palm.

THE ANALYSIS AND READ OUT

I leveraged the curiosity that other cross-functional stakeholders had about the new palm scanner and was able to engage not only the Engineering stakeholders, but also Marketing, Sales, Customer Success and Implementation Engineering to start a more holistic discussion around the launch of our new hardware.

IMPACT OF THE ANALYSIS

My hypothesis of adding audio feedback for when a scan is successfully captured to significantly reduce the back and forth between the patient and the registrar, thereby improving both our user groups’ experience resonated strongly with my Engineering and Product Management partners.

In the span of 1 week we built the following enhancements in the test environment –

 

  • A soft “beep” played from the moderator’s computer when a scan was captured. The authentication guidance message was altered to “Please hold your right hand above the scanner until you hear a beep”
  • Introduction of the “Place palm all the way down” guidance message during authentication
  • Introduction of the “Bring hand closer” guidance message during authentication

Phase 2: Testing the hypothesis

I made some tweaks in the study script to maintain consistency across all sessions and recruited 5 new participants. All the new sessions were designed and conducted exactly like the previous sessions to minimize bias.

The results:

THE OUTCOME OF THE STUDY

The analysis from the usability study helped inform the direction of the new hardware and its relationship with both our primary (registrar) and secondary (patient) users. It also helped determine –

Focused direction on verbiage and UI improvements to help the registrar clearly communicate with the patient

Additions to palm placement guidance and animations to ensure that patients have a friction-free scanning experience

Patient-facing marketing collaterals

Registrar-facing training materials

Clarification of terms during sales demos

General recommendations to the customers to ensure high PatientSecure usage during a pandemic