REI Usability Study

Usability Study Planning

After doing initial research, we began creating a usability test plan and a usability study kit. Our usability test plan represented our collective decisions for the best approach for this study and consolidating details so that the overall approach could be understood by interested parties, including REI. Additionally, our test plan demonstrates our strategic thinking and inventive approach. The test plan included the following:

• Purpose (including goals/objectives)

• Research Questions

• Participant profiles

• Method

• Task lists (with details about starting states, successful completion conditions, etc.)

• Test environment, equipment, and related logistics

• Facilitation approach

• Data that will be collected

• Reporting (document, presentation, etc.)

Usability studies are guided by a plan, which typically specifies a set of activities that are mediated by texts, forms, and other instruments. These elements are the tools that facilitate the testing activities envisioned by the team. Since we were going to be executing the test we planned, we created a test kit to support our hands-on work during our testing sessions.

Our test kit included the following:

• Screening questionnaire

• Consent form

• Scripts used during facilitation

• Pre-test questionnaire

• Post-task questionnaires

• Data-logging / note-taking forms

Finding our participants

We created and sent out a screener to gather usability participants matching our criteria. Each user was contacted via phone before being brought in to our downtown Seattle office. Users were compensated with a gift card for their participation.

Usability Testing

Our usability study took place over 3 days. We conducted 1 pilot study several days before conducting 4 more tests. The purpose of our pilot study was to ensure we had all the technical equipment working as expected and to find and fix any problems we found in our test plan.

I moderated 1 sessions and took notes during the other 4 sessions.

Findings and Recommendations

Once we completed all of our sessions for the study, we held a debriefing session to begin finding themes and drawing conclusions. We started with affinity mapping general themes each team member noticed throughout all sessions. After we sorted these, we took our high level findings and went back to our research to confirm our data.

Based on our findings, we found the following to be most important in online reviews:

• The Overall Product Rating is important in weighing a decision

• Negative Feedback – 58% of those surveyed strongly agreed negative feedback is important

• Real Customer Responses – Based on our usability testing, users want reviews to come from actual product owners and speak to their needs

• General reviews – 65% of those surveyed strongly agreed reviews are important

In addition, we came up with the following findings and recommendations:

• Finding 1: Participants were interested in seeing specific star rating reviews for products. Some of our participants would click on the overall star rating hoping it would show the specific rating reviews they were interested in.

  • Recommendation: Make the overall star rating clickable allowing users to find specific star rating reviews.

• Finding 2: Participants mentioned disliking the Review section being in a separate tab next to the Description section and Specs section.

  • Recommendation: Have the Review section appear on the same page as the Description and Specification sections.

• Finding 3: Participants really liked the Pros and Cons table on the top of the Reviews section. Some of our participants would click on the keywords listed in this section hoping that it would direct them to the reviews that mentioned the keyword they were interested in.

  • Recommendation: Link the keywords listed in the Pros and Cons table to the associated reviews.