USABILITY TESTING

Weee! Usability Case Study

June 2022 - July 2022 (6 weeks)

Project Overview

I want to explore Weee's mobile interfaces for discovering new products of interest and finding a specific product, which is essentially the shopping experience without making a purchase. Study participants will be given a series of tasks and both quantitative and qualitative measures will be recorded to access and validate the design of the interface.

My Goals

I wanted to practice designing study tasks that are purposeful and relevant to daily use, and gain experience in conducting remote usability studies , refining my skills at breaking the ice and staying neutral and consistent throughout the different sessions.

My Tasks

Study design
Remote moderated usability testing
Data analysis

Context

Weee! is an online Asian and Hispanic grocery shopping platform founded in 2015 in Fremont, California. The company aims to make groceries and Asian specialty food more accessible for Asian families across North America.

With the pandemic and the "new normal" of the hybrid working model, I anticipate that online/virtual shopping platforms will continue to grow even after the pandemic ends. I want to perform a study on this application because I want to see how it could be improved and what lessons we could learn from it that are also applicable to other eCommerce and grocery delivery platforms. Identifying any potential user experience problems throughout this typical user flow could help with establishing a good first impression of the platform and retaining existing customers.

Participants + research methods

The screener survey was sent to 10 people, and I chose 4 to be my study participants. I made the decision based on their answer to the screener question and their demography of interest to make it inclusive and more representative.

Each research study participant will complete one usability test containing 9 tasks that cover features including navigation, product search, and browsing. I plan to do a remote moderated test with sound and camera on so that I could gather information such as the time it takes to attempt a task (quantitative data) and micro expressions (both facial and vocal) that indicates feelings and emotions as qualitative data. I will also include a short debrief meeting with each participant to clarify and confirm my interpretation of their interactions with the interface.

Quantitative + qualitative data

Key findings

Based on the quantitative data chart, it was clear that the current design performs well in supporting key features like setting/changing delivery time and location, adding products to shopping cart, and searching for commonly seen items.

Product search

In terms of searching for products, searching by brand works pretty well for the participants. However, searching for a specific product doesn’t work as well for out-of-stock products, especially without the knowledge of its verbatim product name. During the study sessions, we discovered that

the search keyword that makes sense to the user does not always return the right products, suggesting that the search engine needs optimization improvement, and
the mechanism of putting out-of-stock products at the bottom of the search result list becomes problematic for search keywords that are associated with many products by its search algorithm logic.

For new and/or impatient users, this often leads to early termination of the search process and concludes that the platform does not offer the product they want at all.

Promotion banner

One particularly interesting takeaway from the data is the ineffectiveness of the promotion banners on the homepage. None of the participants in the study attempted the task using those banners, and we found that they rarely look at those banners through follow-up questions. One participant thought the banners are paid advertisements by the product manufacturers or distributors while another participant found the number of banner images overwhelming (there are 22 images at the time of the study). During the follow-up debrief session, some participants also found the promotional content too lengthy and not engaging.

Restock reminder

Another feature that comes up during the follow-up discussion quite often is the restock reminder. Three out of four participants have tried to use this feature and concluded that this feature doesn’t work as expected. The participants agreed that they never received a reminder or a notification for back-in-stock items, and one participant reported that these restocked items would automatically be added to the shopping cart, which takes away the user's agency over the interface.

Design recommendations

The tasks in the usability study have made it clear what is working for the applications and what areas need improvement. I have some recommendations based on the key findings:

Identify and isolate the root cause of the problems caused by the search algorithm and the search result page sorting mechanism. Specifically, when users type in the search box, suggested search terms should include brand names and product names, and those could be highlighted. Instead of showing out-of-stock items at the bottom of search results, sold-out items with matching keywords should still display at the top of the page, and similar items (i.e. products relating to this search) could be grouped together with an appropriate section heading under those items;
Redesign the promotion banner section on the homepage and reorganize its content, including each promotional article. This might involve redesigning part of the homepage and experimenting with different presentations of content. The results of these improvements could be measured with A/B testing using analytics like click rate and conversion rate;
Implement the restock reminder functionality so the users would receive a notification for restocked items, and the application should ask for the users’ permission before adding those items to the shopping cart (Schneiderman's 8 golden rules: support internal locus of control);
Replicate this research study with participants who has less technical abilities, because user 4 did not perform as well as other participants. This issue is worth investigating if this group of users is part of the target users of this application.