top of page
Wegmans : Top

USABILITY EVALUATION

​

WEGMANS CURBSIDE PICKUP APPLICATION

Wegmans Project Home.png

This project was completed by me and my six teammates as part of Usability Testing coursework between  February 2020 and April 2020. 

wegamns background.png
C.png

PROJECT PURPOSE

In January 2020, Wegmans launched a service called curbside pickup and delivery within its mobile application such that their users can add products in a list, order them through this application and pickup their order from the nearest Wegmans store as per their convenience.

​

We were assigned to conduct Heuristic Evaluation and analysis on their curbside module to gain a deeper insight on how to improve overall user experience.

timeline background.png
Wegmans timeline.png
Cart.png

KICK-OFF MEETING

undraw_Meeting_h2df.png

The project commenced with a kick-off meeting at the client's office.

​

Purpose of this meeting  -

  • Discuss the Wegmans mobile application and user feedback.

  • Identify user groups.

  • Gather project requirements.

  • Determine different stages of the project and their outcomes.

  • Setting deadlines for each stage.

HEURISTIC EVALUATION

undraw_collaborators_prrw.png

Each team member conducted individual evaluation of the application on their respective smartphones and then discussed the observations as a group. The below report is a cohesive document created as a result of this collaboration. 

Heuristics

15

9

High priority issues

7

Medium priority issues

13

Low priority

issues

TEST PLAN

undraw_contract_uy56.png

A Test plan defines the objective of the project, methodology, participant and evaluator characteristics, test environment and structure of deliverables. It serves as a means of communication between the evaluators and the stake holders.

  • Why is the test conducted?

  • What are we going to test?

  • Who are we going to test those with?

  • How are we going to conduct the test?

  • Where are we going to conduct the test?

  • What data are we going to collect?

icons8-important-note-96.png

Due to COVID-19 situation, we conducted remote usability tests, where participants were asked to connect with evaluators via ZOOM, as they interact with the application on their smartphones to perform few assigned tasks. 

​

It was both a challenge and an opportunity for each one of us on the usability evaluation team as we diverted from the conventional method of usability testing. This experience gave us some useful insights and a whole different perspective towards this process. 

TEST MATERIALS

Following are the documents used before, during and after the test. We ensured that all the materials are checked and ready to be sent to participants before the start of every test session. 

Screener.png

Screener

A questionnaire forwarded to users to screen and select participants based on set criteria

Orientation Script.png

Orientation Script 

Brief introduction of the process that will be followed during the study

Inform Consent Icon.png

Informed Consent

Describes the study, risks associated with the study and also states moderator and user rights.

undraw_fill_forms_yltj.png

Background Questionnaire

A Questionnaire for participants before the test, to understand user practices related to the system being tested.

Think Aloud.png

Think Aloud Practice

Demonstration for participants on how to verbalize thoughts

while they perform tasks.

Post-Test Questionnaire.png

Post-task Questionnaire

A survey for participants after they perform the tasks to understand their experience with the system and the process. 

scenario description.png

Scenario Introduction

A set of tasks for participants to complete as a part of a given scenario

undraw_printing_invoices_5r4r.png

Compensation Receipt

A $25 Wegmans gift card was provided to the participants for their contribution.

PILOT TEST

undraw_detailed_analysis_xn7y.png

Pilot tests were conducted with 2 participants, one belonging to group A (used Wegmans application at-least once) and the other belonging to group B (not used Wegmans application at all).

Performing these tests helped us identify issues related to our test plan and materials which were then modified for the final usability tests. 

Key Takeaways:

​

  • Inform the participants about the major details of consent form at the beginning of the session.

  • Transfer of test materials to the participants via email before the test commences was preferred.

  • Two devices on hand during the test, one to access test materials and  the other to conduct the task was considered important after pilot test as it was inefficient for the users to alternate windows of their phones between the test materials and the Wegmans application. 

 USABILITY TEST

Each usability test session had one participant and four members from the evaluating team.Evaluation team members took turns playing the roles of various personnel for the sessions as described below -

EVALUATING TEAM

Moderator

Primary point of contact as well as a guide for the participant before and during the whole session. Was visible to participant at all times. 

undraw_mobile_testing_reah.png

Technical Supervisor

In-charge of setup, recording and any other Zoom related technical support. Was responsible for sending all test materials to the participant prior to the start of the study. Not visible to the participant and was in contact with moderator via direct messaging. 

Observer 1

Responsible for noting time stamps for every Time-on-task activities during the study. Also took a note of other important observations. Not visible to the participant.

Observer 2

Responsible for observing participant expressions, their minute interactions with the application and noting them. 

PARTICIPANT PROFILES

Group A - Expert Users

 

icons8-long-brunette-hair-lady-100.png

Age : 50+ Years

icons8-brown-hair-business-lady-in-red-s

Age : 50+ Years

Group B - Novice Users

 

icons8-brown-long-hair-lady-with-red-gla

Age : 24-26 Years

icons8-man-in-blue-jacket-100.png

Age : 28-30 Years

TEST SETUP

Participants logged in to Zoom on two devices for the study - their laptop and smartphone. Only the moderator and the participant turned on their laptop's camera for the entirety of the evaluation, which was used to observe participant reactions. The participant presented their phone screen which enabled us to observe their interactions as they navigated the application. The session was recorded by the Technical supervisor. 

​

For more details about the setup and the procedure check the Test Process.

Test Setup.png

TASK DESCRIPTION

Task Intro.png

DATA COLLECTION AND RESULTS

undraw_all_the_data_h4ki.png

COLLECTED DATA 

​

Quantitative Data(QT): 

  • Number of clicks taken to achieve a task. 

  • Likert chart ratings on post-test debrief questionnaires.

  • Time taken to complete tasks.

​

Qualitative Data(QL): 

  • Participant comments and expressions while thinking aloud and conducting the tasks. 

  • Participant questions during test sessions.

  • Participant responses to debrief questions after tasks.

undraw_business_plan_5i9d.png

RESULTS 

​

Consistent with our heuristic evaluation, usability issues were primarily found in the customization and checkout process, particularly regarding the following heuristics:

  • Perceptibility of system state and feedback.

  • Error prevention and recovery.

  • Internal and external consistency.

  • User Control and Freedom.

​

See the Final Presentation presented by us to the client to know more about the research questions, results and design recommendations.

​

Video clippings of the usability test sessions with participants have been removed from the presentation slides under the conditions of non-disclosure agreement with Wegmans and also to protect participant rights. 

​

​

©2019 crafted by Madhura Valvaikar.

bottom of page