Project: Competitive Analysis of Virtual Reality Headsets
Background
Virtual reality (VR) headsets are novel and convoluted to the general public. The main goals of this study was to examine what features of VR headsets are essential for a great user experience. A competitive analysis was conducted to compare the pros and cons of the VR headsets Oculus Go and Lenovo Mirage Solo with Daydream.
Recruitment
A between-subjects design was conducted using a convenience sample of university students. Half was assigned the Oculus Go and the other half was assigned the Mirage Solo with Daydream.
Demographics and background information were collected using the online survey Qualtrics. The survey included display and skip logic depending on participants' responses. The goal of the survey was to identify the users' current experiences with virtual reality and to screen participants with prior VR headset knowledge.
Standalone headsets like the Oculus Go and Mirage Solo are accessibility to the general public (i.e., you don't need fancy computers or phones to run them). People with little or no experience should easily set up and use these headsets. I conducted an out-of-box experience (OOBE) combined with an usability test to evaluate the user experience of the set up and initial use. OOBE is especially important for first impressions when considering a new product.
Testing
Out of Box Experience includes:
1. Unboxing
2. Set up
3. Initial Use
The three main areas of usability (effectiveness, efficiency, and satisfaction) were measured through quantitative data using success rate, time on task, and difficulty ratings through the single ease questionnaire (SEQ). System Usability Scale (SUS) and the raw NASA Task Load Index (NASA-TLX) scores were also collected. NASA-TLX was implemented because virtual reality is a inherently complex task. This scale was used to understand the users better by creating a comprehensive picture of different subjective measures. Users were instructed to talk out loud as they conducted different tasks (think aloud protocol). Qualitative data was collected by user comments to understand specific pain points and what features they liked.
Unboxing Tasks
The shortened form of the Microsoft's product reaction cards was used in the beginning and in the end of the study. The shortened 64 word list was used instead of the 118 word list to make the study more manageable (condensed length of study). Participants generated adjectives describing their experiences which gave deeper insight in what they were thinking.
Product Reaction Cards
Analysis
Overall scores for the SUS and NASA-TLX were compared between the different headsets. Success completion rate, time on tasks, and difficulty ratings were analyzed with qualitative data to pinpoint which features were positive and which were negative. Qualitative data was analyzed using a thematic analysis.
Oculus Go NASA-TLX
Mirage Solo NASA-TLX
Product reaction cards were developed into word clouds to visualize the data. The bigger the word, the more often it was used.
Oculus Go Word Cloud
Mirage Solo Word Cloud
Results
Short summary table
Final Notes
In the future, a within-subject design can be used to understand how different headsets compare directly in comparison with one another. A between-subjects design was conducted because it was unfair to ask students to run a 3 hour + long study without compensation. Further research could compare more virtual reality headsets to create a comprehensive list of positive and negative features. For a more in-depth look in this study, view the slide deck below.
View Other Projects
