Investigating usability for 3D art creation in TiltBrush VR.
Overview
As part of a 5-member team, we evaluated the usability of Google’s Tilt Brush VR app by combining usability testing, user feedback, and observational analysis.
3D digital design varies across diverse platforms. The objective of this usability testing was to investigate the different aspects of 3D art in a Virtual Reality medium
Tasks I did
Participant recruitment, Leading Usability Testing & Data Analysis
Team
5 product designers
tools used
Pen & Paper, MS Excel, Google Forms
Project Timeline
4 months
process timeline
Understanding the Users
Discovering Our Audience
Before we directly putting on the VR headset. We started with a pre-task questionnaire to understand our participants’ backgrounds and VR experience
01
Some had prior VR exposure, adapting quickly.
02
All were first-time Tilt Brush users and required tutorials for a smooth start
03
Participants were artists familiar with 2D concepts but unfamiliar with VR art tools
Diving Into the Experience
Usability Testing: Using the think-aloud method, I observed users as they completed key tasks
As the users completed the tasks mentioned below, I screen mirrored their headset to my laptop and noted observations. My observations consisted of both the positives and challenges they faced while being immersed in their creation.
The entire process is detailed in the study protocol document: Link to the Study Protocol.
01
Creating basic shapes in 3D space.
02
Experimenting with the selection tool to move art
03
Using the erase tool to refine their creations
Key observations
Users found the app engaging and immersive but they faced challenges as first time users of the app
01
Dark objects were hard to see against the dark background.
02
The selection tool didn’t allow users to move or rotate their artwork
03
The erase tool was unintuitive, often deleting more than intended.
Listening to the Users
Unstructured Interviews: To dive deeper into pain points, I conducted casual, open-ended interviews. Here’s what stood out
The users were frustrated while creating 3D work and fussed about specific details. So, I did an unstructured interview targeting these difficulties to gauge their feelings about it.
Manipulation Frustrations
Users struggled to move or adjust their art once created.
Erase Tool Limitations
Experimenting with the selection tool to move art
Comfort Concerns
Drawing for extended periods caused arm fatigue and discomfort.
gathering feedback
Usability Questionnaire
The users rated a few statements from 1 (strongly disagree) to 5 (strongly agree). These statements contributed to measuring the Usability, Immersion, Feedback, and other aspects of the application.
Sickness Questionnaire
A Sickness Questionnaire measures the extent to which users experience sickness after being in a Virtual Environment and using the Headsets for an extended time. Users rated the level of sickness they experienced using a scale from 0 (None) to 3 (Severe).
Eyestrain
Users complained that the dark environment was disturbing and caused eyestrain.
Headache
Users experienced headaches because of the dark environment and the closeness of the drawings to their eyes.
General Discomfort
Users complained about exhaustion in their arms by having to extend their arms to draw for extended periods.
Feature recommendations for the future improvements
Participants' most common problems were the dark environment, the selection & movement of artwork, and the confusing eraser tool.
Users loved the Tilt Brush VR application and rated it a 4 out of 5 for its ease of use. However, They expressed reduced interest when asked if they see themselves using this in the future.
Based on these, the following feature enhancements will improve the Tilt Brush application's usability by allowing the users better control over their 3D art and reducing discomfort.
Artwork Manipulation
Drawing 3D art at a comfortable distance, selecting it all together, and moving it will allow more control and reduce arm exhaustion.
Providing Adjustable Background
Offering an adjustable environment - light or dark backgrounds - could reduce eyestrain and fatigue, potentially lengthening user sessions.
Erase Tool Precision
Implementing a more precise erase tool and confirm-delete option reduces errors due to accidental deletions.
testing gallery
Learnings from the project
Visualizing the Path to Improvement
Studying results revealed valuable insights about the app's strengths and weaknesses. Though time-consuming, extracting qualitative data from observations, think-aloud sessions, and interviews proved enriching, helping me enhance and prioritize features.
Unlocking Natural User Insights
Ditching the notepad during interviews fostered a more natural conversation, easing user self-awareness and enhancing information flow. This approach elicited subconscious feedback and deeper insights, uncovering pain points not explicitly expressed by users.
Teaching myself new techniques
This project challenged me to explore and adopt new methodologies independently. Outside the project scope, I proactively acquired knowledge of feature prioritization methods, enabling me to offer valuable recommendations for future iterations.