User Testing and Iteration
Breakdown of Final Product
Project Puzzler is both a game and an educational opportunity. Built as a class project in the Udacity VR Nanodegree Program, Puzzler’s creation was a framework towards learning the fundamentals of VR design. It is also, as the name indicates, a VR puzzle game that introduces people to the new experience that is VR. Using feedback, it guides users in understanding how VR interfaces work, and what to do when you don’t have a controller in front of you.
(Audio may be out of sync in the video.)
Puzzler is built for the Android OS and Google VR experience. You can download and try the .apk with this link. The experience is deliberately not too complex, consisting of a room with the puzzle, an entry corridor, an exit corridor, some light ambient sound, and a few scenery pieces. Too much would overload the user, but having too little would create a bland, uninteresting experience that would turn people off of VR in general.
The puzzle itself is a “Simon Says” type, where the player must follow the pattern of the flashing orbs in order to proceed. The orbs are arranged in a five point array that takes advantage of the 3D space, spreading out over the X, Y, and Z axes. Simple tones and textures provide feedback to the user, making it easy to follow the pattern without using too many complicated assets that could be distracting.
III.) Process history
The first step in creating Puzzler was writing a mission statement, to lay out a goal and an early stab at the scope of this project. The mission statement is:
Puzzler is a mobile VR application for new VR users which challenges them to solve a familiar type of puzzle in a new way.
Using that as my basis for the project, I then proceeded to write up a persona of a possible user. Who would want to play this game? It would likely not be someone familiar with VR, as they would want something far more in-depth, so this would be oriented towards someone who lacked VR experience. I figured that the most likely user is someone who wants an easy introduction to VR, but it fascinated by its possibilities and wants to dip his or her toe in it. As such, I created the following persona:
Online Marketing Manager
“There’s always a way to reach someone, sometimes it just takes awhile to find it.”
Veronica specializes in online marketing, a field that is constantly changing and adapting. Many days are exhausting with seemingly no progress made, but Veronica keeps at it. She needs something to take her mind off of work, but at the same time she’s always looking for inspiration in new ways of outreach. VR could kill two birds with one stone – exactly what she needs.
VR Experience: None
Then, I moved forward with sketching out a basic picture of what I wanted the environment to look like. Although the class used a fantasy environment, I wanted to develop a sci-fi look instead, both to challenge myself as a designer and to explore other aesthetic possibilities.
After drawing the sketch, however, I realized that this posed a problem. Originally, I wanted it to be a station in space, with the idea being that interconnected modules would each hold an individual puzzle (in some far future iteration of this project.) I thought that space would be an ideal environment for a VR experience, as very few have been in space. However, examining what resources I had on me, I realized that creating such an environment that would meet my own standards and the user’s would be infeasible. It would take too much time and effort to develop my own models for the station, as I’m not a modeler myself. At this point, I decided to switch gears and rely on a free “starter kit” from the Unity Asset Store to build an indoor scifi environment.
The next step was building the environment itself. I started by focusing on the room, getting everything level and making sure all the polygons lined up. I created prefabs for the corner pieces and the ends of the hallways, as they were the most problematic. I also turned off the default directional light, being unsure if that would work for an indoor scene. I added in some ceiling lights to compensate. Then I put in the data orbs, that were the core of the puzzle experience. Initially I went with nine, as I liked the diamond aesthetics. I also wrote a small script to make them rotate, instead of just being still.
With the environment created, I made a build of just the scene, with the camera in the middle of the room, to use as a testing tool. Due to time constraints and miscommunication, I was unable to get user testing initially, so I kept the build and brought into work later.
I moved on to implementing the mechanics. First I sketched out some designs for the UI, which I decided would be very simple.
I then created the UI in the environment itself. Placing the UI panels side by side, I created a build of this for further testing. I initially had some trouble as I had misplaced my Udacity Cardboard viewer, and was using another viewer I had purchased previously, which lacked a button. Since I (based on my own experiences) expected the button trigger to be a time based gaze, I was puzzled, until I realized it required an actual click and found the original Cardboard viewer.
Moving on from that, I imported the movement code and tested it. I had already chosen a rails-type system for my design, as there were only three points to move to. The default speed seemed fine, though I was a bit worried that it might be because I already had VR experience. Fortunately, user testing (in the next section) dispelled my doubts.
With the movement in place, I then imported audio assets. I purchased a sound effects library from ASoundEffect.com that had great scifi tones, and put them into the environment. To make the room less bland, I also put in a holographic monitor model, and attached the ambient noise to that to give the user a sense of place. Now that the audio assets were there, I put in the puzzle mechanics, and wired in feedback. At this point, I also realized that nine orbs was probably too much. So I dialed it back to five. I had also seen, while perusing the forums for the answer to a bug, that another student had arranged the orbs so as to approach the player. I thought that was genius, as it fully utilized the 3D space that VR offers. I didn’t go as far as the other student did, and I also took greater advantage of the Y-axis.
With the audio finally in place, the project was complete. The only thing left were to add some more set pieces, namely adding crates and another console to the corners. I really enjoyed making the crates; they reminded me of playing sci-fi adventure games as a child, and where I always wondered: just what the hell are in those crates?
iv.) user testing and iteration
User testing did not proceed as ideally as I wanted it to, due to some miscommunication between my testers and I. However, I was able to do user testing at my office. I used the separate builds to focus my testers on each of the elements of the app: environment, UI, motion, and finally the audio and completed game.
Having my coworker Jason use the app on my phone, I asked him questions about the various elements of the app. He noted that the height felt exactly as he was in real life (then, perhaps, maybe just a tad shorter). He described the environment as clean, clinical, even industrial. He also said it was like “on Star Trek,” which matched up with other testers who said it was “like a room on a Mass Effect ship.” Since that is pure scifi, it was the right atmosphere I wanted to engender.
The movement test was the best. Asking my tester how he felt with the motion, he said it was perfectly fine, and then added:
I could be drunk and this would be perfectly okay.
One of the problems that appeared in user testing was that the orbs were difficult to place in the scene, depth-wise. They also were quite close to the camera, so I moved the point for the camera to play the puzzle back a bit, and also went from the 9 orb diamond to the 5 orb arc.
Interestingly, height was never a real problem. Although Jason and I are both over 6′, I also gave the game to a coworker who is about 5’0″, and she thought it was fine as well (except for the UI, which had to be tweaked just a little.)
Finally, testers thought it was just a smidgen too dark, so in the final version, I reactivated the directional light to make it just a tad brighter.
Further testing also indicated that I had to be much more explicit in my instructions. I had said “follow the orbs with your eyes,” naively assuming people would understand to click. I rewrote those instructions to be more clear.
v.) breakdown of final product
Puzzler is simple. The player begins in a hallway, with a panel in front of them introducing them to the project and warning them that motion is involved. The player clicks on the Start button, and they are moved forward into the room. The orbs begin to light up in sequence, emitting a small tone as they do so. The same tone plays as the player hovers over and clicks on the orbs. If the player fails, a warning sound emits and the sequence begins again. If the player succeeds, they are sent forward into the next corridor, where they are greeted by another panel that asks if they want to play it again.
In developing Puzzler, I learned a great deal about VR design and programming. Looking at both the Udacity and the Google VR code has shown me how to think about solving problems in VR space. Iterating through user testing highlighted blind spots in my thinking and made my work better as a result. As a bonus, my friends and coworkers who used the Puzzler app were impressed and amazed at what I had created.
If I were to continue developing Puzzler, it would grow to more rooms with more puzzles, each one highlighting a new way of thinking in VR. I would likely switch to a raycast form of travel, giving the user more choice so they could utilize the space more effectively. It would also have a greater variety of environments to be more visually appealing, which would necessitate either purchasing more model sets on the Unity Asset Store, partnering (or outright hiring) someone to develop them for me, or trying to do so myself.