Last week we talked about how to measure ROI on a VR experience. This week, we’re going to look at a related topic: how do you measure what is a successful VR use by a person? That is the fundamental goal of why we build VR, so it’s important to quantify and track.

The criteria you use will likely be found in the Analytics section of InstaVR. Our platform captures virtually every interaction within an app, from scene loading to Hotspots opened to Navigation Links clicked. That plethora of data may seem overwhelming, but is actually quite easy to interpret if you properly name scenes/hotspots/links.

Once you have all the data in your Analytics section of InstaVR, you’ll want to export that report to CSV. This makes for easier organization and querying.

An initial feeling might be to look at total downloads or app starts or users. Those are easy numbers to grab which can be found in Analytics without a lot of effort. It’s probably the least reliable success indicator though. You will have people that will download an app but never open it, or that will start a VR experience and end it in very little time. Both of those scenarios are hard to call successes. So it’s best to consider users or app starts as more of a vanity metric.

Below, we posit three potential success criteria for you to consider. These are based on feedback from our clients, particularly those doing employee training. If there’s more you think should be included, let us know!

If you have created a linear VR experience, or a multi-threaded VR experience with only one ending, then VR experiences completed is a great measure. It’s also relatively easy to track. If a user gets to the final scene, that means they completed your VR experience.

Why would you use this success criteria?

It shows the user followed the entire path that you wanted them to. They didn’t quit or end early. They saw everything you intended them to see. Much of the learning may come from completing a number of steps, so it’s important to track users who reach this final scene.

How to Track It?: In Uploading your media, you’ll want to give the final scene a recognizable file name (ie “End of VR App”). Then, you can do a CTRL-S search of your Analytics fields in the CSV export file to find how many times that particular scene was accessed. Adding up the number of times it appears, which Excel should give you in the upper right hand corner, will tell you how many people completed your VR app.

To get specific names of users, add the User ID feature of InstaVR when building the app. That way you can correlate app finishes to names, with a complete Excel spreadsheet of users who completed your VR experience.

This metric is a bit trickier to track, but necessary if you have a non-linear VR app you’ve created. If you were to give a user say a menu of training videos to view, and they’re not expected to complete all of them, the threshold might be based on number of videos viewed or total time spent viewing.

Neither of these metrics are default analytics within InstaVR, but they can be pulled out from your CSV that shows all actions with a little work.

How to Track It?: First, you’ll have to add the aforementioned User ID screen. This allows you to differentiate users. A User ID doesn’t have to be someone’s name or email address — it can be any alphanumeric combination. So if you wanted to do anonymous testing, just give each test subject a number like “1”, “2”, etc to enter in the headset prior to starting the VR experience.

Once you have User IDs in Analytics, you can separate sessions in your CSV file. For instance, you can look at all of the engagement data for just “Person 1” — including all the Excel rows (ie each action taken in the VR app) that feature that User ID.

To look at total time spent on site, you would just subtract the timestamp of the last Excel engagement entry for the user from the starting one. You can then put together a list of users and time spent in VR experience. This should give you an idea of the threshold you’d like to set for what was a “completed” VR experience is.

If you want to do this by number of scenes or 360 videos watched per user, you would again organize the CSV file for just the one user. Then in the “Event” horizontal column, you can view all scenes that they watched. You’ll want to de-duplicate — remember, a user may return to a scene multiple times — and then add up the unique scenes viewed. Like with time on the site, based on aggregate data, you can determine a threshold for how many scenes or videos viewed comprise a “completed” user experience.

ASK THE USER QUESTIONS

Data has shown that user recall of information is substantially better in Virtual Reality than standard forms of learning. A University of Maryland study pegged this number at an 8.8% lift, but others have found even higher deltas between traditional and VR learning (ie orthopedics performing 38% more accurately after using VR). The generally accepted reason is because people learn better when immersed in a situation through VR. People learn better through experiencing.

Given that, it’s still good to add some basic questions at the end of your VR app experience — either in-app or after taking of the headset — to ensure users were focused and retained the information you were trying to convey.

How to Track It?: For in-app, you would use smartly named questions via Hotspots (aka Quizzes) — which can be text, image or audio-based. These could be multiple choice or Yes/No. Combined with the Analytics and User ID features, you can in your CSV file see which users answered the question(s) correctly, and assign them as having successfully finished your VR experience.

How you measure Offline questions + answers is obviously up to you. But we do suggest you cross-correlate with User ID session information, to notice any patterns. For instance, did users who spent longer in VR do better at answering your questions?

CONCLUSION

Virtual Reality is achieving mainstream success within businesses. Basic metrics such as Downloads or Starts only scratch the surface for those looking at successful VR uses. You’ll want to dig deeper into your Analytics to figure out how many users are effectively completing your VR experience and achieving a desired outcome.

What that metric is depends in part on what type of VR experience you’ve created. A quick overview is:

1. If you have a single linear storyline or multi-threaded VR that all ends at the same scene, quantify which users reached the last scene.
2. If you have multi-threaded VR storylines or options on a menu — with no defined endpoint — use time spent in VR or scenes viewed to quantify success.
3. If you’re looking to teach a specific lesson (or lessons) through your VR, utilize Hotspots to ask questions. Or you can do so Offline, after the VR experience, and correlate it back to User ID.

Let us know if you have any questions!