Augmented reality is exactly what the name implies — a medium through which the known world fuses with current technology to create a uniquely blended interactive experience. While still more or less a nascent entity in the frequently Luddite education industry, more and more teachers, researchers, and developers contribute their ideas and inventions towards the cause of more interactive learning environments. Many of these result in some of the most creative, engaging experiences imaginable, and as adherence grows, so too will students of all ages.
[The following is the second of two articles by college professor and researcher Ben Lewis-Evans on games user research methodology (see Part 1, which covered focus groups, heuristics, and questionnaires, as well as giving a grounding in the topic of user research in general. In this article, Lewis-Evans covers interviews, observational methods (including think out loud and contextual inquiry), game metrics, and biometrics.]
Much like a questionnaire — a topic covered in the last installment — an interview is for collecting subjective data. However, the face-to-face nature of an interview means that you can be more interactive in your data collection, which if done correctly, can lead to very rich data. However, it is also obviously quite time-consuming, and it is harder to analyze and quantify the data you get at the end.
The quality of what you get out of an interview will also depend greatly on your own skill as an interviewer, so here are some tips.
[In this first article in a new series, college professor and user research Ben Lewis-Evans takes a look at different methods of game user research, offering up a handy guide to different ways you can collect useful information about your game.]
This article, and its forthcoming followup, is intended to give a rough idea to developers of several different methods that can be used in games user research.
However, many, many books have been written on research methodology and I cannot cover everything. Therefore these two articles cannot be taken as completely comprehensive.
In the first of the articles I will be covering a few general points about Games User Research and then discussing three methods, focus groups, heuristic evaluation and questionnaires in some detail.
Summary: Better to accept a wider margin of error in usability metrics than to spend the entire budget learning too few things with extreme precision.
Last week, I made a slide for the new User Experience (UX) Basic Training course with the recommended number of test users for different types of studies. I like teaching foundational courses because they afford me just this kind of opportunity — to distill 25 years of usability process research into a single table. Patterns crystallize when complex topics are condensed to the essence.
Some people think that usability is very costly and complex and that user tests should be reserved for the rare web design project with a huge budget and a lavish time schedule. Not true. Elaborate usability tests are a waste of resources. The best results come from testing no more than 5 users and running as many small tests as you can afford.
In earlier research, Tom Landauer and I showed that the number of usability problems found in a usability test with n users is:
where N is the total number of usability problems in the design and L is the proportion of usability problems discovered while testing a single user. The typical value of L is 31%, averaged across a large number of projects we studied. Plotting the curve for L=31% gives the following result:
The most striking truth of the curve is that zero users give zero insights.
When Glenn Harrington donned a motion capture suit complete with more than 40 reflective spheres he wasn’t being turned into the latest video game character, but helping to design car manufacturing jobs that are less physically stressful on workers.
A critique of a common method used in video game usability research
Many video game usability practitioners employ a method to test usability within video games, called the ‘RITE’ method, short for Rapid Iterative Testing and Evaluation (RITE). Pioneered at Microsoft Games Studios and Microsoft Research, the RITE method has been adopted by many usability research organizations besides the teams at Microsoft.
While the RITE method has some advantages, such as the ‘rapid iterative’ ability to suggest changes to designers and test them in successive passes, it may fall short when looking for usability issues that lie beneath the surface.
From: Score One for Quality! Using Games to Improve Product Quality
by Joshua Williams and Ross Smith
In this paper we describe how using a game can improve both the quality of a product, but the quality of life of the employees as well. We call this kind of game a “Productivity Game.”
Small companies should consider outsourcing the facilitation of their usability testing projects. On the surface, it makes sense to have a designer/developer who has a deep understanding of the project be in charge of usability testing, but in fact this can cause serious problems.