The Gaming Experience Using 3D Spatial UI Technologies
Three-dimensional (3D) spatial user interface technologies have the potential to make games more immersive and engaging and thus provide a better user experience. Although 3D user interface (3DUI) technologies such as stereoscopic 3D display, head tracking, and gesture-based control are available for games, it is still unclear how their use affects gameplay and if there are any user performance benefits. A systematic study of these technologies in game environments is necessary to understand how they affect gameplay and how we can use them to optimize the gameplay experience.
User interface design in games differs from other UI design because it involves an additional element — fiction. The fiction involves an avatar of the actual user, or player. The player becomes an invisible, but key element to the story.
Stereoscopic 3D is not a new technology, but it has not been readily available to consumers until recently. Most recent video games are designed in 3D game engines, so the 3D data is already present in games. A stereoscopic driver (such as Nvidia 3D Vision or Tridef Ignition) uses this 3D data to create stereoscopic images that can be rendered on a stereoscopic display.
Diegetic user interface elements exist within the game world (fiction and geometry) so the player and avatar can interact with them through visual, audible or haptic means. Well executed diegetic UI elements enhance the narrative experience for the player, providing a more immersive and integrated experience.
A previous study of PC games showed that playing games with 3D stereoscopic display does not provide any significant performance benefits over using a 2D display.1 However, that study used a traditional game controller (the Xbox 360 controller) as the interaction device, and the games used were not designed with 3D stereoscopic display in mind.
Effects of Simultaneous Usage
So far, we have discussed experiments that focused on exploring 3DUI technologies in isolation, but it is still unclear how the gaming experience might be affected if several 3DUI technologies are used simultaneously.
To explore this area, we custom designed an air-combat game that integrates several 3DUI technologies (stereoscopic 3D, head tracking, and finger-count gestures) to understand the interplay among them and study their combined effect on the gaming experience. Our game design was based on principles for optimizing the use of these technologies in isolation.
We conducted a within-subject experiment with our air-combat game to evaluate the combined effect of stereoscopic 3D, head-tracking, and finger-count shortcuts on the gaming experience.