Common User Interaction Systems for Virtual Simulations

Virtual Simulations represent a specific category of simulation that utilizes simulation equipment to create a simulated world for the user. Virtual Simulations allow users to interact with a virtual world. Virtual worlds operate on platforms of integrated software and hardware components. In this manner, the system can accept input from the user (e.g., body tracking, voice/sound recognition, physical controllers) and produce output to the user (e.g., visual display, aural display, haptic display) .^[12] Virtual Simulations use the aforementioned modes of interaction to produce a sense of immersion for the user.

[edit]Virtual Simulation Input Hardware

There is a wide variety of input hardware available to accept user input for virtual simulations. The following list briefly describes several of them:

Body Tracking The motion capture method is often used to record the user’s movements and translate the captured data into inputs for the virtual simulation. For example, if a user physically turns their head, the motion would be captured by the simulation hardware in some way and translated to a corresponding shift in view within the simulation.

• Capture Suits and/or gloves may be used to capture movements of users body parts. The systems may have sensors incorporated inside them to sense movements of different body parts (e.g., fingers). Alternatively, these systems may have exterior tracking devices or marks that can be detected by external ultrasound, optical receivers or electromagnetic sensors. Internal inertial sensors are also available on some systems. The units may transmit data either wirelessly or through cables.
• Eye trackers can also be used to detect eye movements so that the system can determine precisely where a user is looking at any given instant.

Physical Controllers Physical controllers provide input to the simulation only through direct manipulation by the user. In virtual simulations, tactile feedback from physical controllers is highly desirable in a number of simulation environments.

• Omni directional treadmills can be used to capture the users locomotion as they walk or run.
• High fidelity instrumentation such as instrument panels in virtual aircraft cockpits provides users with actual controls to raise the level of immersion. For example, pilots can use the actual global positioning system controls from the real device in a simulated cockpit to help them practice procedures with the actual device in the context of the integrated cockpit system.

Voice/Sound Recognition This form of interaction may be used either to interact with agents within the simulation (e.g., virtual people) or to manipulate objects in the simulation (e.g., information). Voice interaction presumably increases the level of immersion for the user.

• Users may use headsets with boom microphones, lapel microphones or the room may be equipped with strategically located microphones.

Current Research into User Input Systems Research in future input systems hold a great deal of promise for virtual simulations. Systems such as brain-computer interfaces (BCIs)Brain-computer interface offer the ability to further increase the level of immersion for virtual simulation users. Lee, Keinrath, Scherer, Bischof, Pfurtscheller ^[13] proved that naïve subjects could be trained to use a BCI to navigate a virtual apartment with relative ease. Using the BCI, the authors found that subjects were able to freely navigate the virtual environment with relatively minimal effort. It is possible that these types of systems will become standard input modalities in future virtual simulation systems.

[edit]Virtual Simulation Output Hardware

There is a wide variety of output hardware available to deliver stimulus to users in virtual simulations. The following list briefly describes several of them:

Visual Display Visual displays provide the visual stimulus to the user.

• Stationary displays can vary from a conventional desktop display to 360-degree wrap around screens to stereo three-dimensional screens. Conventional desktop displays can vary in size from 15 to 60+ inches. Wrap around screens are typically utilized in what is known as a Cave Automatic Virtual Environment (CAVE) Cave Automatic Virtual Environment. Stereo three-dimensional screens produce three-dimensional images either with or without special glasses—depending on the design.
• Head mounted displays (HMDs) have small displays that are mounted on headgear worn by the user. These systems are connected directly into the virtual simulation to provide the user with a more immersive experience. Weight, update rates and field of view are some of the key variables that differentiate HMDs. Naturally, heavier HMDs are undesirable as they cause fatigue over time. If the update rate is too slow, the system is unable to update the displays fast enough to correspond with a quick head turn by the user. Slower update rates tend to cause simulation sickness and disrupt the sense of immersion. Field of view or the angular extent of the world that is seen at a given moment Field of view can vary from system to system and has been found to affect the users sense of immersion.

Aural Display Several different types of audio systems exist to help the user hear and localize sounds spatially. Special software can be used to produce 3D audio effects 3D audio to create the illusion that sound sources are placed within a defined three-dimensional space around the user.

• Stationary conventional speaker systems may be used provide dual or multi-channel surround sound. However, external speakers are not as effective as headphones in producing 3D audio effects.^[12]
• Conventional headphones offer a portable alternative to stationary speakers. They also have the added advantages of masking real world noise and facilitate more effective 3D audio sound effects.^[12]

Haptic Display These displays provide sense of touch to the user Haptic technology. This type of output is sometimes referred to as force feedback.

• Tactile Tile Displays use different types of actuators such as inflatable bladders, vibrators, low frequency sub-woofers, pin actuators and/or thermo-actuators to produce sensations for the user.
• End Effector Displays can respond to users inputs with resistance and force.^[12] These systems are often used in medical applications for remote surgeries that employ robotic instruments.^[14]

Vestibular Display These displays provide a sense of motion to the user Motion simulator. They often manifest as motion bases for virtual vehicle simulation such as driving simulators or flight simulators. Motion bases are fixed in place but use actuators to move the simulator in ways that can produce the sensations pitching, yawing or rolling. The simulators can also move in such a way as to produce a sense of acceleration on all axes (e.g., the motion base can produce the sensation of falling).