Virtual Reality Technology
3. Edition August 2024
736 Pages, Hardcover
Textbook
Short Description
Since the second edition of this widely acclaimed book was published, virtual reality technology continued to evolve, with new applications emerging in medicine, oil exploration, defense, manufacturing, and video games. The Third Edition brings the field completely up to date, giving scientists and engineers a thorough understanding of virtual reality and its current state of the art. Focusing equally on hardware, software, and applications, this work also discusses key design issues involving human factors. Many supplementary materials, including a Laboratory Manual and companion website are also included.
Thorough overview of virtual reality technology fundamentals and latest advances, with coverage of hardware, software, human factors and applications, plus companion Laboratory Manual in Unity 3D.
The Third Edition of the first comprehensive technical book on the subject of virtual reality, Virtual Reality Technology, provides updated and expanded coverage of VR technology, including where it originated, how it has evolved, and where it is going. Its primary objective is to be a complete, up-to-date textbook, as well as a source of information on a rapidly developing field of science and technology with broad societal impact.
The two highly qualified authors cover all of the latest innovations and applications that are making virtual reality more important than ever before. Unlike other books on the subject, the book also includes a chapter on Human Factors, which are very important in designing technology around the human user.
Virtual Reality Technology provides readers with a website-accessible Laboratory Manual using the Unity 3D game engine and programming language. Unity 3D is the preferred VR language these days and will prepare the student for the VR gaming and mobile applications industry. For universities Unity 3D is cost-effective as its student license is freely available.
With comprehensive coverage of the subject, Virtual Reality Technology discusses sample topics such as:
* Input and output interfaces, including holographic displays, foveated head-mounted displays, neural interfaces, haptic and olfactory feedback
* Computing architecture, with emphasis on the rendering pipeline, the graphics processing unit and distributed/edge rendering
* Object modeling, including physical and behavioral aspects, Artificial Intelligence controlled characters, and model management techniques
* Programming toolkits for virtual reality and the game production pipeline
* Human factors issues such as user performance and sensorial conflict, cybersickness and societal impact aspects of VR
* Application examples in medical education, virtual rehabilitation, virtual heritage, gaming, and military use of virtual reality.
Virtual Reality Technology provides thorough and complete coverage of an in-demand sector of technology, making it a highly valuable resource for undergraduate and graduate students in computer science, engineering, and science, along with a variety of professionals across many different industries, including but not limited to engineering, gaming, and defense.
Dedication
Foreword
Preface to the 3rd Edition
1. Introduction
1.1 THE THREE I'S OF VIRTUAL REALITY
1.2 EARLY VIRTUAL REALITY
1.3 FIRST COMMERCIAL VIRTUAL REALITY
1.4 VIRTUAL REALITY AT THE TURN OF THE MILENIUM
1.5 VIRTUAL REALITY IN THE 21ST CENTURY
1.5 COMPONENTS OF CLASSICAL AND MODERN VIRTUAL REALITY SYSTEMS
1.6 REVIEW QUESTIONS
REFERENCES
2. Input Devices: Trackers, Navigation, Gesture and Neural Interfaces
2.1 THREE-DIMENSIONAL POSITION TRACKERS
2.1.1 Tracker performance parameters
2.1.2 Electromagnetic trackers
Trackers using alternating current (AC) magnetic fields
Trackers using direct current (DC) magnetic fields
Comparison of AC and DC Magnetic Trackers Accuracy Degradation
2.1.3 Optical Trackers
Passive Camera-based Optical Trackers
Active Camera-based Optical Trackers
Time-of-Flight Optical Trackers
Consumer-Grade Optical Trackers
Inside-looking-out Optical Trackers
Eye Trackers for Head-Mounted Displays
2.1.4 Optical Tracker Accuracy Degradation
2.1.5 Hybrid Inertia Trackers
Hybrid inertial-optical trackers
2.1.6 Global Positioning System Trackers
2.2 NAVIGATION AND MANIPULATION INTERFACES
2.2.1 Desktop Navigation Interfaces
2.2.2 Hand-held Navigation and Manipulation Interfaces
The FlightStick3
Game controllers are navigation and manipulation interfaces
2.2.3 Locomotion interfaces
The Cybershoes Arcade
The Omni Treadmill
2.3 GESTURE INTERFACES
2.3.1 Sensing Gloves
The 5DT Ultra Glove Ultra 14
The CyberGlove II
The Cyberglove-HT
2.3.2 Natural hand tracking
Natural vs Sensing Glove-based Hand Tracking
2.4 NEURAL INTERFACES
2.4.1 Research-grade Brain-Computer Interfaces
2.4.2 Consumer-grade Brain-Computer Interfaces
2.5 CONCLUSIONS
2.6 REVIEW QUESTIONS
REFERENCES
3. Output Devices: Graphics Displays
3.1 THE HUMAN VISION SYSTEM
3.2 GRAPHICS DISPLAY CHARACTERISRICS
3.3 DISPLAY TECHNOLOGIES
3.3.1 Displays using LCD technology
3.3.2 Displays using OLED technology
3.4 PERSONAL GRAPHICS DISPLAYS
3.4.1 Smart phones as VR displays
Smart phones as input devices
Smart phone use in HMDs
3.4.2 Head-Mounted Displays
Fixed-resolution HMD optics
Foveated HMD optics
Foveated HMDs which reduce native resolution
Foveated HMDs which increase native resolution
Head-Mounted Displays weight and weight distribution
The FOVE 0 HMD
The Oculus Quest 2 HMD
The HTC VIVE Focus 3
The Pimax Vision 8K X HMD
3.4.3 Desk Supported Personal Displays
Autostereoscopic monitors
3D Monitors
3.5 LARGE VOLUME DISPLAYS
3.5.1 Liquid Crystal Tiled Large-Volume Displays
Tiled wall displays
Tiled CAVE displays
3.5.2 Projector-based Large-Volume Displays
Dome-type large volume displays
Tiled Display Scene Continuity
Geometrical continuity
Visual continuity
Temporal continuity
3.6 MICRO-LED WALLS AND HOLOGRAPHIC DISPLAYS
3.6.1 Micro-LED Walls
3.6.2 Holographic displays
3.7 CONCLUSIONS
3.8 REVIEW QUESTIONS
REFERENCES
4. Output Devices: Three-Dimensional Sound,
Haptic Displays, Olfactory Displays
4.1 THREE-DIMENSIONAL SOUND DISPLAYS
4.1.1 Localized vs. Non-Localized sound
4.1.2 The Human Auditory System
Azimuth Cues
Elevation Cues
Range cues
4.1.3 Head-Related Transfer Function
4.1.4 Generic Convolvotron Architecture
Spatialized sound for all-in-one HMDs
4.1.5 Speaker-based 3-D Sound
4.1.6 Wearable Sound Interfaces
4.2 HAPTIC DISPLAYS
4.2.1 The Human Haptic System
Human Haptic Sensing System
Human Sensory-Motor Control
4.2.2 Tactile Feedback Interfaces
The CyberTouch II glove
The HaptX DK2 touch feedback glove
The Tactsuit X whole body tactile feedback
Temperature Feedback Interfaces
4.2.3 Force Feedback Interfaces
The Touch X Arm
The Delta.3 Force Feedback Arm
The Armeo Power
SenseGlove Nova Force Feedback Glove
4.3 OLFACTORY DISPLAYS
4.3.1 The Human Olfactory System
4.3.2 Olfactory Systems for VR
Ambient Olfactory Displays
Air Cannon-based Olfactory Display
Wearable Olfactory Displays
4.4 CONCLUSIONS
4.5 REVIEW QUESTIONS
REFERENCES
5. Computing Architectures for VR
5.1 THE RENDERING PIPELINE
5.1.1 The Graphics Rendering Pipeline
Traditional Graphics Rendering Pipeline
Modern Graphics Rendering Pipeline
A graphics pipeline example
Anti-aliasing
Graphics Pipeline Bottlenecks
Graphics Pipeline Optimization
5.1.2 The Haptic Rendering Pipeline
5.2 GAMING DESKTOP ARCHITECTURES
5.2.1 The 12th Generation Core CPU and Chip Set
5.2.2 Cooling of Gaming PCs
5.2.3 Graphics Accelerator Cards for PCs
The NVIDIA GTX 1080 Founder's Edition graphics card
NVIDIA GeForce RTX 4090 graphics card
The NVIDIA GeForce RTX 4090 graphics card
5.3 GRAPHICS BENCHMARKS
5.3.2 The SPECviewperf 2020 benchmarks
5.3.3 A Frame Capture Analysis Tool for Virtual Reality
5.4 DISTRIBUTED VR ARCHITECTURES
5.4.1 Split Rendering
5.4.2 Co-located Rendering Pipelines
5.4.3 Multi-pipeline synchronization
Synchronization of haptic and graphics pipelines
Synchronization of co-located graphics pipelines
Pipeline synchronization using the NVIDIA Quadro Sync II card
5.4.4 Cloud Rendering
A Cloud Rendering Architecture Example
Cloud Rendering Latency
5.5 CONCLUSIONS
5.6 REVIEW QUESTIONS
REFERENCES
6. Modeling Virtual Environments
6.1 GEOMETRIC MODELING
6.1.2 Virtual Object Shape
Using a 3D Authoring Software
Importing CAD files
Creating surfaces with 3D scanners
Using online 3D object databases
6.1.3 Object Visual Appearance
Scene Illumination
Texture Mapping
6.2 KINEMATICS MODELING
6.2.1 Homogeneous transformation matrices
6.2.2 Object Position
6.2.3 Transformation Invariants
6.2.4 Object hierarchies
6.2.5 Viewing the 3-D World
6.3 PHYSICAL MODELING
6.3.1 Collision Detection
Collision detection for large virtual environments
6.3.2 Collision Response Involving Object Surfaces
Topology-preserving collision response
Topology-altering collision response
Surface cutting and stitching
Object morphing
6.3.3 Contact Force Modeling
Contact Forces when Deforming Elastic Objects
Contact Forces when Deforming Plastic Objects
Contact Forces when Interacting with Virtual Walls
6.3.4 Force Smoothing and Mapping
6.3.5 Haptic Texturing
Haptic textures produced by non-wearable interfaces
Haptic textures produced by wearable interfaces
6.4 BEHAVIOR MODELING
6.4.1 Simple behavior models
6.4.2 Enhanced behavior models
6.4.4 Crowd behavior models
6.5 MODEL MANAGEMENT
6.5.1 Level-of-Detail Management
Discrete Level-of-Detail Management
Continuous Level-of-Detail Management
Adaptive Level-of-Detail Management Using Foveated Rendering
Adaptive Level-of-Detail Management Guaranteeing Frame Time
6.5.2 Cell Segmentation
Automatic Cell Segmentation
3-D Cell Segmentation
6.6 CONCLUSION
6.7 REVIEW QUESTIONS
REFERENCES
7. Virtual Reality Programming
7.1 SCENE GRAPHS AND TOOLKITS
7.1.1 Scene Graphs
Internal scene graphs
Distributed scene graphs
7.1.2 Toolkits
7.2 JAVA3D
7.2.1 Java 3D Model Geometry and Appearance
7.2.2 Java3D Scene Graph
7.2.3 Java3D Sensors and Behaviors
7.2.4 Java3D Networking
7.3 THE VIZARD TOOLKIT
7.3.1 Vizard Model Geometry and Appearance
Model Geometry
Model appearance
7.3.2 Vizard Scene Graph
7.3.3 Vizard Sensors and Behaviors
Vizard Physics Engine
Vizard OpenHaptics Plug-in
7.3.4 Vizard Networking
7.4 THE OPENHAPTICS TOOLKIT
7.4.1 OpenHaptics Integration with the Graphics Pipeline
7.4.2 OpenHaptics QuickHaptics Micro API
7.4.3 OpenHaptics Haptic Device to Screen Mapping
7.4.5 OpenHaptics Unity Plugin
7.5 UNITY 3D GAME ENGINE
7.5.1 The Game Engine
7.5.2 Game Production Pipeline
The pre-production pipeline stage
The production pipeline stage
The post-production pipeline stage
7.5.3 Unity Game Programming
Creating a New Project in Unity
The Unity Editor
Unity Game Objects
Physics Programming in Unity
Scripting in Unity
7.5.4 Artificial Intelligence in Unity Gaming
Non-Player Characters, Finite State Machines and Machine Learning
Unity Implementation
7.6 CONCLUSION
7.7 REVIEW QUESTIONS
REFERENCES
8. Human Factors in Virtual Reality
8.1 METHODOLOGY AND TECHNOLOGY
8.1.1 The Experimental Protocol
8.1.2 Institutional Review and Participant Consent
8.1.2 Data Collection and Analysis
Objective performance measures
Modality-specific objective performance measures
Task-specific objective performance measures
Subjective performance criteria
8.1.3 Usability Engineering Methodology
8.2 USER PERFORMANCE STUDIES
8.2.1 Testbed Evaluation of Universal VR Tasks
Influence of navigation and text display techniques on user performance
Influence of locomotion techniques on user performance
8.2.3 User Performance Dependency on Feedback Modality
User performance during object manipulation with substituted and redundant haptic feedback
Participant sense of presence during tasks involving sensorial substitution
Olfactory and haptic feedback contribution to participant quality of experience in VR
8.2.4 Sensorial Illusion
Illusion of weight
Illusion of Scent
8.3 HEALTH AND SAFETY ISSUES IN VIRTUAL ENVIRONMENTS
8.3.1 Direct Effects of Virtual Environments on Users
Direct effects on the Visual System
Direct Effects on the Auditory, Musculoskeletal and Olfactory Systems
8.3.2 Indirect Effects of Virtual Environments on Users
Neural conflict between inner ear, visual and proprioceptive sensorial cues
Ways to measure cybersickness severity
Influence of user's characteristics
Influence of VR system characteristics
Temporal factors influencing cybersickness
Adaptation and Aftereffects
Guidelines for proper VR usage
8.4 SOCIETAL IMPLICATIONS OF VIRTUAL REALITY
8.4.1 Virtual Reality Impact on Professional Life
8.4.2 Virtual Reality Impact on Private Life
8.5 CONCLUSION
8.6 REVIEW QUESTIONS
REFERENCES
9. Applications of Virtual Reality
9.1 MEDICAL APPLICATIONS OF VIRTUAL REALITY
9.1.1 Medical Education
Anatomy trainers using Virtual Reality
Surgical Skill Acquisition when Using VR Trainers
9.1.2 Virtual Rehabilitation
Virtual Rehabilitation Game Design Principles
Virtual Rehabilitation for Management of Chronic Pain
Virtual Rehabilitation for Cognitive Impairments
Autism Spectrum Disorder
Dementias
Remote Therapeutic Monitoring Using Home Virtual Rehabilitation
Chronic stroke
Cerebral Palsy
9.2 VIRTUAL REALITY IN EDUCATION, ARTS AND ENTERTAINMENT
9.2.1 Virtual Reality in Education
Constructionist Learning in Virtual Reality
Constructionist 3D Dynamic Geometry
The Plasma Engineering Playground
Constructivist Learning in Virtual Reality
9.2.2 Virtual Reality and the Arts
The Virtual Florentine Pietà
The House of Greek Epigrams in ancient Pompeii
Visitor fixation vs. feedback modality - The Honk Kong Heritage
Olfactory Heritage
9.2.3 Entertainment applications of Virtual Reality
Gaming on Personal Computers
Location-based Virtual Reality Entertainment
Virtual Reality Arcades
Virtual Reality Lounges
Cloud Gaming
General Cloud Gaming
Virtual Reality Cloud Gaming
9.3 MILITARY VIRTUAL REALITY APPLICATIONS
9.3.1 Army Use of Virtual Reality
Small Arms Tactical Trainers
Armored Personnel Carrier Driver Trainer
9.3.2 Air Force Use of Virtual Reality
Aircraft Deployable Tactical Trainers
Mixed-Reality Flight Simulators
Transfer of Training from VR and MR Flight Simulators
9.3.3 Virtual Reality Applications in the Navy
Naval Ship and Bridge Virtual Reality Simulator
Shipboard Firefighting Trainer
9.4 CONCLUSIONS
9.5 REVIEW QUESTIONS
REFERENCES
Index
Laboratory Manual (online on book site)
