Virtual Reality (VR) systems can improve the training of first responders and soldiers in multi-domain operations in a number of ways. Realistic simulation of physical objects, however, is challenging. The huge variety of equipment pieces and other objects specialists in first responder units - and in particular CBRN-troops - interact with further increases the effort. In this paper, we present a novel and flexible Mixed Reality (MR) training system for first responders that enables the integration of physical objects by using Augmented Virtuality (AV) and binary tracking". A Head Mounted Display (HMD) immerses the user in VR, while augmenting the visualization with 3D imagery of real objects, captured by an RGB-D sensor. In addition, a RFID-reader at the user's hand detects the presence or absence (binary response) of certain equipment items. Our proposed MR system fuses this information with data of an inertial motion capture suit to an approximate global object pose and distributes it. Our solution provides a wide range of options for physical object interaction and collaboration in a multi-user MR environment. In addition, we demonstrate the training capabilities of our proposed system with a multi-user training scenario, simulating a CBRN crisis. Results from our technical and quantitative user evaluation with 13 experts in CBRN response from the Austrian Armed Forces (National Defense Academy and Competence Center NBC Defense) indicate strong applicability and user acceptance. Over 80% of the participants found it easy or very easy to interact with physical objects and liked the multi-user training much or very much.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.