Augmented reality environments allow users to interact naturally with 3D objects, including robots. Many robots have been used in the automated sector for painting, picking, packing, and palletizing tasks. The Baxter robot is an example of an industrial robot ideal for research and education. Baxter robots can offer multiple benefits compared with regular robots. In this study, we designed an augmented reality system that makes users intuitively interact in 3D environments by using a Leap Motion controller as a hand tracker and performing a basic human-robot coordination task with the Baxter robot. And the Baxter robot with a stereo camera is connected to a Linux computer, which was programmed with python language. The augmented reality world was programmed in the Unity software. The human robot-coordination task consisted of an augmented reality alignment. We asked the subject to wear the head-mounted display and move the hands. Every hand motion was translated into the robot’s limb motion in real-time. The subject and robot had to align two augmented reality markers. Here are two different experimental conditions: visual information activated and deactivated. The subject performed three trials under each condition. The experimental results showed that the subject under the visual information activated mode improved the average time by 70.63 %.
Decreasing hand tremor is crucial for sensitive micromanipulation during micro-surgery. Virtual reality (VR) technology
is playing an important role in many biomedical applications. These applications enable the subject to gain
valuable experience in accurate tasks. This study proposes a VR-based system of a handheld gripper combined
with a long short-term memory (LSTM) architecture. Our VR-based system shows an image of forceps in a
virtual space merged with an LSTM model to precisely track the tool’s position. We applied the LSTM as
sensor fusion between a VR controller and an inertial measurement unit. Also, this study compared the LSTM
model with similar models such as the gated recurrent units (GRU) and VR controller raw data. The trained
models used a linear motor attached to a stage as reference data. The training data used different velocities and
accelerations provided by the linear motor control. Experimental results indicate that the LSTM can provide
better precision in both stationary and dynamic scenarios.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.