Mirror-based ultrasound system for hand gesture classification through convolutional neural network and vision transformer

Keshav Bimbraw; Haichong K. Zhang

doi:10.1117/12.3005247

1 April 2024 Mirror-based ultrasound system for hand gesture classification through convolutional neural network and vision transformer

Keshav Bimbraw, Haichong K. Zhang

Author Affiliations +

Proceedings Volume 12932, Medical Imaging 2024: Ultrasonic Imaging and Tomography; 129320Z (2024) https://doi.org/10.1117/12.3005247
Event: SPIE Medical Imaging, 2024, San Diego, California, United States

Abstract

This research presents an innovative mirror-based ultrasound system designed for hand gesture classification using Convolutional Neural Network (CNN) and Vision Transformer (ViT) architectures. Hand gesture recognition using ultrasound has garnered significant interest due to its potential applications in various fields such as prosthetic control and human-machine interfacing. Traditionally, ultrasound probes are placed perpendicular to the forearm causing discomfort and interference with natural arm movements due to the center of mass of the wearable ultrasound system being distanced from the body. To address this challenge, a novel approach utilizing the advantages of acoustic reflection is proposed. A convex ultrasound probe is strategically aligned with the forearm, and ultrasound waves are transmitted to the forearm, and received back using a mirror placed at 45 degrees to the imaging region and the forearm. By aligning the probe parallel to the arm, the center of mass is brought closer to the body, ensuring enhanced stability and reduced strain on the user's arm during data collection. A dataset comprising 5 hand gestures was collected to train and evaluate the performance with Support Vector Machines with linear kernel, CNN, and ViT based approaches. It was observed that the performance of the mirror-based ultrasound system is comparable to the traditional perpendicular approach for hand gesture classification. The experimental results demonstrate the potential of the system in assisting with data acquisition and device development for hand gesture recognition using ultrasound in the field of human-machine interfacing, prosthetic control, human-computer interaction, and beyond.

Conference Presentation

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Keshav Bimbraw and Haichong K. Zhang "Mirror-based ultrasound system for hand gesture classification through convolutional neural network and vision transformer", Proc. SPIE 12932, Medical Imaging 2024: Ultrasonic Imaging and Tomography, 129320Z (1 April 2024); https://doi.org/10.1117/12.3005247

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available