MiShape
MiShape: Accurate Human Silhouettes and Body Joints from Commodity Millimeter-Wave Devices
Aakriti Adhikari, Hem Regmi, Srihari Nelakuditi, Sanjib Sur Computer Science and Engineering University of South Carolina
MiShape is a millimeter-wave wireless signal based imaging system that generates high-resolution human silhouettes and predicts 3D locations of body joints.
Overview
We propose MiShape, a millimeter-wave (mmWave) wireless signal based imaging system that generates high-resolution human silhouettes and predicts 3D locations of body joints. The system can capture human motions in real-time under low light and low-visibility conditions. Unlike existing vision-based motion capture systems, MiShape is privacy non invasive and can generalize to a wide range of motion tracking applications at-home. To overcome the challenges with low-resolution, specularity, and aliasing in images from Commercial-Off-The-Shelf (COTS) mmWave systems, MiShape designs deep learning models based on conditional Generative Adversarial Networks and incorporates the rules of human biomechanics. We have customized MiShape for gait monitoring, but the model is well adaptive to any tracking applications with limited fine-tuning samples. We experimentally evaluate MiShape with real data collected from a COTS mmWave system for 10 volunteers, with diverse ages, gender, height, and somatotype, performing different poses. Our experimental results demonstrate that MiShape delivers high-resolution silhouettes and accurate body poses on par with an existing vision-based system, and unlocks the potential of mmWave systems, such as 5G home wireless routers, for privacy-noninvasive healthcare applications.
Publications
- MiShape: Accurate Human Silhouettes and Body Joints from Commodity Millimeter-Wave Devices Aakriti Adhikari, Hem Regmi, Sanjib Sur, Srihari Nelakuditi IMWUT'22 Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Atlanta, USA, September 2022. [Paper] [Slides] [Talk]
- MilliFit: A Millimeter-Wave Wireless Sensing Based At-Home Exercise Classification Edward Sitar, Sanjib Sur MSN'22 IEEE International Conference on Mobility, Sensing and Networking, Guangzhou, China, December 2022. [Paper] [Slide] [Talk]
- A Millimeter-Wave Wireless Sensing Approach for At-Home Exercise Recognition Edward Sitar, Moh Sabbir Saadat, Sanjib Sur MobiSys'22 Proceedings of the ACM International Conference on Mobile Systems, Applications, and Services, Portland, Oregon, June 2022. [Paper] [Poster]
- MilliPose: Facilitating Full Body Silhouette Imaging from Millimeter-Wave Device Aakriti Adhikari, Sanjib Sur UbiComp-ISWC'21 ACM International Joint Conference on Pervasive and Ubiquitous Computing and ACM International Symposium on Wearable Computers, Virtual, September 2021. [Paper] [Poster]