top of page
actsonic.png

ActSonic: Recognizing Everyday Activities from Inaudible Acoustic Waves Around the Body

Saif Mahmud , Vineet Parikh , Qikang Liang , Ke Li , Ruidong Zhang , Ashwin Ajit , Vipin Gunda , Devansh Agarwal , François Guimbretière , and Cheng Zhang

Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), Accepted with minor revision 2024

arXiv Preprint

Supported by NSF 

SonicID.jpg

SonicID: User Identification on Smart Glasses with Acoustic Sensing

Ke Li , Devansh Agarwal , Ruidong Zhang , Vipin Gunda , Tianjun Mo , Saif Mahmud , Boao Chen , François Guimbretière  , and Cheng Zhang

Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), Accepted with minor revision 2024

arXiv Preprint

Supported by NSF 

ring-a-pose.png

Ring-a-Pose: A Ring for Continuous Hand Pose Tracking

Tianhong Catherine Yu , Guilin Hu , Ruidong Zhang , Hyunchul Lim , Saif Mahmud , Chi-Jung Lee , Ke Li , Devansh Agarwal , Shuyang Nie , Jinseok Oh , François Guimbretière , and Cheng Zhang.

Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), Accepted with minor revision 2024

arXiv Preprint

Supported by NSF 

GazeTrak.jpg
Paper

GazeTrak: Exploring Acoustic-based Eye Tracking on a Glass Frame

The 30th Annual International Conference on Mobile Computing and Networking (MobiCom 24), 

Ke Li, Ruidong Zhang, Boao Chen, Siyuan Chen, Sicheng Yin, Saif Mahmud, Francois Guimbretiere,  Cheng Zhang

Supported by NSF 

SeamPoseHero.png

SeamPose: Repurposing Seams as Capacitive Sensors in a Shirt for Upper-Body Pose Tracking

Tianhong Catherine Yu, Manru Mary Zhang , Peter He, Chi-Jung Lee, Cassidy Cheesman, Saif Mahmud, Ruidong Zhang, François Guimbretière , and Cheng Zhang

The ACM Symposium on User Interface Software and Technology (UIST) ’24

arXiv Preprint

Supported by NSF 

munchsonic.jpg

MunchSonic: Tracking Fine-grained Dietary Actions through Active Acoustic Sensing on Eyeglasses

Saif Mahmud , Devansh Agarwal , Ashwin Ajit , Qikang Liang , Thalia Viranda , François Guimbretière , and Cheng Zhang

Proceedings of the 27th Annual International Symposium on Wearable Computers (ISWC' 24)

arXiv Preprint

Supported by NSF 

glasses.jpg
arXiv

AI-Powered Eyewear for Routine Facial Expression Analysis in Parkinson’s Disease
Hwai Yin Ooi , Cheng Zhang, Ke Li , Jordan Narins , and Harini Sarva
International Congress of Parkinson’s Disease and Movement Disorders, 2024

echoguide.jpg

EchoGuide: Active Acoustic Guidance for LLM-Based Eating Event Analysis from Egocentric Videos

Vineet Parikh , Saif Mahmud , Devansh Agarwal , Ke Li , François Guimbretière , and Cheng Zhang

Proceedings of the 27th Annual International Symposium on Wearable Computers (ISWC' 24)

arXiv Preprint

Supported by NSF 

Beyond-Voice.jpg
Paper

Beyond-Voice: Towards Continous 3D Hand Pose Tracking on Commercial Home Assistant Devices

The ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN 24), 

Award: Best Poster Award for the Accompanying Poster Presentation

Yin Li, Rohan Reddy, Cheng Zhang, Rajalakshmi Nandakumar

Partially Supported by NSF 

EyeEcho_Figure.jpg
Paper

EyeEcho: Continuous and Low-power Facial Expression Tracking on Glasses

Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI 24), 

Conditionally Accepted. 

Ke Li, Ruidong Zhang, Siyuan Chen, Boao Chen, Mose SakashitaFrancois Guimbretiere,  Cheng Zhang

Supported by NSF 

EchoWrist.jpg
Paper

EchoWrist: Continuous Hand Pose Tracking and Hand-Object Interaction Recognition Using Low-Power Active Acoustic Sensing On a Wristband

Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI 24), 

Conditionally Accepted. 

Chi-Jung Lee, Ruidong Zhang, Devansh Agarwal, Catherine Yu, Vipin Gunda, Oliver Lopez, James Kim, Sicheng Yin, Boao Dong, Ke Li, Mose Sakashita,  Francois Guimbretiere,  Cheng Zhang

Supported by NSF 

HPSpeech.jpg
Paper

HPSpeech: Silent Speech Interface for Commodity Headphones

Proceedings of the 27th Annual International Symposium on Wearable Computers (ISWC' 23)

Ruidong Zhang, Hao Chen, Devansh Agarwal, Richard Jin, Ke Li, Francois Guimbretiere,  Cheng Zhang

Awards: Highest Review Score among all submissions, Best Paper Honorable Mention Award

Supported by NSF 

EchoNose.jpg
Paper

EchoNose: Sensing Mouth, Breath and Tongue Gestures inside Oral Cavity using a Non-contact Nose Interface

Proceedings of the 27th Annual International Symposium on Wearable Computers (ISWC' 23)

Ruijia Sun, Xiaohe Zhou, Benjamin Steeper, Ruidong Zhang, Sicheng Yin, Ke Li, Shegnzhang Wu, Sam Tilsen,  Francois Guimbretiere,  Cheng Zhang

Supported by NSF

C-Auth.jpg
Paper

C-Auth: Exploring the Feasibility of Using Egocentric View of Face Contour for User Authentication on Glasse

Proceedings of the 27th Annual International Symposium on Wearable Computers (ISWC' 23)

Hyunchul Lim, Guilin Hu, Richard Jin, Hao Chen, Ryan Mao, Ruidong Zhang, Cheng Zhang

Supported by NSF

posesonic-03.png
Paper

PoseSonic: 3D Upper Body Pose Estimation Through Egocentric Acoustic Sensing on Smartglasses
Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT)/UbiComp ’23

Saif Mahmud, Ke Li, Guilin Hu, Hao Chen, Richard Jin, Ruidong Zhang, Francois Guimbretiere, Cheng Zhang

Selected Media Coverage: Cornell Chronicle,  Engadget,  Hackster, Tech Explorist, Tech Xplore

Supported by NSF

Ruidong_EchoSpeech.png

EchoSpeech: Continuous Silent Speech Recognition on Minimally-obtrusive Eyewear Powered by Acoustic Sensing

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI'23)

Ruidong Zhang, Ke Li, Yihong Hao, Yufan Wang, Zhengnan Lai, Francois Guimbretiere,  Cheng Zhang

Selected Media Coverage: Fast Company,  Cornell Chronicle,  Engadget, Popular Science, NeuroScience News, Hackster, AZO Robotics , New Atlas, Tech Xplore, Interesting Engineering

ReMotion.png
Paper

ReMotion: Supporting Remote Collaboration in Open Space with Automatic Robotic Embodiment

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI'23), 

Mose Sakashita, Ruidong Zhang, Xiaoyi Li, Hyunju Kim, Michael Russo, Cheng Zhang, Malte F. Jung, François Guimbretière

DTouch.png
Paper

D-Touch: Recognizing and Predicting Fine-grained Hand-face Touching Activities Using a Neck-mounted Wearable

The 26th Annual Conference on Intelligent User Interfaces (IUI ’23)

Hyunchul Lim, Ruidong Zhang, Samhita Pendyal, Jeyeon Jo,  Cheng Zhang

EatingTrak.png

EatingTrak: Detecting fine-grained eating moments in the wild using a wrist-mounted IMU

The ACM International Conference on Mobile Human-Computer Interaction MobileHCI ’22 

Ruidong Zhang,  Jihai Zhang,  Nitish Gade,  Peng Cao,  Se Yun Kim,  Yizhuo ChenJunchi Yan,  Cheng Zhang

bodytrak2.png

BodyTrak: Inferring Full-body Poses from Body Silhouettes Using a Miniature Camera on a Wristband

Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT)/UbiComp ’22 

Hyunchul Lim, Yaxuan Li, Matthew Dressa, Fang HU,  Jae Hoon Kim,  Ruidong Zhang Cheng Zhang

Selected Media Coverage: Cornell Chronicle , CNET , HacksterInteresting Engineering, News Atlas , Tech Xplore.

EarIO_Demo.png

EarIO: A Low-power Acoustic Sensing Earable for Continuously Tracking Detailed Facial Movements

Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT)/UbiComp ’22

Ke Li, Ruidong Zhang, Bo LiangFrancois Guimbretiere,  Cheng Zhang

Selected Media Coverage: Cornell Chronicle , Engadget , Hackster News Atlas , Tech Xplore.

SpeeChin2.png

SpeeChin: A Smart Necklace for Silent Speech Recognition

Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT)/UbiComp ’21

Ruidong Zhang,  Mingyang Chen,  Benjamin Steeper,  Yaxuan Li,  Zihan Yan,  Yizhuo Chen,  Songyun Tao,  Tuochao Chen, Hyunchul Lim,  Cheng Zhang

Selected Media Coverage: Cornell Chronicle , Gizmodo , Hackster News Atlas , Tech Xplore.

Fig1_research_idea.png

HandyTrak: Recognizing the Holding Hand on a Commodity Smartphone from Body Silhouette Images

The ACM Symposium on User Interface Software and Technology (UIST) ’21

Hyunchul Lim, David Lin , Jessica Tweneboah,  Cheng Zhang

ThumbTrak-1.png

ThumbTrak: Recognizing Micro-finger Poses Using a Ring with Proximity Sensing

 ACM International Conference on Mobile Human-Computer Interaction (MobileHCI) ’21

Wei Sun, Franklin Mingzhe Li, Congshu Huang, Zhenyu Lei, Benjamin Steeper, Songyun Tao, Feng Tian, Cheng Zhang

NeckFace-1.png

NeckFace: Continuously Tracking Full Facial Expressions on Neck-mounted Wearables

Published on Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT)/UbiComp ’21

Tuochao Chen, Yaxuan Li, Songyun Tao, Hyunchul Lim, Mose Sakashita, Ruidong Zhang, Francois Guimbretiere, Cheng Zhang

Selected Media Coverage: Cornell Chronicle , News Atlas , Hackster

ScreenshotTeethtap.png

TeethTap: Recognizing Discrete Teeth Gestures using Motion and Acoustic Sensing on an Earpiece.

The 26th Annual Conference on Intelligent User Interfaces (IUI'21)

Wei Sun, Franklin Mingzhe Li, Benjamin Steeper, Songlin Xu,  Feng Tian, Cheng Zhang

Demo_HeadPhone.png

C-Face: Continuously reconstructing facial expressions by deep learning contours of the face with ear-mounted miniature cameras

 ACM Symposium on User Interface Software and Technology (UIST) UIST ’20

Tuochao Chen, Benjamin Steeper, Kinan Alsheikh, Songyun Tao, Francois Guimbretiere, Cheng Zhang

Selected Media Coverage: Cornell Chronicle , Engadget , New Scientist, NowThis(Twitter, Facebook, YoutubeYahooMashableNew Atlas, South China Morning Post.

All_Activities.png

VibroSense: Recognizing Home Activities by Deep Learning Subtle Vibrations on an Interior Surface of a House from a Single Point Using Laser Doppler Vibrometry

Published on Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT)/Ubicomp’20

Wei Sun, Tuochao ChenJiayi ZhengZhenyu Lei, Lucy WangBenjamin Steeper, Peng He, Matthew Dressa, Feng Tian, Cheng Zhang

Selected Media Coverage: Cornell ChronicleEngadgetNewAtlas,  Yahoo, TechExplore, PhysicsWorldThe Engineer, Hackster

12333.png

FingerTrak: Continous 3D Hand Pose Tracking by Deep Learning Hand Silhouettes Captured by miniature Thermal Cameras on Wrist

Published on Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT)/Ubicomp’20, ECCV'20 Best Demo Honorable Mention (TOP 3 Demos)

Fang Hu, Peng He, Songlin Xu, Yin Li, Cheng Zhang

Selected Media Coverage: BBC, Forbes, Cornell Chronicle, Engadget, Gizmodo, VentureBeat, Yahoo, VisionSystemDesign.

WX20200619-131656@2x.png

WristWash: Towards Automatic Handwashing Assessment Using a Wrist-worn Device  

Published on 2018 International Symposium on Wearable Computers (ISWC’18)

Hong Li, Shishir Chawla, Richard Li, Sumeet Jain, Gregory D. Abowd, Thad Starner, Cheng Zhang, Thomas Plötz

System_edited.png

FingerPing: Recognizing Fine-Grained Hand Poses Using Active Acoustics Extraction from the Body  

Published on Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI'18)

Cheng Zhang, Qiuyue Xue, Anandghan Waghmare, Ruichen Meng, Sumeet Jain, Yizeng Han, Xinyu Li, Kenneth Cunefare, Thomas Plötz, Thad Starner, Omer Inan, Gregory D. Abowd

FingerSound.jpeg

FingerSound: Recognizing unistroke thumb gestures using a ring 

Published on Proc. ACM Interact. Mob. Wearable Ubiquitous Technology(IMWUT). Issue 3 /Ubicomp’17

Cheng Zhang, Anandghan Waghmare, Pranav Kundra, Yiming Pu, Scott Gilliland, Thomas Ploetz, Thad E. Starner, Omer T. Inan, Gregory D. Abowd

SystemOverview.jpg

FingerOrbits: Interaction with Wearables Using Synchronized Thumb Movements 

Published on 2017 international symposium on wearable computers (ISWC’17).

Cheng Zhang, Xiaoxuan Wang, Anandghan Waghmare, Sumeet Jain, Thomas Ploetz, Omer T.Inan, Thad E. Starner, Gregory D. Abowd

thumbnail-1.jpg

SoundTrak: Continuous 3D Finger Tracking Using Active Acoustics

Published on Proc. ACM Interact. Mob. Wearable UbiquitousTechnol. 1, 2, Article 30 (June 2017) / Ubicomp 2017.

Cheng Zhang, Qiuyue Xue, Anandghan Waghmare, Sumeet Jain, Yiming Pu Sinan Hersek, Kent Lyons, Kenneth A. Cunefare, Omer T.Inan, Gregory D. Abowd

GestureHand.jpg

TapSkin: Recognizing On-Skin Input for Smartwatches

Published on 2016 ACM International Conference on Interactive Surfaces and Spaces.

Cheng Zhang, AbdelKareen Bedri, Gabriel Reyes, Bailey Bercik, Omer T. Inan, Thad E. Starner, Gregory D. Abowd

bottom of page