Haptics-Centered Multi-Sensory Multi-Modal Interfaces for Motor Skill Training and Immersive Digital Twin Interaction in Virtual Reality

Bldg: College of Engineering, Lawrence Technological University, E101, Engineering Building , 21000 W 10 Mile Rd, Southfield, MI 48075, Southfield, Michigan, United States, 48075

The growing demand for immersive, intuitive, and physically meaningful interaction in Virtual Reality (VR) and Human–Robot Interaction (HRI) underscores the limitations of traditional visual–auditory interfaces. By integrating tactile and kinesthetic haptic feedback with synchronized audio‑visual cues, this work enhances realism and interactivity in both virtual and remote environments. We present a unified framework for haptics‑enhanced multi‑modal, multi‑sensory interfaces designed to improve digital twin interaction and motor skill training. Elevating touch to a primary modality addresses a critical gap in current VR and teleoperation systems, where visual feedback alone cannot convey essential physical properties such as texture, stiffness, or weight. Our initial pilot study focuses on vibro‑tactile texture perception, demonstrating how enriched tactile cues improve user engagement and perceptual accuracy. The proposed system integrates bilateral force feedback, high‑fidelity cutaneous actuation, vibro‑tactile rendering, and a novel sensory fusion strategy that prioritizes haptic information during contact‑rich events, mitigating issues such as visual occlusion and video latency. This approach enhances embodiment and realism when interacting with digital twins used for simulation, training, and remote operations. Comparisons with vision‑dominant baselines show measurable gains in task completion time, force regulation accuracy, and cognitive load reduction. Looking forward, integrating advanced AI techniques will enable adaptive synchronization of multi‑modal feedback, supporting next‑generation enactive interfaces with broad implications for surgical robotics, hazardous material handling, remote exploration, and high‑fidelity digital twin applications. Speaker(s): Dr. James, Bldg: College of Engineering, Lawrence Technological University, E101, Engineering Building , 21000 W 10 Mile Rd, Southfield, MI 48075, Southfield, Michigan, United States, 48075

Documentary Night: Vannevar Bush

Virtual: https://events.vtools.ieee.org/m/539619

[]Vannevar Bush (March 11, 1890 – June 28, 1974) was an American engineer, inventor and science administrator, who during World War II headed the U.S. Office of Scientific Research and Development (OSRD), through which almost all wartime military R&D was carried out, including important developments in radar and the initiation and early administration of the Manhattan Project. He emphasized the importance of scientific research to national security and economic well-being, and was chiefly responsible for the movement that led to the creation of the National Science Foundation. { note: Actually 3 short documentaries which have been strung together } AFTER the documentary - we can have a brief discussion session. NOTE: You must supply your own soda pop and popcorn! :-) Trivia may also follow, so bring your Jeopardy hats too! Agenda: 7:00 PM - Welcome and Introductions, Chapter business update; (on your own) Pizza, Popcorn and Soda Pop 7:05 PM - Documentary Start 8:05 PM - End of Documentary; Start of Q & A; Group Discussion 8:15 PM - Wrap Up ALL times are in EST/EDT (UTC-4 or UTC-5) depending upon local day light savings times in when effect Virtual: https://events.vtools.ieee.org/m/539619