A tiny inertial transformer for human activity recognition via multimodal knowledge distillation and explainable AI

over time. While this introduces more noise compared to depth-based sensors like Kinect, it offers a more realistic representation of visual input variability encountered in real-world scenarios. Concurrently, inertial data is collected via wearable IMUs placed on various body parts, such as the wrists, ankles, and torso, measuring both acceleration and angular velocity at high sampling rates. For our purposes, the teacher model is trained using both modalities, and the student model uses only (2020)."). As these systems are increasingly adopted in sensitive applications such as elderly care or physical rehabilitation, stakeholders must trust and understand the system’s decisions17 in healthcare. Appl. Sci. 13(24), 13009 (2023)."). Current transformer-based HAR models provide little transparency into their inner workings18, 1–28 (2023)."), making them difficult to interpret or debug in practice. (1) ViT-inspired temporal modeling