Immersive feedback in fencing training using mixed reality

Abstract

During sports training, providing athletes with real-time feedback that is based on the automatic analysis of motion is both useful and challenging. In this work, a novel system that is based on mixed reality is proposed and verified. The system allows for immersive and real-time visual feedback in fencing training. Novel methods have been introduced for 3D blade tracking from a single RGB camera, creating weapon-action models by recording the actions of a coach and evaluating the trainee’s performance against these models. Augmented reality glasses with see-through displays are employed, and a method for coordinate mapping between the virtual and real environments is proposed, this will allow for the provision of real-time visual cues and feedback by overlaying virtual trajectories on the real-world view. The system has been verified experimentally in fencing bladework training (with the supervision of a fencing coach). The results indicate that the proposed system allows novice fencers to perform their exercises more precisely.