The Project
Developing mixed reality training applications traditionally requires a large amount of manual work, including task analysis, 3D content creation, animation, scripting, and testing.
Such development cycles are often slow, expensive, and difficult to adapt when industrial workflows change. As a result, creating dedicated MR training material for every new task or machine is rarely feasible.
The AIXRIG project, developed by the HUN-REN Cloud Applications Research Group and the HSN Labor Kft, aims to address this challenge through an AI-based generation framework for immersive training.
Technology
The concept is to let the system automatically create a training application from a single expert demonstration. During this process, the expert performs the task once while the system continuously tracks their motion, gaze, and hand interactions using mixed reality sensors. At the same time, the expert’s spoken instructions are captured and synchronized with the visual data.
All of these multimodal inputs are analyzed by AI-based algorithms that identify and segment the workflow into meaningful subtasks, extracting the corresponding spatial and temporal relationships.
From this structured representation, the system automatically generates a mixed reality step-by-step tutorial, including guidance and an animated expert avatar that demonstrates each motion in 3D space.
This automated process dramatically accelerates MR content creation. What once required months of manual design and programming can now be completed in just a few minutes, enabling scalable, adaptive, and cost-effective training across industrial and educational domains.
