A Human-in-the-Loop Digital Twin Architecture for Real-Time Safety-Control Simulations in Manufacturing Systems

Abstract

Digital twins are increasingly used to simulate and optimize manufacturing systems; however, current implementations are largely machine-centric, excluding human operators from virtual representation. This results in safety risks, reduced situational awareness, and limited support for human-machine collaboration central to Industry 5.0. This paper presents a human-in-the-loop digital twin architecture that integrates real-time human motion data into a simulated manufacturing environment for safety monitoring and human-centric production. The architecture spans three modular layers: a vision-based physical layer, a robust communication layer, and a simulation layer. Human motion is captured via the Occurrence multi-camera 3D pose fusion system, detecting 17 joints at 20 Hz, and streamed via MQTT to the Emulate3D simulation platform. A custom JSON parser and coordinate calibration pipeline transform incoming pose data into the simulation’s coordinate space, with a dual-protocol fallback strategy ensuring connection reliability across varied network configurations. The live pose data is mapped into a hierarchical 14-joint human proxy model using a hybrid forward- and inverse-kinematics approach, enabling stable real-time replication of full-body motion. A spatially aware, dual-zone safety monitoring system computes real-time pelvis-to-conveyor distances using an Axis-Aligned Bounding Box (AABB) model, triggering warning alerts and latched automated emergency stops on zone violations. The architecture is demonstrated and validated at the drone assembly workstation in the SII lab at Chalmers University of Technology. Results from experiments confirm that all performance targets were met or exceeded, like the pose update rate of 20 Hz, end-to-end latency of 100 ms, etc. This demonstrates the potential of real-time human proxy integration with digital twin environments for adaptive safety monitoring and lays a scalable foundation for human-centric manufacturing