Gesture-Based Control for Human-Centric Manufacturing in Industry 5.0

Abstract

Industry 5.0 emphasizes human-centric manufacturing, where digital technologies should support operators in industrial work. In this context, interaction methods are needed that allow operators to communicate with automated assets in a simple and practical way. This thesis investigates a gesture-based control layer for connected manufacturing systems, where camera-based 3D pose estimates are translated into discrete command actions and routed to industrial assets. The aim of the work was to design, implement, and evaluate an integration blueprint for gesture-based human-system interaction in a laboratory manufacturing environment. The system uses a small vocabulary of static upper-body gestures with holdtime confirmation to reduce accidental triggering. Recognized gestures are processed by a Python-based gesture module and routed either through ThingWorx and Kepware or, for selected AMR actions, through a direct ARCL command path. The implementation was evaluated in a use case involving an OMRON LD-250 Autonomous Mobile Robot (AMR), a conveyor, and an Emulate3D-based digital shadow. The final implementation included AMR mission commands, direct AMR stop handling, an optional robot-facing activation condition, and a stop-and-replace workflow that clears current and queued AMR mission requests after a stop gesture. For the conveyor, the implemented gesture commands included toggle, speed increase, speed decrease, and quick-stop actions. The digital shadow was used for monitoring and repeatable observation of AMR behavior, but it was not treated as a full digital twin. The system was evaluated through controlled practical trials and a preliminary user survey. Under the tested laboratory conditions, the gesture vocabulary was recognized as intended, accepted gestures were mapped to the correct command actions, and the connected assets responded according to the implemented command logic. The main limitations were related to the simplified virtual AMR model in Emulate3D, the two-camera setup, pose-estimation jitter, occasional unexplained response-time delays, privacy and data-governance considerations, and the limited number of user participants. Overall, the thesis shows that gesture-based control can be used as a complementary interaction layer for selected manufacturing commands, provided that the sensing conditions, command logic, and safety limitations are clearly defined.