Video as Observational Data for Machine Resetting

Abstract

Machine resetting is a growing share of lost production time in high-mix, low-volume manufacturing. At SKF, resets on the SGP 320 shoe centerless grinding machine have become more frequent, longer, and more variable, motivating the search for new observational data sources to inform a future multimodal AI-based operator assistance system. Within this context, this exploratory single-case study evaluates video as such a source. Reset events were recorded with a wide-angle RGB-D camera and an action camera. These recordings were analyzed at four levels: manual phase annotation in BORIS, operator tracking using YOLOv8 with depth-based zone analysis, R3D-18 video embeddings with cosine similarity and PCA, and visibility detection using Qwen3-VL 8B. Video analysis was further complemented by two expert interviews, the official machine manual, an OPC parameter-snapshot pipeline, and a VNC-based HMI screen recording. While video reliably captured the observable execution of a reset, it missed internal machine states and experience-based decisions. Its usefulness depended on camera placement, visibility, and analytical method. The findings position video as a valuable observational layer for capturing the execution of a reset, but one that must be combined with machine-state and interface data to support a future multimodal system.