Novelty-Aware Frame Selection for Streaming Object Detection: A Centralized and Federated Study on Driving Data

Abstract

Modern vehicle fleets produce continuous camera streams, but training on every frame is impractical under compute and bandwidth constraints. Many frames are redundant, while frames from altered lighting conditions, adverse weather, or unfamiliar road contexts carry most of the adaptation signal. This thesis studies noveltyaware frame selection for streaming object detection, where frames are scored by Mahalanobis distance to a periodically refreshed reference Gaussian. A bootstrap anchor keeps novelty tied to an initial urban daytime reference, and the scoring snapshot, reference, and threshold are refreshed together so scores stay meaningful as the detector evolves. Using the Zenseact Open Dataset (ZOD) and an FCOS/ResNet-50 detector, the method is evaluated in centralized streaming and in a four-client federated setting. Each variant is paired with a random baseline at the same empirical acceptance rate to isolate selection quality from data volume. Results show three consistent patterns. The filter is domain-sensitive, with acceptance varying strongly across stream conditions. Periodic refresh is essential for calibration; without it, the static filter drifts from a 20% target acceptance rate to 77% and saturates on novel segments. At matched acceptance rate, detection gains are positive but modest. The primary practical benefit is systematic per-domain coverage, with frames from novel or under-represented driving conditions reliably prioritized over familiar content. The federated setting reproduces the same three patterns.