Multi-modal Machine Listening for Monitoring Guillemot Behaviour at Stora Karlsö

Abstract

Automated analysis of seabird behaviour is challenging due to the complexity of dense colony environments and the practical limits of manual annotation over continuous recordings. This thesis presents a machine listening pipeline trained on acoustic recordings collected at the Karlsö AukLab on Stora Karlsö, Sweden, during the 2025 breeding season, distinguishing three classes: arrivals with fish, arrivals without fish, and non-events (background colony noise). Accurate classification of these events is directly linked to chick provisioning rates and breeding success, making it a valuable tool for long-term ecological monitoring of seabird populations. A labelled dataset was constructed from YOLO detections on colony camera footage and verified through manual inspection, replacing the labour-intensive manual annotation previously performed by researchers at the lab. Audio segments were extracted from DPA microphones and preprocessed. Three frozen pre-trained CNN models (BirdNET, PANN, and Perch) were evaluated as feature extractors with lightweight classifier heads, trained on the manually curated dataset and evaluated on both the curated and an automatically extracted dataset. BirdNET achieved the strongest performance (F1-score 82%), with fish arrivals being the most acoustically separable class. Embedding inspection revealed ecologically interpretable acoustic signatures, with chick vocalisations and adult call patterns playing a key discriminative role. Incorporating weather data further improved classification performance. The model generalised well to a held-out, non-curated dataset, demonstrating robustness beyond the manually verified training distribution. These results demonstrate that event-driven multimodal annotation combined with pre-trained bioacoustic embeddings enables reliable, scalable acoustic monitoring of prey delivery in dense seabird colonies, offering a practical path toward automated and passive long-term ecological monitoring.