An FPGA-Based Visual Processing System for Autonomous Driving under Extreme Lighting

Abstract

Extreme lighting conditions degrade camera image quality and may cause fatal autonomous driving accidents through perception failures. This thesis presents an FPGA-based front-end visual processing system that mitigates such degradation in real time while preserving downstream model recognition accuracy. The system architecture contributes two innovations: the traditional image signal proces sor pipeline is pruned from over ten hardware modules to five essential modules based on literature consensus, and black-level correction, auto exposure gain, and localized bloom suppression are consolidated into a single Bayer-domain affine correction applied before demosaicing, severing the saturation-energy diffusion cascade at its source. Auto exposure control operates as an independent side channel with zero pixel-throughput overhead. Validation follows a task-driven paradigm using contrastive-learning-based style transfer, where downstream model accuracy rather than pixel-level metrics serves as the evaluation criterion. FPGA implementation achieves low resource consumption, deterministic fixed latency, and throughput far exceeding real-time requirements. Style-transfer-trained models match the accuracy of those trained on original data in both object detection and semantic segmentation tasks, and visual comparison against a professional action camera confirms comparable overall image quality under both daytime and nighttime conditions, while achieving substantially lower and deterministic pipeline latency.