An FPGA-Based Visual Processing System for Autonomous Driving under Extreme Lighting - Wu, Chen; Wu, Xingchen (My University, 2026)
Extreme lighting conditions degrade camera image quality and may cause fatal autonomous driving accidents through perception failures. This thesis presents an FPGA-based front-end visual processing system that mitigates such degradation in real time while preserving downstream model recognition accuracy. The system architecture contributes two innovations: the traditional image signal proces sor pipeline is pruned from over ten hardware modules to five essential modules based on literature consensus, and black-level correction, auto exposure gain, and localized bloom suppression are consolidated into a single Bayer-domain affine correction applied before demosaicing, severing the saturation-energy diffusion cascade at its source. Auto exposure control operates as an independent side channel with zero pixel-throughput overhead. Validation follows a task-driven paradigm using contrastive-learning-based style transfer, where downstream model accuracy rather than pixel-level metrics serves as the evaluation criterion. FPGA implementation achieves low resource consumption, deterministic fixed latency, and throughput far exceeding real-time requirements. Style-transfer-trained models match the accuracy of those trained on original data in both object detection and semantic segmentation tasks, and visual comparison against a professional action camera confirms comparable overall image quality under both daytime and nighttime conditions, while achieving substantially lower and deterministic pipeline latency.
Keywords: FPGA, image signal processor, extreme lighting, bloom suppression, autonomous driving, machine vision