FPGA Implementation of Machine Learning Based Nonlinear Equalizer with On-Chip Training

Abstract

In fiber-optical communication, the linear polarization mode dispersion (PMD) effect and the nonlinear optical Kerr effect have a combined detrimental effect on the transmitted signal, and the time-varying nature of PMD also means that the PMD-Kerr effect can change over time. Therefore, a nonlinear equalizer that compensates for both PMD and Kerr effects and is adaptive to the time-varying PMD-Kerr channel is needed at the receiver side. In recent years, machine learning algorithms, especially neural networks, have been introduced to the fiber-optical field and they offer a promising way to construct a nonlinear equalizer in the digital signal processing (DSP) part in the receiver. In this work, we have implemented an adaptive machine learning based nonlinear equalizer on a field-programmable gate array (FPGA). A model-based machine learning algorithm is adopted and modified for FPGA implementation, and the training of the equalizer is also implemented on the same chip. The equalizer contains multiple layers, which has essentially the same structure as a neural network. The on-chip training realizes the backward propagation to optimize the weights in the equalizer, which allows the equalizer to adapt to the time-varying PMD-Kerr channel in real-time. Logic simulations of a 3-layer equalizer show that its static performance is very close to the ideal upper limit of the original 3-layer software model, and that the 3-layer equalizer can remain stable until the varying speed of each principal state of polarization (PSP) reaches above 1×105 rad/s in the same rotation direction. Final FPGA implementation results show that a 3-layer equalizer utilizes around 79.72% of the total DSP resources on a medium size FPGA, and the DSP becomes the bottleneck of FPGA resources. The on-chip training constitutes most of the used resources due to its complexity and intensive arithmetic computation.