Multi-dimensional assessment of machine-learning based nonlinear equalizers for optical interconnects

Abstract

This paper presents a multi-dimensional analysis of machine-learning-based nonlinear equalizers for coherent optical communication systems. Focusing on the well-known sequentialized multi-layer perceptrons (MLPs), we evaluate the trade-offs among compensation performance, computational complexity, and processing latency in a 112 Gbps dual-polarization 16-QAM system. A simulation framework combining VPI TransmissionMaker and Python was employed to assess 240 MLP configurations, varying the number of taps, neurons, and hidden layers. Performance metrics include bit error ratio (BER), total number of floating-point operations, and the number of sequential processing operations. Results show that single-layer MLPs offer superior Pareto-optimal behavior—particularly in low-latency and low-complexity regimes.