Processing-in-Memory based CNN Acceleration: Application Characterization and Simulation

Abstract

Convolutional neural networks (CNN) are widely used in different machine learning tasks, especially computer vision. Large amounts of computation in CNN cause intensive data movement, which makes traditional compute-centric architectures (CPU and GPU) less capable. There is a demand for new memory-centric architecture, such as processing-in-memory (PIM).

Two subcategories of PIM, processing near memory (PNM) and processing using memory (PUM) are discussed in this report. Starting from the application level, we characterize the layers. Then we use the PNM simulator, ramulator-pim, to simulate CPU and PNM architecture and use the PUM simulator, DNN+NeuroSim, to simulate PUM architecture. The CNN models we use are VGG8, UNet, ResNet18 and MobileNetV3. Based on the characteristics of software and advantages of hardware, we aim to accelerate CNN with PIM technologies and estimate the performance improvement by using PIM technologies.

By observing the behavior of different layers on CPU and PNM architecture, we find the correlation between application characteristics and the performance improvement we can get from PNM. The characteristics are memory footprints, number of floating point operations and arithmetic intensity. PUM is another promising technology that provides high speedup compared to CPU with a cost of energy.