QuadriSparse: RISC-V Sparse Matrix Accelerator and ISA Extension - A Tightly-Coupled Sparse-Dense Matrix Multiplication Accelerator for RISC-V

Abstract

Sparse dense matrix multiplication (SpMM) is an important operation in many applications such as inference and training of pruned large language models, graph analytics and scientific computing. These applications often operate on data that is inherently sparse. Applying dense matrix multiplication (GEMM) to sparse data wastes computations on zero-valued elements, which SpMM avoids by skipping them. However, accelerators designed for dense matrix multiplication do not necessarily support sparse matrix multiplication efficiently. This motivates extensions that can exploit sparsity while retaining computation capabilities for dense workloads. This thesis explores the prospect of extending a small dense matrix multiplication accelerator with additional hardware for SpMM, evaluating the the performance benefits against the hardware overhead. We introduce QuadriSparse, a SpMM extension for unstructured sparsity that adds a partly new datapath to the small and efficient RISC-V accelerator Quadrilatero. The accelerator includes three new instructions to load a tile of the sparse matrix (SPLD_W), load a tile of the dense matrix (DLD_W), and multiply the two using Gustavson’s algorithm (SPMAC_W). We evaluate the resulting accelerator in terms of execution time across different sparsity levels and matrix sizes and measure the FPGA resource utilization through synthesis. QuadriSparse achieves up to 6.6x lower execution time than dense execution on Quadrilatero at 99%sparsity and first outperforms the dense baseline at 95% sparsity. FPGA synthesis shows an increase in resource utilization of 4.6% in LUTs and 23% in DSP-blocks relative to the baseline design.