Data Prefetcher Based on a Temporal Convolutional Network

Abstract

Cache memory serves a crucial role in alleviating the difference in speed between the computer’s processor and main memory, which has become a growing problem over the years. However, the cache can only hide the whole memory access latency if the requested data is present in it, and only parts of it if the data is already on its way. For this reason, the technique called data prefetching has proven to be an effective way of increasing performance. This technique entails predicting which memory addresses will be accessed in the future and bringing the corresponding data to the cache ahead of time. This thesis explores the design of a data prefetcher based on a Temporal Convolutional Network (TCN), focusing on low storage overhead to make its corresponding implementation size realistic for hardware implementation. In performance simulation tests performed on 15 memory-intensive benchmarks, the TCN prefetcher achieved an average speedup of 30.5 % over a no prefetching baseline, while adding only 14.4 KB of storage overhead. The result shows that the TCN architecture can be a contender for future ML-based prefetchers and that it might work as a good substitute for larger multilayer perceptron (MLP) models. However, the results also suggest that the trade-offs necessary for practical implementation size of a neural network prefetcher make it challenging to advance the average performance beyond rule-based offset prefetchers.