Hybrid Quantum Transformer for Natural Language Processing

Abstract

Large language models, based on the transformer architecture, have revolutionized the field of Natural Language Processing by using deep learning techniques to capture complex linguistic patterns. This thesis explores a hybrid architecture for a transformer model for natural language processing. Specifically, a hybrid quantum transformer which incorporates a quantum self-attention mechanism for sentiment classification on the IMDB dataset. The hybrid quantum transformer leverages the strengths of both classical and quantum computing to enhance the performance of the sentiment analysis for natural language processing. The model aims to capture more complex semantic relationships and addresses the potential of quantum computing compared to classical computation. The quantum self-attention module computes the similarity measure between input tokens by first embedding the data in the Hilbert Space of quantum states, followed by an inner product between quantum states. Comparative analyses are performed against current work on quantum and classical transformer architectures for sentiment analysis. The quantum kernel archictecture is benchmarked against an attention-less transformer and previous implementations for a quantum transformer [27]. Although the quantum kernel transformer performed marginally better in terms of accuracy on the test set over five epochs, the runtime for a simplified quantum kernel transformer significantly highlighted the drawbacks for kernel computations. The long run-time ruled out the feasibility for a more complex set of hyperparameters to match the capabilities of a classic transformer. Future work would explore tuning of the hyperparameters and adding a trainable parameterized layer in the quantum circuit for the kernel to allow for a trained element to the architecture.