Hybrid Quantum Transformer for Natural Language Processing
Publicerad
Författare
Typ
Examensarbete för masterexamen
Master's Thesis
Master's Thesis
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
Large language models, based on the transformer architecture, have revolutionized
the field of Natural Language Processing by using deep learning techniques to capture
complex linguistic patterns. This thesis explores a hybrid architecture for a
transformer model for natural language processing. Specifically, a hybrid quantum
transformer which incorporates a quantum self-attention mechanism for sentiment
classification on the IMDB dataset. The hybrid quantum transformer leverages the
strengths of both classical and quantum computing to enhance the performance of
the sentiment analysis for natural language processing. The model aims to capture
more complex semantic relationships and addresses the potential of quantum
computing compared to classical computation. The quantum self-attention module
computes the similarity measure between input tokens by first embedding the data
in the Hilbert Space of quantum states, followed by an inner product between quantum
states. Comparative analyses are performed against current work on quantum
and classical transformer architectures for sentiment analysis. The quantum kernel
archictecture is benchmarked against an attention-less transformer and previous
implementations for a quantum transformer [27]. Although the quantum kernel
transformer performed marginally better in terms of accuracy on the test set over
five epochs, the runtime for a simplified quantum kernel transformer significantly
highlighted the drawbacks for kernel computations. The long run-time ruled out
the feasibility for a more complex set of hyperparameters to match the capabilities
of a classic transformer. Future work would explore tuning of the hyperparameters
and adding a trainable parameterized layer in the quantum circuit for the kernel to
allow for a trained element to the architecture.