Hybrid Quantum Transformer for Natural Language Processing

dc.contributor.authorConcepcion, Wesley
dc.contributor.departmentChalmers tekniska högskola / Institutionen för fysiksv
dc.contributor.departmentChalmers University of Technology / Department of Physicsen
dc.contributor.examinerGranath, Mats
dc.contributor.supervisorFitzek, David
dc.date.accessioned2025-04-15T08:47:07Z
dc.date.issued2025
dc.date.submitted
dc.description.abstractLarge language models, based on the transformer architecture, have revolutionized the field of Natural Language Processing by using deep learning techniques to capture complex linguistic patterns. This thesis explores a hybrid architecture for a transformer model for natural language processing. Specifically, a hybrid quantum transformer which incorporates a quantum self-attention mechanism for sentiment classification on the IMDB dataset. The hybrid quantum transformer leverages the strengths of both classical and quantum computing to enhance the performance of the sentiment analysis for natural language processing. The model aims to capture more complex semantic relationships and addresses the potential of quantum computing compared to classical computation. The quantum self-attention module computes the similarity measure between input tokens by first embedding the data in the Hilbert Space of quantum states, followed by an inner product between quantum states. Comparative analyses are performed against current work on quantum and classical transformer architectures for sentiment analysis. The quantum kernel archictecture is benchmarked against an attention-less transformer and previous implementations for a quantum transformer [27]. Although the quantum kernel transformer performed marginally better in terms of accuracy on the test set over five epochs, the runtime for a simplified quantum kernel transformer significantly highlighted the drawbacks for kernel computations. The long run-time ruled out the feasibility for a more complex set of hyperparameters to match the capabilities of a classic transformer. Future work would explore tuning of the hyperparameters and adding a trainable parameterized layer in the quantum circuit for the kernel to allow for a trained element to the architecture.
dc.identifier.coursecodeTIFX05
dc.identifier.urihttp://hdl.handle.net/20.500.12380/309270
dc.language.isoeng
dc.setspec.uppsokPhysicsChemistryMaths
dc.titleHybrid Quantum Transformer for Natural Language Processing
dc.type.degreeExamensarbete för masterexamensv
dc.type.degreeMaster's Thesisen
dc.type.uppsokH
local.programmeComplex adaptive systems (MPCAS), MSc

Ladda ner

Original bundle

Visar 1 - 1 av 1
Hämtar...
Bild (thumbnail)
Namn:
W_Concepcion_Master_Thesis_Report.pdf
Storlek:
3.75 MB
Format:
Adobe Portable Document Format

License bundle

Visar 1 - 1 av 1
Hämtar...
Bild (thumbnail)
Namn:
license.txt
Storlek:
2.35 KB
Format:
Item-specific license agreed upon to submission
Beskrivning: