Text summarization using transfer learnin: Extractive and abstractive summarization using BERT and GPT-2 on news and podcast data
Typ
Examensarbete för masterexamen
Program
Computer systems and networks (MPCSN), MSc
Publicerad
2019
Författare
RISNE, VICTOR
SIITOVA, ADÉLE
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
A summary of a long text document enables people to easily grasp the information of
the topic without having the need to read the whole document. This thesis aims to
automate text summarization by using two approaches: extractive and abstractive.
The former approach utilizes submodular functions and the language representation
model BERT, while the latter uses the language model GPT-2. We operate on
two types of datasets: CNN/DailyMail, a benchmarked news article dataset and
Podcast, a dataset comprised of podcast episode transcripts. The results obtained
using the GPT-2 on the CNN/DailyMail dataset are competitive to state-of-the-art.
Besides the quantitative evaluation, we also perform a qualitative investigation in
the form of a human evaluation, along with inspection of the trained model that
demonstrates that it learns reasonable abstractions.
Beskrivning
Ämne/nyckelord
transformer , BERT , GPT-2 , text summarization , natural language processing