Using pre-trained language models for extractive text summarisation of academic papers
| dc.contributor.author | Hermansson, Erik | |
| dc.contributor.author | Boddien, Charlottte | |
| dc.contributor.department | Chalmers tekniska högskola / Institutionen för mekanik och maritima vetenskaper | sv |
| dc.contributor.examiner | Selpi, Selpi | |
| dc.contributor.supervisor | Selpi, Selpi | |
| dc.date.accessioned | 2020-03-02T12:28:38Z | |
| dc.date.available | 2020-03-02T12:28:38Z | |
| dc.date.issued | 2020 | sv |
| dc.date.submitted | 2019 | |
| dc.description.abstract | Given the overwhelming amount of textual information on the internet and elsewhere, the automatic creation of high-quality summaries is becoming increasingly important. With the development of neural networks and pre-trained models within natural language processing in recent years, such as BERT and its derivatives, the field of automatic text summarisation has seen a lot of progress. These models have been pre-trained on large amounts of data for general knowledge and are then finetuned for specific tasks. Datasets are a limiting factor for training summarisation models, as they require a large amount of manual summaries to be created. Most of the current summarisation models have been trained and evaluated using textual data mostly from the news domain. However, pre-trained models, fine-tuned on data from the news domain, could potentially also be able to generalize and perform well on other data as well. The main objective of this thesis is to investigate the suitability of several pre-trained language models for automatic text summarisation. The chosen models were finetuned on readily available news data, and evaluated on a very different dataset of academic texts to determine their ability to generalise. There were only slight differences between the models on the news data. But more interestingly, the results on the academic texts showed significant differences between the models. The results indicate that the more robustly pre-trained models are able to generalise better and according to the metrics perform quite well. However, human evaluation puts this into question, showing that even the high-scoring summaries did not necessarily read well. This highlights the need for better evaluation methods and metrics. | sv |
| dc.identifier.coursecode | MMSX30 | sv |
| dc.identifier.uri | https://hdl.handle.net/20.500.12380/300719 | |
| dc.language.iso | eng | sv |
| dc.relation.ispartofseries | 2020:01 | sv |
| dc.setspec.uppsok | Technology | |
| dc.subject | natural language processing | sv |
| dc.subject | nlp | sv |
| dc.subject | machine learning | sv |
| dc.subject | deep learning | sv |
| dc.subject | automatic text summarisation | sv |
| dc.subject | extractive summarisation | sv |
| dc.subject | transformer | sv |
| dc.subject | bert | sv |
| dc.subject | roberta | sv |
| dc.subject | xlnet | sv |
| dc.title | Using pre-trained language models for extractive text summarisation of academic papers | sv |
| dc.type.degree | Examensarbete för masterexamen | sv |
| dc.type.uppsok | H | |
| local.programme | Computer systems and networks (MPCSN), MSc |
Ladda ner
Original bundle
1 - 1 av 1
Hämtar...
- Namn:
- 2020-01 Erik Hermansson & Charlotte Boddien.pdf
- Storlek:
- 1.28 MB
- Format:
- Adobe Portable Document Format
- Beskrivning:
- Master thesis
License bundle
1 - 1 av 1
Hämtar...
- Namn:
- license.txt
- Storlek:
- 1.14 KB
- Format:
- Item-specific license agreed upon to submission
- Beskrivning: