Finding Top-k Similar Document Pairs - speeding up a multi-document summarization approach

Examensarbete för masterexamen

Please use this identifier to cite or link to this item: https://hdl.handle.net/20.500.12380/199411
Download file(s):
File Description SizeFormat 
199411.pdfFulltext1.43 MBAdobe PDFView/Open
Type: Examensarbete för masterexamen
Master Thesis
Title: Finding Top-k Similar Document Pairs - speeding up a multi-document summarization approach
Authors: Toft, Johan
Bogren, Emma
Abstract: Today there exist many approaches to multi-document summarization, which is the task of automatically creating a summary from multiple sources. This can be a complicated problem, but in this thesis we have focused on a simple approach that is currently being researched. The idea is to find the pairs of documents with the largest overlaps in words and use these to produce a summary. Originally, the algorithm used for this was very naive, comparing each word of every possible pair of documents, and the aim of this project has been to make it more efficient. We have reviewed existing algorithms used for similar problems, and found two useful ones: TOP-MATA and Topk Join. We have also created a new algorithm which we call the Segment Bounding Algorithm (SBA). The approaches were evaluated on two data sets - TREC and Opinosis - and the experimental results showed that SBA was the most efficient on the documents of TREC, while Top-k Join performed slightly better than SBA on the shorter documents of Opinosis. SBA was in the end proposed as an improvement of the simple summarization approach.
Keywords: Data- och informationsvetenskap;Informations- och kommunikationsteknik;Computer and Information Science;Information & Communication Technology
Issue Date: 2014
Publisher: Chalmers tekniska högskola / Institutionen för data- och informationsteknik, Datavetenskap (Chalmers)
Chalmers University of Technology / Department of Computer Science and Engineering, Computing Science (Chalmers)
URI: https://hdl.handle.net/20.500.12380/199411
Collection:Examensarbeten för masterexamen // Master Theses



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.