Patch-wise image similarity search Searching for small regions in collections of large images
Typ
Examensarbete för masterexamen
Program
Complex adaptive systems (MPCAS), MSc
Publicerad
2022
Författare
Arvidsson, Christoffer
Davidsson, Ebba
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
In autonomous driving it is important that a neural network performs well even for
examples that do not occur often in the dataset. One method to improve performance
is to find and add examples similar to those the network struggles with, thereby
increasing the training data available. Similarity search is an automatic method for
searching large datasets for the most similar examples to some target. This thesis
describes a similarity search algorithm for locating different sized objects in a large
dataset of high-resolution images. The algorithms uses patches, which are small
regions in an image, in order to enable precise searches for small objects. Each patch
is embedded to 512 dimensional vectors with CLIP[1] that captures the semantic
meaning of the content in the patch. The main contribution of this thesis is a method
to reduce the potential number of patches resulting from each image, by selecting
the most visually interesting patches in each image. We evaluate the combined patch
selection and similarity search on three classes of objects relevant for autonomous
driving: ambulances, animals and a specific traffic sign marking an upcoming road
narrowing, to measure the fraction of relevant patches retrieved. Further, we show
that searching with the average vector representation of several images of the same
object improves the result, while searching with a text string gives varying results
depending on the object class.
Beskrivning
Ämne/nyckelord
thesis , nearest-neighbor search , deep learning , image retrieval