Zero-Shot Detection and Classification of Symbols in P&IDs

Abstract

Piping and instrumentation diagrams (P&IDs) are a type of industrial schematic used to represent complex industrial processes. Digitization involves extracting information from a P&ID, such as locations and symbol interpretations. This thesis demonstrates how a vision-language model (VLM) can be leveraged for domainspecific zero-shot detection and classification of symbols. A problem with training models is that symbols vary between projects, combined with a scarcity of annotated training data from real-world projects. Training a model on insufficient data would limit its ability to generalize. For this reason, VLMs are leveraged to provide general knowledge, enabling zero-shot detection and classification without requiring any training data. The thesis presents a novel approach for transforming the detection of symbols into a classification task performed by a VLM. For symbol classification, the pipeline matches a detected symbol with one of the possible symbol categories. The symbol classifier must be provided with a list of possible categories. The digitization pipeline showed strong performance despite no training and no examples given. The proposed digitization pipeline provides an adaptable solution where symbol variability is high and annotated data is scarce.