The Rise of Hydra-BERT - A Multiheaded Approach for Multiclass Event Extraction on a Single Language Model Body

Abstract

Every day, millions of pieces of text hit the internet. A fraction of these describe events which can be invaluable in the right context. Recorded Future uses a platoon of event extraction models, attempting to find information nuggets in a sea of digital noise. Each model is only trained on a specific event type, leaving a lot of potential data synergies unexplored. This thesis proposes an alternative model, trained on all event types. The model should be able to detect events and tag roles equal to or better than models dedicated to a specific event type. It should also be a continual learner, not deteriorating on old event types as new ones are added. The resulting model, called Hydra2, was trained on six different event types. It outperformed the baseline models in all event detection and role tagging tasks. Furthermore, the observed increase in performance also hints at hidden similarities among the event types utilized in these tasks. A smaller version, called Hydra2b, showed potential for continual learning, though further studies are required before declaring it a definite success.