Emergence of Agency from a Causal Perspective

Abstract

Causal models of agents and agentic behavior allows for safety analysis of machine learning systems. Understanding how goal-directed behavior emerges from adapting to an environment is however non-trivial. This thesis addresses the gap between theoretical models and real-world implementations of machine learning systems, though a framework that formalizes the connection between system dynamics and goal-driven behavior. This thesis introduces novel probabilistic graphical models for describing system dynamics involving learning agents, based on dynamic bayesian networks, which allows for a flexible representation of causal relationships in the training environment. To analyze goal-directed behavior that emerges from interacions between agents and the environment, the thesis also introduces temporally abstracted models. Such a model captures the dynamics of a system after the learning process has converged, derived from a model of the learning process. A temporally abstracted model describes potential outcomes involving equilibria between agents and the environment, and can under certain conditions be viewed as a model of goal-directed behavior in the system.