A quality-focused explainability approach for robot mission planning

Abstract

Self-adaptive systems adjust their behavior in response to changes in their environment and plan their actions by prioritizing different quality attributes, such as travel time, safety, and privacy. These systems are seen as black boxes, and stakeholders find it challenging to understand their behavior. This study aims to help humans better understand how the prioritization of quality attributes affects the actions of self-adaptive systems. We used design science research to identify, explore solutions for, and mitigate key challenges stakeholders perceive in understanding the behavior of self-adaptive systems. Our research led to the development of a tool that uses a combination of visual and text features to provide both a descriptive and a contrastive explanation of the system’s behavior. The tool was evaluated in a thinkaloud study, and our findings suggest that using our developed tool can significantly mitigate stakeholders’ perceived challenges, especially the complexity.