A Framework for Verification and Validation of Simulation Models in esmini

Abstract

As the use of simulations is becoming more prominent in the industry, the need for verified and validated simulation models becomes more important. For the simulator esmini, there are no concrete methods or techniques for the V&V process, which this thesis aims to unravel. This thesis presents a verification and validation (V&V) framework for the traffic simulator esmini, with the purpose of providing a foundation for the V&V process and contribute to the research field of simulation validation. The framework was developed by performing desk research on existing V&V methodologies and combining found techniques with software engineering concepts such as quality attribute scenarios (QAS) and acceptance test-driven development (ATDD). The research was done in collaboration with Volvo Cars. Simultaneously, a lane independent routing model for esmini was developed to solve a case provided by Volvo Cars, to which the framework was applied in order to provide an example of how the framework could be applied to a simulation model and to evaluate the framework in terms of usefulness and effectiveness. The resulting framework consisted of six iterative steps: creating an assumptions document, defining requirements and acceptance tests, developing the model, testing and refactoring, reconnecting with stakeholders, and performing empirical and visual validation. The fundamental concept of the framework is to perform the steps iteratively, in order to continuously improve the simulation model. The evaluation of the framework was primarily done through fault injection and developer feedback from Volvo Cars which were encouraging. Overall, the conclusion is that the framework provides necessary guidance for the V&V process in esmini and that it could be used for V&V in other similar simulators