On automated testing of ship control and monitoring system in submarine

Abstract

As technology advances, increasingly complex and autonomous engineering systems emerge. Given our daily dependence on them, these systems must work as intended. Testing is used to verify that the systems behave correctly according to given specifications. In industry, testing is often performed manually by test engineers. However, this approach is expensive, time-consuming, repetitive, and error-prone. A potential solution to these issues is test automation. This thesis employs a design-science research methodology to investigate test automation at a case company. This resulted in the development of an automated testing system for a ship control and monitoring system in a submarine. The automated testing system consists of three different variants of test automation. The first variant involves creating scripts corresponding to manual test procedures in the current documentation. The second and third variants of the automated testing system comprise the testing approach falsification. Optimization-based falsification involves using a simulation model to automatically identify input signals that cause a system to violate given specifications. Quantitative semantics are used to assess how close a scenario is to violating the specifications. The optimization problem includes decision variables that determine the type and shape of the generated input signals. The proposed automated testing system has been assessed according to different measures. Combined, the automated testing system has proved that it confronts the business needs of the case company, where it enhances the productivity and quality of the testing activities. While the initial time spent on creating test scripts may be considerable, the long-term benefits become evident with saved efforts and costs, especially when most systems are tested frequently.