Behaviour evaluation, virtual reality fire extinguisher training Master’s thesis in Maritime Management ANDREAS BACKMAN DEPARTMENT OF MECHANICS AND MARITIME SCIENCES CHALMERS UNIVERSITY OF TECHNOLOGY Gothenburg, Sweden 2024 www.chalmers.se http://www.chalmers.se/ MASTER’S THESIS IN MARITIME MANAGEMENT BEHAVIOUR EVALUATION, VIRTUAL REALITY FIRE EXSTINGUISHER TRAINING Andreas Backman Department of Mechanics and Maritime Sciences Division of Technical and Maritime Management CHALMERS UNIVERISTY OF THECHNOLOGY Gothenburg, Sweden 2024 Behaviour evaluation, virtual reality fire extinguisher training. Andreas Backman © ANDREAS BACKMAN, 2024-06-06 Supervisors: Monica Lund and Henrik Ringsberg, Department of Mechanics and Maritime Sciences. Examiner: Henrik Ringsberg, Department of Mechanics and Maritime Sciences. Department of Mechanics and Maritime Sciences Division of Technical and Maritime Management Chalmers University of Technology SE-412 96 Göteborg Sweden Telephone +46 (0)31-772 1000 Department of Mechanics and Maritime Sciences Göteborg, Sweden 2024 Abstract Fire onboard a ship have serious consequences. The fire scenario can cause life-threatening situations for crewmembers and passengers. Further have an environmental impact, and material damages to both the ship and its cargo. Access to and receiving reinforcement of personnel to a vessel in distress is in many cases very limited. Research have shown that the first minutes after a detected fire determines the risk and outcome of the situation. Therefore, it is crucial that the crewmember can safely operate and efficiently use different type of fire extinguishers onboard. The need of fire extinguisher training is important to properly handle the equipment and safely extinguish an ongoing fire. Improper use of fire extinguishers can result in serious danger for the crewmember and negatively affect the extinguish progress and the overall outcome of the situation. A qualitative approach was used in this master thesis. A triangulation analysis method was used based on the data, primary in the form of questionnaires and observations. Secondly in the form of a systematic literature review that relates to virtual reality safety training. The purpose of this master thesis was to explore educational transferable knowledge and skills from VR-fire extinguisher training. The main results of this study are that the use of a physical VR-fire extinguisher combined with a virtual fire scenario ensured that participants were more confident with the extinguisher. Secondly the participants were to a greater extent able to correctly use the carrying handle to safely hold the extinguisher during the operation. VR- fire extinguisher training also improved the participant learning outcomes as sweeping the fire extinguisher discharge nozzle from side to side, covering the area of the fire better than the control group. The result from this study also provides evidence that the VR-fire extinguisher training is effective for training in a hazardous environment. The conclusion is that VR-fire extinguisher training could be used as a complement to the basic safety training. One of the key advantages are training in a safe environment and the possibilities for crewmembers to rehearse different fire scenarios that are difficult to simulate. Finally, this master thesis has shown the relationship between rehearsal training with the fire extinguisher in a virtual environment and the improved handling result when conducted under real-life conditions. Key words: Maritime Safety Education, Virtual reality, Virtual environments, Just-in-time training, Fire extinguisher training, Safety of life at Sea, International Convention on Standards of Training Certification and Watchkeeping for Seafarers. Acknowledgements I would like to thank my supervisors, associate professor Monica Lund and associate professor Henrik Ringsberg at Chalmers University of Technology for their support and valuable knowledge. Many thanks to all participants from both the master mariner programme and marine engineer program at Linnaeus University. I would also like to thank ICT-Teacher Brian Kottonya at Linnaeus University for technical support and advises. I would like to thank my opponents for reviewing my master thesis and sharing their opinions with me. Finally, I would like to thank my family, good friends, and colleagues. Thank you all for keeping me motivated and supported. Thank you, Andreas Backman Rörö, Sweden, 2024-06-06 Table of Contents 1. Introduction .................................................................................................................................... 1 1.1 Background .................................................................................................................................... 1 1.2 Research purpose and research questions .................................................................................... 3 1.3 Limitations ..................................................................................................................................... 4 2. Frame of reference .......................................................................................................................... 5 2.1 The incipient stage in firefighting .................................................................................................. 5 2.1.1 Technical fire prevention onboard and its limitations ............................................................ 5 2.1.2 Psychological stress and constrains in fire fighting................................................................. 7 2.1.3 Fire extinguisher training onboard ......................................................................................... 7 2.2 Virtual Reality Training .................................................................................................................. 8 2.2.1 Virtual Reality-fidelity ............................................................................................................. 8 2.2.2 Perception of geometric layouts created in a virtual environment ........................................ 9 2.2.3 Educating and training in a safe virtual environment ............................................................. 9 2.2.4 Potential risk of frequent training in engaging virtual environments .................................. 10 2.3.5 Fire extinguisher training in virtual reality ........................................................................... 11 3. Method .............................................................................................................................................. 12 3.1 Research approach ...................................................................................................................... 12 3.2 Data collection methods .............................................................................................................. 13 3.2.1 Literature review .................................................................................................................. 13 3.2.2 Case Study ............................................................................................................................ 15 3.3 Data analysis ................................................................................................................................ 19 3.4 Research Quality .......................................................................................................................... 22 3.4.1 Validity .................................................................................................................................. 22 3.4.2 Reliability .............................................................................................................................. 22 4. Results and analysis .......................................................................................................................... 23 4.1 Level 1 Reactions ......................................................................................................................... 23 4.2 Level 2 Learning ........................................................................................................................... 28 4.3 Level 3 Behaviour ........................................................................................................................ 34 4.4 Level 4 Results ............................................................................................................................. 35 5. Discussion.......................................................................................................................................... 36 5.1 Discussion .................................................................................................................................... 36 5.2 Methodological discussion .......................................................................................................... 39 6. Conclusions ....................................................................................................................................... 41 6.1 Theoretical contributions ........................................................................................................ 41 6.2 Practical implications ............................................................................................................... 42 References ............................................................................................................................................. 43 Appendix I ............................................................................................................................................. 48 50 List of Tables Table 1 ................................................................................................................................................... 18 List of Figures Figure 1 .................................................................................................................................................. 13 Figure 2 .................................................................................................................................................. 14 Figure 3 .................................................................................................................................................. 16 Figure 4 .................................................................................................................................................. 19 Figure 5 .................................................................................................................................................. 20 Figure 6 .................................................................................................................................................. 23 Figure 7 .................................................................................................................................................. 24 Figure 8 .................................................................................................................................................. 25 Figure 9 .................................................................................................................................................. 26 Figure 10 ................................................................................................................................................ 27 Figure 11 ................................................................................................................................................ 28 Figure 12 ................................................................................................................................................ 29 Figure 13 ................................................................................................................................................ 30 Figure 14 ................................................................................................................................................ 30 Figure 15 ................................................................................................................................................ 31 Figure 16 ................................................................................................................................................ 31 Figure 17 ................................................................................................................................................ 32 Figure 18 ................................................................................................................................................ 33 Figure 19 ................................................................................................................................................ 34 Figure 20 ................................................................................................................................................ 35 Table of Abbreviations BST Basic Safety Training HMD Head mounted display IMO International Maritime Organization RE Real Environment SOLAS Safey of Life at Sea STCW International Convention on Standards of Training, Certification and Watchkeeping for Seafarers VE: s Virtual environments VR Virtual Reality AR Augmented Reality EMSA The European Maritime Safety Agency SLR Systematic literature review IMCA The International Marine Contractors Association 1 1. Introduction In this chapter the reader is introduced to the background, research questions, purpose and limitations. 1.1 Background The safety onboard has always been related to the performance of the master and crew. The master is regarded as common carrier, liable for damage on the cargo whether caused by an insufficient vessel, negligence or other faults of the officers and crew under his command (De Oliveira Torres, 2014). Access to and receiving reinforcement of personnel to a vessel in distress is in many cases very limited. The vessels own capabilities and trained crew is the only resources available. For example, in firefighting response time, knowledge and use of equipment depend on their previous training and experience (Stokoe & Russell, 2016). The Nordic association of marine insurance has published analyses regrading fire-trends on vessel types. They conclude that even if the fire/explosion frequency is low in percentage compared to other types of insurance claims in 2021, the cost of such claims is typically high, and they effect the overall annual claim cost (Cefor, 2021). Further research from The European maritime safety agency (EMSA) states that fire and explosions are in the top five causes of maritime accidents. Therefore, the relevance of the potential consequences on fires onboard is self-explaining, as well the potential benefits achievable by new effective technological solutions (Ricci et al., 2023). Firefighting procedures onboard includes that the first responder arrive on the scene, he or she must then decide whether to extinguish the fire or begin containment procedures and wait for the smoke divers and the rest of the crew in the safety organisation (Leonard, et al.,1991). According to the EU-project Lash Fire (2023) aiming at significantly reduce the risk of fire onboard ships, a guideline has been developed for the first responder onboard vessels. The definition of a designated first responder onboard are, a crewmember, specially prepared for response activity, (e.g., a fire patrol member, an able seaman, or engine control room staff). Further according to Lash Fire, (2023) the first responder action is to extinguish a fire in its initial stage without specified protection and use of only working clothes. Finally, the fire must be in its incipient stage and could be suppressed by a fire extinguisher. In addition, the international maritime organisation (IMO) has in many reports concluded that although there have been enough rules, there are implementation problems at an operational level onboard vessels. There are still concerns about the ship’s emergency preparedness and the most important factor are insufficient firefighting practise (Tac et al., 2020). The international convention for the safety of life at sea (SOLAS) is constantly amended, and the latest consolidated version is from 2020. The SOLAS consolidated version from 2020 regulated that fire drills should be conducted and recorded in accordance with the provisions of regulations 2 III/19.3 and III/19.5. Further onboard training in the use of fire extinguishing system and appliances should be planned and conduct in accordance with the provisions of regulation III/19.4.1. Despite the developed safety requirements and records of onboard training, the fire onboard is still a relevant safety factor, generating accidents causing damages and fatalities and material damages onboard modern vessels (Ricci et al., 2023). To comply with mandatory STCW regulations, each crewmember associated to the safety organisation onboard must have a valid Basic Safety Training certificate (BST). Each certificate is valid for five years and is renewed by the crewmember at refresher course at approved training facility. A seafarer is often required to work in situations where conditions are continuously changing and demanding (Riahi et al., 2012). The main component for emergency preparedness for the crewmember is fire training and fire drills. Key factors for good performance in the case of onboard fire are good working conditions and effective training, meaning working environments, systems, organizations, and routines that fit the needs of the crew (Bram et al., 2019). Further an extreme situation differs from normal day to day operations by removing procedures and routines. The individual could experience that the situation is dangerous, obnoxious, and threatening. Further unpredictability and complexity with an overwhelming number of open problems (Lindholm, 2006). Traditional firefighting training methods are based on classroom lessons and multimedia contents and may not be completely effective for teaching practical skills and behaviours (Calandra et al., 2023). Further investigation concludes that live firefighting extinguisher training are difficult to safely operate onboard a vessel. The learning curve for live fire extinguisher training, the accurate number of trials in which an individual has an acceptable level of performance with a fire extinguisher is approximately between 5-10 real trials (Oviedo- Trespalacios et al., 2013). With the introduction of Virtual Reality (VR), new fire training scenarios has been created in an artificial environment. Stefan et al. (2023) analyses that VR training is an alternative to traditional training methods and VR-training has become more widely used, mainly for is proven repetition and use in safe virtual environment. Stefan, et al. (2023); Strojny & Dużmańska-Misiarczyk (2023) further describes that VR-technology presents an important opportunity to improve the effectiveness of safety and safety-relevant training due to its increased level of presence, ability to fail safely in a virtual environment. This is also confirmed by Pedram et al. (2017) that explains that VR training occurs in a safe and controlled environment which allows the replicable testing of extreme scenarios. This is particularly relevant in the safety training due to its ability to safely simulate dangerous and in some form extreme scenarios that would otherwise be difficult to access or duplicate in the real world. Pedram et al. (2017) further analyses that training in a virtual environment has the benefit of creating endless of scenarios that in the real world would not be possible due to economic resources. Pedram et al. (2017) describes examples of difficult economic decisions for instance putting a building on fire with the intention for fire fighters to train them. Further Pedram et al. (2017) describes that there is no guarantee to have a successful training in only one fire training session and repetition might be needed and this would not always be possible when conducted in the real world. 3 Feng et al. (2018) explains that traditionally, safety training has been delivered using different methods. These include safety manuals, videos, in-person or online lectures, and drills. These approaches may not effectively transmit knowledge due to their lack of instant feedback assessing their safety behaviour. Strojny & Dużmańska-Misiarczyk (2023) describes that there are many categories of possible learning outcomes to measure the effectiveness. Skills can be taught and measured in a target environment, which is referred as skill transfer. Such measurement is particularly beneficial because it makes it possible to determine if the acquired skill can be transferred from an artificial environment to a real one that is similar or identical to the one which the skill will be actually used in practice. Further behaviour skills training is maintained due to activities in environment through demonstration, repetition, and feedback. Behaviour skill training is one of the effective methods to obtain natural responses to corresponding emergency behaviours (Strojny & Dużmańska-Misiarczyk, 2023). According to Pedram et al. (2017), in an VR- environment participants can experience and repeat all scenarios as much as they need to become professionals and there is always room to make mistakes without the risk of personal safety. Further appropriate safe behaviour training are key elements for increasing the chance of survival (Chen & Chien 2022). Another aspect relates to VR-training and its validity and real-world performance benefits. From an educational perspective, it is difficult to reinforce crewmembers’ theoretical knowledge with hands-on applications as fire extinguishers. Further onboard drills have little effect if no feedback is achieved afterwards (Bram et al., 2019). Pedram et al. (2017) also warns about the risk with VR-training as with other training methods and the need of evaluation. Unlike traditional operational training it is not possible to evaluate the success or usefulness of the safety training in a virtual environment solely based on the particular training session. Engelbrecht et al. (2019) argue that the worst-case scenario would be the implementation of unproven VR-training, which does (on the face of it) train valuable skills but results in worse overall training outcomes due to limited skill transfer to real scenarios. Additionally, there are also new and more likely new educational challenges with VR due to its rapidly development. Scorgie et al. (2023) determined that Virtual Reality (VR) could provide an engaging and exciting training experience and there is a need to evaluate its application and effectiveness in safety training. 1.2 Research purpose and research questions The purpose of this master thesis is to explore educational transferable knowledge and skills from a VR-fire extinguisher training. Based on the stated purpose, the following questions have been formulated. 1. What are the participants self-reported reactions to VR- fire extinguisher training? 2. What are the participants learning outcomes from a VR-fire extinguisher training? 4 1.3 Limitations This study was conducted based on a fixed time frame. This resulted that the number of participants that were included in both groups were limited. For this study it was decided that only the first-year student at the master mariner and marine engineer program at Linnaeus University was selected as participants. This because their presence at the Linnaeus University and their similar maritime educational background and age. The second year, third year and fourth year students at the Linnaeus University were not included in this study. Further the requirements were only first year student with an approved basic safety training certificate (BST). There are always room for including more participants however based on the time restriction and available resources this study was conducted by a total of 24 participants. Another aspect included that the participants previous fire extinguisher training was conducted at the same training facility. Other limitations were available ships at the port of Kalmar. The M/V Calmare Nyckel were chosen due to its geographical location by the Linnaeus University. In this study, only the three levels in Kirkpatrick model have been considered and presented. This study does not utilize Kirkpatrick model level four. However, Kirkpatrick model level three which concerns behaviour has been addressed but additional time and resources are needed for a more comprehensive result. Economical calculations between different virtual fire extinguisher training solutions will not be reviewed in this study. Further there are not any comparison between different VR-manufacturers hardware specifications and their different performance. 5 2. Frame of reference In this frame of reference chapter, the necessary literature connected with the subject and research questions are presented. Subjects included are, the incipient stage in firefighting, technical fire prevention onboard and its limitations, psychological stress and constrains in firefighting, fire extinguisher training on-board, virtual reality training, virtual reality-fidelity, perception of geometric layouts created in a virtual environment, educating and training in a safe virtual environment, potential risk of frequent training in engaging virtual environments and fire extinguisher training in virtual reality. 2.1 The incipient stage in firefighting Within minutes, an incipient fire can develop into a life-threatening scenario. Consequently, it should be extinct as soon as possible (Thielsch et al., 2021). A recognition skill is the ability to rapidly size up a situation and have the knowledge to act accordingly. A firefighter must be able to cope with recognition in an emergency. Recognition skills are normally refined at the job via long time experience. These contexts are difficult to address in traditional training. Experienced persons are often able to quickly recognise a situation as familiar and based on that recognition generate an effective course of action. Further in many cases the person is not aware that they have decided anything at all (Militello et al., 2023). For example, Militello et al. (2023) address the awareness of the fireground commander. The commander described an incident in which he arrives on the scene of a fire and immediately recognised it as a fire in the kitchen of a single- family house. He had seen this type of fire many times before and it was instantly recognizable. Based on his rapid assessment of the situation he ordered his crew to enter directly through the front of the house and extinct the ongoing fire in the kitchen. Because of this rapid reaction they were able to save much of the structure of the house. Further the fireground commander did not recall making a decision when he was told to explain his actions (Militello et al., 2023). To quickly recognise a dangerous situation and act accordingly are also highly desirable for crewmembers onboard a ship. Santarpia et al. (2019) argues that an assessment of the situation must be done by the first crewmember on the scene. This includes proceeding with the correct fire extinguisher to the possible location of the origin fire. Further the use of correct fire extinguishers together with the use of the fire and safety plan. However, depending on the actual location of the fire and fire sensors technical standard an incipient fire could be difficult to handle with a fire extinguisher. According to Santos & Son (2024) there have been little research efforts to understand what indication and information firefighters use to assess ongoing situations and predict future dangerous conditions. The firefighter needs to possess the ability to assess a new situation rapidly and accurate generate new possible courses of action. 2.1.1 Technical fire prevention onboard and its limitations According to Statens räddningsverk, Fartygsbrandsläckning (1994) the fire prevention onboard a ship is primarily structured upon its automatic fire alarm system. An automatic fire alarm system is placed onboard the bridge with its connected fire detectors designated at different sections of the ship. The detectors are connected to a display, visible for crew members. In case of a fire alarm, the system should directly release the electromagnetic door holders on the fire 6 doors inside the ship. The electromagnetic door holders’ purpose is to prevent smoke and fire from spreading to new sections in the ship. The onboard fire detection is also based on manual fire alarm activation through a call point at various places onboard. Further in new installation of fire alarm system, the location of each fire detector could be displayed at the bridge. This modern system has the advantage of a precise fire detection in the ship. However not all ships have this location fire detection system but is based on an alarm that indicates fire in an entire sector of the ship (Statens räddningsverk, Fartygsbrandsläckning 1994). Another problem arises with the specific type of fire detectors installed onboard. As described by Li et al. (2023) the fire detection methods are mainly physical and based on smoke, thermal and flames. However, these types of detectors could cause false alarms and the threshold for trigger the fire-alarms are difficult to set. The response time of the individual detector depends on the sampling rate of the discrete signals. This results that a fire detection could be delayed due to technical limitations of the fire detector. Thielsch et al. (2021) describes that many fires can be discovered at an early stage thanks to modern smoke detection systems. The problem arises how far offset the origin of the fire are in relation to the distance from the nearest fire detector. This additional time error due to either technical limitation of the detector reduces the available time of suppression of the fire with the fire extinguisher. This could also lead to an increased risk of a fire developing out of control. Thielsch et al. (2021) further analysed that a dangerous full fire develops quickly and in modern buildings a flashover could occur in less than five minutes. Further fire gases and dangerous smoke are also a direct danger for the person before the typical flashover period. The time of the flashover point are depending on the specific fire load and ventilation. In many modern apartments the flashover and lethal concentrations of fire gases could occur in 3-5 minutes (Kerber, 2012). A problem described by Kerber (2012) are that modern apartments and houses consist of synthetic and modern material that react differently from traditional wood or other natural materials. Nermin (2023) describes that ships and especially passenger ships carry large amount of textiles as beddings, furnishing fabrics and floorcoverings. Due to this large amount of flammable textiles there are a high potential of an increasing development of a fire. The fire threat increases if use of a low flammability and flame protective equipment is used onboard. Nermin (2023) also analyses that the flame resistance standard and regulations need to be high onboard ships due to escape restrictions, narrow corridors and low ceilings that causes panic to many passengers in a fire accident. The textiles in marine applications need to have a define level of fire resistance and is set by IMO Resolution A.471(XII) and SOLAS 2010 FTP code. Nermin (2023) also describes that many products are after-treated by solutions and that those kinds of treatments are semi-durable. Exactly how long the time for flashover or lethal concentrates of gases onboard a specific type of ship accommodation or area is varying depend on both structure and material being used. However, the first minutes of an incipient fire are the most crucial for effective use with a fire extinguisher. In some cases, there will only be a short available time period for first responder to assess and extinguish the ongoing incipient fire. This is also determined by Thielsch et al. (2021) who describes that people on site should assess the situation and if possible, without endangering themselves, attempt to extinguish the fire on their own before it have developed to a life-threatening full fire scenario. 7 2.1.2 Psychological stress and constrains in fire fighting Another factor arises when crewmembers encounter a real fire. Some persons are more vulnerable to interact and act accordingly during a real fire scenario. Some factors that influence the performance negatively are caused by educational under-achievements (Oviedo- Trespalacios et al., 2013). Bearman & Bremner (2013) describes that social psychological pressure and pressure from strong situations seems relevant to firefighters in their profession. Strong pressures are situations that constrains the person’s perceived options, such as making a poor decisions become more likely. Another reaction during the acute phase of a fire can be of a psychological nature that involves severe agitation, depression, and stress. Reale et al. (2023) analyses how trained professionals make naturalistic decisions under pressure and identified five themes. These vital themes are the decision-making strategy, time pressure analysis, stress, uncertainty, and errors. According to McLennan & Omodei (1996) the decision strategy for a fire officer is based on a recognition-primed decision model. The fire officer uses and visualise cues to recognize an emergency as an instance of a class of familiar problems. Further the fire officer used any available prior knowledge to simulate the likely situation and consequent actions. Then the fire officer compares what he sees at the location with his pre-primed simulations. If there is a match, initial actions are taken based on his already considered possibilities for the action. McLennan & Omodei (1996) also describes that precognition simulations can explain how sound decisions can be made very rapidly by experienced decisionmakers. An important study was conducted by Thielsch et al., (2021) regarding persons without any professional firefighting training that were confronted with a real incipient fire. Their results showed the importance of training and reacting accordingly to an incipient fire at a stage when it is not too dangerous. Further there are positive effect of even simple fire training. According to Thielsch et al. (2021) the participants skills needed to extinct a fire was mainly calmness and an overview of the situation. Further basic fire-extinguishing knowledge, risk assessment skills and self-confidence. Thielsch et al. (2021) results are in line with Reale et al. (2023) conclusion regarding that time pressure, stress, uncertainly are major factors in decision making during crisis events. In training a skilled professional must adapt the decision process depending on the time available, the type and level of uncertainly and finally their own experience and skill level. 2.1.3 Fire extinguisher training onboard Minimum training requirements are covered by international conventions and regulations covered by IMO. According to International Convention for the Safety of Life at Sea, SOLAS (2020) Chapter III: Part B –Regulation 19, every crew member shall participate in at least one abandon ship drill and one fire drill every month. The drills of the crew shall take place within 24 h of the ship leaving a port if more than 25% of the crew have not participated in abandon ship and fire drills on board that particular ship in the previous month. According to International Convention for the Safety of Life at Sea, SOLAS (2020) Chapter II-2 - Part E - Regulation 15, crew members shall be trained to be familiar with the arrangements of the ship as well as the location and operation of any fire-fighting systems and appliances that they may be called upon to use. This direct regulations by IMO shows the importance of safe operation of both fixed and portable fire extinguishers onboard. Scorgie et al. (2023) analyses that VR- fire extinguisher training includes actions, and proper utilization of the equipment as the main 8 training outcome after a performed VR-session. Chittaro & Buttussi (2015) also explain that professionals working in these fields benefit to gain, refresh, and maintain knowledge and skills necessary for an accurate operation. Finally, according to Thielsch et al. (2021) a too long time- period between fire extinguisher training might in the worst-case result in serios danger for the participant. This due to the participant having a higher intention to extinguish a fire, but then potentially endanger them self by an incorrect behaviour. 2.2 Virtual Reality Training As a possible professional training solution, virtual reality (VR) has become more widely used in sectors as military, aviation, medical and firefighting education (Buttussi & Chittaro, 2018). According to Narciso et al. (2021), training is a vital process for any organisation around the world regardless of size, as they all have a constant need to transfer knowledge and skills to both existing and new employees. Further other traditional training methods possess several limitations regarding engagement. According to Stefan et al. (2023) VR technology present an important opportunity to improve the effectiveness of safety-relevant training due to its increased level of presence and the ability for the user to fail safely or make errors in a safe environment. Further allowing the user to a training design that is related to different scenarios that are difficult to replicate in the real world. Example of this training design is based on economical constraints or safety-based concerns (Stefan et al., 2023). This includes too expensive or dangerous fire scenarios. Another example of VR-training related to the possibility of repeatability. This includes that the conditions do not change in a virtual environment between the participants training sessions. Further there are no limit on the number of training sessions that can be experienced by the participant (Narciso et al., (2021). According to Mathysen (2021) VR has already existed for decades but today VR is still very much a technology under development. This is due to the large investments from large tech-companies as Google, HTC, Microsoft, Samsung, and Facebook. At the present time Mathysen (2021) explains that VR has reached a point where it can offer photorealistic experiences and in the meantime being consumer friendly and relative affordable. Stefan et al. (2023) also describes that besides the reduction of price, improvement has also been made in the usability and ergonomics of VR-headsets. This includes reduction in weight of the VR-system, simpler user setup and visual graphic with a higher refresh rate and a wider field of view. Another advantage with VR-training is its access on demand, reducing any scheduling issues with the use of just- in-time training. Another aspect of VR-training consists of the specific training sessions that are delivered through the VR-system, removing any discrepancies that may exist between different instructors and their individual human factors (Stefan et al., 2023). 2.2.1 Virtual Reality-fidelity Buttussi & Chittaro (2018) refers to Ragan et al. (2015) who identifies three types of VR- fidelity. The first one is interaction fidelity that relates to the realism and the reproduction of real-word interaction. LaValle (2023) describes the importance of the quality of the realistic VR-simulated environments and that it could have an impact on the learning and training outcome. The second display fidelity relates to the realism of the output devices and their reproduction of sensory stimuli. Another word for display fidelity is also referred as 9 “immersion”. For example, a highly immersive VR-system such as the head-mounted display (HDM) allows its user to perceive visual and auditory information. This “immersion” includes natural actions such as reaching a hand to touch objects and turning the head to change viewpoint in a virtual environment. Furthermore, in comparison, a low immersive VR-system such as a desktop VR, requires unnatural actions such as using a computer mouse and keyboard to observe the same information. The third type are scenario fidelity that relates to the realism of the simulated scenario and reproduction of behaviours and properties in the simulation (Stefan et al., 2023). 2.2.2 Perception of geometric layouts created in a virtual environment Another positive educational aspect regarding training in VR relates to how the user can get familiar with its surroundings created in a virtual environment. Training in a virtual environment can be used to help people acquire spatial knowledge about real-world places and navigate those places more effectively and safely. For example, printed evacuation and safety plans used in airplanes provide only an abstract diagrammatic representation that can present comprehension problems for the passengers. One of these problems consist of a cognitive complex geometric operation to translate the knowledge provided by the map into actual routes in the environment (Burigat & Chittaro, 2016). For increasing the learning results, subjective usability such as enjoyment might have a significant impact on the training and its acceptance. Serious gaming and the important sense of being in a virtual environment can play a significant role of gaining and application of procedural knowledge (Buttussi & Chittaro, 2018). However, VR- serious gaming is relatively a new genre. From the year 2016, the start of a new wave of VR-technology signified by the commercial release of VR head-mounted-displays. This includes VR-system as Ocuclus Rift in March 2016 and HTC Vive in April 2016. Under this wave of VR-system there were an observed increase of interest in VR-technology from both public and academic domains (Stefan et al., 2023). 2.2.3 Educating and training in a safe virtual environment According to Chen & Chien (2022) behaviour skill training is one of the effective methods to obtain natural responses to corresponding emergency behaviours. Further in educational skill training, the trainees interact more actively and could make mistakes in a comfortable way in dangerous situations. Behaviour skills training is maintained due to activities in environment through demonstrations and feedback. Chen & Chien (2022) also argue that appropriate safe behaviours of participants are key elements for increasing the chance of survival. VR-training could also be used to train cognitive skills as the ability to build situational awareness to act accordingly in an emergency situation. Militello et al. (2023) analyses the concept of sizing up a situation and acting requires recognition-primed decision making. This action is often taken with specific expectations about how different actions will influence the situation. Decision- makers continues to assess the situation and its possible outcomes based on their previous decisions and refine their understanding of the situation. The decisions-makers assessment is in many ways a hypothesis that can be tested by taking new actions and observing the results. The goal of recognition training is to help participants to recognize crucial and dangerous patterns for a quick intervene with corresponding appropriate actions. Further Militello et al. (2023) argue that the effective simulated training designs should extend naturally from the environment that the participant normally works. 10 Tusher et al. (2024) describes that VR used in professional education had a pronounced emphasis on skill-based learning outcomes. Çakiroğlu & Gökoğlu (2019) determine that fire safety training involves the adoption of VR-scenarios that could be created for real-life conditions that involves both life threat and material damages. A scenario-based training refers to training that placed the participant in situations they must assess and manage. These scenarios have dedicated learning objectives. Further learning objectives are associated with recognition skills and specific tasks in the simulation environment (Militello et al., 2023). A study regarding fire and safety behaviour was carried out by Çakiroğlu & Gökoğlu (2019). Their study was based on results from 10 participants who previously had an VR-fire safety training with mentors and teachers and then were assigned a real fire drill. The residential fire drill was created by observing the participants. Çakiroğlu & Gökoğlu (2019) result showed that 6 of 10 participant could transfer the skills that they had previous trained in virtual environment to a real-life condition completely and correctly. Çakiroğlu & Gökoğlu (2019) results indicates that behaviour skills acquired in a virtual reality environment could be transferred into real-life. Further Buttussi & Chittaro (2018) describes that emotional intensity caused by experiences can increase the upholding of memory allowing for a better recall. As the virtual environments become closer to reality, the perceived presence and learning could increase due to a higher degree of immersion. Buttussi & Chittaro (2018) results are in line with Engelbrecht et al. (2019) that there are promising findings that relate to VR and its transferable skills for creating affective and multi-sensory experiences. Finally, in all education, feedback is important. Hedin (2006) analyses that feedback is a receipt of present knowledge and possible knowledge gap. Militello et al. (2023); LaValle (2023) analyses the important of feedback and debriefing to encourage participants to training reflection. 2.2.4 Potential risk of frequent training in engaging virtual environments Engelbrecht et al. (2019) also addresses that more research is needed to establish methods of skill transfer from virtual reality to real life scenarios. Further evaluate potential risk of frequent training in engaging physiologically stimulating virtual environments. Engelbrecht, et al. (2019) describes that the worst-case scenario would be the implementation of unproven VR- training, which does (on the face of it) train valuable skills, but results in worse overall training outcomes due to limited skill transfer to reality. Scorgie et al. (2023) also argues that only a few studies have enclosed multi training outcomes combined with hazard identification and safety training. Chittaro & Buttussi (2015) analyses that many research studies have been published based on knowledge tests immediately after the VR-experience, without assessing the knowledge over a longer time period. VR-fire training could be a false safety barrier for reality due to lowered perception of threats. According to Santarpia et al. (2019) most victims die of the asphyxiation and intoxication due to a high concentration of lethal products of the combustion. Further the hot gases and temperature causes burns, eye irritations and severely creates a difficult environment, even if the person was not directly exposed to the flames. The smoke and its different colour make it difficult for the person to escape and for the rescue personnel to assist in the situation. Haight (2023) analyses that from a life safety perspective, the concerns are not about to which extent a person are in direct danger of death, but instead when the person becomes sub-lethal or incapacitated by the fire gases and its effects. Secondly a person without professional or specialized knowledge in fire extinguishers must correctly 11 learn how to make a risk assessment before approaching an incipient fire. Militello et al. (2023) also describes that the participant is required after the simulated VR-training to interact and operate in a real-world dynamic, high-stake environment and therefore it is important that the education maintains the desired level. One of the problems arises if the VR-training scenarios does not represent the real task problem. Further persons who try VR for their first time are unaware of technical flaws that would be obvious to some experienced engineers and developers (LaValle, 2023). 2.3.5 Fire extinguisher training in virtual reality The VR-technology makes it possible for the user to train with fire extinguishers in a virtual environment. The fire extinguisher training includes the safe use and practice with different handheld types of extinguishers. Research in the area has shown that the number of trials and repetition with the fire extinguisher has an overwhelming factor on the final performance and outcome of the fire scenario. This includes an improved time ratio for the user related to practice and experience with the fire extinguisher (Oviedo-Trespalacios et al., 2013). Kriger (2002) also analyses a larger dimension of training performance, namely the ability to imitate demonstrated behaviours, in other words to perform a fluid performance with few errors after continued repetition and practice. This is also in accordance with Alvarez et al. (2004) research regarding that training performance is the ability to perform a newly acquired skill and is measured through an observable demonstration. Oviedo-Trespalacios et al. (2013) analyses the accurate number of trials in which afterwards an individual has an acceptable level of performance with a real fire extinguisher. Their results showed 5-10 attempts with the fire extinguisher for a required acceptable level of performance. Oviedo-Trespalacios et al. (2013) result shows the importance of repetition training with the hand-on use of the fire extinguisher. 12 3. Method In this method chapter, the chosen approaches and method are described. Further the chapter provide an explanation of how the research has been conducted and how information was collected. The research approach, data collection methods, data analysis and a discussion of research quality are presented. 3.1 Research approach Despite the increased research following the effectiveness of VR-training, the research remains limited. As described by Narciso et al. (2021) evaluations supporting the effectiveness of VR is either limited or inferior in quality. Examples are a limited number of participants or questionable research design. This conclusion is also described by Buttussi & Chittaro (2018) that concludes that even with new improved VR-technology, more research is needed to understand VR-fidelity and its impact on the effectiveness. Stefan et al. (2023) describes that although the consensus of the prospect of VR for training is positive any perceived benefits need to be validated. Furthermore, Stefan et al. (2023) describes that less rigorous evaluation of the VR has been established in maritime and offshore domains and the utilization of VR- training in these fields is still uncommon and largely exploratory. Learning in virtual reality environment is being explored as an alternative to present learning however, evaluating the learning outcomes are a challenge for VR-training (Tusher et al., 2024; Menin et al., 2022). Further Narciso et al. (2024) argue that there is a gap between literature concerning the comparison of a training in a virtual environment (VE) with the same training in a Real Environment (RE). This master thesis and its research design are focusing on the problems with research regarding VR as both stated by Narciso et al. (2024); Tusher et al. (2024); Menin et al. (2022). To further answer the research questions for this study an abductive research approach was used (Kovacs & Spens, 2006). An abductive research approach can explain, develop, or change the theoretical frame of reference during or after the research progress (Dubois & Gadde, 2002). The theoretical frame of reference used in the master thesis was based on a systematic literature review. The research begins with an exploratory study with the purpose of summarizing the problem regarding fire extinguisher training. 13 According to Denscombe (2009) method triangulation is the combination of two or more methods to investigate the research questions. The method of triangulation were used during the research and consisted of a literature review, questionnaire and a case study observation. The method of triangulation is shown in in figure 1. Figure 1 Method Triangulation Note. Adopted from Denscombe, (2009), p. 188. 3.2 Data collection methods The qualitative research method is an umbrella term covering different research traditions which have in common a reliance on the collection of qualitative data (Robson, 2007). Further Stenbacka (2001) describes qualitative methods as seek for a quality that is typical for a phenomenon or that makes the phenomenon differ from others. 3.2.1 Literature review In this thesis a systematic literature review (SLR) on VR safety training was used to develop the frame of reference as suggested by Gessler & Siemer (2020). The main purpose of a literature review is selecting different texts, concepts and theories, arguments relevant to the research frame of reference and use of a particular method consisting of the steps: 1. Definition of the scope (Specify the research problem or research questions). 2. Data selection (Systematic search, define the sources and search terms, finally read the selected research, and exclude unsuitable research). 3. Data processing (Asses the quality of the selected research to exclude unsuitable research, further select the relevant data from different texts and critically evaluate the extraction and synthesise the data). 4. Data reporting (Present the findings and discuss the conclusions and limitations of the review). (Gessler & Siemer, 2020) Literature Review Questionnaire Results Case Study Observation 14 Following Gessler & Siemer (2020) guidelines this master thesis utilized a backward snowball approach in conducting the SLR. Backwards snowballing was used with focus on the reference list of previous VR-studies. A recently published VR-article lead to other articles and their reference lists. Wohlin (2014) describes that the efficient of systematic literature reviews depend on the number of included papers and articles in relation to the total number of candidate paper examined. Included in this study were review of literature and articles from primarily the database Scopus. Wohlin (2014) arguments that one of the main advantages of snowballing is that if it starts of relevant papers and then uses these to drive the study further. When performing a literature review, mainly in the database Scopus, three articles were the start set for this research, named candidate, C1-C2-C3 (figure 2). Due to the many search results of articles, the selection process of C1-C3 articles is based on different inclusion and exclusion criteria. • C1. Stefan, H., Mortimer, M. & Horan, B. (2023). Evaluating the effectiveness of virtual reality for safety-relevant training: A systematic review. Virtual Reality 27, 2839–2869 https://doi.org/10.1007/s10055-023-00843-7 • C2. Scorgie, Dorothy & Feng, Zhenan & Paes, Daniel & Parisi, Fabio & Yiu, Kenneth Tak Wing & Lovreglio, Ruggiero. (2023). Virtual Reality for Safety Training: A Systematic Literature Review and Meta-Analysis. Safety Science. 10.1016/j.ssci.2023.106372. https://doi.org/10.1016/j.ssci.2023.106372 • C3. Paweł Strojny, Natalia Dużmańska-Misiarczyk. (2023). Measuring the effectiveness of virtual training: A systematic review, Computers & Education: X Reality, Volume 2, https://doi.org/10.1016/j.cexr.2022.100006. Selection process of articles The literature search progress was performed during the months of November-December 2023 and covered one interdisciplinary database. (Scopus). This research focus on previous studies that performed effectiveness evaluations on VR-safety training and research based on a systematic literature review. REF REF REF REF REF REF REF REF REF C1 C2 C3 Figure 2 Main articles, named candidate, C1-C2-C3. https://doi.org/10.1007/s10055-023-00843-7 https://doi.org/10.1016/j.ssci.2023.106372 https://doi.org/10.1016/j.cexr.2022.100006 15 STEP 1 Inclusion criteria of articles and search strategy In step 1, the total number of articles were reviewed by its relevance. By reading the title, article info and its abstract, an overview of the article were created. Further the sample and respondent size of the article were then analysed for the validity. Finally, the assessing of the data’s credibility, i.e., the data were accurate and unbiased. Search keywords used in Scopus consisted of terms as, Virtual Reality, Virtual Environment, VR-safety training, VR-fire extinguisher training, Immersive virtual reality, and Kirkpatrick’s Levels of Training Criteria. Many articles, conference papers and reviews were presented in Scopus based on the keywords. To narrow down and reduce the possible number of total document articles were chosen based on mainly their comprehensive reference list and summarizing method of data. This included a meta-analysis of Virtual Reality for safety training to systematically assess the result of previous related articles. The other two articles were evaluating the effectiveness of VR-safety training by a systematic review. A systematic review is often ranked high in the academic research due to that they compile the results from all original studies within a subject. As the research progressed, articles were selected beyond the reference list from the three main articles C1-C3 were based on their scientific value as detailed information that couldn’t not been forsaken for the outcome of this research. This includes method and educational literature and articles related to specific fire extinguisher training. STEP 2 Exclusion criteria of articles In step 2, documents and literature were excluded in the literature review based on publishing date. The requirements were that the three main candidate, C1-C3 was published within 2023. (Topicality). This because a summation of the problem area in real time were required. Had the articles been published previous before 2023 their meta-analysis and systematic review would not have been as detailed. Further excluded in this research were articles that were not published in English, about a few percent of the articles related to VR were written in other languages. Scientific journals that were not published in Scopus were not included in this research. 3.2.2 Case Study Case studies are preferred when relevant behaviours still cannot be manipulated and when the desire is to study contemporary events or a set of events (Yin 2018). Robson (2007) describes a case study as something that throw light on wider issues, parallels, or possible similarities. Further Hammond et al., (2021) analyses that some case studies take in a very large number of cases in order to generalise and is called large-n studies. The idea of casing involves systematically selecting examples of phenomenon. As mentioned by Hammond et al. (2021) a smaller case study has the advantage of in-depth exploration by using qualitative methods. Case studies can have different purposes, Yin (2018) distinguishes between critical cases that might challenge prevailing research and the unique case that could prove new events or phenomena. The case study in this study used a fire drill at the vessel accommodation. The layout onboard the ship was chosen due to its realistic background and known source of reported fires internally across the merchant navy. The International Marine Contractors Association (IMCA) has made 16 safety issues in 2016-2018 regarding fire in accommodation with focus on electronic items in the crew cabin. Those reports include overheating notebook computer, laptop battery fire and mobile phone charger failures. To conduct an electrical fire drill scenario as realistic as possible, previous accidents were set as reference and the layout of the exercise. The case study focused on a crew cabin consisted of a work desktop. The desktop computer screen was set as a primary source of the electrical fire. The scenario was combined with smoke-generator in the cabin. A smoke-generator were used to lightly fill the cabin with non-toxic smoke. The door to the cabin were closed to ensure records of the participant behaviour for determining the cause of the fire when investigating the accommodation area. Further electrical outlet and electrical cords were visible ahead of the fire to enhance the information that this were an electrical fire. The overall process used for the data collection is described in figure 3. Figure 3 Process used for the data collection. Direct observations of participants As with all research, the researcher has the responsibility of protecting all participants from potential harmful consequences that could affect them because of their participation in this study. Robson (2007) argue that the researcher has a high level of responsibility for its participants. This includes the possible results from the study and its possible impacts on the participants after the research from different stakeholders. To ensure a high level of transparency and ethics, the participant in this study has signed and understood the document of consent before participating in the research. It was important that the document of consent were signed and witness by other participants and me as researcher. The document was then saved and archived. Warm up, Familirization •VR-experimental group, participants received Virtual Reality familiarization Direct Observation •VR-experimental group, participants completed three individual fire extinguisher training sessions in Virtual Reality Direct Observation •Participants of the VR-experimental group completed individually the fire drill. •Participants of the controllgroup completed individually the same fire drill. Questionnaire •Participants of the VR-experimental group filled out the questionnaire. 17 Fangen (2005) exemplifies observations as one of the most central methods in social research. The method includes collection of data by studying situations that people interact and behave in certain situations. Patel & Davidson (1991) also argue that in the everyday life, observations are one of the primary methods for data collection of the outside world. Further Yin (2018) describes that direct observations take place in the real word, setting the purpose of the case study. Another aspect is that observational evidence is often useful in providing additional information about the topic being studied. Similar observations can add new dimensions for understanding actual uses of new technology and problems being encountered (Yin, 2018). Researchers also collect observation data more or less randomly and from our own experiences, needs and expectations. However, in the scientific research, observations cannot be randomly but should follow academic and research methods (Patel & Davidson 1991). Observations are also useful in data collection regarding natural situations and behaviour differences (Patel & Davidson 1991). By addressing observations as one of the methods in this study, Patel & Davidson (1991) argue that researchers who are interested in crisis behaviour could not follow a participant and wait for an actual crisis to develop. This could take years and would be non- economical and time consuming. To shorten the time needed and narrow down the observation method, this study used a case study based on an onboard fire drill. In this study a direct observation method was performed on a VR-experimental group and a control group of participants. They were all assigned at the same fire drill onboard a ship. The experimental group had previous VR-fire extinguisher training sessions and the control group had not completed this VR-training session. Yin (2018) applies that observation method provides certain opportunities for collecting case study data. The study was conducted at Kalmar Rescue Training Centre at Linnaeus University. A total of N= 24 students (21 male, 3 female, age: M = 22,1) were assigned in this research. Participant selection was based on a general rule regarding its external validity. In order to make a generalisation of the research result, the participants were not based on randomization but based on inclusion criteria. One of the main inclusion criteria were that the participants needed to have a background as maritime officer cadets, year one, with previous approved basic safety training certificate. The concept of extern validity of this research was that the participants and the situation studied should be similar to the possible situations that the participants encounter onboard their ships. (Mundane Realism). Participants were self-assigned and volunteer to the two groups in such a way that each group had the minimum of 10 participant each. This were made by the students E-mail for respective scheduled times available. In this case study the participant were assigned randomly without the influence of me as researcher to determine or affecting the placement of participant in the two groups. As described by Stefan et al. 2023, a true-experimental design includes studies that have two or more independent groups, and the participants allocation is randomized. When the allocation is not randomized it is called a quasi- experimental design. A true-experimental design is the ideal research design for maximizing the internal validity. In this study a true-experimental design was conducted to prove or disprove a cause-and-effect relationship between two variables. In this case the VR experimental group compared to the control group. Validity of this study depends on the well-chosen participants. As seen in table 1, there were a total of N=13 in the VR experimental group and a total of N=11 in the control group. 18 Table 1 Participant age and gender based on VR experimental group A and control group B. VR-experimental group A Age Gender Control group B Age Gender P1 35 Male P14 19 Female P2 35 Male P15 20 Female P3 19 Male P16 20 Male P4 20 Male P17 21 Male P5 25 Male P18 22 Male P6 25 Male P19 25 Male P7 20 Male P20 20 Male P8 20 Male P21 20 Male P9 19 Male P22 23 Male P10 20 Male P23 20 Male P11 24 Female P24 20 Male P12 19 Male P13 20 Male Age distrubution 19-35 Age distrubution 19-25 Average age 23 Average age 21 VR-Equipment and Software In the Virtual reality-condition, the study used the equipment HTC Vive Focus 3, combined with DAFO Virtual Fire extinguisher system (https://www.dafo.se/vr-utbildning/). The system was purchased by the manufacturer as a complete kit in a waterproof case for use anywhere and anytime. The HTC Vive Focus 3, were able to charge within a few hours for just-in-time- training and had 5K resolution, wide 120˚ field of view, and up to 90Hz refresh rate. The VR application from the HTC Vive Focus 3 were streamed by Miracast 4K device to Windows 10 operating system and its compatible projectors and monitors. One of the VR-systems primary features where the physical fire extinguisher. Both the HTC Vive Focus 3 hand-controllers were fitted on the external fire extinguisher. Applications on the VR-fire extinguisher as handle, safety pin, operating handle, and discharge nozzle were identical to real existing fire extinguishers. The technical differences between the DAFO VR-system and real extinguishers where dimensions, possible weight differences and the specified nozzle. During the simulated VR-training sessions, the nozzle changed for each extinguisher in the computer-generated virtual environment. Example of different nozzles and fire extinguisher weight differences are seen in figure 4. https://www.dafo.se/vr-utbildning/ 19 Figure 4 Fire extinguishers weight and nozzles differences compared to DAFO VR-system Note. Photograph taken by author. 3.3 Data analysis Yin (2018) recommends collecting and present and analyse the data fairly and compare it with compelling articles, reports, and literature. Yin (2018) describes that research should press for a high-quality analysis and strategy. At least four principles underlie all good social science research. First the analysis should attend all the evidence and cover the key research questions. The analyse should show as much evidence as was available, and the interpretations should account for all the evidence. Second the analyse should investigate all plausible rival interpretations. Thirdly the analyse should address the most significant aspect of the case study. Finally, the fourth principle, the analysis strategy should demonstrate a familiarity with the prevailing research about the case study topic (Yin 2018). In the analysis of observations and interviews Kirkpatrick’s model was used in this thesis. Stefan et al. (2023) has presented results of 188 studies that were evaluating the effectiveness of virtual reality for safety‑relevant training regarding Kirkpatrick’s model for training evaluation level 1 (reaction) and level 2 (learning). Very few VR-studies evaluated level 3 (behaviour) and level 4 (results). To further understand Kirkpatrick’s model for training evaluation Stefan et al. (2023) describes training effectiveness as the desired result depending on what is being trained. Kirkpatrick divided the process of evaluating the effectiveness of training into achievable measures in his four-level model. Stefan et al. (2023) further describes that there are four levels in the model. At level (1) the trainees’ attitudes are measured towards the training. At level (2) the measure of learning determines the level of knowledge and skills that had been acquired. At level (3) the measure of behaviour determines the level of change in trainees’ related behaviour after the training. Finally at the last level (4) the measure of results 20 relates to noticeable improvements, such as a reduced number of safety incidents, increased revenue, or reduced operational costs. See figure 5 for the Kirkpatrick’s model for training evaluation. Note. Adopted from Stefan et al., (2023), p 2. Stefan et al. (2023) further describes that the Kirkpatrick model is based on a fixed hierarchy. This indicates that the previous levels are focus on and measured before advancing to the next level. This is based on the concept that higher-level measurements are not expected to change if the lower level of the model have not been satisfactory (Salas et al., 2012). Further there are two primary strategies for increasing the impact of training evaluation. The first strategy is to clarify the purpose of the evaluation and to specify what and how to evaluate training. Salas et al. (2012) describe the outcomes as a) To make a decision of the training, whether a new training should be kept or eliminated. b) By provide feedback to trainees, trainers, or training designers. c) To market training outcomes, future trainees, or new organizations. The second strategy is related to how training outcomes are based on assessment. To evaluate training outcomes as reactions, learning, behaviour and results, there are requirements of cognitive and behaviour indicators for the intended learning outcome (Salas et al., 2012). This research has analysed the Kirkpatrick level 1, the measure of reaction and attitudes toward the VR-training by the answers from the participant in the questionnaire. To advance upwards in the Kirkpatrick’s hierarchy at level 2 and 3, observations has been performed during the case study fire drill. The last level of Kirkpatrick’s hierarchy relates to fewer safety accidents and less operational costs. Those aspects are not measured in this research but is suggested to further research questions. Results Behaviour Learning Reaction Attitudes of trainees towards training Knowledge and skills acquisition Changes in the job behaviour of trainees after training Noticeable improvements as a result of the training such as reduced number of accidents. Level 1 Level 2 Level 3 Level 4 Figure 5 Kirkpatrick’s four-level model of training evaluation 21 Kirkpatrick’s hierarchy at level 2 focus on learning outcomes. This includes the measure of increased knowledge or intellectual capacity. The study implemented an observational schedule in order to validate each individual participant. This observational schedule was also used for analysing the VR-experimental group against the control group. At Kirkpatrick’s hierarchy level 3, behaviour evaluation is the extent to which the trainee applied the learning and changed their behaviour. This study has used behaviour markers to identify the student’s situational awareness during the case study fire drill. Behaviour evaluation is the extent to which the participant applied the learning and changed their behaviour. Further if there was noticeable and measurable behaviour change in the activity during the case study. All participants of both the VR-experimental group and control group were analysed by the same assessment matrix. Effective strategies to protect personal data in this research included a safe storge of data on two personal physical hard drive disks. None of the data was available at servers abroad other countries. Removal of identifiers of the participants as names were conducted and replaced with numbers. After this research had been conducted, all recordings and data were erased to ensure future confidentially of the participants. Thematic data analysis The empirical data in this thesis was analysed using thematic analyse method for coding and interpretation. The thematic process includes raw data and the method for draw meaningful insights. By using inductive coding further new ideas and concepts were analyses as inductive coding has nonpredetermined set of codes. Inductive coding includes a process of applying tags, names, or labels to item of data. It is often applied on quantitative data, in particular unstructured data. There are many approaches to coding depending on the research focus, however its typically begins with a simple descriptive label to summarize the meaning of a unit (Hammond et al., (2021). The coding of this qualitative study began with a production of conceptual headlines that “opened up” the data to identify properties as the participants safe operation of the extinguisher or their own perceptions of a specific task. The initial coding was subjected to sub-themes, due to the ongoing abductive research approach. Then data from the previous literature review combined with the questionnaire were coded to fill conceptual gaps in the study. Semi structured questionnaires were analysed to obtain data and empirical findings regarding Kirkpatrick level 1, these includes the participants own reflection of their VR-training and performed case study. Further the questionnaire had the purpose of evaluate how the participant experienced their own VR-training and if they enjoyed the training session. The questionnaire was analysed for collecting data on the participant own reflection regarding the potential practicality of the VR-training. The method of coding relied on remarks and coloured highlights the segregate the data between different categories. As the analyse proceeded, data sources involve the backward snowballing from the three main articles C1-C3. With an abductive research approach regarding behaviour and learning outcomes, the case study observations were analysed by an observation schedule. The observation records were analysed to obtain data and empirical findings regarding Kirkpatrick level 2 and level 3. 22 3.4 Research Quality 3.4.1 Validity According to Robson, (2007); Stenbacka (2001) validity refers to whether something measures what it claims to measure. Robson (2007) argue that a measure can be reliable without being valid. Stenbacka (2001) describes that the validity of the data and if the “data is good” first and foremost depends on the purpose of the study. In this study a total of 24 participants were assigned a case study fire drill. The participants background as maritime officer cadets were representative for this study due their previous same basic safety training and former experience. All participants were also first year’s students from the same two classes with a relative same age. If the participants had been assigned the case study fire drill from different classes and stages in their own education level, the validity could have been lowered due to the participants different prior knowledge and experiences. The interaction between the researcher and the participant leads to circumstances and the possibility of getting good data. In other words, providing the information needed according to the purpose of the research. Stenbacka (2001) describes that to understand if a phenomenon is valid, the informant is a part of the problem area and if he/she is given the opportunity to speak freely according to his/her own knowledge is vital. It is also important during the research progress that the data is analysed with no bias or prejudice that could change the outcome of the results. Further the participant was based on self-randomization during the case study into the two groups. If I as researcher had established all students in one of the groups by my own opinion, there had been a low internal validity and the possible for manipulation through the research results. This study has always been focussing on a high validity and to increase the validity of the observations, all participants were assigned with the same information before the commence of the case study fire drill. This included the question from the researcher if the participant had read the fire and safety plan before commencing the fire drill. This to assume that all the participants have the same basic knowledge of the crew accommodations and location of the portable fire extinguishers according to the fire and safety plan. Further the disclosure that none of the participant should inform the other participant about their own progress or performance of the individual fire drill. 3.4.2 Reliability According to Robson (2007) the data is reliable if you get essentially the same data when a measurement is repeated under the same conditions. The problem lies of exact repetition of a measurement when working with people. Robson (2007) argue that the situation of the respondent may have changed during time and their opinion could be changed if the same questions arise again at a later stage. Fusch et al. (2018) emphasized that the important of triangulation cannot be underestimated to ensure validity and reliability of data and results. However, the empirical data presented in this thesis is not a picture of all possible real-world scenarios or possible outcomes regarding fire extinguisher training. Rather it is a set of interpretations produced by a triangulation of methods and a pre-set sampling group of participants from the Linnaeus University. The number of participants is 24 and for a wider range of generalization a larger study might be necessary. 23 4. Results and analysis In this chapter results are presented based on Kirkpatrick’s model. The results from the questionnaires, and case study observations are presented to give the reader a detailed understanding to answer the two research questions. 4.1 Level 1 Reactions Subjective usability such as enjoyment could be depended on the participants previous VR and computer game experiences. The participants were asked to rate on a scale of 0-5, where 0 is no experience, 1 very inexperienced, 2 inexperienced, 3 somewhat experienced, 4 experienced and 5 most experienced. According to the analysed questionnaire results five of the participants had no experience with use of virtual reality. The participants previous experience with the use of virtual reality (figure 6). However, the other participants in the group had less to a very high degree of previous experience with the use of virtual reality. Figure 6 Participants previous experience with the use of virtual reality Note, the experience is rated from 0 to 5. Where 0 is no experience, and 5 is most experienced. Further the participants had to some extent previous experience with computer and videogames (figure 7). The participants previous experience is important regarding generalization of the results into a larger population. Furthermore, the average age of the VR-experimental group were 23 years. Their previous experience with both virtual reality, computer and video games are important when analysing transferable skills from a virtual environment into a real- life condition. 0 1 2 3 4 5 0 1 2 3 4 5 N u m b e r o f p ar ti ci p an ts Previous experience, self rated 0-5 PREVIOUS EXPERIENCE WITH THE USE OF VIRTUAL REALITY 24 Figure 7 Participants previous experience with the use of computer and video games Note, the experience is rated from 0 to 5. Where 0 is no experience, and 5 is most experienced. The participants previous experience with the use of fire extinguisher (figure 8). The participants were asked to rate on a scale of 0-5, where 0 is no experience, 1 very inexperienced, 2 inexperienced, 3 somewhat experienced, 4 experienced and 5 most experienced. According to results presented in this figure all the participants had some previous experience with the use of a fire extinguisher. 0 1 2 3 4 5 0 1 2 3 4 5 N u m b e r o f p ar ti ci p an ts Previous experince, self rated 0-5 PREVIOUS EXPERIENCE WITH THE USE OF COMPUTER & VIDEO GAMES 25 Figure 8 Participants previous experience with the use of fire extinguisher Note, the experience is rated from 0 to 5. Where 0 is no experience, and 5 is most experienced. The participants were asked to rate on a scale of 0-5, their previous experience with fire extinguisher exercises, where 0 is no experience, 1 very inexperienced, 2 inexperienced, 3 somewhat experienced, 4 experienced and 5 most experienced. According to the analysed questionnaire results all the participants had some previous experience with fire extinguisher exercises (figure 9). 0 1 2 3 4 5 0 1 2 3 4 5 N u m b e r o f p ar ti ci p an ts Previous experince, self rated 0-5 PREVIOUS EXPERINCE WITH USE OF FIRE EXSTINGUISHER 26 Figure 9 Participants previous experience with fire extinguisher exercises Note, the experience is rated from 0 to 5. Where 0 is no experience, and 5 is most experienced. When asked about participants reactions to VR- fire extinguisher training as an educational complement, the questionnaire result shows that 77% of the participants in the VR experimental group tended to believe that the VR-training had a significant engaging and useful role to their present training. However, 23% of the participants in the VR- experimental group disagreed that there was anything that were value added to their own education using the VR- fire extinguisher training (figure 10). They concluded that their own previous experience was sufficient to not learn anything new during the VR fire extinguisher training. One of the participants replied that his previous basic safety training (BST) at an approved training facility was sufficient. However, the VR fire extinguisher training somehow made him feel safer even if he directly did not agree that the VR- training had any added value. The participants answer is in line with previous studies which show that more research is needed to establish methods of skill transfer from virtual reality to real life scenario. Further one participant described the limitations of the VR-equipment and that it could not entirely replace the mandatory on-site fire drills at approved fire training facilities. 0 1 2 3 4 5 0 1 2 3 4 5 N u m b e r o f p ar ti ci p an ts Previous experince, self rated 0-5 PREVIOUS EXPERINCE WITH FIRE EXSTINGUISHER EXERCISES 27 Figure 10 Percentage of participants who describes the VR-fire extinguisher training as value added education. In addition, the questionnaire results highlighted the different examples of VR-fire extinguisher implementation in the present fire education. According to analysed questionaries 46% of the participants suggested that the main purpose of the VR-fire extinguisher training was as an introduction course or pre-training course before the mandatory basic safety training (BST). Furthermore, 23% of the total of participants suggested that the VR-fire extinguisher training could primarily be used as a refresh course. Additional 8% of the participants suggested that the VR-fire extinguisher training could be custom made by using a realistic ship in the simulation. About 15% did not have any specific suggestions regarding the implementation of the VR-fire extinguisher training. Finally, approximately 8% of participants suggested that the virtual fire extinguisher training could completely replace the mandatory real fire extinguisher training (figure 11). 77% 23% PARTICIPANTS OWN REACTIONS TO THE VR-FIRE EXSTINGUISHER TRAINING AS AN VALUE-ADDED EDUCATIONFORM YES NO 28 Figure 11 Suggestions from the participants regarding VR-fire extinguisher training and its implementation in the education. 4.2 Level 2 Learning The second level (learning) is called the assessment of learning outcomes. Therefore, for analysing the learning outcomes, this study focus on changes in knowledge that has been learned and skills that have been developed or improved by the VR- fire extinguisher training. In this case analysis the VR experimental group were compared to the control group. The case analysis consisted of learning outcomes from a VR-fire extinguisher training. These learning outcomes were individually analysed. The result presented that 82% of the participants in the control group had chosen the correct CO2 extinguisher according to the ongoing electrical fire. The same number for the VR- experimental group were 85% (figure 12). 46% 23% 8% 8% 15% VR-FIRE EXTINGUISHER TRAINING AND ITS IMPLEMENTATION IN THE EDUCATION VR-Introduction course VR-Refresh course VR-Replacment training Custom VR fire exstinguisher training content No suggestions 29 Figure 12 The total percent of participants that used the appropriate CO2 fire extinguisher. According to results from the questionnaire, most of the participants (54 %) replied that the VR- fire extinguisher training was useful and practical to maintain the “hands on training” with the fire extinguishers. This was also confirmed by one of the participants who replied; , “I think VR was a good supplement in the fire training because it describes very well how different types of extinguishing work differently depending on location, environment and material” (Participant 9). According to the questionnaire, the remaining 38% of the participant did not describe if they felt more or less confident after the VR- fire extinguisher training. Examples of positive educational aspects described from the VR-experimental group were the ability to physically practice in a low position with the fire extinguisher. Further how to position and interact with an ongoing fire at a recommended safety distance. Further how the extinguisher operates on different materials and surfaces and that it gives a better understanding and reduces the time it takes to decide in a “sharp situation”. Only 8% of the participant did not agree with the majority of the group regarding the positive educational effects from VR-fire extinguisher training. The percentage of the participants that used the fire extinguisher at a recommended safety distance to the fire were almost identical between the control group and the VR- experimental group (figure 13). 82% 85% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1 2 Pe rc en ta ge o f p ar tic ip an ts Control group /VR-experimental group The percentage of participants that uses the approperiate exstinguisher to the actual fire 30 Figure 13 The percentages of the participants that used the fire extinguisher at a recommended safety distance to the fire. Further the percentage of the participants that described pulling the safety pin at the top of the extinguisher, and breaking the tamper seal are higher in the control group compared to the VR- experimental group. 91% compared to 62% (figure 14). Figure 14 Percentage of participants that describes pulling the safety pin at the top of the extinguisher, breaking the tamper seal. Further the percentages of the participants that aimed the nozzle at the base of the fire were 91% in the control group compared to 100% in the VR experimental group (figure 15). 64% 69% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1 2Pe rc en ta ge o f p ar tic ip an ts Control group /VR-experimental group The percentage of the participants that used the fire extinguisher at a recommended safety distance 91% 62% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1 2 Pe rc en ta ge o f p ar tic ip an ts Control group /VR-experimental group Percentage of participants that describes pulling pin at the top of the extinguisher, breaking the seal 31 Figure 15 The percentage of participants that aimed the nozzle at the base of the fire. The percentage of participants that describes the function of the handle to discharge the extinguishing agent inside the fire extinguisher were 91% from the control group compared to 77% from the VR experimental group (figure 16). Figure 16 The percentage of participants that describes the function of the handle to discharge the extinguishing agent. 91% 100% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1 2 Pe rc en ta ge o f p ar tic ip an ts Control group /VR-experimental group Percentage of participant that aimed the nozzle at the base of the fire 91% 77% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1 2Pe rc en ta ge o f p ar tic ip an ts Control group /VR-experimental group Percentage of participants that describes the function of the handle to discharge the extinguishing agent inside 32 Furthermore, the percentage of participants that sweep the fire extinguisher nozzle from side to side directly on the fire were 55% from the control group compared to 92% from the VR- experimental group (figure 17). Figure 17 Percentage of participants that sweep the fire extinguisher nozzle from side to side on the fire. Finally, the percentage of participants that easily removed the fire extinguisher from its mounting brackets were 82% from the control group compared to 77% in the VR-experimental group (figure 18). 55% 92% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1 2 Pe rc en ta ge o f p ar tic ip an ts Control group /VR-experimental group Percentage of participants that sweep the fire extinguisher nozzle from side to side on the fire 33 Figure 18 The percentage of participants that easily removed the fire extinguisher from its mounting brackets. One interesting finding were that 31% of the participants in the VR- experimental group disagreed that there was anything new that were value-added to their own education. They concluded that their own previous experience was sufficient enough to not learn anything new during the VR fire extinguisher training. Although they all agreed that the VR fire extinguisher training would be a positive training form for beginners and repetition. Another result showed that very few of the total 24 participants mentioned to cut the main electric supply to the electrical fire before use of the fire extinguisher. This result indicates that the participant focuses on the handling of the fire extinguisher as being trained both in virtual environment and in reality, but not with specific dangers with an electric circuit. An interesting result from this study also showed that the percentage of participants that sweep the fire extinguisher discharge nozzle from side to side on the fire were 55% from the control group compared to 92% for the VR- experimental group. 82% 77% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1 2 Pe rc en ta ge o f p ar tic ip an ts Control group /VR-experimental group The percentage of participant who easily removed the fire extinguisher from its mounting brackets 34 4.3 Level 3 Behaviour The measure of behaviour determines the level of change in trainees’ related behaviour after the VR-fire extinguisher training. The main behaviour changes that were observed were an improved situation awareness. The result showed a difference between the participants of the two groups when entering the smoke-filled room safely by utilizing a low position. 27% in the control group held a low position compared to the much higher 85% in the VR- experimental group (figure 19). Figure 19 The percentage of participant that entering the smoke-filled room safely by utilizing a low position. Based on the results from the case analysis, the control group had a lower percentage of participants (45%) that were able to confidently use the fire extinguisher compared to the (69%) of VR-experimental group (figure 20). 27% 85% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1 2 Pe rc en ta ge o f p ar tic ip an ts Control group /VR-experimental group The percentage of participants that entering the smoke-filled room safely by utilizing a low position 35 Figure 20 The total percentage of participants that could confidently handle the fire extinguisher. To further analyse a larger pattern of changes in the participants behaviours, another second analyse of the participants are required after a longer period of time. This was not possible to conduct due to practical and available resources. 4.4 Level 4 Results Only the three first levels have been considered in the presented research underneath the 1.3 limitations section. The measure of results relates to noticeable improvements, such as a reduced number of safety incidents, increased revenue, or reduced operational costs is not a part of this research. 45% 69% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1 2 Pe rc en ta ge o f p ar tic ip an ts Control group /VR-experimental group The percentage of participants that could confidently handle the fire extinguisher 36 5. Discussion In this discussion chapter the results will be discussed and analysed with the literature review. This section will also be linked to the research questions and for the upcoming conclusion and its theoretical contribution and practical implications. 5.1 Discussion Within minutes, an incipient fire can develop into a life-threatening scenario. Because of this it is important to act accordingly and safe during these circumstances. To handle an extreme situation the crewmember must be positioned low when using a fire extinguisher. Heavy smoke and toxic gases rise and poses a direct danger to the crewmember. 27% in the control group held a low position compared to the much higher 85% in the VR- experimental group. This emphasises the importance of training in diverse environments and smoke conditions, and it is here that the VR-fire extinguisher training has an advantage compared to traditional education forms. These results from this study are in line with Militello et al. (2023) who describes that recognition skills are normally refined at the job via long time experience and that these contexts are difficult to address in traditional training. The result from this study also provides evidence that the VR-fire extinguisher training made a difference by letting the participant explore the virtual environment with different smoke scenarios. The virtual smoke expanded vertically and horizontally through the air in each VR-fire extinguisher training, and this made the participant to adapt to the current situation. According to Santarpia et al. (2019) most victims die of the asphyxiation and intoxication due to a high concentration of lethal products of the combustion. Another example is also described by Santarpia et al. (2019) and relates to the problem with hot gases and temperature that causes burns, eye irritations and severely creates a difficult environment, for the participant even if the person was not directly exposed to the flames. In other word the pa