Design for Integrating Explainable AI for Dynamic Risk Prediction in Ambulance IT-systems Master’s thesis in Interaction Design David Wallsten & Gregory Axton Department of Computer Science and Engineering CHALMERS UNIVERSITY OF TECHNOLOGY Gothenburg, Sweden 2023 Master’s thesis 2023 Design for Integrating Explainable AI for Dynamic Risk Prediction in Ambulance IT-systems David Wallsten Gregory Axton Department of Computer Science and Engineering Chalmers University of Technology Gothenburg, Sweden 2023 Design for Integrating Explainable AI for Dynamic Risk Prediction in Ambulance IT-systems David Wallsten & Gregory Axton © David Wallsten & Gregory Axton 2023. Supervisor: Mikael Wiberg, Department: Interaction Design and Software Engineer- ing, Computer Science and Engineering co-Supervisor: Thommy Eriksson, Department: Interaction Design and Software Engineering, Computer Science and Engineering Supervisor: Stefan Candefjord, Care@Distance co-Supervisor: Eunji Lee, Care@Distance Examiner: Palle Dahlstedt, Department: Interaction Design and Software Engineer- ing, Computer Science and Engineering Master’s Thesis 2023 Department of Computer Science and Engineering Chalmers University of Technology SE-412 96 Gothenburg Telephone +46 31 772 1000 Printed by Chalmers Reproservice Gothenburg, Sweden 2023 iv Design for Integrating Explainable AI for Dynamic Risk Prediction in Ambulance IT-systems David Wallsten & Gregory Axton Department of Computer Science and Engineering Chalmers University of Technology Abstract Demographic changes in the West with an increasingly elderly population puts stress on current healthcare systems. New technologies are necessary to secure patient safety. AI development shows great promise in improving care, but communicating AI decisions requires more research. In this study, a prototype of eXplainable AI (XAI) was designed for an ambulance IT system, based on an AI model for risk prediction of severe trauma to be used by Emergency Medical Services (EMS) clinicians. Knowledge was gathered for the design through ethnography, expert interviews, and a literature review. Then several ideas developed through stages of prototyping and were verified by experts in prehospital healthcare. Finally, a high-fidelity prototype was evaluated by the EMS clinicians. The design was then evaluated by seven EMS clinicians. They thought that XAI was necessary for them to trust the prediction. They make the final decision, and if they can’t base it on specific parameters, they feel they can’t make a proper judgement. In addition, the design helped in reminding EMS clinicians of things they might have missed. If given a prediction from the AI that was different from their own, it might cause them to think more about their decision, moving it away from the normally relatively automatic process and likely reducing the risk of bias. While focused on trauma, the design should be applicable to other AI models. Current models for risk prediction in ambulances have so far not seen a big benefit from using artificial neural networks (ANN) compared to more transparent models. This study can help guide the future development of AI for prehospital healthcare and give insights into the potential benefits and implications of its implementation. The report also explores the ethical implications, the complexity of the ambulance work environment and possible implications for cognitive decision processes. Keywords: Computer, science, computer science, engineering, project, thesis. v Acknowledgements We would like to thank everyone at the Care@Distance research group for staying interested. Especially Stefan Candefjord and Eunji Lee, who have worked as our mentors in the group, guiding us extensively throughout the process and helping us with contacts and inspiration. Also, a special thanks to Anna Bakidou, the primary developer of the AI model this research is partially based on, Beng-Arne Sjöqvist for his support in understanding the current market and the future and Mattias Seth for his knowledge contribution to interoperability. We want to thank PICTA, especially Elin Maxstad, who did a similar project last year. With her report, this project reached nearly as far as it did the contacts she provided. We are also grateful for Andreas Dehre’s contribution of knowledge and contacts in Gothenburg’s ambulance services. Also, we would like to express our gratitude to Paratus and Jonas Borgström, who welcomed us to visit the company and gain insights about their current systems. We are very thankful to Stefan Jönsson at VGR for his expertise and for informing us of the potential future of healthcare at VGR. Lastly, thank you to all the ambulance personnel who have willingly given us their time and effort. The work would have been almost worthless without your contribution. Everyone we met has shown excitement for our project, giving us the energy to push forward. There is much kindness and engagement in this field, with people working hard to save lives in many ways. David Wallsten & Gregory Axton, Gothenburg, 2023-06-18 vii Contents List of Figures xiii List of Tables xv 1 Introduction 1 1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Stakeholders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.4 Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.5 Deliverable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.6 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.7 Research questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 Context 7 2.1 Ambulance organisation . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 Ambulance workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.3 Triage systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.4 AI-driven decision support for Trauma . . . . . . . . . . . . . . . . . 10 3 Theoretical framework 13 3.1 Human factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.1.1 Cognitive processes and bias . . . . . . . . . . . . . . . . . . . 13 3.1.2 Gestalt principles . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.2 User Experience Design . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.3 Double diamond model . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.4 Swedish healthcare regulations, guidelines and ethics . . . . . . . . . 18 3.5 Ethical challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.6 Artificial intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.6.1 Artificial Intelligence (AI) . . . . . . . . . . . . . . . . . . . . 20 3.6.2 AI in healthcare . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.6.3 Interaction and Artificial Intelligence (AI) . . . . . . . . . . . 21 4 Methods 23 4.1 Literature study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 4.2 Ethnography research . . . . . . . . . . . . . . . . . . . . . . . . . . 23 4.3 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 ix Contents 4.4 Personas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 4.5 Sketch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 4.6 Prototyping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 4.7 Usability test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 5 Results 27 5.1 Literature study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 5.1.1 Current use of it-support . . . . . . . . . . . . . . . . . . . . . 27 5.1.2 Current triage decision support in Sweden . . . . . . . . . . . 27 5.1.3 Status of Explainable AI in Healthcare . . . . . . . . . . . . . 28 5.2 Ethnography research . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 5.2.1 Empathy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 5.2.2 Strategy and product objective . . . . . . . . . . . . . . . . . 29 5.2.3 Identify and understand user . . . . . . . . . . . . . . . . . . . 29 5.3 Expert interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 5.4 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 5.5 Prototyping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 5.5.1 Low fidelity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 5.5.1.1 Sketch and tangible Prototype . . . . . . . . . . . . 38 5.5.1.2 Digital prototype . . . . . . . . . . . . . . . . . . . . 38 5.5.1.2.1 Components . . . . . . . . . . . . . . . . . . 40 5.5.1.2.2 User experience (Emotions) . . . . . . . . . 41 5.5.1.2.3 Physical format . . . . . . . . . . . . . . . . 41 5.5.2 Wireframe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 5.5.2.1 Decision support presentation . . . . . . . . . . . . . 44 5.5.2.2 Visual design . . . . . . . . . . . . . . . . . . . . . . 46 5.5.2.2.1 Accessibility . . . . . . . . . . . . . . . . . . 49 5.5.2.2.2 Interaction . . . . . . . . . . . . . . . . . . 49 5.5.2.2.3 Layout . . . . . . . . . . . . . . . . . . . . . 49 5.5.2.2.4 Styles . . . . . . . . . . . . . . . . . . . . . 51 5.5.3 High fidelity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 5.6 Test and evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 5.6.1 Expert evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 59 5.6.2 User tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 5.6.2.1 WEST . . . . . . . . . . . . . . . . . . . . . . . . . . 63 5.6.2.2 Scenario . . . . . . . . . . . . . . . . . . . . . . . . . 63 5.6.2.3 AI notification and interaction . . . . . . . . . . . . . 64 5.6.2.4 AI page . . . . . . . . . . . . . . . . . . . . . . . . . 64 5.6.2.5 AI’s prediction . . . . . . . . . . . . . . . . . . . . . 65 5.6.2.6 Vital Parameters . . . . . . . . . . . . . . . . . . . . 66 5.6.2.7 General interface . . . . . . . . . . . . . . . . . . . . 67 5.6.2.8 System Usability Scale (SUS) . . . . . . . . . . . . . 67 6 Discussion 69 6.1 Research questions and answers . . . . . . . . . . . . . . . . . . . . . 69 6.2 Project scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 6.3 Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 x Contents 6.4 User study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 6.5 Tests and evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 6.6 Human factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 6.6.1 Wickens attentional processing model . . . . . . . . . . . . . . 75 6.6.2 4-D multiple resource model . . . . . . . . . . . . . . . . . . . 75 6.6.3 Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 6.6.4 Gestalt principles . . . . . . . . . . . . . . . . . . . . . . . . . 76 6.7 Ethical aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 7 Conclusion 79 Bibliography 81 8 References 81 A VGR paper journal 89 B Video introduction and tasks for testing 91 B.1 Script for video introduction . . . . . . . . . . . . . . . . . . . . . . . 91 B.2 Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 C Consent form: Usability test 93 D SUS results I xi Contents xii List of Figures 1.1 Population pyramids, EU 2007 and 2022 (Eurostat, 2022) . . . . . . . 2 2.1 The flowchart represents EMS general workflow and routines. . . . . . 8 2.2 Adult SATS Chart, The South African Triage Scale (Cheema and Twomey, 2012) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3 NEWS system from VGR Paper journal: Each parameter is given a score, they are then combined. If the score is seven or higher, the severity is red. 5-6, orange. 0-4, green. Yellow has separate preconditions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.1 The human information processing model adapted from Wickens, Hollands, Banbury and Parasuraman (2016) . . . . . . . . . . . . . . 14 3.2 4-D multiple resource model by Wickens et al.(2008) . . . . . . . . . 14 3.3 System 1 and System 2, adapted from Kahneman (2011). . . . . . . . 15 3.4 Three examples of Gestalt principles, adapted from Todorovic (2008). 16 3.5 Disciplinary map of user experience design to other design fields (Interaction Design Foundation, 2002). . . . . . . . . . . . . . . . . . 17 3.6 The double diamond of design, adapted from Sharp et al. (2019) . . . 18 5.1 Gap analysis map of this project’s literature review . . . . . . . . . . 28 5.2 Ethnography research; Observation & contextual inquiry from regions, Halland and VGR) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 5.3 SOTA table showing product diversity. . . . . . . . . . . . . . . . . . 31 5.4 The second latest version from Paratus. . . . . . . . . . . . . . . . . . 32 5.5 One out of four developed personas. Fictional profiles that represent user behaviour patterns based on collected data. . . . . . . . . . . . . 32 5.6 Journey Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 5.7 An example of Dynamic Risk Prediction implementation structure with added active decision support, based on an example from Bengt- Arne Sjöqvist. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 5.8 MoSCoW mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 5.9 Sketching based on fictitious scenarios. . . . . . . . . . . . . . . . . . 39 5.10 Tangible prototype, brainstorming and generating ideas. . . . . . . . 39 5.11 Example of number entry solutions. . . . . . . . . . . . . . . . . . . . 40 5.12 Example of free text entry solutions. . . . . . . . . . . . . . . . . . . 41 xiii List of Figures 5.13 Example of option picker solutions. From left: Checkboxes, Dropdown menu, Two versions of input chips . . . . . . . . . . . . . . . . . . . . 41 5.14 Diagram of MDA framework . . . . . . . . . . . . . . . . . . . . . . . 42 5.15 Early wireframe AI presentation with multiple cards representing different values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 5.16 Early wireframe AI presentation overlay with a chart comparing different values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 5.17 The WEST paper journal and an early digitised version of it. For the full-sized paper journal, see Appendix A . . . . . . . . . . . . . . . . 44 5.18 Drill down functionality during triage. . . . . . . . . . . . . . . . . . 46 5.19 The wireframe in different stages in the system. . . . . . . . . . . . . 47 5.20 Interactive prototype flow created in Figma . . . . . . . . . . . . . . 48 5.21 The graphic describes UX design, including Visual Design. Adapted from Interaction Design Foundation (2002). . . . . . . . . . . . . . . 48 5.22 A mood board presenting early visual interface and potential design. . 49 5.23 AI Overlay. More cards can be seen by panning from the side . . . . 50 5.24 Sketching ideas for the design layout . . . . . . . . . . . . . . . . . . 51 5.25 The selected colour pallets for ambulance work service. . . . . . . . . 52 5.26 Font Roboto, used throughout design . . . . . . . . . . . . . . . . . . 53 5.27 WEST page of the high fidelity prototype, based on the WEST paper journal for triage decision support . . . . . . . . . . . . . . . . . . . . 55 5.28 Regions as a part of the AIS system to measure the severity of trauma injuries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 5.29 AI output overlay. The predictor cards should be intractable, allowing one to change them and giving quick information. . . . . . . . . . . . 58 5.30 A second display to view the top three predictors for and against. . . 58 5.31 Additional variables view of the AI currently lacks but deems important. 59 5.32 Vital parameters chart . . . . . . . . . . . . . . . . . . . . . . . . . . 60 5.33 Feedback from a UX designer . . . . . . . . . . . . . . . . . . . . . . 61 5.34 User test setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 5.35 Usability testing flowchart . . . . . . . . . . . . . . . . . . . . . . . . 63 A.1 VGR paper journal . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 D.1 SUS results 1-5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . II D.2 SUS results 6-10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . III xiv List of Tables 5.1 Distribution of testers . . . . . . . . . . . . . . . . . . . . . . . . . . 62 5.2 The table of participants who participated in user testing. Experience shown in years . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 xv List of Tables xvi 1 Introduction 1.1 Background Healthcare in Europe is facing a major crisis with an ageing population. From 2007 to 2022, the median age has risen from 41.9 years 2007 to 44.1 years 2022(Eurostat, 2022) Figure 1.1. The same applies to Sweden, with the age group over 90 increasing the most by percentage (SCB, 2022). This increases the demand for healthcare systems with limited resources, creating a need for new solutions. The Swedish Government has a goal to be leading in e-Health by 2025 because of this reason (Blix & Levay, 2018). It creates demands on simple e-health solutions like SMS reminders but also more robust data management (e.g. big data solutions) as well as advanced decision support. While necessary, the digital transformation of the Swedish healthcare system faces many challenges. It’s important to take into account the usability and the work environment of healthcare professionals when digitising healthcare. Otherwise, it risks increasing stress levels of healthcare professionals and decreasing patient safety, as stated by Sveriges Läkarförbund (The Swedish Medical Association, 2021). This is just as relevant for IT systems in prehospital care and care outside the hospital. A research group at Chalmers University, Care@Distance – Remote and Prehospital Digital Health, are investigating multiple new technologies with the potential to improve prehospital care. They include AI-driven decision support (Candefjod et al., 2021), video-supported assessments in ambulances (Vicente et al., 2021), and voice-controlled augmented reality interfaces (Kemppainen, 2022). The hope is to make decisions faster, more efficiently and safer, often using big data tools with wide combinations of data. It can be both structured, such as tabular data, and unstructured such as long-form text or images. However, further user experience(UX) research is needed to fully integrate these technologies into the workflow of ambulance workers and determine the most effective means of presenting the results of the decision support systems. The field of explainable AI (XAI), in particular, is very new, with big needs for user studies (Antoniadi, et.al., 2021). XAI lacks a proper technical definition, but key elements are interpretability, how much the model can be understood, transparency, about how the model works, and explainability, giving insight into the reasons for the AIs’ decisions (Antoniadi, et.al., 2021). Currently, ambulance care uses different triage decision support systems, such as 1 1. Introduction Rapid Emergency Triage and Treatment System (RETTS) or West Coast System for Triage (WEST), a kind of checklist based on vital parameters, patient information and warning symptoms. The problem with these systems is that they can be imprecise. In the case of children, RETTS can lead to an over-triage of around 30% and the same level of under-triage (Magnusson, 2021). Over-triage is when a case is rated more severe than it is, leading to resource drain. Under-triage is when a case is rated as less severe, leading to a high risk to patient safety. AI decision support systems have the possibility to have higher precision than current systems. A particular AI model is under development by Anna Bakidou et. al. (2023) at Care@Distance. It is a general model for trauma triage, assessment and prediction for urgency and severity. Bakidou’s model uses data usually collected at a hospital to make predictions for the ambulance. Thus, a novel design is needed. With Bakidou’s model as a base for the design, this project aims to investigate how data from AI models could be presented in a way that fits into the ambulance workflow. More importantly, the question of how to display the AI model’s prediction is important to build trust and understanding with the ambulance workers, who are ultimately responsible for making the final decision. And could this information be displayed earlier than current systems, leading to quicker decision-making? Figure 1.1: Population pyramids, EU 2007 and 2022 (Eurostat, 2022) 2 1. Introduction 1.2 Stakeholders Care@Distanc ASAPs Care@Distance ASAP (Acute Support Assessment and Prioritising) is a research group based at Chalmers University with a vision to increase decision precision, limiting errors in assessment, prioritisation and handling while also reducing time to make decisions. Aims to improve remote and prehospital care using data fusion, clinical decision support, AI/ML (Machine learning), telemedicine, and innovative user interaction. Care@Distance ASAP is the main stakeholder for this project. It provided valuable resources and insights within the healthcare sector input to drive this project forward. The expectation is to gain influence by this study as a part of the more prominent research. It may raise a discussion on the design aspect of future systems and ongoing research. PICTA PICTA is a prehospital innovation arena funded by the region Västra Götaland (VGR) as a part of Lindholmen Science Park. Their goal is to be a platform for research sharing in the prehospital setting, to make individual projects inform each other, driving progress forward. It is a collaborative platform for health care, academia, and business organisations. Different governmental foundations, such as Vinnova, fund projects run or supervised by PICTA. PICTA is not directly involved in the development of this project. Elin Maxstad introduced us to her previous report and provided us with the necessary information and contacts to drive this project forward. Andreas Dehre, a former nurse and current teacher of ambulance personnel, has provided us with his expertise and contacts as well, including inviting us to the ambulance station in Gothenburg. PICTA has solid knowledge and experience within prehospital settings and the implementation of new technology in this field. PICTA does not expect anything in terms of deliverables and is involved solely by self-interest, and will act as continuous support throughout the project. Västra Götaland Region (VGR) The Västra Götaland Region (VGR) is the governmental organisation that funds healthcare in Västra Götaland including the city of Gothenburg. VGR is planning to implement a new comprehensive healthcare system for all healthcare in VGR, including prehospital care. The latter haven’t started development yet. To narrow the scope of this project, it is focused on VGR instead of other regions, and basing its design on the capabilities of the new IT system. Thus, this project could work as a source of influence for any future ambulance system when it starts development. Hospital The emergency department (ED) at hospitals in Sweden are the departments that have direct contact with the ambulance. ED collects valuable information such as ambulance action and data like transportation, first medical treatment, patient transfer and mission confirmation. In VGR, most of the data is given to the ED through the delivery of a paper journal. In Halland, most of the data updated in the ambulance are displayed in real-time at the ED station. The ambulance system needs to cooperate with the system at the hospital, including phone calls directly through the ambulance system, and synchronise the data that need to be reported to 3 1. Introduction a doctor; therefore, the hospital needs to understand AI risk prediction and general UI to gain a better workflow between them. Third-party service developers In the prehospital setting in Sweden, there are multiple IT system providers. The biggest is CSAM with Paratus and Ortivus with Mobimed, according to our interview with Bengt-Arne Sjöqvist, a healthcare researcher and industry veteran. The project aims for the design to be able to be integrated into current systems, thus it is important to take these companies into consideration. The design may then work as an inspiration for future implementation of AI decision support. 1.3 Users Primary users The primary user is the EMS personnel who intervenes in EMS services. In this study, We have divided the user into two groups: the Primary user and the secondary user, based on who may have direct contact with the system. The primary users include EMS nurses (usually taking decisions), EMS paramedics, EMTs, EMS volunteers, EMS students, and EMS trainees. The primary user generally works in pairs that vary in age, gender, experience and skill sets. Secondary users The secondary user consists of those who may display information indirectly through the system under various circumstances, e.i. The EMS personnel share some info with secondary users according to their needs. This may include the patient, the patient’s family or relative, and the caretaker. In addition, the secondary users may be described as witnesses and have less direct contact with the system. 1.4 Aim This study aims to investigate a possible design for a future IT-This study aims to investigate a possible design for a future IT-system with an integrated dynamic risk prediction AI decision support tool. It will be based on an AI model trauma risk prediction but should be applicable to other conditions and AI models. It aims to act as inspiration for those working with developing the models, to get an idea about how their models could be applied and how medical professionals might react to them. It also aims to act as an inspiration for the healthcare services and industry, to help them in generating ideas on how to integrate AI into their existing systems and get inspiration for the interaction design of their general ambulance IT system. The design should take cognitive resources into account, be based on usability research and be adapted to the ever-changing work environment of ambulance personnel. 4 1. Introduction 1.5 Deliverable The project is located in the city of Gothenburg, a part of VGR. Their current work towards a new digital interface and the ease of access to local expertise and testers makes this project focused towards VGR. At the same time, the project takes current systems used nationally and internationally into account, likely making findings applicable to a much wider market. The deliverables are: An interactive interface for explainable AI(XAI) output A wider interactive prototype interface, imagining future utilisation of big data to increase speed and ease of use. Needs to be able to be adaptable to any medical emergency where an ambulance might be involved 1.6 Limitations The project will be focused on the VGR region but should be adaptable to other areas. It will take into account currently available technology but might adapt them in ways currently not done in the prehospital field. This places the project with a possible implementation of an estimated 5-8 years in the future. Primary users, meaning ambulance personnel, will be the focus of the report. The system however will have a heavy impact on secondary users, meaning patients, relatives and bystanders. Having that in mind is important. In addition, ambulance personnel might need to show the display to the secondary users in case they want to explain something, or, it might be placed in a location where secondary users might look at it. These factors need to be taken into account but won’t be a focus of this study. Due to a tablet being the primary hardware for digital interfaces in ambulances, this will be the focus of the design. Phones should be a strong consideration since they are also used in the ambulance but were out of scope for this project. The AI input and output will primarily be based on trauma patients to create a focus of the project and make the interaction based on real scenarios, but the interface aims to work towards other scenarios as well. However, these will not be designed or tested. The user test will only test the initial experience of using the interface. A real interface would be repeatedly used each day. How this daily interaction would impact user behaviour will not be investigated. The same applies to the long-term use of AI, and how this impacts trust and decision-making biases. Further, actual decisions based on AI will only be investigated on a superficial level, since it would likely require an unfeasible amount of users for significant results. 5 1. Introduction 1.7 Research questions RQ1: How can a dynamic AI-driven decision support be designed to align with the ambulance personnel workflow and work environment? RQ2: How can a dynamic AI-driven decision support be designed to align with the structure of current IT systems used in ambulances? RQ3: How important is explainability for ambulance AI-driven decision support? RQ4: How can AI decision support be explained and visualised to increase trust and understanding of the system? RQ5: How could AI-driven decision support impact ambulance workers’ cognitive decision process? 6 2 Context The following chapter goes through the context of the ambulance work environment. While the aim of the report is to apply to all ambulance systems, the focus is on Sweden generally and the Swedish region Västra Götaland(VGR) specifically, mostly due to its proximity to the authors, making it easier to study. Most of the contents of this chapter are based upon information gathered from the interviews with experts and ambulance personnel and relevant documents. 2.1 Ambulance organisation Prehospital care is care that occurs outside of a hospital setting e.g. home, emergency medical service(EMS) vehicle, and incident hot spots. In the case of emergency ambulance care, providers include; EMS nurses or paramedics and emergency tech- nicians (EMTs). The care provided in this setting is typically for patients who are experiencing an acute medical emergency or trauma, such as a heart attack, stroke, and vehicle accident. However, it can also be provided to less emergent cases only requiring transport or patients that appear to be healthy enough to stay at home. The purpose of ambulance care, in the case of acute cases, is primarily to stabilise the patient’s condition while providing rapid transport to a hospital for further treatment. Most of the time, there are two personnel in every ambulance, one driving and one taking care of the patient. In most cases, they both take part in medical decisions, although only one is fully responsible. The roles can be switched between cases, or based on who has the most medical expertise. 2.2 Ambulance workflow The following workflow (Figure 2.1) is primarily based on our ethnography research conducted in Halland and VGR regions in Sweden. Notification from SOS Alarm centre The communication between SOS Alarm Centre and the EMS is done via the national RAKEL system; however, this is used by the police and fire departments. EMS personnel usually carry a telecom tool with this system, and every ambulance is equipped with one unit. Whenever a mission is sent out from the SOS Alarm Centre via the RAKEL system, this can be a prioritisation between 1-3 where a 1 is very 7 2. Context Figure 2.1: The flowchart represents EMS general workflow and routines. 8 2. Context acute and needs to be responded to as early as possible. Then the EMS personnel needs to confirm it; an estimated arrival time is reported back to the SOS Alarm centre, and that is when the mission begins. Reach patient’s location The EMS personnel read through a brief description of the mission from the Emer- gency Medical Dispatch Centre (EMDC). If there is enough time, they can also view some of the patient’s medical history through a laptop or a mobile phone in the ambulance. With this information, the EMS personnel may get a better view of the patient and prepare the appropriate equipment. Assessment When arriving at the patient’s location, an initial assessment examines the most important vital signs, such as breathing and circulation. Suppose a life-threatening acute emergency has not been found, the EMS personnel collects information by interview and begins a physical exam for vital parameters (e.g. pulse and blood oxygen saturation). In the Halland region, these parameters can be transferred wirelessly to the IT system, while in VGR, the parameters need to be written down on paper. Triage systems such as Rapid Emergency Triage and Treatment System (RETTS) or West Coast System for Triage (WEST) are used through a digital interface such as the one in Halland or a pen & paper system practice in VGR. An encyclopaedia of symptoms and treatment guidelines can help EMS personnel throughout the process. The parameters will be noted, and when the investigation is done, the EMS personnel converse about further actions in consultation with the patient, relative and or healthcare staff through telecommunication. The patient must approve the care decision when consent is possible, as the EMS nurses cannot force any treatment on a patient unless necessary. Treatment There are four primary choices of action; the patient could be treated on the scene, the patient should seek further care at a primary healthcare centre, the patient needs to go to a hospital but can wait for a taxi or hospital transportation to take them there or the patient needs to go to a hospital in an instant and is taken to the Emergency department (ED) by the ambulance. Transfer patient When the patient is taken to a hospital; an acute patient medical record must be completed beforehand and handed over to the emergency department in the hospital. Region Halland and some other regions (e.g. Växsjö, Kronoberg) utilise a digital record that can be transferred in real-time. In contrast, region VGR uses paper journals that must be handed over in person. Parameters, medical history, and the patient’s status are noted. The EMS personnel who provide the treatments on the scene are noted as responsible for the medical decisions, such as triage and treatment. Write a patient medical record journal When the mission is completed after the patient is transferred to a hospital, the rule of thumb is the EMS personnel has 20 minutes to finalise writing the patient’s 9 2. Context medical record in a web-based system, such as Ambulink. This takes place where it has access to a stationary desktop computer, often at the hospital or the ambulance station, but most ambulance vehicles are equipped with a laptop where it can be performed as well. The ambulance typically receives a new assignment after 20 minutes. However, if there is a new assignment with the highest priority no. 1, it can be set directly after the previous one is completed. In these cases, the writing of the patient’s medical record is postponed. In the worst-case scenario, the whole shift can go on without having enough time to transfer the notes on paper (VGR) to the patient’s medical record. Thus, it has to be done at the end of the shift instead. The regions like VGR allow access to patients’ medical journal outcomes after visiting the hospital. This allows EMS personnel to check the results (optional). 2.3 Triage systems The triage system is utilised as a tool for nurses to help them prioritise patients. Typically, the severity of cases is colour coded, with blue, green, yellow, orange or red. From yellow and upwards, patients should be sent to a hospital. In the case of red, the case is very urgent, with severe consequences if the patient does not receive healthcare in time. Orange is also urgent but less urgent than red. Yellow means the patient should be sent to the hospital but is not time-critical. Several triage systems have been developed and established globally. However, the most common use of the triage system in Swedish healthcare is RETTS, according to Magnusson (2021). The system is based on vital parameters (VP), emergency systems and signs, and patient information such as age. VGR uses its system, WEST (Tran, Winkler, 2021). It is based on the South African warning scale(SATS) (Figure 2.2), which categorises symptoms by severity, and guides triage based on a step-by-step method, first looking for the most severe symptoms and then going further. WEST also utilises the national early warning score (NEWS) (Figure 2.3) that continuously considers VP, giving a severity number based on how much lower or higher they are outside regular intervals. If one score is very severe, it can be enough to give high priority to a patient. Otherwise, a combination of scores is taken into account, and depending on how high the score is, the patient has ascribed a coloured prioritisation level. 2.4 AI-driven decision support for Trauma Although the project is supposed to work with different AI models, for the sake of having a focus point, the project was based on an AI model for trauma prehospital triage, developed at Care@Distance by Anna Bakidou(2023). Similar models have been developed in the research group before such as an On Scene Injury Severity Prediction (OSISP) model for motor vehicle trauma (Candefjod et al., 2021). Baki- dou’s model is similar but used for other kinds of trauma as well, both blunt and 10 2. Context Figure 2.2: Adult SATS Chart, The South African Triage Scale (Cheema and Twomey, 2012) 11 2. Context Figure 2.3: NEWS system from VGR Paper journal: Each parameter is given a score, they are then combined. If the score is seven or higher, the severity is red. 5-6, orange. 0-4, green. Yellow has separate preconditions. penetrating, like falling injury, stab injury, explosion, etc. For training and validation, data from the Swedish national trauma registry (SweTrau) from between 2013-2020 with 47357 patient registrations after exclusion criteria are utilised. The model uses a total of 13 different predictors, including body region, injury mechanism, intent of injury, age, etc. The combination of these predictors for each patient is then compared to the patient outcome during training. When validating, the model is granted a dataset that hasn’t been trained on, is given the predictors for each case and makes its own prediction. Performance-wise, when set to the recommended level of over-triage at 25-35%, the model yielded under-triage at 8- 25%. That is potentially a big improvement to current triage results, with current over-triage at 45.8% and under-triage at 40.4% according to the data in the SweTrau registry. This is also before the algorithms received any optimization for the trauma model which in this case only used the default settings. Five different machine learning techniques were used to train the model, including more transparent techniques and those more like a black box such as an artificial neural network. However, both types delivered similar performance, meaning a transparent method could be used, which would allow for knowing more about what caused the AI to come to a conclusion, for example, which predictors were the most important. This is paramount in the case of healthcare, for allowing healthcare workers with understanding why the AI came to a specific conclusion. 12 3 Theoretical framework The theoretical framework for this thesis formed a base for important design consid- erations and the general process of the project. This includes human factors, user experience design, design theory, healthcare regulations and guidelines, and ethical challenges concerning the use of artificial intelligence. 3.1 Human factors 3.1.1 Cognitive processes and bias Wickens’s attentional processing model(2021) shown in Figure 3.1, describes several information processing and decision-making stages. Stimulus creates sensory input through our sensory organs. This is then perceived based on knowledge stored in the long-term memory, which utilises working memory to process stored information, connecting it to the sensory input. Then, working memory is utilised to consider possible responses to the stimulus, finally leading to a response execution, which creates feedback with new information. Working memory is minimal, both in the number of units and time. Thus, throughout the design process, this will need to be considered to ensure ambulance workers only need to hold small sets of information at a time. At every stage in the process model a pool of attentional resources is spent, meaning that if many resources are used in one stage, it can affect other stages, leading to errors (e.g., in perception or response selection) which could be detrimental to decide optimal care for a patient in an emergency situation. The project should aim for a design limiting the usage of attentional resources. Ambulance personnel can often need to work with time-shared tasks, such as listening to information from a patient or monitoring vital signs while putting the data into a journal. According to Wickens (2008), the 4-D multiple resource model, Figure 3.2, predicts how different cognitive modalities can be used simultaneously while reducing the expenditure of resources from each other. (e.g., talking to a patient and reading instruction are both considered verbal/linguistic modalities). At the same time, the spatial task of putting on equipment and speaking are two different modalities and can share time-space. Thus, what modalities are used can be important considerations for the design. 13 3. Theoretical framework Figure 3.1: The human information processing model adapted from Wickens, Hol- lands, Banbury and Parasuraman (2016) Figure 3.2: 4-D multiple resource model by Wickens et al.(2008) 14 3. Theoretical framework A prominent decision-making theory is that we have two modes of thinking, system 1 and system 2, seen in Figure 3.3 (Kahneman, 2011). System 1 is quick and automatic, taking mental shortcuts called heuristics in order to make conclusions. System 2 is analytical, slow, and requires more energy and concentration. Most decisions are of system 1, and in a situation such as an ambulance where stress, fatigue and time limits can be significant factors, system 1 is a likely major factor. This can lead to a more decisive influence of biases and incorrect heuristics. This includes biases such as anchoring, where the first piece of information influences the rest, meaning when and how to present information in an IT system can lead to different consequences. Another is the framing effect, where talking about the risk of death has a different effect than the chance of survival, although the numbers could be the same. Investigating the most apparent biases in prehospital care and using the design to limit them could improve decision-making and reduce errors. Figure 3.3: System 1 and System 2, adapted from Kahneman (2011). 3.1.2 Gestalt principles The gestalt principles describe how we perceive group objects (Todorovic, 2008). For example, the proximity principle tells us that objects close together are within the same group. Similarity says the same for objects that are similar to each other, and the common region principle says the same for objects that share the same closed region. See Figure 3.4. An interface in the medical field can include several kinds of information, such as vital parameters, personal information, and guidelines. To make sure these are not accidentally grouped together as the same objects or objects of the same importance, following Gestalt principles are necessary. 15 3. Theoretical framework Figure 3.4: Three examples of Gestalt principles, adapted from Todorovic (2008). 3.2 User Experience Design User experience (UX) design is acknowledged as a framework for understanding the interaction between users and artefacts, see Figure 3.5. UX design typically focuses on creating compelling and satisfying experiences for them (Norman & Nielsen, 2010). User experience can be defined as "a person’s perceptions and responses that result from the use or anticipated use of a product, system or service" (ISO 9241-210). According to Jesse James Garrett (2010), the term ’user experience’ is generally applied to the positive, neutral, and negative emotions felt whilst interacting with computer systems and user interfaces and is equally applicable to any other instance where a human uses a product, object, or service. He stated, "Every product used by someone has a user experience: newspapers, ketchup bottles, reclining armchairs, cardigan sweaters". The UX design relies on various principles, concepts and methods and is set on various disciplines such as psychology, sociology, and human-computer interaction (HCI) (e.g., Figure 2.). UX design emphasises the importance of understanding user needs and goals and designing products that meet their needs seamlessly and intuitively. UX design provides a powerful framework for creating digital products that are both effective and enjoyable for users (Garrett, 2010). The process of UX design practice will involve a deep understanding of user behaviour, preferences and pain point issues, including the broader social and cultural context in which the product will be used. The process generally interacts and continues improving products, systems or services. Designers must be willing to test and refine their designs based on user feedback and to adapt to the change in user needs and market trends. This would encourage designers to experiment, try new approaches, and be open to feedback and criticism. 3.3 Double diamond model The activities and methods will be performed for this project following the double diamond model, see Figure 3.6. It is a process introduced in 2004 and has become a well-known model for designers and non-designers to systematically engage in problem-solving (British design council, 2019). Regarding this project, developing a future IT system is a complex and dynamic 16 3. Theoretical framework Figure 3.5: Disciplinary map of user experience design to other design fields (Inter- action Design Foundation, 2002). 17 3. Theoretical framework project requiring an understanding and effective approach. Therefore, the double diamond model is a framework that can offer several advantages. First, the divergent part of the model allows for the exploration of these complex tasks related to the work environment, current decision support tools, diverse technical solutions and organisations behind such solutions. When such findings have been made, the project can develop into a narrow scope, a comprehensive system to improve patient outcomes that meet the user’s needs and stakeholder’s expectations. However, the limitation of the double diamond model that Sharp et al. (2019) mentioned is; It can be time and resource intensive, relying heavily on user feedback which may be limited in some cases. The model may only sometimes fully address broader social and cultural factors, and its flexible approach may need more explicit direction. It was crucial to consider the potential limitations of the double diamond model for this project and utilise it based on the context and goals. Figure 3.6: The double diamond of design, adapted from Sharp et al. (2019) 3.4 Swedish healthcare regulations, guidelines and ethics The project will focus on the Swedish EMS system. While not necessary for this research to succeed, if the project is to be able to present a feasible solution according to Swedish standards, it will need to try to follow Swedish regulations. Swedish healthcare is governed by Hälso- och Sjukvårdslagen (HSL) [The Health and medical care act], decided by the Swedish parliament; Chapter 5, 1 § states that all healthcare shall be, as translated by the author of this paper: 18 3. Theoretical framework 1. Be good quality with excellent hygienic standards. 2. Satisfy the patient’s need for security, continuity and safety. 3. Build on respect for the patient’s self-determination and integrity. 4. Promote good contact between the patient and the healthcare staff. 5. Be easily accessible. In particular, points 2-4 are relevant since they can inform a focus on the patients’ needs in the design. While the project will focus on the user/paramedics, the results should promote the patients’ needs to be stated in HSL. An important consideration could be how to design to keep respecting the self-determination and autonomy of a patient while an AI makes a recommendation that might be more difficult for humans to interpret. In addition to HSL, ambulance-oriented healthcare has its own set of regulations, as translated by the author of this paper (SOSFS 2009:10, chapter 2, 1 §); • Information cannot be changed by mistake, unauthorised use or due to mal- function or any other disturbance. • Every incoming and outgoing alarm, as well as other communication via logs, can be traced to operators at the alarm centre as well as health and medical personnel inside or outside an ambulance. • The requirements of regulation, EU 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free flow of such data and the repeal of Directive 95/46/EC (general Data Protection Regulation) as well as the requirements in the law (2018:218) with supplementary provisions to the EU’s data protection regulation and in the patient data act (2008:355) are met (HSLF-FS 2018:38). It will be impossible for this project without judicial expertise to abide by all regulations, but they can still act as guidelines for the design process. In particular, the first paragraph here highlights the importance of the design to decrease the risk of user error, either through prevention or easy correction. When deciding which patients to help, healthcare workers are regulated to follow the prioritisation platform with three guiding principles(Prop. 1996/97:60), as translated by the author of this paper: The human value principle is that all humans are equal in value and have the same rights independent of personal attributes and roles in society, Needs-Solidarity principle; resources should be distributed depending on the need, Cost principle when choosing between different operations or measures, a reasonable relationship between costs and effectiveness, measured in improved health and increased quality of life, should be sought. The design needs to support these principles because paramedics constantly have to make decisions and prioritizations during the triage process. It also allows for 19 3. Theoretical framework a better understanding of which principles guide EMS decision-making during the empathising part of the project. 3.5 Ethical challenges Healthcare is a sector where consideration of ethics is central. While doing ob- servations of ambulance workers, there is a need to ensure to not interrupt their work process in any way since this could cause risks for them and their patients. Situations can be witnessed where respect for the patient’s privacy is paramount and information that can be linked to them can not be spread under any circumstance. Carefully considering the design’s effects on healthcare workers’ working environment was required throughout the design process, as well as patient safety and privacy. For instance, some designs might give an AI algorithm more data, allowing it to make better decisions in the future. Still, it is possible that the interaction would be too slow, risking patient safety and requiring information that might go against the ethics of patient privacy. The design would be used continually every work day, so considering mental load and stress factors is paramount for the well-being of ambulance workers. 3.6 Artificial intelligence 3.6.1 Artificial Intelligence (AI) Russell& Norvig (2010) describe in the book that AI as a field of study aims to develop intelligent machines capable of performing tasks that typically require human intelligence. AI covers various approaches and techniques, (e.g. machine learning, natural language processing, and robotics). The stage of AI is based on the idea of creating algorithms and system models that can process large amounts of data and learn from collected data to make predictions or decisions. Russell and Norvig (ibid.) also mention that unsupervised machine learning involves developing algorithms capable of learning from data without being explicitly programmed. This allows algorithms to improve their performance over time without needing direct input from humans. A rising number of AI developments are also leading to ethical discussions and social issues, such as the potential for bias and discrimination (Zuiderveen Borgesius, 2018). 3.6.2 AI in healthcare AI has played a significant part in the public consciousness, gaining enormous publicity through models, for instance, chatbot ChatGPT and picture generators, e.g. Dall-E 2 and Midjourney. AI has become a more natural part of ordinary life through different tools and natural language processing with speech dictation. In addition to general use cases, specific AI tools have great potential to be used in professional fields like healthcare. Much research has been done on AI image classification (Razzak et al., 2018), such as detecting diabetic retinopathy (Abràmoff et al., 2018), cancer 20 3. Theoretical framework diagnosis and prediction (Goldenberg et al., 2019) and Covid-19 diagnosis(Hassan et al., 2022). Overall, AI can potentially improve diagnosis and prognosis compared to healthcare professionals, allowing for improved decision-making. In healthcare sector has specific requirements for AI healthcare. For example, WHO (2021) gives guidelines for using AI development, including transparency, explainabil- ity and the need to work against bias. However, AI can often be described as a black box, where we do not have the tools to understand too complex algorithms. Thus, AI needs to be developed with this in mind because it is essential for both developments of AI, patient autonomy and safety. Furthermore, it requires explainability; thus, medical professionals must make informed decisions. Finally, AI is often trained on data that already are formed from human bias, which needs to be heavily considered to reduce significant errors in AI decision support. Emergency services (EMS) play a pivotal role in healthcare outcomes and introduce new challenges compared to the AI tools previously mentioned. For example, an ambulance rarely has the same tools for diagnosis as a hospital, there is a wide variety of patient symptoms, and time can be minimal with acute symptoms. AI can potentially help triage patients, both in speed and in the case of reducing over-triage and under-triage (Buendia et al., 2015; Keselman et al., 2022), increasing patient safety and reducing resource depletion. AI models require real-time digital data to function, and while many current IT systems have the potential to provide it, they often need to be appropriately used in practice (Porter et al., 2020). In addition, they can be suboptimal for the complex workflow of EMS personnel, resulting in the input being done after the patient has already been sent to a hospital. A design that considers the many factors impacting EMS and looks into cases where IT implementation has been successful is thus paramount. 3.6.3 Interaction and Artificial Intelligence (AI) Interaction design and AI are two things that are increasingly intersecting or often come across each other. Interaction design focuses on designing effective, intuitive, and feasible products, systems and services for users. In contrast, AI involves developing intelligent machines capable of performing tasks that require human intelligence. As AI becomes more advanced and more involved in direct human interaction, it is used to enhance and improve the user experience in digital products and services. For instance, AI-powered chatbots can provide personalised and responsive customer service. At the same time, machine learning algorithms are used to personalise content, recommendations and search results. However, He et al., (2022) discussed designing effective and engaging interactions between humans and AI-powered systems may encounter several challenges, such as designing transparency and understandable interactions with users so that they can trust and feel in control of the system, including designing inclusive and equitable interactions that prevent biases or discrimination. To address these challenges, designers might consider AI’s ethical and social implications in their design process 21 3. Theoretical framework and develop user-centred, inclusive and transparent interactions. 22 4 Methods This chapter goes through the methods used during the study. It provides an overview of the methods themselves and why they are used for this project. Some of the methods had minor modifications to be better adapted to the context of the study. 4.1 Literature study A literature study provides an overview of the current state of knowledge and identifies gaps or areas for further research. The literature study will include an extensive analysis of research articles, books, reports, and other sources of information related to the project. Considering this, conducting a literature study would be in several stages. The first stage is identifying a research question or topic that is specific and narrow enough to allow for a comprehensive search of literature but also broad enough to capture the key concepts and issues related to the topic The next step would involve searching for relevant literature using databases, online libraries, and other sources of information. The search terms should be carefully selected to capture all relevant literature. After identifying potentially relevant literature, the quality and relevance of each source need to be filtered and evaluated. The evaluation process included reading each source carefully, assessing each study’s methodology, research design, data analysis and conclusions and determining its relevance to the research questions. Finally, the findings should include synthesised analysis, summarising each study’s key findings, identifying patterns and themes in the literature, and drawing conclusions (Martin et al., 2012). 4.2 Ethnography research Ethnography is a qualitative method to study people and their culture in their natural settings, typically over an extended period. Hammersley and Atkinson (2007) explain that this method aims to provide a comprehensive understanding of a culture or subculture and its beliefs, practices, values and social norms. Ethnography research involves prolonged engagement and observation, including conducting interviews, taking notes, collecting artefacts, and participating in their cultural environments. In this case, ethnography research was done in two ways: following ambulance workers for a day in Halland and visiting the primary ambulance central in Gothenburg. 23 4. Methods Halland uses a digital interface, and Gothenburg uses pen and paper; thus, visiting both regions was to gain a more comprehensive view of the field. In addition, both were important to understand the broader context of the ambulance work environment. This includes gaining insight into what tools are used, how personnel interact with patients and relatives, how they react to emergencies, work culture, and interaction with the IT-/pen & paper system. 4.3 Interview Interviews are a great way to empathise with the users. Therefore an interview is an excellent method to gain an in-depth understanding of users’ values, perceptions and experiences. The method allows us to ask specific questions while remaining open to exploring the participants’ points of view. The interview methods are flexible and often combine with other research methods, such as usability tests or surveys, to gain better insights into objective results by asking users about them and to elicit their subjective opinions on the products or interactions (Martin et al., 2012). To avoid bias, during this project’s exploration phase, the context expects to be a busy environment. For better results, The interview shall be combined with the ethnography method in terms of contextual inquiry. Contextual inquiry can be made in different manners. For instance, the designers ask participants questions simultaneously while observing their interaction with a product or while they are performing a daily activity. The user is also asked to explain the interaction as though explaining it to novice users. Another way is to observe first and refrain from asking the participant any question until afterwards to avoid influencing or interrupting their behaviour. Interaction design foundation (2002) has addressed the limitations of this method. "User interviews can be very informative and helpful, but only if they are used correctly and for the right things. Therefore, it’s important to know what you can expect to get out of interviews and what you shouldn’t expect to get out of interviews." "What users say and what they do are different," said Jakob Nielsen, Usability Expert and co-founder of Nielsen Norman Group. 4.4 Personas Personas are a method to consolidate archetypal descriptions of user behaviour patterns into visual profiles that aim to humanise design focus to use in test scenarios and design communication. Personas typically present in short profile descriptions include a name, a photograph or chest, and a narrative story describing in detail such as living situations, goals, and behaviours relevant to the design question. Personas are used as human references by the team throughout the design. Often helpful in all project phases, e.g., developing, discussing, and presenting product or system design in the definition and ideation phase. They are also valuable for checking use scenarios and highlighting positive experiences and potential break- 24 4. Methods points. Personas provide a persuasive human reference when communicating research summaries and scenarios to clients. (Hanington & Martin,2012) 4.5 Sketch Sketching is an important activity in the design process, as it facilitates designers in exploring and conveying ideas visually. Crazy 8’s and Solution Sketch are two sketching exercises frequently employed in the ideation phase of the design process to generate and refine design concepts. Crazy 8’s is a high-velocity sketching technique- and Solution Sketch focuses on iterating and refining a single design concept. Crazy 8 involves folding a sheet of paper into eight equal sections and setting a timer for eight minutes. Designers sketch as many ideas as possible within the timeframe, with the goal of generating at least eight unique concepts. This technique’s rapid and voluminous idea generation stimulates creativity and helps designers overcome design blockages (Reynolds, 2018). Once a set of ideas has been generated through Crazy 8’s, Solution Sketch can be used to develop a specific concept further. Solution Sketch is a method that involves detailed sketching of a single design concept while also considering its feasibility and implementation. The primary objective of Solution Sketch is to create a well- defined and elaborated design concept that can be presented to stakeholders for their feedback (Meroni et al., 2019). The combination of Crazy 8’s and Solution Sketch provides a robust ideation and design iteration framework. Crazy 8’s fosters designers to generate many ideas rapidly, while Solution Sketch provides a structure for refining and developing a specific design concept. Together, these techniques can assist designers in overcoming creative barriers, producing innovative solutions, and refining design concepts into practical solutions (Cooper et al., 2014). 4.6 Prototyping Typically, prototyping involves creating an initial version of a product or feature that allows people to interact with it. The purpose of a prototype is to convey abstract concepts such as features, functionality, potential benefits, risks, and implementation costs. Then, testing prototypes with real users provides insights into what works, what does not, and whether it is the right solution to pursue. It also allows continuous iteration without spending needless resources on something that will not work satisfactorily. A prototype is great for gathering feedback to keep improving the design. However, prototypes can vary in fidelity, and the quickest and most cost-effective ones are typically based on initial wireframes. Wireframes are in grayscale layouts that capture the structure, navigation, information architecture, layout details, and interfaces. Using sketches, task flows, and sitemaps to consolidate this information in wireframes is often an intermediate step between sketching and prototyping. However, depending on the design stage and prototype objectives, a prototype can be created by rapid paper prototypes, clickable wireframes, hand-coded interfaces, 25 4. Methods or high-fidelity prototypes resembling the final product. As a general guideline, it is beneficial to prototype early and frequently. Then, most importantly, test the prototypes and iterate based on feedback and insights from these tests (Houde and Hill, 1997). 4.7 Usability test Usability testing is commonly used in design practice to evaluate systems by testing with end users or representative users. However, there are different ways of conducting usability testing, such as formal, informal and pilot (“down the hallway”) tests. • A formal usability test is a traditional way. Typically the researcher invites participants to the lab and performs the test or testing in their work environment to get a closer result to the end product. • Informal testing is usually minor testing that can be done over time, with one or two users per week. • The “down the hallway” test is “quick and dirty” testing similar to a pilot test that involves friends or colleagues who are testing the designs. Usually, this type of test can lead to bias; therefore, it is helpful before properly testing a design on behalf of representative users. The usability test includes creating a plan to scoop out the test. The plan consists of user tasks and the sample issue, which will be given to the user to solve; recruitment criteria, scripts and discussion questions, and logistical information followed by facilitating or moderating the test. Then we can analyse data and finalise the test report to communicate the findings. Documenting the result is essential to help other people who were not involved in the testing and is a reminder for researchers when they need to revisit it (Rubin & Chisnell, 2008). Interaction design foundation (2002) also describes usability tests as structured research sessions that help us understand apparent problems with a particular design. They are a trial run for a change to our product where we can see if potential users are likely to need help to complete specific tasks. Such a test usually involves observing users attempting to complete tasks and can be done for different designs. It is often conducted repeatedly, from early development until a product’s release. In this project, we have combined usability testing with the Think-aloud protocol. The think-aloud protocol is a method that requires participants to verbalise what they are doing and thinking as they complete a task, revealing aspects of an interface that delight, confuse, or frustrate (Hanington & Martin, 2012). 26 5 Results This chapter starts with the results from background research. It then moves on to the design process, followed by the final design and its evaluation. 5.1 Literature study During the divergent phase of the project, a literature study was conducted. The main aim of the literature study was to gain insights into the current state of IT systems used in ambulances, the performance of current triage systems, as well as current uses of AI in healthcare in general, and explainable AI(XAI) in particular. The secondary aim was to find gaps in current research. The main sources of searches were Google Scholar, Pubmed and Chalmers Library. Also, some articles were added as per recommendations from experts, supervisors or stakeholders. Keywords included, but were not limited to: “AI”, “AI healthcare”, “explainable AI”, “IT-support ambulance”, “triage models”, “over-triage” and “under-triage”. In addition to work directly related to the field, literature studies were used throughout the project to gain insights into relevant work within user interface (UI) and human factors to help guide the design process. Many findings and articles are used throughout the text, but some key findings are summarised below. 5.1.1 Current use of it-support Current it-systems used in ambulances in the UK are often underwhelming, leading to workers noting information on their hands or pen and paper before putting it into the system only after the patient has been sent to the hospital (Porter et al., 2020). Thus the information can’t be used for digital decision support in an early stage. Key issues were that many it-systems were still in a transitory phase and that there were few standards for hardware and software, providing challenges for interoperability. Providing the correct information to the correct medical professionals was also a challenge. 5.1.2 Current triage decision support in Sweden Magnusson (2021) compares two dominant triage systems in Swedish ambulance care, the Rapid Emergency Triage and Treatment System(RETTS) and the National Early 27 5. Results Warning Score (NEWS) (Royal College of Physicians, 2017), showing that RETTS has higher sensitivity for time-critical patients but less specificity than NEWS. Both systems have issues with both over-triage and under-triage, with about one-third of children being either over- or under-triaged and under-triage in general being an issue when evaluating the elderly. 5.1.3 Status of Explainable AI in Healthcare While several applications for AI were found, described more in the background section, research on explainable AI (XAI) was very limited. Antoniadi, et.al., (2021) reviews the current literature and finds that it is only in very recent years that XAI in healthcare has really started to be explored. The majority of XAI was used for tabular data while research on XAI used for text analysis was the least common. XAI can improve decision-making and increase trust in the system. However, XAI research in clinical decision support was extremely limited, especially when it came to user research. Figure 5.1: Gap analysis map of this project’s literature review 28 5. Results 5.2 Ethnography research This session will discuss ethnography research on how we used the methods adapted to the project in best practice. And the results were interpreted from this particular field study. However, in a phase, we aim to understand the user’s point of view and emphasise as much as possible their role to be able to learn their daily routine. This project got an opportunity to explore two different regions in Sweden. One is Halland, and the other is VGR. Furthermore, due to the positive response from the Halland region, the team had the opportunity to follow up their routine for a day from 7.00 o’clock to 20.00 o’clock (one full shift). 5.2.1 Empathy The primary approaches used in ethnography research are observation and contextual Inquiry. The observation is typically “learning by watching” while taking notes on the essential things happening that are considered valuable later on. The most important thing is to act calmly, dress like them, be neutral as much as possible, try to understand what is happening and interfere less with the EMS work routines, in combination with the form of an interview (contextual inquiry) that is described as “asking while doing it”. If they are at some point wondering about their action or confused in some ways, we rather question “the right way or can ask later on as appropriate” (Hanington & Martin,2012). Figure 5.2 shows the results from EMS’s general context in the ambulance workflow and work environment interim of understanding and empathy. Unfortunately, some images are not allowed to be shared in this report. There were several actions and tools we didn’t understand at first, but everything was noted after receiving answers from them. 5.2.2 Strategy and product objective Figure 5.3 displays an overview of the existing system on the market. It compares clear user Interfaces from different products used in emergency service. This overview provides additional insights beforehand that can be considered as State-of-the-Art (SOTA). Oxford Dictionary (n.d.) describes SOTA as “the most recent stage in the development of a product, incorporating the newest technology, ideas, and features.” SOTA’s input has motivated us to learn the design patterns and the current trends of the existing system. It allowed us to analyse the pros and cons of the popular systems currently on the market that may lead to a better design later on. Learning their strategy also notified us about the most obvious errors and weaknesses that might be prioritised in the next system update. 5.2.3 Identify and understand user To be able to get a better comprehension of the user and their workflow, we have created personas to consolidate archetypal descriptions of user behaviour patterns, 29 5. Results Figure 5.2: Ethnography research; Observation & contextual inquiry from regions, Halland and VGR) 30 5. Results Figure 5.3: SOTA table showing product diversity. 31 5. Results Figure 5.4: The second latest version from Paratus. see Figure 5.5, and a journey map, see Figure 5.6, that analyses the context based on the observation data. The journey map is meant to provide a heuristic analysis narratively for the team. For example, it may begin with a brief explanation of users and behaviours in their work environment setting, followed by phases, actions, trends, and narrative facts. Figure 5.5: One out of four developed personas. Fictional profiles that represent user behaviour patterns based on collected data. As a result, the journey map should review a clear understanding of user touch points, a clear understanding of the channels in which actions occur, an understanding of any other actors who might alter the user experience, and a timescale. In conclusion, the journey map presents a heuristic view of the observation addressing problems and identifying activities, action, interaction, and satisfaction in the EMS workflow based on the user personas, similar to the AEIOU framework which stands for activities, environments, interactions, objects and users. Hanington & Martin (2012) describe this framework as an organisational framework reminding the 32 5. Results Figure 5.6: Journey Map researcher to attend—document, and code information under the guiding taxonomy the name is based on. 5.3 Expert interview Stefan Jönsson - Future of healthcare in VGR Highlights: • New IT-solution in VGR called Millenium on the path to be implemented • Live data transfer with high interoperability • Ambulance part of Millenium planned but hasn’t started proper development An interview was conducted with Stefan Jönsson, process manager of emergency care, working with regional development at VGR. He has responsibilities for the implementation of a new IT system for ambulances and emergency departments. Currently, different hospitals in VGR use different journaling systems, making it difficult or impossible to transfer data between them. A new system called Millenium is currently in the process of being implemented, allowing for higher degrees of interoperability. The planned launch date is 2025 but without all functionalities. The system for ambulances hasn’t started development yet, however, some func- tionalities already existing in Millenium might be used for it. The plan is to have a joint journaling interface, both for the ambulance and the ED. Both should be user-friendly and quick to use. The journaling works in real-time, with different 33 5. Results healthcare systems being able to add information to a patient. For example, the emergency dispatch centre might already have information on the patient- such as the type of accident and that could already be filled into the same system used by the ambulance. When passing through the healthcare chain, the patient should be continuously evaluated. Access should be granted to previous journals within the system, combining data from different sources such as the 1177 medical advisory call centre, local hospitals or specialist hospitals. Finally, since the EMS IT system will be integrated with the hospital system, ambulance personnel should be able to book a time for a patient to visit a hospital instead of just recommending the patient to book a time themselves. This should relieve pressure from ambulance workers to solve less severe cases while providing better healthcare service to the patient. When it comes to UI in the experience of the expert, healthcare professionals want explainable AI to be able to understand the decision process and make informed decisions themselves. In addition, an input system for AI should work alongside the general EMS Clinical Decision Support Systems (CDSS) such as NEWS. Borgström, CSAM - Paratus demonstration Highlights: • Demonstration of the next to latest Paratus ambulance IT-system • Flexible system, being able to adapt to any device size • Live data collection from the whole healthcare chain, allowing for better data integration. An interview was conducted with Jonas Borgström, product manager at CSAM, working on the Paratus ambulance IT system. CSAM has the advantage of having products in the whole healthcare system, allowing them to work with multiple kinds of data in real time and continuously utilise that in different ways. A demo of one of the latest editions of Paratus was given (Figure.), showing how it currently gives live feedback based on the RETTS triage system. At all times the user can also access healthcare guidelines, based on what type of situation it is in that the user preselects. During the majority of the interview time he showed us the demo of the system, which in general seems to have a high degree of usability. It was shown to be flexible, adapting to any window size provided, from desktop to tablet, to phone. The customer, for example, in different regions in Sweden can have different preferences for what contents should be a part of the system so the system is modifiable. Bengt-Arne Sjöqvist, Professor of Practice Emiratus, industry veteran Highlights: • Several current systems, all working similarly, often with a touchpad-based interface. 34 5. Results • Sees potential in collecting data live from several sources, both structured and unstructured. • Future of prehospital decision support should be able to be dynamic to facilitate as correct and early decisions as possible. • Should be able to drill down and get more information. Figure 5.7: An example of Dynamic Risk Prediction implementation structure with added active decision support, based on an example from Bengt-Arne Sjöqvist. Sjöqvist has extensive industry experience, a big part of the development of the Mobimed system, an IT system for ambulances that have become the industry standard and is widely used globally. When starting out, the goal was to do everything that was currently done on paper but digitally. Several other systems are currently on the market, but all of them share a similar interface, with main navigation on the side and with similar contents. The UI that is being used currently works at a satisfying level. The current level of data access and the possibility to gather and evaluate data in real-time brings many possibilities to decision-making. Sjöqvist has always worked towards making decision-making in the ambulance quicker and more efficient. One major area of improvement he sees in current digital decision support systems is for them to be more active and dynamic. Current decision support follows a decision tree structure where one step leads to the next. A first initial assessment is made. Then, if no immediate emergency is found, the workflow moves on to a more detailed assessment, then triaging, etc. Instead, the future of healthcare should actively in real-time check the information, and notify the user when enough information has been gathered to make a decision, thus quicker and more correctly coming 35 5. Results to a conclusion of what to do with the patient. Sjöqvist called this dynamic risk prediction, see Figure 5.7. Time is often the difference between life and death or a much longer time for rehabilitation, thus more efficient and faster decision support systems would be a great benefit for patient safety. In addition, Sjöqvist sees the potential of such a system to allow the user to “drill down” on the provided information, to understand what causes the system to give a recommendation and provide sources to it. Thus it becomes more transparent and the user can gain a greater understanding of causes for a higher or lower risk prediction, especially if the system goes against the experience of the ambulance personnel. During the interview, we were also given a quick demonstration of an older version of Mobimed to see how it was structured and how it handled information input. Anna Bakidou, the PhD student in trauma algorithm Highlights: • Both transparent and “black box” algorithms were tested with similar success rates. • Trauma has a high degree of triage error, making it particularly interesting for research. • Model based on information that could be gathered from a prehospital setting. • Not fully verified since it doesn’t take distance to the hospital into account for under-/over-triage. Anna Bakidou is a PhD student at the Care@Distance research group who has developed an AI model for trauma patients. There have previously been developed models for traffic-based trauma in the group, but her model is more general and attempts to work on all trauma cases. A primary reason for Bakidou to be interested in trauma is that it can be very complex. The variety of cases is big, making it a unique problem to solve. There is also a big problem of both under-triage and over-triage, increasing the need for better decision support. Various algorithms have been tested, from advanced neural networks to simpler and open algorithms. Both types performed similarly, making it possible that simpler algorithms are a better choice since neural networks are much more complex and thus lack the same level of transparency and explainability. However, since the algorithms haven’t been optimised and variables could change in future development, more research is required before being able to be sure that the simpler algorithm works better in this context. The models are trained and verified using the Swetrau database. It contains a variety of variables, such as what caused the injury and what body regions were impacted. The severity of the injury is also calculated using the new injury severity score. Much 36 5. Results of this information is currently not collected by ambulance workers, but the model is based on what kind of information could be collected in the ambulance setting. Currently, the model appears to be better on average than average at triage, compared to Swedish ambulance personnel. A major limitation of the model currently is that it doesn’t take distance into account. Thus, if a hospital is too far away, the ambulance might decide not to transport the patient to a hospital, and this could make it appear in the system as an under-triage. 5.4 Requirements Through the findings from our interviews and ethnography, we came up with a list of requirements for the system. • Integrate well into current and future systems: Ambulance IT systems already exist on the market. The goal of Care@Distance is to bring their technologies to ambulances as soon as possible. Thus, creating an entirely new system is not feasible. To make the implementation of their system realistic, the design should be made in a way for it to be adaptable to existing systems. • Blend into the EMS workflow: Obviously, for any system used in a professional environment, it needs to fit into the workflow of the personnel. The complex and ever-changing work environment of ambulance personnel makes this a big challenge. In addition, while AI is being developed and evaluated, it will need to work beside current triage systems. • Increase trust and understanding of AI: AI is new and more complicated than existing triage systems, making it more difficult to trust and understand. The decisions made are often about life and death, making trust in the system is essential in order for it to function since it is still the personnel that is responsible for the decision. • Efficient and effective regarding stressful environments: In order for the system to work, it needs to be used. This is currently not the case with many existing triaged based IT-systems, both according to our ethnography and a meta-study (Porter et al., 2020). In order for it to be used and not increase the cognitive load of the user, efficiency is essential. Thus, reduced steps to complete an action and easy overviews will always be paramount when making design decisions. • Versatile, support various kinds of situations: Should be able to be used both in very acute and less acute cases. In addition, the system will be based on trauma. However, for it to be relevant, it needs to be a possible design for other areas as well. For System requirements, We used the MoSCoW technique for system requirements to prioritise and distinguish what we must, should, could, and will not have in the design, see Figure 5.8. 37 5. Results Figure 5.8: MoSCoW mapping 5.5 Prototyping Throughout the project, feedback was gathered from stakeholders, users and experts, either through showing off our progress through online video conferences or through recorded videos. Thus we could gather information on what parts of the design were liked and which parts needed improvement. 5.5.1 Low fidelity This section presents the result of the low-fidelity prototyping for this project divided into tangible prototypes and digital prototypes. 5.5.1.1 Sketch and tangible Prototype We began with a tangible prototype where we structured, restructured, added, and discarded the design from the sketch, see Figure 5.9, as well as explored the actual size of the device by paper cutting, and experimented with different components that may fit within the UI concept. The tangible prototype provided hints of early stages of interaction that reflect the system flow and simulates the UI in the lower realism in a physical environment and allowed for rapid changes and testing of new ideas. Later on, the most relevant ideas were developed further into digital wireframes. See Figure 5.10. 5.5.1.2 Digital prototype The digital prototype allowed us to explore the interaction of a digital device, but it also demanded more time to build. In this phase, we used the interactive prototype to communicate the ideas with stakeholders and gain their early feedback. The prototype was reformulated according to the feedback and the new findings. 38 5. Results Figure 5.9: Sketching based on fictitious scenarios. Figure 5.10: Tangible prototype, brainstorming and generating ideas. 39 5. Results 5.5.1.2.1 Components They corresponded to four input categories: number entry, free text entry, and option picker. The categories were chosen based on what type of data EMS personnel currently need to enter for patient assessments. Each data entry method is based on a material design suggestion for an Android device; the guideline was developed by Google in 2013 (Material Design, 2023). However, We have adopted some elements to this project for ambulance services. For instance, The button and the free text entry require a bigger size than the standard mentioned in the Material Design due to the shaking of a moving ambulance; dark colour needs stronger contrast to notice in stressful environments. Number entry Figure 5.11 • Keypad, Allows one to select a subject and enter values through a keypad. • Dictate, Allows the users to add input numbers during hands-full tasks. Figure 5.11: Example of number entry solutions. Free text entry Figure 5.12. • Keyboard, To allow users to select the field in which they want to enter data and use the digital keyboard to do it and a physical extension keyboard is available if required. • Dictation, To provide an alternative option while physical contact with the device is not possible. Users can dictate by simply pressing the microphone icon or using word commands to activate it. Option picker Figure 5.13 • Checkboxes, Typically similar to a conventional checklist on paper, allowing the user to check on each box by tapping the icon or the text connected to it. • Dropdown menu: A solution to hide the options under a menu and reduce required space. It allows the user to click the dropdown icon and select it from the menu. • Chip: Allows the user to make a selection, filter content and trigger action. The chip can show multiple interactive elements together in the same area. 40 5. Results Figure 5.12: Example of free text entry solutions. Figure 5.13: Example of option picker solutions. From left: Checkboxes, Dropdown menu, Two versions of input chips 5.5.1.2.2 User experience (Emotions) Another brainstorming session was set to discuss the UX aspect of the design. For example, to understand and have an overview of what emotions to evoke or what dynamic should be emphasised to motivate positive experiences for the users. Therefore, the Mechanics-Dynamics- Aesthetics (MDA) framework was adopted in this project to gain an overview of the relationship between interaction and emotions and to know the needed mechanics for enabling it. See Figure 5.14. 5.5.1.2.3 Physical format The design considers adaptive design as the main, user interface adapted to different screen sizes. The EMS personnel use multiple devices according to the circumstances and situations. However, this project is mobile tablet-oriented based on the result from the exploration phase that tablets are used the most in the ambulance. 5.5.2 Wireframe The wireframe was presented in a neutral tone with a grey scale but with good enough contrast and well enough for communication. However, the focus was on how to display the AI in a comprehensive way in the ambulance setting. A few suggestions were granted and allowed to combine ideas (e.g., cards vs chart) Here seen as both an individual page in Figure 5.15 or as an overlay in Figure 5.16. Besides the main focus of AI, for input to the system we have digitised the pen & paper acute journal utilising WEST, which is being used in the VGR today and is used by the EMS personnel in making prioritisation decisions for the patient cases, 41 5. Results Figure 5.14: Diagram of MDA framework Figure 5.15: Early wireframe AI presentation with multiple cards representing different values. 42 5. Results Figure 5.16: Early wireframe AI presentation overlay with a chart comparing different values. 43 5. Results see Figure 5.17. We reformulated the design a few times. This allows for a more complete workflow for the nurse, with both input into the system, and then the AI results as output. Figure 5.17: The WEST paper journal and an early digitised version of it. For the full-sized paper journal, see Appendix A The wireframe strategy testing not only allows the project to move forwards faster but also enables us to make several better adjustments in the design by trying this early prototype on the actual device. Unfortunately, things were not as smooth as expected; at this stage, we did not have access to the actual device yet, but we speculated using Figma software in the testing mode and continued development while aiming attention to reliability and functionality to make the system more efficient. This solution aimed to solve the following; 1. to solve the issue with misinterpreted handwriting in the current paper journal, 2. to include most information in one device, 3. to reduce time consumption by fewer interactions or a number of clicks, 4. to increase visibility and reduce overwhelming information, 5. to emphasise the most critical information. 5.5.2.1 Decision support presentation The Decision support consists of four main parts: 1. The top bar displays the overall risk prediction using the same colour codes currently in use by ambulance services. It also includes the confidence level 44 5. Results of the decision support, which can increase or decrease depending on the information input. 2. The guideline is a part where the decision support provides feedback and recommendations. It also provides further details accessed by pressing "more" as an option. 3. Decision explainable AI (XAI), this part is a passive presentation of what the AI decision is based on. The view can switch into two modes, both displaying the same info. The card mode shows all predictors that the AI was using for the risk prediction. It shows them in two rows, one for a serious condition and one against a serious condition. They are sorted from highest to lowest contribution. The chart view is more limited in that it only shows the top three predictors of for or against a serious condition but allows for a quick overview of their individual contribution. A horizontal chart was chosen instead of a vertical one to be able to contain all the required information about the predictor on the side. An exploration about the possibility to drill down information on the predictors was explored, see Figure 5.18. This could be important for learning the logic behind the system. However, after the ethnography research where interaction with the system was very quick and after asking a nurse for feedback, this feature would seem to be unlikely to be used during triage. This, combined with the added knowledge acquisition this feature would require and time constraints, the feature was not investigated further. Instead, the functionality was imagined to be moved to the summary page of the journaling system where nurses would have time to investigate. 4. Missing value allows the user to input information that the AI thinks is the most important for increasing its confidence level in the prediction. It allows for quick input without going through the whole system or changing views. Figure 5.19 displays a result wireframe as a whole. It includes pages that develop according to the imagined scenarios and builds for usability testing. There are several thoughts from this process on how the system should look. 1. The AI page should be able to assist at any time. 2. The AI prediction should be visible at all times. 3. The top bar should present the patient’s brief info and the EMS personnel responsible. 4. Pages that are long must be able to pan down. 5. More visuals (Icons) should be added to reduce the focus on reading text. 6. Having much space in between content helps against overloading users. 7. Size of text and icons needs to be bigger for the ambulance environment. The more developed version of the wireframe, see Figure 5.20, was used to communicate with the stakeholders and an experienced EMS nurse. A video 45 5. Results Figure 5.18: Drill down functionality during triage. demonstration for a workflow was recorded using voice and screen recording and was then sent to gather feedback. The same video was used in supervision as well. Points from the feedback included the need to create a versatile design and have multi-optional choices due to the requirement of individual nurses in the ambulance context. The feedback included discussions about UI elements like how to present the confidence rate for AI risk prediction (e.g., score system, scaling system and dot system). Another point was about the XAI presentation, where we discussed how to visualise and present it. Two options were presented; the cards view or the chart view. The card view gathered more positive feedback, but it was concluded that both options should remain for the ambulance personnel to decide. 5.5.2.2 Visual design Visual design covers several important aspects which increase usability and positive user experience by establishing a visual hierarchy: the most probable should be most prominent, position (top, middle, and bottom), visual attribute (colour, size, space, typography, readability, and aesthetics), alignment (use of the grid system to minimise visual excise and group object) and balance (left and right of UI). 46 5. Results Figure 5.19: The wireframe in different stages in the system. 47 5. Results Figure 5.20: Interactive prototype flow created in Figma Figure 5.21: The graphic describes UX design, including Visual Design. Adapted from Interaction Design Foundation (2002). 48 5. Results 5.5.2.2.1 Accessibility In the following design approach, we focused on inclusive design (Imrie & Hall, 2003). aim to increase the design quality in respondent behaviour to the needs of individuals. The design intends to consider information visualisation, which considers some adjustments toward ambulance services based on information visualisation perception for design (Ware, 2019). Thus, the mood board was created to brainstorm and generate ideas for visual communication inclusivity; colour, lightness, brightness, contrast, and constancy. See Figure 5.22 Figure 5.22: A mood board presenting early visual interface and potential design. 5.5.2.2.2 Interaction This part is about improving the interaction to ensure the design has a high functionality standard for an ambulance and enhances the user’s positive experience. First, however, avoiding switching the page for the leading AI page is crucial. We are considering the data manipulation loop into account. According to Ware (2019), At the lowest level is the data manipulation loop through which objects are selected and moved using the basic skills of eye-hand coordination. Delays of even a fraction of a second in this interaction cycle can seriously disrupt the performance of higher-level tasks. The option in the AI page to gain time reduction and voice switching page to reduce the reload screen, e.g. where the card is located, the user can continue to pan the card to beagle to view the rest of the card. See Figure 5.23. Another general UI included Icons, checkboxes, a change in surface colour, or a combination of highlighting to minimise mistakes and have better control over the interface. 5.5.2.2.3 Layout The layout, see Figure 5.24, is designed to fit the different screens. Therefore, it should be