Safer secondary tasks in partially self-driving vehicles Master’s thesis in Industrial Design Engineering CHENGLIN SONG RAN AN DEPARTMENT OF INDUSTRIAL AND MATERIALS SCIENCE DIVISION DESIGN & HUMAN FACTORS CHALMERS UNIVERSITY OF TECHNOLOGY Gothenburg, Sweden 2022 www.chalmers.se http://www.chalmers.se/ MASTER’S THESIS 2022 Safer secondary tasks in partially self-driving vehicles CHENGLIN SONG RAN AN Department of Industrial and Materials Science Division of Design & Human Factors CHALMERS UNIVERSITY OF TECHNOLOGY Gothenburg, Sweden 2022 Safer secondary tasks in partially self-driving vehicles Master of Science Thesis In collaboration with Autoliv Sverige AB © Chenglin Song & Ran An, 2022 Department of Industrial and Material Science Chalmers University of Technology SE-412 96 Göteborg, Sweden Phone +46(0) 31 - 772 10 00 Cover illustration: Steering wheel screen concept, by Chenglin Song & Ran An Printed by Repro Service Chalmers Gothenburg, 2022 Abstract This Master Thesis was conducted at the Industrial Design Engineering program at Chalmers University of Technology in collaboration with Autoliv Sverige AB. Self-driving cars from Level 1 to Level 3 that are currently ready to be put into the market, the proportion of time that the system controls the car gradually increases. In the future, higher-level autonomous driving technology can further reduce the distraction of perception factors, freeing people's hands to do other things, such as checking emails, watching videos, and reading. We believe that there is a possibility of a screen on the steering wheel of the car to perform secondary tasks. The aim was to come up with conceptual graphical user interface design alternatives for the touch screen control that could provide safer secondary tasks during SAE L3 automation. Academic research in related fields, user study and benchmarking were performed before designing. The deliverables were two concepts for steering wheel screens in different directions, including concept sketches and high-fidelity prototypes of the user interfaces for Gmail and Youtube respectively of the two concepts. Concept I uses a mobile phone as a screen mounted on the steering wheel to present information and provide control functions. The phone could be connected with the vehicle system to display the user interface through the center screen. In addition the user can use a stick and a voice assistance physical button to control the center screen for secondary tasks. Concept II integrates a touchpad screen on the right-hand side of the steering wheel as a small display and controller for the central display. The screen can switch between a display and a touchpad when performing secondary tasks. A final user testing was performed after building the high-fidelity prototypes. The test results show that the proposed concepts can meet the project objectives to a certain extent, and further professional testing is needed. This thesis provides a feasible scheme for level 3 autonomous vehicles to perform secondary tasks, and open up new possibilities for level 3 and other autonomous vehicles to perform secondary tasks. Acknowledge Over the process of researching and writing this paper, we would like to express our thanks to all those who have helped us. First, we would like to express our gratitude to our examinar and supervisor, Bijan Aryana, who gave us lots of suggestions during the process of researching. Sincere gratitude should also go to Autoliv and our colleagues, especially our manager Eric Moragues who gave us utmost help and encouragement for researching and designing. It is an honor to work with Autoliv and Eric on the master thesis project. We would also like to thank all the participants in the user studies and user tests. Thanks for their time and effort they invested in this project. Finally, our gratitude should also go to Jinhong Guo and Yodit Kinfe, our opponents, for your helpful feedback on our work. Gothenburg, June 10th 2022 Ran An & Chenglin Song Table of Contents 1 Introduction 1 1.1 Background 1 1.2 Aim and Scope 3 2 Literature Review 4 2.1 Aim 4 2.2 Findings 4 2.2.1 The potential advantages of a screen on the steering wheel 4 2.2.2 Autonomous Vehicles 10 2.2.3 Human Factors For Driver-Vehicle Interfaces 12 2.2.4 UX guidelines of Android for cars 16 2.3 Discussion 20 2.4 Conclusion 21 3 Benchmarking 22 3.1 Ethical 23 3.2 Process 23 3.2.1 Steering wheel screen product or concept in the market 23 3.2.2 Functions of steering wheels in the market 24 3.2.3 Functions of the vehicle center display in the market 25 3.3 Result and conclusion 26 4 User Study 28 4.1 Define target group 28 4.2 Ethics 29 4.3 User study 1 30 4.3.1 Method and purpose 30 4.3.2 Process 30 4.3.3 Results 31 4.4 User study 2 32 4.4.1 Purpose 32 4.4.2 Method 33 4.4.3 Process 33 4.4.4 Result 34 5 Use scenario 35 5.1 Purpose 35 5.2 Scenarios 37 5.3 Analysis 38 6 Design goals 39 7 Physical Concept Generation 40 8 Concept I 48 8.1 Wireframe 50 8.2 Low fidelity prototype 1 52 8.2.1 Home page 53 8.2.2 Gmail 53 8.2.3 Youtube 56 8.3 Low fidelity prototype 2 59 8.4 High fidelity prototype 60 9 Concept II 66 9.1 Wireframe 66 9.2 Low fidelity prototype 1 68 9.2.1 Touchpad screen 68 9.2.2 Center screen 69 9.3 Low fidelity prototype 2 76 9.3.1 Gmail 78 9.3.2 Youtube 79 9.4 High fidelity prototype 80 9.5 Connection of interfaces 85 10 User Test 87 10.1 Method and purpose 87 10.2 Process 88 10.3 Results 88 11 Discussion 91 12 Conclusion 95 13 Reference 97 Appendix A: Definitions for Terms Related to Driving Automation Systems 100 Appendix B: Questions in User study 1 103 Appendix C: Participants’ answers of the interview in User study 1 105 Appendix D: User requirements statistics 113 Appendix E: Precess of User study 2 117 Apendix F: Results of User study 2 121 Apendix G: User test questions 124 1 Introduction 1.1 Background With the development of AI technology and deep learning algorithms, as well as the arrival of the Internet of Things era, autonomous vehicles are the general trend. People are looking forward to the arrival of fully self-driving cars. Self-driving cars from Level 1 to Level 3 that are currently ready to be put into the market, the proportion of time that the system controls the car gradually increases. As self-driving cars are likely to significantly improve road safety, countries want to be at the forefront of this development (Vellinga, N. E., 2017). In the future, higher-level autonomous driving technology can further reduce the distraction of perception factors, freeing people's hands to do other things, such as checking emails, watching videos, and reading. This will transform our experience of commuting and long-distance travel, keep people away from high-risk work environments, and allow industries to grow and collaborate to a higher degree.In the future, our reliance on and relationship to cars will be redefined - reducing carbon emissions and paving the way for a more sustainable way of life. The development of autonomous driving technology will bring different benefits and challenges to the existing transportation system. Compared to the advantages mentioned above, there are problems with making rules, re-examining highway codes, public opinion, improving the infrastructure of streets, towns and cities, and so on. Determining ultimate responsibility for road accidents is a bigger issue. As an increasing number of road tests of autonomous driving technology are implemented, countries have successively introduced corresponding laws to adapt to the changes brought about by this new technology as soon as possible. On 6 January 2016, EU member states signed a declaration on strengthening cooperation in the field of connected and autonomous driving. Through this declaration, member states recognize the importance of a coordinated approach to facilitate the cross-border use of connected and autonomous vehicles One of the objectives of the Member States is (…) to work towards a unified European framework for the deployment of interoperable connected and autonomous driving (…). This will promote legal consistency, recognizing that the legal framework is flexible enough to facilitate the introduction and cross-border use of autonomous and connected vehicles (Vellinga, 2017). It can be seen from the speed of response in various regions that autonomous driving technology has been widely concerned and researched. Many car companies and related companies have begun to study new energy vehicles and autonomous driving technologies and related fields. Autoliv, the world's largest supplier of automotive safety equipment, has also conducted corresponding research. Because autonomous driving technology provides drivers with time and space to perform secondary tasks, there is a possibility of a screen on the steering wheel of the car. In the past years, steering wheels in passenger vehicles include physical buttons to trigger some functions while driving. However, the transition from manual to automated driving is creating lots of assisting functions and features (e.g., speed limiter, adaptive cruise control, lane-keeping assist systems, etc.), which increase the complexity of the driving activity. Consequently, there are possibilities of replacing physical buttons for touch screen displays. Touch screen steering wheel controls enable the possibility of 1 creating a flexible user interface, where functions and buttons can change and adapt according to the driving context. SAE is the leader in connecting and educating mobility professionals to enable safe, clean, and accessible mobility solutions. The standards set by SAE represent the best technical content developed in a transparent, open, and collaborative process. To advance automation technology, clarify the role of the driver, answer legal questions, and clarify definitions, the SAE On-Road Automated Driving (ORAD) Committee and ISO TC204/WG14 through a Joint Working Group established a working group in 2018. This collaboration brought to bear the knowledge and expertise of global experts in driving automation technology and safety. Several new terms and definitions have been added and multiple corrections and clarifications have been made to address frequently misunderstood concepts and improve the utility of the document. The concept of level 3 mentioned in the article comes from the document proposed by the organization. As a conclusion, a Level 3 self-driving car is a car capable of self-driving in some circumstances but needs the occupant to be able to take-over in a short amount of time. Figure 1 shows the summary of level 3 driving automation in terms of five elements. Figure 1:Summary of level 3 of driving automation (Made by SAE INTERNATIONAL) While the safety implications of this concept in the context of purely assisting systems (where the driver is responsible for the driving activity all the time) are still explored, this concept could have potential benefits in a car with SAE L3 of automation. In this case, while the car is performing the driving the system could provide embedded applications (e.g., reading emails) that allow the performance of safer secondary tasks (compared to e.g., using a smartphone), since the eyesight would be closer to the road and hands would remain on the wheel. Therefore, it is worthy to explore putting a steering wheel screen on a Level 3 self-driving car. 2 1.2 Aim and Scope The exploration of the steering wheel screen starts with academic research in related fields so that the knowledge gap can be filled by the study of related technologies and guidelines. We need to study the existing similar products, discover design regulations, and design products that meet the market and users. In addition, we need to think about the design from the user's point of view, user study is required to obtain requirements and build analysis models. The next task is to come up with a conceptual graphical user interface design alternative for the touch screen control that could provide safer secondary tasks during SAE L3 automation. Besides, evaluate them from the user experience and cognitive ergonomics point of view. In addition to the conceptual graphical user interface design, we must consider the combination of hardware and software. Because most electronic systems, whether self-contained or embedded, have a predominant digital component consisting of a hardware platform that executes software application programs (De Michell, G., & Gupta, R. K. 1997). To achieve a complete interactive user experience, the design of the user interface is inseparable from the physical implementation, that is, the specific form and implementation of the screen need to be considered. Therefore, in addition to the conceptual graphic user interface design, we also need to design the physical level of the steering wheel screen, explore its specific style, display mode, interaction mode, etc. Therefore, the aim and scope of the project is to conceive and design a steering wheel screen that meets the design specifications and ergonomics to help users complete secondary tasks for SAE L3 autonomous vehicles, including the industrial design of the screen and the user interface design as a combination. 3 2 Literature Review 2.1 Aim The research field of this project is mainly in autonomous vehicles and in-vehicle human-computer interaction systems. Although the team members have basic knowledge and experience in ergonomics and human-computer interaction design, we lack the background and knowledge for autonomous vehicles and in-vehicle interaction. Therefore, the main purpose of this literature study is to fill the knowledge gap in related fields and understand related design specifications. Some of the specific topics that the literature study investigated are: ● Learn about the field of autonomous vehicles, especially how Level 3 autonomous vehicles work. ● Explore the potential benefits of having a touchscreen on the steering wheel. ● Learn the design guidelines of in-vehicle human-machine interface from the perspective of ergonomics ● Learn and analyze UX guidelines of Android for cars In addition, in the process of literature study, research vacancies in related fields may be found, which may have guiding significance for the follow-up research direction. 2.2 Findings 2.2.1 The potential advantages of a screen on the steering wheel Reduce the distractions from using mobile phones About 1.35 million people lose their lives each year due to road traffic accidents around the world. Between 200 and 50 million people suffered non-fatal injuries from these accidents, many of them permanently disabled. Road traffic accidents have a significant economic impact on victims, resulting in a loss of approximately 3% of GDP for the entire country. Supported by data, driver distraction is the biggest cause of road traffic accidents (Hua, Q., Jin, L., Jiang, Y., Guo, B., & Xie, X., 2021). As the most commonly used multi-functional electronic device, mobile phones are considered to be an important factor causing driving distraction. Drivers need to use mobile phones to complete specific secondary tasks during driving, such as making calls on the highway, sending text messages when waiting for red lights, etc. According to the World Health Organization, the use of mobile phones and other electronic products quadruples the likelihood of road accidents (Chand, A., Jayesh, S., & Bhasi, AB, 2021), and mobile phone use (texting or talking) is associated with 26% of car crashes are related and the frequency is increasing. Cell phone use while driving is directly related to work obligations and overconfidence in the ability to drive while talking/texting 4 (Engelberg, J. K., Hill, L. L., Rybar, J., & Styer, T., 2015). Due to busy personal affairs or traffic jams, many drivers use fragmented time to read emails and make phone calls in the car. Research shows that drivers text at least some of the time (30%) and at red lights (66%) while driving on the highway (Engelberg, et.al, 2015). The National Highway Traffic Safety Administration (NHTSA) has categorized three types of driving distraction: visual (eyes off the road), manual (hands off the wheel) and cognitive (mind off of driving) in their policy statement (NHTSA, 2011). So Texting while driving (i.e. any direct manipulation of a handheld device) is considered particularly problematic because the behavior combines manual, cognitive and visual forms of distraction. Likewise, multi-resource theory suggests that visual-manual auxiliary tasks (such as smartphone use) may negatively affect primary visual-manual primary tasks (such as driving a car) because both tasks use the same cognitive resources (Naujoks, F., Purucker, C., & Neukum, A., 2016). Therefore, when the driver looks at the mobile phone, his vision is off the road ahead, at least one hand is off the steering wheel to hold the mobile phone, and his thinking is separated from the actual dynamic driving task, and instead, a lot of cognitive ability is applied to how to reply to text messages. When encountering an emergency, the driver's cognitive ability is limited, and it is impossible to completely separate the cognition from the second task in a short period of time, resulting in the inability to respond in time and the occurrence of danger. With the advancement of communication technology, autonomous driving technology, and entertainment devices, different levels of autonomous vehicles and manual vehicles will share the road in future mixed traffic environments. Hence, multiple factors will increase the cognitive burden of drivers and lead to distraction (Hua, et.al, 2021). Avoiding danger to drivers requires avoiding the use of mobile phones while driving, but the popularity of automated driving can provide drivers with an opportunity to complete a secondary task in certain situations. This project is based on the L3 autonomous driving technology provided in specific situations, the completion role of the driving task requires frequent switching between the driver and the automatic driving system, which requires the driver to quickly get out of the distracted state to avoid danger in a timely manner. However, using a mobile phone to complete the second task in the process of automated driving still occupies a lot of cognitive resources of the driver, so it is worth exploring adding more interactive functions that can complete the second task to the car itself, and the current car has come with more functions beyond the driving function. In order for the driver of an automated car to safely complete the second task of driving, it is first necessary to make the driver's field of vision as close to the road ahead as possible, thereby reducing the degree of visual distraction. The degree of manual distraction can be reduced by keeping the driver's hands on the steering wheel as much as possible. So exploring the steering wheel user interface allows the driver to quickly switch between their roles (ie, action initiator and system supervisor). This is also the starting point of this project, providing the driver with a user interface on the steering wheel that can complete the secondary task interaction. We need to explore its possibility to ensure the life safety of the driver of the automated car. This is in line with the goal of UN sustainable development goal 3 to reduce future fatalities from traffic accidents through the development of vehicle safety equipment. 5 Steering wheel related factors Since the design objective is the steering wheel, there are many factors to consider when a touch screen is installed on the steering wheel. We should understand the basic structure of the steering wheel to decide where to install the screen without affecting the basic functionsl. In addition, the ergonomics of the steering wheel is also important, which affects the operability of the screen and the user's willingness to use it. So we decided to study the basic structure of the steering wheel and the way drivers generally hold the steering wheel. Through the study of the steering wheel, we intend to further understand the form of the screen and the possibilities of how to use it. As shown in figure 2, take the steering wheel of Porsche 997 II as an example, which consists of basic structure, steering wheel corpus, steering wheel cover with horn function and airbag (Harrer, M., & Pfeffer, P. (Eds.)., 2017). It is well known that the airbag module in the center of the steering wheel is a mature and effective life safety protection device, and its position cannot be changed at will, which adversely affects the driver's safety. So the airbag is a compromising factor in the design of the steering wheel screen, which limits the size, material and shape of the user interface. At the same time, the car horn is currently in the form of a bulky physical button. For the driver, requiring them to hold the steering wheel at the 9 o'clock position with the left hand and the right hand at the 3 o'clock position or lower prevents the airbag from pushing the driver's hand back onto the driver's body in the event of a injury (Meschtscherjakov, A., 2017). Which is also one of the factors that limit the form of the steering wheel screen. Figure 2:Exploded view of a steering wheel (Porsche 997 II) (Harrer & Pfeffe, 2017) Steering wheels have been added more and more electric functions and elements. One of the most widely used components is the multifunction switch, for example, to operate a car computer or a navigation system/radio. Gearbox operation/manually selected automatic or double-tight gearboxes tend to be moved to the steering wheel, etc. In addition, modular design and customized steering wheels are a future development trend (Harrer & Pfeffe, 2017), especially after the popularization of automated driving technology, more functions 6 are added to the car and even the steering wheel. When the steering wheel is fully customized, it can achieve stronger inclusiveness, such as accommodating the use of skilled left-handers, or it can satisfy drivers of different types of work. This is another reason to explore the possibilities of steering wheel user interfaces. Hruška, M. (2018) studied the ways and actions of the current left-hand drive car drivers to grip the steering wheel through experiments, and expressed the proportion of the number of people holding the steering wheel and the gender ratio in the corresponding position. From the study we can see that drivers of different genders hold the steering wheel differently. According to the experimental conclusion, the number of males holding the steering wheel with one hand accounts for more than 50% of the total male subjects in the experiment, 40% of the males hold the steering wheel with both hands, and nearly 75% of the females hold the steering wheel with both hands. In addition, the experiment obtained the position combination of the left and right hands of all-gender drivers holding the steering wheel, of which R3 and L9 accounted for the highest proportion, followed by R2 and L10. Through this experiment, we understand the user's holding habits, and the most suitable position on the steering wheel to place the interactive interface, which can help the user to achieve the purpose function faster and more accurately. Figure 3:Schematic depiction of the resulting grip percentage representation values in individual positions. Note: The values in percentages are calculated separately for the left and right hands. In addition to the experiments on the driver's holding habits, Hruška (2018) also conducted experiments on the body posture of the driver who gripped the steering wheel with one hand. Holding in this position for a long period of time results in significant lateroflexie with significant muscle strain, in particular m. quadratus lumborum, m. obliquus externus abdominis, m. obliquus internus abdominis and m. erector spinae. Long-term driving in such a position can lead to pain in the lumbar spine and, in extreme cases, to permanent damage to the postural system. In the study, males using this unhealthy posture to drive are more than females. However, the design of the steering wheel interface can affect the driver's holding posture. Encouraging drivers to use both hands to hold the steering wheel through the design of the steering wheel screen is a direction worth exploring. It will help more drivers stay healthy, especially for long-term driving drivers. However, the emergence of 7 autonomous driving will inevitably free the hands of drivers, but as far as the current situation is concerned, L3 autonomous vehicle drivers still need to frequently take over driving tasks, which highlights the importance of steering wheels beneficial to health. Location of visual display and controls The location of the user interface also affects the driver's sight and operation. The location of the in-vehicle user interface has an impact on both the driver's reception and input of information. The suitable location of the user interface makes it easy for the driver to get information and perform actions. Because the steering wheel is located in front of the driver, it is one of the few things inside the car that offers excellent visibility and reachability (Meschtscherjakov, 2017). For conditional self-driving vehicles, a touch screen control on the steering wheel can provide drivers with a safer secondary task path in self-driving mode than looking at a cell phone or electronic device in other places. Current research shows that the position of a visual display has a big impact on how easy it is for drivers to get information. With the continuous development of vehicle assistance systems, more and more information needs to be displayed to drivers. The appropriate location of the information display can reduce the impact of viewing the information on the driver. For manually driven cars, reducing the frequency and time of looking away from the road and increasing the readability of information are possible ways to reduce the impact of displays on the driver. Human Factors Design Guidance For Driver-Vehicle Interfaces from NHTSA approves the study by Lind, H. (2007), arguing that locating the visual warning near the primary driving activity will enhance the likelihood of it being noticed and lower the amount of time required to glance at that information. More specifically, as shown in figure 4, the specification proposed by NHTSA (2016) requires that critical displays for continuous vehicle control or critical warnings related to vehicle forward-path are located within ± 15 degrees of the central line of sight but as close to the central line of sight as practicable. For conditional autonomous vehicles, we expect the driver to perform a secondary task while being able to have a certain degree of perception about the driving situation and can respond promptly when the car requests the driver to do the driving task. Although there is no specific research at present, referring to the standard of traditional vehicles, the driver's eyesight when performing the secondary task should be as close to the road conditions as possible. In addition, taking the Mercedes-Benz S-class vehicle as an example, as shown in figure 5, it will display a red light on the steering wheel when it requires the driver to take over the driving task. If the driver's sight is right on the steering wheel when performing the secondary task, the driver can detect the red light signal on the steering wheel and take over the driving task promptly. 8 Figure 4: Examples of visual display locations (Made by Human Factors Design Guidance For Driver-Vehicle Interfaces from NHTSA) Figure 5. The button on the steering wheel will display a red light when Drive Pilot requires the driver to take over the driving task 2.2.2 Autonomous Vehicles This project is based on Level 3 autonomous vehicles. In order to fully understand the autonomous driving system and its classification principles and driving methods, we studied and analyzed the document written by SAE International: Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. Below is our summary of definitions that may be used, and analysis of Level 3 autonomous driving systems. The detailed explanation of the definition can be found in Appendix A. 9 Definition AUTOMATED DRIVING SYSTEM (ADS) DYNAMIC DRIVING TASK (DDT) [DYNAMIC DRIVING TASK (DDT)] FALLBACK OBJECT AND EVENT DETECTION AND RESPONSE (OEDR) OPERATIONAL DESIGN DOMAIN (ODD) REQUEST TO INTERVENE [HUMAN] DRIVER [DDT] FALLBACK-READY USER Level 3 - Conditional Driving Automation The sustained and ODD-specific performance by an ADS of the entire DDT under routine/normal operation with the expectation that the DDT fallback-ready user is receptive to ADS-issued requests to intervene, as well as to DDT performance-relevant system failures in other vehicle systems, and will respond appropriately. Limitations from Level 3 autonomous vehicles L3 autonomous driving technology does not fully provide convenience and safety for human life. In an L3 self-driving car, the driving environment does not need to be monitored all the time, but intervention requests need to be responded appropriately. The self-driving system needs to provide "sufficiently comfortable transition times" where the dynamic driving task is handed over from the self-driving system to manual control, in that case, the driver acts as a backup measure to perform dynamic driving tasks. This means that the higher the level of automation, the less the driver will focus on traffic and the system, and the less able to regain control (Seppelt & Victor, 2016). In addition, compared to imperfect automation, manual driving by the driver is further from perfect, with 94% of crashes attributed to key driver-related causes such as identification errors, decision errors, and performance errors. This seems like a classic dilemma, if we don't automate we're stuck with human contributions to collapse, but if we choose to automate, human performance will get worse as automation gets better. The higher the level of autonomous driving, the more time the driver has to complete the secondary task, so that the more cognitive resources are occupied by the secondary task, it is more difficult for drivers to quickly switch from the secondary task to dynamic driving task. Similarly, L3 autonomous vehicles require the driver to be the supervisor and backup of the autonomous driving system, and also need to pay attention to dynamic driving tasks for a certain period of time, which is different from the autonomous driving systems below L2 and above L4. Level 3 autonomous vehicles will disguisedly increase the cognitive load on the driver as they have to focus on so many things besides secondary tasks during autonomous driving. This is also the reason why Seppelt & Victor (2016) promoted for changing L3 autonomous driving technology or resisting L3 autonomous driving. 10 Through a critical analysis of L3 autonomous driving technology, the current limitations of this technology and what additional considerations need to be made during the design phase are recognized. Although the automatic driving technology is graded according to the level of automation, the improvement of this system level is not linear with the driver's cognitive load and safety factor. Compared with other levels, L3 will increase the driver's cognitive load. But compared to using mobile phones and other in-vehicle screens to complete secondary tasks during autonomous driving, the advantage of the steering wheel screen is that the user's hands will not be too far away from the driving task, because the steering wheel itself and its position are not too far away from the vehicle control system and the effective field of view, the driver can take over faster. The steering wheel screen may be a rare possibility for L3 autonomous driving technology to help users safely complete secondary tasks. The Level 3 autonomous car produced by Mercedes-Benz The Mercedes-Benz Drive Smart Level 3 autonomous driving feature will become available in the S-Class luxury sedan and EQS electric luxury car in Europe by early 2022. Germany has taken a pioneering role in this with the opening of the Road Traffic Act (StVG) for Level 3 systems in 2017. On December 09, 2021, Mercedes-Benz became the first automotive company in the world to meet the demanding legal requirements of UN-R157 for a Level 3 system. Level 3 or conditional driving automation enables the vehicle to react to its environment and make decisions without urging the driver to take control. The autonomous driving of Mercedes-Benz is achieved through the Drive Pilot system.Mercedes-Benz's updated Drive Pilot system can take over the driving chores while the vehicle is traveling at the legally permitted speed of 37 mph (60 kph). After activating Drive Pilot via a pair of haptic buttons above the steering wheel, the system controls the driving speed and following distance while independently performing evasive or braking maneuvers – all without physical driver intervention. The Drive Pilot can be activated via two buttons on the steering wheel rim. If self-driving is available, they will turn white. If the Drive Pilot is activated, they will turn turquoise.If the buttons in the steering wheel rim turn red, the vehicle requests the driver to retake control within ten seconds. The S-Class is capable of reliable conditionally automated driving in traffic jams or when traffic density is high. The Drive Pilot controls the speed, the distance to the vehicle ahead, and confidently steers the car within its lane. As it does so, it can also recognize unexpected traffic situations and handle them on its own by means of braking or evasive action within the car’s lane. However, the Drive Pilot is designed to recognize its limits of conditionally automated driving. When the traffic environment does not conform to the conditions predetermined by the system, the system will request the driver to intervene and regain control of the vehicle. Since the Drive Pilot system can perform vehicle driving tasks, the driver can have time and energy for secondary tasks. The S-Class provides drivers with a variety of secondary tasks including working and recreation. For example, they could communicate with colleagues 11 online via In-Car Office, answer emails, surf the Internet, or read the news. The tasks are mainly completed through the media display. As the Mercedes-Benz's S-Class car has been put on the market, its autonomous driving scenarios and driving mode switching methods can provide a reference for our project. Since this project is to design a steering wheel screen for L3 self-driving cars, it does not specify which car it is designed for, so the existing L3 self-driving cars are considered to be the most reasonable reference targets. Some parts of this project that do not need to be redesigned will refer to the car's design and standards. 2.2.3 Human Factors For Driver-Vehicle Interfaces A key element of vehicle automation is the driver-vehicle interface (DVI). DVI is the vehicle display that provides information to the driver and allows the driver to control the entire vehicle and various vehicle components and subsystems status control device. The safe and efficient operation of any motor vehicle requires that the DVI be designed in a way that is consistent with the driver's constraints, capabilities, and expectations. This document is intended to help DVI developers of Level 2 (L2) and Level 3 (L3) autonomous vehicles achieve these results from a human factors perspective. The reason we chose to study this document was that it provided design guidance on the aspects of automotive ergonomics, which complemented our knowledge. At the same time, it can pave the way for further learning UX and UI design guidelines. Only by understanding the background For reasons from the perspective of human factors, we can better learn and apply design specifications. Below is the information in this document that we think will be helpful to our project, and our thoughts on how that information relates to our project. General Design Guidance for Level 3 Automation This part of the document highlights Designing Messages for Driver Comprehension. For L2 and L3 autonomous vehicles, developing and presenting messages that support accurate and timely comprehension by the driver are considered to be the design goals. The document proposes a design guideline for the comprehension process of drivers. Figure 6 shows what the documentation suggests to be considered at various stages of the driver's comprehension process. 12 Figure 6: Questions should be considered when designing messages from Campbell et al.(2018) This document then discusses driver needs related to message complexity and identifies the characteristics of visual, auditory, and haptic messages that affect message complexity. Message complexity refers to the number and types of basic information elements contained in a message, as well as the relationship between these elements. Overly complex messages may not be properly perceived, understood or acted upon by the driver. In order to design the presentation of information as simply as possible, while keeping the message accurate, the document makes the following recommendations: A. Visual Messages consist of simple icons and fonts with only the necessary detail included. In text displays, the number of lines of text per-message is minimized. B. Auditory Messages are simple when an immediate response is required. This could be single or grouped frequencies presented simultaneously; such as a simple tone that consists of a square wave. C. Haptic Messages are simple and perceptible. Research relevant to the topic of haptic message complexity is limited. Based on current industry trends, it is expected that many messages in L2 or L3 automation systems will be multisensory (presenting auditory or tactile messages accompanied by visual messages). This topic suggests matching the format of messages to the driver's tasks, needs, and expectations in order to improve driver comprehension and performance, and provides guidelines for supporting design. Visual information, for example, is best for presenting more complex information that is not safety-critical and does not require immediate action. Audible information quickly grabs the driver's attention and can be used to provide high-priority alerts and warnings. This means that the display on the steering wheel is more suitable for secondary tasks unrelated to the driving task. It is important to avoid the auditory information generated by the secondary task interfering with the driver's recognition and understanding of important driving-related alert information. Visual Interfaces The visual modality is of primary importance in the driving task, and can use various sensory dimensions, such as color, brightness, and contrast, as well as stimulus dimensions, such as 13 location, size, shape, and periodicity (e.g., flickering). Semantic content that benefits from persistence are better suited to be displayed through visual form. In addition, characteristics such as color, size, spacing, and temporal characteristics (e.g., flashing or noticeable motion) can be used to maximize the prominence, legibility, and understandability of warning messages. The document argues for the need to place the visual interface in a location that facilitates rapid extraction of information while minimizing line of sight off the road and negative impact on drivability, and gives potential visual display locations as shown in Figure 7. We believe that some of the visual displays in these examples might have a good linkage with the display on the steering wheel we are going to design. Figure 7:Examples of potential visual display locations Display Locations in Image: A. Head-Up Display B. High Head-Down Display C. Head-Down Display/Instrument Panel D. Center Console The document also emphasizes that when designing DVI, minimize glare, both on and from, visual displays. Glare on visual displays can originate from a variety of sources in the driving environment and can make visual displays difficult to read. In addition, the light emitted by the display can be harsh at night, causing discomfort or, in some cases, reducing the visibility of the external driving environment. The document also provides some strategies for addressing such issues. For example, mitigating glare on the display in daytime driving by providing sufficient display brightness and using high-contrast display technology to ensure sufficient contrast. Mitigating glare that emanates from the display while driving in darkness by displaying content with a dark background to minimize the amount of light emitted by the display. 14 In addition, the document emphasizes the application of color. Color is a characteristic of a visual display that can be used to convey the meaning or urgency of an alert signal. Compared with words and symbols, color has certain advantages in the immediacy of recognition. Here are some common relationships between color and information categories. Red is usually associated with danger or critical situations; white is usually associated with caution; green is usually associated with normal operation. One of our initial thoughts on this part is that when the interactive system switches between autonomous and non-autonomous driving modes, it may use a combination of color and other visual displays to achieve driver attention. The Temporal Characteristics of Visual Displays are also mentioned in this document, such as flashing, blinking or apparent motion, to command visual attention. Some related design guidelines are also mentioned in the document, for example, Flashing is used for important, suddenly-occurring, situations (optimal rate is 3-4 times/s); Multiple flash mode is used for more urgent situations (this mode uses rapid pulses of flash for each flash cycle). Driver Inputs As autonomous driving is an emerging topic, there is limited information on driver input methods for autonomous vehicles. Therefore, the document has written this section based on basic human factors information and current interface standards. Poorly designed controls can adversely affect or impair the operation of primary driving controls. It is important for designers to carefully consider the placement and operation of various types of controls (Pomerleau et al, 1999; Stevens et al., 2005). Controls that are easy to understand and operate can minimize the distraction during the transition from manual to autonomous driving. Here are some examples of relevant design guidelines given by the document. Controls should provide timely and clear feedback (visual, tactile or auditory) (AAM, 2006; Bhise, 2011). Have identifiable labels (symbols or text) that are visible and located close to the control (Bhise, 2011). The location of the controls should not adversely affect the control of the primary driving system controls. Usage of controls must not hinder the driver's choice to keep at least one hand on the steering wheel at all times. Control Placement is a point in this topic that deserves a separate discussion. Designers should ensure that control placement and operation do not interfere with driving tasks or use other driving controls. Here are some examples of accompanying design guidelines. The placement of controls should be easy to reach and find, and controls should be in a visible area or be able to be found blindly (Bhise, 2011). 2.2.4 UX guidelines of Android for cars The car screens involved in this project are all developed based on the Android system. In order to design a user interface that meets the specifications, we need to understand and learn the design specifications of UI and UX for the DVI of the car. Google Design for Driving is a design hub that provides design guidelines for developers working with the two Android for Cars systems: 15 ● Android Auto: Phone-based infotainment system that’s projected onto the screens of compatible cars ● Android Automotive OS (AAOS): Infotainment platform that car makers can customize, build into their vehicles, and (if they're GAS partners) integrate with Google Automotive Services (GAS) Design for Driving foundations At present, there is a lack of interaction design specifications for autonomous vehicles in the market, thus it should be noted that the design principles mentioned in this part are mainly for manual driving cars. We learned the conventional driving interaction design specifications and extended them to apply to the user interface design of autonomous driving reasonably. At the same time, since the L3 autonomous driving technology requires the role of users to switch driver and the fall-back ready user, the design with reference to the manual driving interaction design specification can make the user switch the driving role faster and safer. Interaction principles The interaction between the driver and the screen must be simple, non-distracting and easily interrupted so that the driver's attention can quickly return to the road. Keep information current and glanceable: To ensure that the information and status presented in the interface are clear at a glance, the driver needs to quickly understand the task or real-time updated system status by a glance at the screen. They should be able to finish reading in 2 seconds and turn their attention to the road. Also ensure that the system response time after user input does not exceed 0.25 seconds. If the content takes longer than 2 seconds to load, a spinner or similar UI change should indicate that the device is responding. Encourage hands-on driving: Achieving safer driving requires keeping the driver's hands on the steering wheel as much as possible. For example, avoid designing interactions that require two hands, and design one-hand gesture interactions when necessary to ensure that the driver has one hand to control the steering wheel and respond in time in an emergency. A hands-free speech interface and simple voice interactions are recommended with minimal visual and manual requirements for the driver to minimize driver distraction and help the driver focus on the road. Prioritize driving tasks and avoid pulling the driver’s attention away from the road for non-essential reasons. The most important tasks a driver performs are those related to driving – everything else must be secondary. Although we need to design the steering wheel user interface to achieve the completion of the secondary task in the autonomous driving process, we need to take into account the priority of the driving task, and the secondary task needs to make compromises for the driving task at all times. And considering the position of the touch screen on the steering wheel, it is close to the effective range of the driver's driving field of vision. Therefore, we need to combine ergonomics and cognitive theory to study 16 carefully about the degree of how the screen occupies the driver's cognitive load in different driving stages. Visual principles Make content easy to read: Content designed for car screens must be clear and easy to read, with a consistent UI and large touch targets that drivers can identify in all viewing conditions (day, night, bright light, etc.). Text legibility in a driving environment can be affected by many factors, such as lighting, time of day, font scale (thin, medium, bold) and contrast. Highly legible text helps drivers reduce browsing time and decision-making time, thereby reducing cognitive and visual distractions. First of all, the key information and long paragraphs should avoid using bold font, the minimum title type characters should be 32dp, the auxiliary text size should be at least 24dp, and the number of Roman characters per line should not exceed 120, otherwise it will seriously increase the visual burden of the driver. In order to ensure the readability of the text, the contrast ratios for text, icons & background should be guaranteed to be 4.5:1, besides, the night mode is necessary and the above contrast ratio needs to be changed appropriately. Generally, the contrast ratio is reduced in the night mode to avoid glare. It is worth mentioning that the black background is recommended as it is suitable for both day and night, because it is consistent with most car interior colors, and does not occupy visual resources to cause visual disturbances during driving, at the same time, can clearly display text and icons. Make targets easy to touch: The smallest touch targets should be 76dp*76dp and the minimum distance between two touchable targets should be 23dp. This prevents drivers from being easily distracted or making mistakes when trying to touch objects that are too small or too close to the screen. Keep UI elements consistent: To help drivers quickly understand their screen options, the user interface must be clear and consistent. First of all, using a consistent icon in the user interface, the color and interaction characteristics of the same category of information can reduce the time and cognitive effort required by the driver and makes decision-making easier. Color is a powerful cue for reinforcing memory and recognition, such as when dialing on the phone interface, green means initiating a call, red means hanging up, and the corresponding green and red colors should be used in the notification window to ensure the color consistency of the same interactive content. Users can quickly judge and react by color. Google Design for Driving presents basic in-car interaction principles and visual design principles in understandable language, each of which needs to be kept in mind in our project advancement, which is related to the safety of the people in the car. As at-a-glance information and simple interactions are required, while self-driving cars can allow drivers to be distracted for a while, using design to keep distraction to a minimum is what L3 self-driving technology hopes for. 17 Android Auto As the user interface of the mobile phone is connected to the vehicle, the user can control the app in the mobile phone by operating the car screen. Google Design for Driving gives strict design specifications in the System Design section, including color, layout, dynamic effects, various sizes and fonts. First of all, in terms of color, the contrast in the two modes of day and night proposed in the Visual principles above is emphasized again, and specific values are given for reference. In addition, the consistency of color usage is also mentioned. The Layout section introduces the layout, pedding space and specific values for different screen sizes, and the Sizing section specifies different button and icon sizes and bleeding values. Different page logic relationships should use different dynamic switching effects. The specification divides different motion effects into ● Switching between apps ● Switching between peer views ● Extending an existing action ● Minimizing and expanding an action ● Disrupting an action Each interactive operation needs to be supplemented with specific dynamic effects to express the logical relationship of the pages, so that users can understand their operation feedback more clearly. For example, in music playback software, the shared axis motion pattern when switching from song to song in a media app reinforces that both songs are in the same playlist. Different user input methods are also added to the design specification. The design of various components of the UI interface of different types of apps (media app, messaging app, navigation app, etc.) has also been standardized. For example, the UI principle includes how to design a standard-compliant music playback soft armor upper navigation column. Automotive OS Automotive OS is the advanced planted native user interfraction system, which was built into the car itself that can be used without additional connectivity. Its design specification is similar to that of Android Auto. However, as it is not restricted by the Android phone system, a lot of detailed content has been added to its design specification, such as the design principles of the dial keyboard and scrollbar. Conclusion Google Design for Driving places special emphasis on interaction principles and visual principles, and is a good resource library for learning and evaluating automotive UI & UX. The touch screen user interface on the steering wheel has many possibilities, its shape, size 18 and interaction method are very uncertain at the current stage. However, it cannot be designed without any design specifications, therefore the principles of interaction & vision still need to be considered. Most of the design norms can be used for reference and application, these design norms may need to be further developed and thought deeply by ourselves to practice and evaluate in our design results. While some design norms are not applicable, we need to study and evaluate more deeply then select what we need after understanding the guidelines. 2.3 Discussion In the above part of the article, numerous literatures helped us understand the possible benefits of designing a touch screen on the steering wheel, and also provided us with design guidance from the aspects of domain background knowledge, ergonomics and interaction design. But they all have their own limitations due to the lack of solid experiments and data. For the steering wheel screen, although we believe that it has potential advantages in reducing driver distraction caused by mobile phone use, helping drivers maintain a better holding gesture and providing a better location for information and control, but there is no relevant experiment to prove that Level 3 autonomous vehicles can achieve such a purpose with a steering wheel screen. We can't verify the reality of these potential advantages until we have a Level 3 self-driving car and a matching steering wheel screen. But these potential advantages can serve as the direction for our in-depth study in the follow-up research. Our design goal is to design an interactive system that provides secondary tasks for level 3 autonomous vehicles. After our design is complete, we can use this system to design experiments to test the potential benefits of previous assumptions. In addition to the advantages, there may be potential disadvantages of using the steering wheel screen, such as whether the position of the steering wheel screen is suitable for long-term viewing, and whether the steering wheel screen and other in-vehicle displays will interfere with each other and affect the driver's judgment? These questions are also the directions that can be explored using the results of the project. The background study on Autonomous Vehicles mainly helps us understand how a Level 3 autonomous vehicle works and how it differs from manual driving. We understand that ADSs of level 3 can perform DDT only after meeting a certain ODD, but the document does not propose what specific ODD is. This is because different ADSs have different ODD limits, and different Level 3 autonomous vehicles may perform DDT in different environments and for different durations. In the literature review section we cannot clearly describe the use environment of a Level 3 autonomous vehicle, so this is of limited help in defining the use scenarios of our product. In the subsequent definition of the use scenarios, we can only select some common situations as the use scenarios of the product. As for the relevant design guidelines, although they provide us with very useful design guidelines and specifications, most of them are based on ordinary manually driven cars. While the document from NHTSA is designed for Level 2 and Level 3 autonomous vehicles, as is often described in the document, due to the lack of relevant experiments, more guidance is based on experiments with manually driven vehicles. The design guidance provided by Google also faces the same limitations, because Google provides design 19 guidance for the interaction system of ordinary manually-driven vehicles, so the specifications in it, such as character size and other visual principles, are not for level 3 autonomous vehicles. Considering that our design is more specific to the state of the user when not performing DDT, design guidelines to reduce user distraction while driving will not needed to be followed or only partially followed. There is no clear judgment standard to help us decide which specifications need to be followed, so all the design guidance must be considered in the design process, and judgments and decisions should be made according to the actual situation. As a conclusion, the biggest limitation of the literature in this field is the lack of recent experiments to provide analysis and design guidance for the interactive systems and steering wheel screens of autonomous vehicles. Therefore, our project is of great importance in this field. After the design results are obtained in this design project, the follow-up research can be carried out using the design results. 2.4 Conclusion The use of mobile phones during driving is the main factor causing traffic accidents. This behavior combines manual, cognitive and visual forms of distraction and consumes a lot of cognitive resources of drivers. For L3 cars and below, more frequent distractions are more dangerous, but it's hard for people to avoid making calls or doing other secondary tasks while driving. Level 3 self-driving technology allows the car to drive by itself under certain conditions, giving the driver an indeterminate period of time to perform secondary tasks. According to the definition of SAE International, the driver as a fallback-ready user needs to take over at any time, and failure to take over in time will lead to an accident or the termination of the journey. Therefore, the closer the driver's visual center is to the road ahead when completing secondary tasks, the safer it will be. This is a limitation to how users accomplish secondary tasks, and one of the technical drawbacks of L3 self-driving cars. Level 3 autonomous vehicles require a secondary task interaction system that can quickly disengage the driver from task content or keep the drivers’ status close to manual driving. The original intention of the steering wheel screen is that for the driver, the position of the steering wheel has the best visibility and reachability. If an interactive screen is provided in this position instead of the mobile phone for the driver to complete the secondary tasks during automatic driving, it can reduce the impact of the distraction of using mobiles, keep the driver's attention and vision as close to the road ahead as possible, and keep hands close to the steering wheel. When the system asks the driver to take over the car again, they can go back to manual driving in time. Considering the structure of steering wheels and how to hold them, our design should encourage or guide the driver to use both hands to hold the steering wheel on L9 R3, which contributes to airbag deployment and physical fitness. In the design stage, we need to define the secondary task according to the user's requirements. It is necessary to consider which can be solved by the steering wheel user interface, and which design specifications need to be referred to to design the user interface. This is one of the difficulties of this project. Because the project involves a wide range of fields and there is no corresponding reference basis, it has considerable freedom and possibilities. 20 21 3 Benchmarking Anand, G., & Kodali, R. (2008) summarize benchmarking as a continuous analysis of strategies, functions, processes, products or services, performances, etc. compared within or between best-in-class organizations by obtaining information through appropriate data collection method, with the intention of assessing an organization's current standards and thereby carry out self-improvement by implementing changes to scale or exceed those standards. Due to the possibilities of the project and the uncertainty of the future, we need to learn from the relevant in-vehicle interaction interface and functions of the vehicle. Since users cannot currently experience L3 autonomous vehicles, we hardly have the possibility to get the real experience of the completion of related secondary tasks from the user's point of view. In the absence of corresponding real user data, studying the relevant behaviors of users and the products that have appeared in the market or product concepts can help us gain insight into the development direction of related technologies in the current market. Therefore, in this project, we need to benchmark as many related vehicle interfaces and functions as possible. For the general direction, the content we study in benchmarking is divided into three directions, which may affect the design goals: 1. Steering wheel screen product or concept in the market 2. Functions of multifunction steering wheels in the market 3. Functions of the vehicle center display in the market First of all, investigating the steering wheel screen concepts that are similar to this project in the current market is required. In order to better analyze the current enterprises' unique understanding of the given direction and different solution concepts, the following aspects are summarized: ● Steering wheel screen size ● Location ● Interactive mode ● Connection with other screens ● Functional group ● The meaning of the screen ● Whether/what kind of self-driving car Due to the limited number of announced and actual products currently in the market, it is difficult to discover the design motivation and design trend of the corresponding concepts, so we need to find possibilities in the human-machine interface of all cars. Therefore, benchmarking in related fields is also necessary. For example, in the design phase, it is necessary to consider which functions of the steering wheel can be integrated into the screen; if the steering wheel screen has a subordinate relationship with other car screens, then what kind of functions of other car displays can be controlled by the steering wheel screen. Therefore, the market research on the necessary functions and the second task functions of the steering wheel is the second part of benchmarking, and the research on the functions of the current mainstream vehicle center control screen is the third part. 22 3.1 Ethical The ethical factors that need to be considered in benchmarking is the confidentiality of the information, so illegal channels are not applicable to obtain the confidential information of the company. Protection of business secrets is an ethical aspect that needs to be considered during the benchmarking. Therefore, in the process of benchmarking, the source of information should be considered, and information that is authentic and credible and does not affect business secrets should be obtained from formal channels such as the official website of the enterprise as much as possible. 3.2 Process 3.2.1 Steering wheel screen product or concept in the market At present, there are no related steering wheel screen products that have been released in the market, but their various functions can be studied logically and reasonably through a wide range of related concepts released by well-known companies in the automotive field. First, we started from different companies, inquired about the latest product releases, and summarized the concepts of the steering wheel screen into a table. The following information in table 1 was found from the official introduction websites of the relevant companies. BYTON m-byte Lixiang L9 Hyundai Scale Rectangle 7’ Rectangle unknown 2 rectangles unknown Position Steering wheel upper center Middle top edge of steering wheel Where the thumbs are How to interact Touch & gesture Touch Touch Press hard for activating Connection between other screens The steering wheel screen and the center armrest screen are used as the controller of the central large screen, Affect the big screen by interacting on these two Unknown Unknown 23 screens Function groups Main menu is customizable. Music, maps, air conditioning, seats, games, photos, meditation mode. Touch Bar: Quickly operate vehicle functions All functions in the car – from radio volume to seat heating Purpose of the screen Apps and functions selection, use as a pad Quickly operate basic car functions Basic car functions Is there an ADS? L3+ L4 Unknown Table 1: Benchmarking of steering wheel screen concepts and products in the market 3.2.2 Functions of steering wheels in the market Considering the comprehensive and valuable information in the market, it is valuable to summarize the steering wheel functions of various vehicles from many brands. In this benchmarking process, the main research object is the multi-function steering wheel. The following content in table 2 comes from the official websites of the corresponding companies. Left function group Right function group Form Basic driving functions Keep distance, speed limit Volume adjustment slider Media controller Voice control switch Heat steering wheel Bump sensation with panels that move ever so slightly so that certain features actually feel like a button being pushed Basic driving functions Speed limit, Keep distance Home, direction, back Phone call Media controller Customize function Voice control switch Home, direction, back Button feedback display Phone call Media controller Customize function Voice control switch Basic driving functions Speed limit, Keep distance Physical buttons Small roller 24 Left function group Right function group Form Keep distance, speed limit Media controller Voice control switch Pressing panel buttons Turn-signal headlight button Scroll: volume (up and down), changes the audio track (left and right) Car horn, wiper, voice control switch Scroll: "Autopilot" (press) Pilot speed (up and down) Touch-sensitive buttons and two scroll wheels Table 2: Benchmarking of functions that the steering wheels have in the market 3.2.3 Functions of the vehicle center display in the market Finally, we took the Tesla Model 3 as an example to study the functions of the in-vehicle central display (figure 8). It is a unique industry benchmark in the current market as the screen in Tesla has the most abundant functions and the best visual effects. Tesla Model 3 center screen includes basic vehicle function controllers that control the vehicle itself through user-friendly visualization operations, such as the air-conditioning wind direction. Other functions like Vehicle Assistance, Navigation, Music, Tricky Entertainment, Music, and Send Messages, are also provided by the center display. The implanted system’s interface first displays the content related to safe driving by default, and other functional content is displayed in the form of modal windows and drawers. The level of the operation interface is reduced as much as possible, and the content is put away after use to significantly improve the use efficiency. The most important and most frequently used function buttons are as close as possible to the driver's side of the display. In the bottom function bar, the one closest to the driver's side is the Model 3 model icon. Clicking it will call up the vehicle's detailed settings page. Because the capacity of the bottom function bar is not enough, Tesla specially designed an application drawer and placed it in the position of the third icon. Drivers can click it to use applications in the drawer, such as browser, contact list, energy consumption curve, and so on. 25 Figure 8: Interface of Tesla Model 3 center screen 3.3 Result and conclusion Through benchmarking in the current market, it can be seen that the steering wheel screen appears in the form of concepts, and the number of related concepts is still small. First of all, some companies in the industry tend to use the screen instead of the traditional physical buttons on the steering wheel, so as to achieve the purpose of simplicity and full customization to allow users to define their own steering wheel. This type of screen is mainly responsible for quick vehicle setup on a smaller scale. In Hyundai's concept, there are two rectangle screens on the spokes where the thumbs are when a driver is holding the steering wheel with both hands. The screen on the left shows the basic function settings of the vehicle. The right screen acts as a controller for the selected function on the left. The concept first guarantees the position of the user's hands on the steering wheel when performing the corresponding operation, making the transition process for the user to take over the driving task shorter and the deployment of the airbag safer. At the same time, the screen is located at the position of the index fingers of both hands (Meschtscherjakov, 2017). In order to prevent accidental touches, the function triggering method of the screen is also different from ordinary clicks. It needs to be pressed hard within a short period of time to trigger the corresponding function. Although this may reduce the operating efficiency of the driver, from the perspective of user-centered design thinking, preventing misoperations also greatly reduces the risk of accident, so similar triggering methods can be considered . Hyundai's concept is not specifically designed for self-driving cars, because more attention is paid to the possibility of using a touch screen to replace the previous physical buttons on the steering wheel. Lixiang L9 is also a recently released car product, as for the steering wheel screen, the physical buttons are replaced by the touch bar above the airbag cover. In this touchbar, the 26 basic control functions of the vehicle are provided. At the same time, since the vehicle adopts the form of HUD to replace the traditional instrument panel, the steering wheel screen also undertakes the display function of some vehicle information. On the whole, the steering wheel screen does not help users complete the second task, but it serves as a replacement for some physical buttons and instrument panels. However, due to the small size of the screen, it is not conducive to the display of information, so it appears on the steering wheel as a functional touchbar. BYTON gave a different answer to the market. A larger screen that is not rotated with the steering wheel is placed in the upper center of the steering wheel area. First, the screen is used more like a tablet, with plenty of secondary task apps. The steering wheel screen also has a frequent connection with other screens in the car. For example, during automatic driving, the driver can project the video selected on the steering wheel screen to the center screen and watch it together with the occupants in the car. Although the screen is fully functional, BYTON still retains the relevant physical buttons arranged on both sides of the screen. During automatic driving, the driver’s hands can leave the steering wheel to complete a variety of secondary tasks by operating the steering wheel screen and the center screen. Compared with the other two concepts, BYTON gives a more complete and effective answer to the user's secondary task requirements during autonomous driving, giving the driver a lot of freedom to use both hands when the vehicle is driving automatically. However, it prevents the user from quickly taking over the driving task and increases the possibility of injury to the user in the event of an emergency. The three steering wheel screen concepts currently released in the market propose three distinct solutions. The current market is not prepared enough to adapt to autonomous vehicles. It will be difficult to find a balance between encouraging users to perform rich secondary tasks and safer autonomous driving. Safer autonomous driving encourages the driver to keep both hands on the steering wheel during autonomous driving, which has obvious benefits both in terms of the speed at which it takes over the driving task and the secondary injury of airbag deployment (Meschtscherjakov, 2017). The steering wheel functions configured in most automotive products have obvious design commonalities. The function groups of the latest steering wheel products are mainly divided into two types. The first function group is responsible for setting the basic functions of vehicle driving, including distance maintenance, automatic cruise switch, light switch, and so on. This part of the function group is set on the left steering wheel spoke of the thumb position held by the left hand in 4 of the 5 brands of vehicles investigated, and most of them appear in the form of physical buttons. Another type of function group is responsible for the secondary tasks, which generally include answering calls, voice control, volume adjustment, and other media control functions. This type of function group appears in the form of physical buttons on the right-hand steering wheel spoke. As a highly integrated steering wheel, Tesla adopts a form of multi-function buttons, through the driver's pressing and rolling in different directions, it can play different control functions. Besides, the left side function button is responsible for the basic driving functions of the vehicle, and the right one controls the media functions, which is similar to the market trend. At present, cars in the market have made a clear distinction between the functions of the left and right-hand controllers, so as to avoid confusion between the secondary task functions and the driving functions. When considering the functions of the steering wheel screen, it is 27 also necessary to follow the general regulations of distinguishing the function groups for the two hands. The function buttons are mainly in the form of physical buttons, which can be operated by the driver without vision, which greatly reduces the cognitive burden of the driver. The button feedback display, which is partially reused in the currently launched cars, makes the entire steering wheel integrated. At the same time, in order to achieve the same effect of reducing cognitive load and driving distraction as physical buttons. Interactive feedback is important to this type of button, they should give users a similar experience through realistic pressing effects and vibrations. Various interactive feedback effects need to be considered, especially when the user uses a screen to perform corresponding operations. 4 User Study Through literature reading and benchmarking, the corresponding design specifications and related design concepts in the market are introduced. Then a user study is considered to be needed, and it is one of the important design processes used to discover user needs. The steering wheel, a product that is closely related to the user, needs a user-centered design method to obtain the user's needs and usage problems. However, the design objective of this project, the L3 autonomous vehicle, is at the conceptual stage in the current market, so we cannot obtain the actual user experience and problems of users. Therefore, it is considered to cut into the problem from the side. Although the results of user study cannot directly reflect the needs of users, regulations can be discovered. The user study was mainly divided into four parts (as shown in figure 9). First, reading papers and meetings with Autoliv were used to determine the target user group for this project. Second, interviews were used for the target group to obtain the corresponding mobile phone and driving habits information, and discover the needs for secondary tasks. Thirdly, through the practice of the Bodystorming method, we studied the relationship between the user's preferences of display position and the secondary task interface. Finally, after obtaining the results of two user studies, potential secondary task requirements were discovered and analyzed. Figure 9: Process of the user study 4.1 Define target group According to literature, compared with higher and lower level autonomous vehicles, under certain conditions and imperfect autonomous driving will significantly increase the cognitive burden of drivers(Seppelt & Victor, 2016) because the driver needs to be the fallback-ready 28 user, who needs to pay attention to the dynamic driving task frequently. This shows that L3 autonomous vehicles have high requirements for the driver's cognitive ability and other related abilities. In addition, Günthner, T., & Proff, H. (2021) studied the acceptance of autonomous driving technology by users of different ages. It was found that most people start to feel a decline in their physical ability and cognitive response-ability from the age of 50. Usually, the elderly will avoid participating in secondary tasks as much as possible for their own driving safety due to their physical decline. Although self-driving technology is beneficial to the elderly with declining physical ability, let them make up for the lack of physical ability through the self-driving system. But for the market, this result means that the usser group of older drivers, in particular, can be persuaded to use advanced driver assistance systems by means of a suitable and user-focused value proposition. For the elderly, higher-level and more perfect self-driving systems are needed. Besides, trust is equally important to all user groups for the usefulness and ease of use of autonomous driving technology, but tends to be slightly higher among younger users and is more willing to engage in secondary tasks. So in this project, young adults (20-35 years old) and adults (36-50 years old) with driving skills are our primary target group, they can quickly adapt to new technology while participating in secondary tasks with more willingness. The occupations and lifestyles of the two target groups are different. Young adults are more willing to use mobile phones, and their needs for secondary tasks are more youthful and trendy. Adults, as employees, have a higher demand for information input. Therefore, the results need to be counted separately in the following interviews. 4.2 Ethics In order to obtain the requirements of users, it is necessary to study the daily behavior of users, which will inevitably involve corresponding personal privacy issues, and some issues may include the daily behavior habits of respondents. Especially for the study of driving habits, due to the current lack of self-driving cars, it is not entirely legal for some drivers to complete secondary tasks while driving. As legal and safe driving is admired and distracting from driving to secondary tasks is not recommended, we need to avoid tying users' actions to the name and identity of the individual. The interviewee's name and address information should not be interviewed and recorded when recording the interview results, and the interview materials should not be screen-recorded during online video interviews. Because the protection of the personal privacy of respondents in the user research stage is one of the ethical factors that need to be paid attention to. On the other hand, it is necessary to avoid the description and guidance in the questions that are contrary to the laws of the country, so as to reduce the guidance of the interview questions to the respondents on unsafe driving habits. This helps promote safe and responsible driving. Finally, gender equality and occupational equality also need to be considered. 29 4.3 User study 1 4.3.1 Method and purpose In order to more effectively analyze the user requirements we need from other perspectives, we need to use a long time to communicate with users. Only by communicating as deeply as possible, we can discover potential needs from the behavior of users. Therefore, it is particularly important to guide interviewees to fully recall their daily behaviors and think deeply. Through the most suitable form of interviews, we help the interviewees to open up their hearts and speak freely through progressive questions and more flexible time. We can also ask follow-up questions and discuss with interviewees to further explore the needs of users. In order to more comprehensively analyze the corresponding secondary task requirements from other perspectives, we need to start with the mobile phone that is currently used to complete secondary tasks to study the corresponding mobile using behavior, including daily use and the use during driving. In addition, we also enriched the research on user expectations in the interview, so that the interviewed drivers can first mobilize their thinking in the form of recalling what they did this morning, then transitioned to daily behavior, and finally imagine future scenarios. Therefore, the purpose of this interview is mainly to obtain the user's requirements for using mobile phones, the needs of users for using mobile phones during driving, and the user's imagined preferences and requirements for secondary tasks during autonomous driving. 4.3.2 Process First, we developed step-by-step interview questions, starting with the user’s travel experience that day, asking the user to recall how they used their phone during their commute, and why. The interviewees were interviewed on their recent long and short trips and asked about the purpose and method of their travel, as well as how and why they used their mobile phones on the road. Then, the questions were directed to the respondents' daily driving habits, including the corresponding driving experience, steering wheel grip, common functions of the vehicle center display they usually use, and the need to use mobile phones in daily driving. Respondents were finally asked to imagine various tasks they would like to accomplish under hypothetical autonomous driving conditions. The questions of the interview are in Appendix B. In order to better explore user needs from the side, additional questions will be added for different respondents and answers, which will not be included in it. Since the target driving user groups are divided into two types: young adults (20-35 years old) and adults (35-50 years old), we found 5 young adult drivers for interviews and then interviewed 6 adult drivers. The results of all interview sites will be summarized after the interview. At the same time, we ensured the gender ratio and occupational diversity of respondents when looking for respondents. The interviews lasted an average of 15 minutes and were conducted in the form of an online video conference. One of the panelists is responsible for asking questions from the list of questions, with additional questions for deep requirements, the other panelist is responsible for recording the interview and the answers. The interview is divided into the following steps: 30 1. Introduction of project background 2. Emphasize the protection of respondents' privacy 3. Formal interview 4. Follow-up questions 5. Feedback time and emphasize the protection of personal privacy again Everyone's answers are included in Appendix C. In each person's answer, there is a need for using a specific mobile app. For different secondary task requirements, we separately counted the types of secondary tasks in different situations mentioned by the respondents of the two target groups, and the number of times the secondary tasks were mentioned by different respondents. Statistics on secondary tasks are included in Appendix D. 4.3.3 Results Young adults (20-35 years old) target user groups are mainly students and young practitioners. On the way to study and work, their main requirements for mobile phones are navigation (including bus route apps, and map navigation), enjoying music and podcasts, followed by news and current affairs. The use of social media software is also a raised user requirement. This group of users has the most obvious need for music and podcasts, although the need for navigation is the most cited. In non-work and study situations, the respondents' needs for mobile phones are also reflected in their needs for navigation and music/podcast. In addition, in this relaxed atmosphere, they communicate more with their friends or use social applications to send texts. In addition, listening to audiobooks is also an option for some young respondents. For driving purposes, the answers given by the respondents did not show a certain pattern, but in general, carrying items that cannot be carried on public transportation is a relatively common reason. Most of the respondents hold the steering wheel with both hands, and more than half of the respondents hold the steering wheel at a position higher than L9 and R3. Their main needs for mobile phones during driving are to listen to music and radio, use navigation, and perform control operations within the software. Among the secondary task requirements proposed by young adult drivers in a hypothetical autonomous driving environment, watching Youtube and playing games are two novel secondary task requirements, while music and podcasts are still relatively basic secondary task requirements. In addition, sending text messages, checking and replying to emails, and using a browser to surf the Internet are also the requirements raised twice by the respondents. Overall, Young adult respondents believe that some secondary tasks that require high cognitive resource consumption can be completed during autonomous driving, and current daily secondary tasks will not be replaced. Similarly, adults have the most prominent need for mobile phone navigation during their daily work trips, followed by music, audiobooks, and news reading. In addition, due to interpersonal and family requirements, they frequently use their mobile phones to send and receive text messages and make phone calls. While driving, listening to audiobooks was one 31 of the most mentioned secondary tasks, texting and listening to music were two other important secondary tasks. Adults drive mostly for vacation trips and family trips, followed by carrying goods. Regarding the steering wheel holding position, the most concentrated on the L10R2. Compared with young adults, they are more inclined to drive seriously and safely and to hold the steering wheel more safely. They have a requirement for active input of daily information, which is reflected in their love for reading and listening to books. Among their expected secondary tasks, reading and podcasts remained the most prominent choices, followed by watching videos, sending messages and emails, and listening to music and news. In general, the two types of user groups have certain similarities in the choice of secondary tasks in daily mobile phone use and driving. Listening to music or audiobooks is the secondary task that occupies less cognitive resources, and was also the most mentioned task. Whereas young people tend to appreciate music, adults tend to read. Regarding the imagination of the two target groups for the secondary task, watching videos is the common choice of the two target groups, and answering emails was also proposed by the two groups. Video media and text input were the most easily associated kind of tasks among users. Therefore, they are the secondary tasks that we decided to develop in-depth. 4.4 User study 2 4.4.1 Purpose After obtaining the possibility of user needs for various secondary tasks from user study 1, some questions and follow-up research directions appeared. First, the possible secondary task requirements were estimated based on the user's mobile phone usage and the user's speculations. So we need some way to verify which secondary tasks are actually required by the user in an autonomous driving environment. Second, as drivers need more information while driving, and as the car wants to provide more secondary tasks, Level 3 autonomous vehicles will likely offer many displays. For example,the Head-Up Display ,the High Head-Down Display, Instrument Panel and Center Console Display may exist in the field of vision of a driver at the same time. In this case, the screen on the steering wheel and other displays need to be clarified in relation to secondary tasks. Display and control are considered to be the two most important relationships between secondary tasks and displays. Therefore, for vehicles with multiple displays, the responsibility of the screen on the steering wheel for secondary tasks and the relationship of secondary tasks to multiple displays needs to be defined. In summary, the overall purpose of further user studies is to validate the results of the previous user study and explore users' thoughts on the relationship between secondary tasks and multiple displays. 32 4.4.2 Method Different from other experiments that study the user experience of drivers in ordinary cars, we hope that the environment in which the experiment is located is a level 3 autonomous driving environment. This means that the user can see the driving environment outside the car while sitting on the driver's seat, but at the same time, they do not perform driving tasks for a period of time. So far, there is no widespread Level 3 autonomous vehicle on the market, nor is there a way to simulate a Level 3 autonomous driving environment. In talking with Autoliv's supervisor we learned that Autoliv has driving simulators that simulate both autonomous and manual driving scenarios. This simulator is considered to be able to help us do better research. An experiment can be designed based on the simulator. And bodystorming is considered the most suitable user research method for this experiment. Bodystorming is an immersive innovation process that involves role-playing and physical involvement with props, prototypes, real items, and physical environments to explore ideas. Its overall aim is to comprehend the interactions between people, their physical location, and the items (e.g., tools, devices, materials) they employ in that context (Wilson, 2011). The main implementation method of this experiment is that the simulator provides a physical environment and an electronic driving environment, and we supplement the environment details by descript, including time and tasks. We print pictures of all possible secondary tasks and their different functional components, show them to the user after simulating entering self-driving mode in the simulator, and let them choose their preferred secondary task and perform the interaction they think makes sense. We observe and record from the sidelines. 4.4.3 Process According to the target group, we found 3 young adult drivers and 3 adults drivers for physical and online experiments. In order to keep the information of the participants confidential, we did not record and take pictures in the physical experiments and online experiments. The average duration of each experiment is 15 minutes, and the following is a simplified version of the experimental process. The full experiment procedure is attached in the appendix E. We followed the same process for both physical experiment and online experiment. First is the introduction of the project and the experiment. After that, we described a detailed scenario for users helping them immerse themselves in the scenario and make decisions. Figure 10 shows how we create scenarios through Miro. They were told to imagine they are driving a Level 3 self-driving car with many screens, including a touch screen on the steering wheel. Then they were required to choose three secondary tasks they want to perform during autonomous driving from the secondary tasks we provide. For each secondary task,they were asked to place different functional components where they see fit. Take email as an example, a reading email page, a writing email page, and a keyboard were provided. Users may believe a bigger screen is more suitable for reading email, and the steering wheel screen is more suitable for a keyboard. And they were asked what kind of touch screen on the steering wheel they would like to have. At the end of the experiment, we invited respondents to ask us questions and give feedback. 33 Figure 10: Scenario created by Miro 4.4.4 Result Although we used onsite and online experiment, because we have the same purpose and process for these two methods, we comprehensively analyze all the results of the two methods. The complete record of this study is attached in Apendix F. Combining the experimental results of the two target groups, some useful conclusions are found. First, we wanted to see if the experimental results showed that some secondary tasks were mentioned significantly more than others. Combining the responses of all participants, we found that Spotify, Messenger, Youtube, and Gmail were mentioned the most. But due to our small number of participants, this result does not indicate that some secondary tasks are especially valued and favored in the self-driving scene. Larger and longer studies may be required to obtain data on secondary task selection. Then for our design, the conclusions of user study 1 are more worth considering when deciding which secondary task should be provided. Regarding the placement of different functional components in the application, although participants chose different types of applications, we found some commonalities. Figure 11 shows the components that we concluded were the most frequently selected by the participants to be placed on the steering wheel screen. Among them, the play function buttons and keyboard are considered to be the most suitable components to be placed on the steering wheel screen. We believe that users tend to place components with control functions on the steering wheel screen, such as the play and pause buttons in playing video. Components with reading or watching attribute are more placed on other larger screens, such as video or email content. In addition, users believe that some content can be displayed on the steering wheel screen and other screens at the same time, which will give users more freedom. This inspired us to think about the relationship between multiple displays in the car. Different alternatives can be proposed later in the design phase to explore multiple possibilities. 34 Figure 11: Components that most frequently selected by the participants to be placed on the steering wheel screen. Additionally, no more consistent preferences were found in the user's perception of the position of the steering wheel screen. But some concerns of users were found. For example, users are more concerned about whether the position of the steering wheel screen provides a comfortable operation and whether they need to keep their heads down to observe the screen. What this inspired us is that users believe comfort and ease of use come first, and we should focus on exploring good controls. 5 Use scenario 5.1 Purpose After obtaining a lot of user behavior and preference information, we hope to use a more vivid way to summarize and analyze the obtained information, and to help us better predict the user's intention and performance when using the products we provide. Persona and use scenarios are considered to be a good way to help us translate user needs into product concepts. Since we have two target user groups, we want to create two persona and their use scenarios according to their respective typical needs, in order to simulate and explore common use scenarios of all target user groups. Persona is a fictional character created from research to represent possible different types of users. Creating personas will help users understand their needs, experiences, behaviors, and goals. (Dam & Siang, 2022). Nielsen (2013) proposed a method “Ten steps to personas” to construct persona, as shown in Figure 12. This process model contains four different main parts: data collection and analysis, persona descriptions, scenarios for problem analysis and idea development, acceptance from the organisation. Nielsen (2013) argues that a project does not need to follow all steps. In this project, the sixth step “defining situations” is considered to be the most efficient approach and we call it creating use scenarios. We want to create some virtual characters and usage scenarios that describe the user's scenario 35 when using our product, including the user's vision and behavior, possible feedback and performance of the product. Figure 12: “Ten steps to personas” to construct persona by Nielsen (2013) We believe this is critical step in the project, because the creation of the use scenario is based on the results of the user research, which represents the patterns of user behavior and needs found in the user research, and is the summary and presentation of the data in the user research phase. At the same time, the use scenario is also the beginning of the 36 product concept generation. In this step, we define the use environment of the product in more detail, the scope of the design, and the results we want to achieve. In the user research stage, we found that media and text input are the most desirable secondary tasks for users, and they are also the two types of secondary tasks that we decided to develop in depth. Therefore, when creating use scenarios, these two types of applications are integrated into the story, and the email application and video application are highlighted. Another reason to choose these two types of apps is that video apps are more of an entertainment and relaxation attribute, while email apps are more of a business attribute. Reflecting on the secondary tasks that users frequently mentioned in the user research they wanted to perform in a Level 3 autonomous vehicle, some of them were more for entertainment purposes, and some were more for office purposes. This gave us the hint to create a relaxation scene and an office scene when creating the use scenario. Therefore, reading and replying emails, and watching videos, respectively, match the two usage scenarios that users expect from a Level 3 autonomous vehicle. According to the results of user research, young adults use more entertainment and relaxation apps on a daily basis, so we designed a scene that is biased towards relaxation for young adults. Adults, on the other hand, showed preferences for self-improvement and work in user research, so we designed an office scenario for adults. To make the story seem more real and specific, we substitute each type of app with a specific product that is most widely used in that type of app. For example in Young adults' use scenario, we use Youtube instead of "video app". 5.2 Scenarios For young adults, we created a couple's persona and a situation where they drive to IKEA to shop. Carl (26 years old) and Anna (25 years old) are students at Chalmers University of Technology and live in the city centre of Gothenburg. On a weekend in April, Carl and Anna decide to go to IKEA to buy some furniture and decorations for their home . Since IKEA is so far away