The Future of User Evaluation 
Exploring how emerging technology can be used in user 
evaluation to investigate driver interaction 
Master’s thesis in Industrial Design Engineering 
 
 
 
LOVISA LJUNGDAHL 
JONNA STENLUND 
 

DEPARTMENT OF INDUSTRIAL AND MATERIALS SCIENCE  
CHALMERS UNIVERSITY OF TECHNOLOGY 

Gothenburg, Sweden 2025 

www.chalmers.se 



 

 

 



MASTER’S THESIS 2025 
 
 
 
 
 
 
 

The Future of User Evaluation 
 

Exploring how emerging technology can be used in user 
evaluation to investigate driver interaction 

 
 

LOVISA LJUNGDAHL 
JONNA STENLUND 

 
 
 
 

 
 
 

 
 

 
 

 

 
 
 
 

Department of Industrial and Materials Science 
Division of Design and Human Factors 

CHALMERS UNIVERSITY OF TECHNOLOGY 
Gothenburg, Sweden 2025  

​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​      i 



 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
The future of user evaluations 
Exploring how emerging technology can be used in user 
evaluation to investigate driver interaction 
 
 
LOVISA LJUNGDAHL 
JONNA STENLUND 
 
© Lovisa Ljungdahl & Jonna Stenlund, 2025 
 
Supervisor: Siw Eriksson 
Examiner: Pontus Wallgren 
 
Master’s thesis  
Department of Industrial and Materials Science 
Division of Design and Human Factors 
Chalmers University of Technology 
SE-412 96 Gothenburg, Sweden 
 
 
Cover: An illustration of a user evaluation supported by various technologies 
Cover made by Lovisa Ljungdahl & Jonna Stenlund 
 
Print: Chalmers Digitaltryck 
Gothenburg, Sweden 2025 

ii 



Abstract 

Technology is advancing rapidly across multiple fields and emerging technologies hold 
significant potential to shape the future of product development and design. This thesis 
explores the intersection of emerging technology and user evaluation in collaboration with 
Volvo Group Trucks Technology. It is particularly focused on the context of human-machine 
interaction and how emerging technologies can enhance future user evaluation of driver 
interaction. The study set out to identify what types of insights and results emerging 
technologies could offer for user evaluation, along with associated practical limitations and 
opportunities in an industrial automotive setting. 

The work was structured in two phases. The first phase involved technology scouting through 
literature research and expert meetings to identify relevant emerging technologies. In the 
second phase, two selected technologies, extended reality and emotion interpretation, were 
tested in practice to further explore their value for user evaluation. A total of 19 expert 
meetings were held, an XR workshop was conducted with both a pilot and main session, and 
three types of emotion interpretation tests were performed. 

Five technologies were identified as particularly relevant for user evaluation in the considered 
context: extended reality, emotion interpretation, EEG sensors, eye tracking and artificial 
intelligence. The results suggested that each explored technology demonstrates distinct 
strengths and opportunities to contribute to in-depth understanding of users and enhanced 
user evaluation. However, the results of the study also highlight technology limitations and 
the importance of complementing, rather than replacing, traditional user evaluation methods.  

The thesis concluded with specific recommendations for each explored technology area, 
along with an overarching recommendation to actively monitor ongoing developments in 
emerging technology for user evaluation. Specific recommendations for user evaluation in the 
automotive context were also presented, as well as the suggestion that a balanced approach 
between new tools and established practices would be most effective. While further research 
in this area is needed, the findings offer promising directions for how industrial actors, like 
Volvo Group, can advance user evaluation practices through emerging technologies. 

​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​   iii 



 

 

iv 



Acknowledgements 
This master’s thesis would not have been achievable without the support we have received 
throughout the project. We would like to express our sincere appreciation to our academic 
supervisor Siw Eriksson for always taking the time to assist us, showing genuine 
engagement, offering valuable advice and guiding us through the complexity of this study. 
We also extend our thanks to our examiner, Pontus Wallgren, for his helpful direction and 
contribution to insightful discussion throughout the thesis process. 

We are deeply grateful to Volvo Group Trucks Technology for hosting our master’s thesis and 
for the meaningful learnings and experiences we gained during our time at the company. A 
special thanks goes to our supervisor at Volvo, Matilda Wiklund, for her commitment to 
continuous supervision, valuable industry knowledge and consistently positive approach. 
Additionally we would like to thank the entire team at Strategy and Governance Office for 
their warm welcome and support whenever needed.  

We would also like to thank everyone who participated in our meetings, interviews and 
evaluations. The time and contributed perspectives were essential to the development of this 
thesis and are deeply appreciated. 

Lastly, we would like to extend our gratitude to Chalmers University of Technology and our 
Industrial Design Engineering program for an incredible five years. This journey has been 
truly rewarding, filled with valuable experiences and knowledge that we will carry with us 
forever. 
 

 
Lovisa Ljungdahl & Jonna Stenlund, Gothenburg June 2025 

 

​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​    v 



 

vi 



Abbreviations 

Instrument panel = The combined interface consisting of the instrument cluster, dashboard, 
center console and side displays in a vehicle. 

HMI = Human-Machine Interaction refers to the communication and interaction between a 
human user and a machine or system. 

XR = Extended Reality, a broad term that encompasses all forms of technology that blend 
real and virtual environments for immersive or interactive experiences. 

VR = Virtual Reality, a form of XR that immerses users in a completely digital environment. 

AR = Augmented Reality, a form of XR that overlays digital content onto the physical world. 

MR = Mixed Reality, a form of XR that blends physical and digital environments. 

EEG = Electroencephalography, a technique used to measure electrical activity in the brain. 

BCI = Brain-Computer Interface, a direct communication pathway between the brain and an 
external device. 

DMS = Driver Monitoring System, a technology that tracks a driver’s behavior, attention and 
state to ensure safe driving. 

AI = Artificial intelligence is the field of computer science dedicated to creating systems that 
can learn from data, reason and perform tasks that normally require human intelligence. 

​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ vii 



 

 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 

viii 



 

Table of contents 

1. Introduction​ 1 
1.1. Background​ 2 
1.2. Aim​ 2 
1.3. Research Questions​ 3 
1.4. Objectives​ 3 
1.5. Report Structure​ 3 

2. Related Theory​ 4 
2.1. Introduction to User Evaluation​ 6 
2.2. Automotive Human-Machine Interaction (HMI)​ 6 
2.3. Introduction to Emerging Technology​ 6 
2.4. Emerging Technologies with Potential for User Evaluation​ 17 

3. Study Methodology and Process​ 19 
4. Phase One - Technology Scouting: Method​ 21 

4.1. Technology Literature Research​ 21 
4.2. Meetings with Experts for Technology Knowledge​ 23 
4.3. Analysis of Technology Scouting​ 23 

5. Phase One - Technology Scouting: Results​ 25 
5.1. Findings from Technology Scouting​ 32 
5.2. Phase One’s Implications for Future User Evaluation​ 32 

6. Phase Two - User Evaluation Testing: Method​ 34 
6.1. Aim of Testing User Evaluation for HMI in Practice​ 34 
6.2. Several Considered User Groups​ 35 
6.3. Exploratory Ideation of User Evaluation Use Cases​ 35 
6.4. User Evaluation Testing in Practice - XR Technology​ 39 
6.5. User Evaluation Testing in Practice - Emotion Interpretation Technology​ 47 

7. Phase Two - User Evaluation Testing: Results​ 48 
7.1. Listing of User Evaluation Use Cases​ 48 
7.2. Insights from XR Workshop Testing​ 54 
7.3. Findings from Tests Using Emotion Interpretation Technology​ 57 

8. Synthesized Results​ 59 
9. Discussion​ 61 

9.1. Aim & Research Question​ 61 
9.2. Results​ 64 
9.3. Process & Execution​ 66 
9.5. Recommendations Regarding Future User Evaluations​ 68 

10. Conclusion​ 69 
References​ 70 
Appendices​ 90 

 



 



1 
Introduction 

 
Technology is advancing at a rapid pace across multiple areas. These forefront technologies 
are labeled emerging technologies. This advancement has the potential to shape the future 
and leveraging the opportunities they offer is in many ways crucial to ensure future 
competitiveness for tech-focused industries at a global scale (European Innovation Council 
Tech Report, 2024).  

The rapid progression of technology has significant implications for product development 
and design and its associated evaluation practices. User evaluation is an integral part of such 
practices which involves assessing how intended users interact with, or experience, a product 
or system (Bertini et al., 2008). Such interaction between a user and a product or system is 
referred to as a human-machine interaction (HMI).  

This study explores the intersection of emerging technology and user evaluation in the 
context of human-machine interactions. It is conducted in collaboration with Volvo Group 
Trucks Technology (Volvo GTT) and its Strategy and Governance Office (SGO). 

1.1. Background 

The following section covers emerging technology in user evaluation and provides a 
company description of Volvo Group. It highlights the evolving nature of user evaluation 
practices and how emerging technology can enhance the practices within Volvo Group’s 
innovation strategy. 

1.1.1. Technological Advancements in User Evaluation  

User evaluation practices within product development and design industries have gradually 
evolved alongside technological advancements. New methods and tools have been 
incorporated to ensure the continued relevance and increased value of user evaluation. Today, 
a variety of methods are commonly used to conduct user evaluations within product 
development and design processes, with the choice of method depending on the specific 
context and objectives. Such methods include traditional interviews and observations, 
usability testing, surveys, focus groups, ethnographic studies and user journey mapping, 
among others.  

    1 



While these methods provide valuable insights, utilizing emerging technologies in user 
evaluation creates the potential for new perspectives and more comprehensive understanding 
of users, their experiences, behaviors and perceptions. Proactively exploring possibilities and 
ensuring continuous evolution of user evaluation is increasingly important given the rapidly 
developing technology landscape. This can be achieved by identifying newly launched 
technologies, those that have been in use but have undergone significant improvements, or 
technologies that can be applied in new contexts to explore how they can be integrated into 
user evaluation methods. 

1.1.2. Company Description: Volvo Group  

As a key player in the global automotive industry, Volvo Group delivers transport and 
infrastructure solutions such as trucks, buses, construction equipment, as well as financing 
and services. This study will be conducted in collaboration with Volvo GTT and its 
department SGO. Within Volvo Group, Volvo GTT handles all technology and product 
development related to truck operations and aftermarket support. In this context, SGO is 
responsible for Volvo Group's strategy regarding emerging technology. They fulfill this role 
by identifying emerging technologies and trends with the greatest potential for value creation 
for Volvo Group. They further manage uncertainty and push boundaries to establish new 
ways of working and enhance innovation and competitiveness.  

Additionally, maintaining its position as an industry-leading innovator requires Volvo Group 
to operate quickly and efficiently while prioritizing aspects like safety, product usability and 
customer satisfaction. Given the human-centered nature of the products and services offered 
by Volvo Group, high-quality human-machine interaction is especially important for 
achieving these goals, as it directly shapes how users experience and interact with the 
complex systems.  

This underscores the importance for Volvo Group to conduct thorough user evaluations. As of 
now, this is done using a variety of comprehensive methods and tools. However, with the 
rapid pace of technological advancement and the strategic focus on future-oriented 
technologies led by SGO, there is growing interest in exploring how emerging technologies 
can be leveraged within user evaluation practices at Volvo Group. By doing so, there is 
potential to discover new ways of gaining deeper user insights and strengthening the 
company’s product development results. Due to the evolving nature of the field of emerging 
technology, the topic is relevant not only to Volvo Group’s innovation efforts but also to 
current research. 

1.2. Aim 

The aim of this study is to investigate how emerging technologies can enhance future user 
evaluation of human-machine interactions in the design- and product development process.  
 

2 



1.3. Research Questions 

The research questions this study aims to answer are as follows: 

●​ How can emerging technology be used in future user evaluation? 

●​ What types of insights and results can be obtained from user evaluation that 

incorporate emerging technologies? 

●​ How can Volvo Group leverage these opportunities?  

1.4. Objectives 

The objectives this study aims to achieve are as follows: 

●​ Provide Volvo Group with insights on how emerging technologies can enhance user 

evaluation practices within the context of human-machine interactions.  

●​ Identify emerging technologies with potential applications in user evaluation methods. 

●​ Identify future opportunities for user evaluation methods.  

●​ Evaluate user evaluation methods that incorporate emerging technology and develop 

recommendations based on the findings.  

1.5. Report Structure 

This report presents both the process and findings of the study, divided into two phases. It 
begins by describing the method and results from the first phase, which focused on 
technology scouting. It is followed by the method and results from the second phase, which 
focused on user evaluation testing. A more in-depth description of the study's overall 
methodology and process will be provided in Chapter 3: Study Methodology and Process. 
Please note that some of the material included, originating from the study process, are in 
Swedish.  

    3 



2 
Related Theory 

 
This chapter provides a theoretical framework for understanding the context of the study by 
covering the concepts of user evaluations, automotive HMI and emerging technologies. 
Additionally, it examines a range of emerging technologies relevant in user evaluations, 
including electroencephalography sensors (EEG sensors), extended reality (XR), artificial 
intelligence (AI), driver-monitoring systems (DMS), emotion interpretation, digital twins and 
wearable devices.  

2.1. Introduction to User Evaluation 

To create products, systems and services that function effectively, it is essential to first 
understand the prevailing circumstances, the context in which they will be implemented and, 
most importantly, the needs and preferences of the users (Wikberg Nilsson et al., 2015). This 
is where user evaluation becomes invaluable, to ensure solutions with seamless, effective and 
intuitive user interaction. User evaluation can also be referred to as user testing, user-based 
testing, user research and more. In this study it will hereafter be referred to as user evaluation. 
As described by Bertini et al. (2008) user evaluation is evaluation conducted together with 
the people intended to use the system or product being evaluated. There can be several 
different users or stakeholders in one project, including the client, investors, and various 
types of end-users (UserBit, 2025). These can be categorized as primary, secondary, tertiary 
users and so on. The goal of user evaluation can vary, but it is often centered around 
understanding the user in relation to the product (Moran, 2019).  

This may involve identifying design issues, learning about user behavior and uncovering 
potential opportunities. By conducting evaluations along with users, designers can identify 
problems, improve products, reduce costs, create a competitive advantage and assure that 
products meet user needs (Interaction Design Foundation - IxDF, 2016). By gaining the user’s 
perspective and hearing their insights the product’s functionality, usability and overall 
experience can be enhanced (Moran, 2019). 

2.1.1. Categories and Methods of User Evaluation 

There are various methods for conducting user evaluation and the choice of method depends 
on various factors including the context, the participants involved, the stage of the 
development process, the product being evaluated and the specific criteria being assessed. 
Examples of user evaluation methods include usability tests, interviews, observations, and 

4 



focus groups (Wikberg Nilsson et al., 2015), as well as the NASA Task Load Index 
(NASA-TLX) (Agency for Healthcare Research and Quality, n.d.), Enhanced Cognitive 
Walkthrough (ECW) (Khasanah & Gunawan, 2023), and Wizard of Oz (Paul & Rosala, 
2021). Practices of user evaluation have evolved over time alongside technological 
advancements. Examples of technologies that have led to the introduction of new methods for 
conducting user evaluation are eye tracking, driving simulators and virtual reality (VR).  

User evaluation can be broadly categorized into two main types: qualitative and quantitative 
testing (Moran, 2019). Qualitative testing focuses on users' subjective insights and 
experiences, making it particularly useful for identifying issues related to user experience. 
This approach relies on a hermeneutical and interpreting approach (Wikberg Nilsson et al., 
2015) and includes methods such as interviews, workshops, and observations. The purpose of 
qualitative evaluations is to develop a deep understanding of specific phenomena, user 
attitudes, and the reasons behind user behaviors. To enhance the chances of achieving this, 
researchers can employ the technique of triangulation by using multiple qualitative methods 
to develop a multifaceted understanding of phenomena (Carter et al., 2014). In contrast, 
quantitative testing collects measurable data, such as task completion rates and time-on-task 
data (Moran, 2019). It involves gathering data from a large number of users and is used in the 
design process to gain insights rather than to prove a point (Wikberg Nilsson et al., 2015). 
Typical quantitative methods include questionnaires, measurements related to human comfort 
(such as light, sound, and climate) and human dimensions (such as height, width, and reach). 
In both design and development processes, a combination of qualitative and quantitative 
evaluations is essential to achieve the most comprehensive understanding of the users and 
their context. 

2.1.2. User Evaluation Stages in the Product Development Process 

User evaluation can take place at different stages of the product development process 
(Wikberg Nilsson et al., 2015). Gathering information about a user is not a task that ends 
before the project is completed, but rather an ongoing process. In the early development 
stages, it is important to explore user needs and gather user insights to shape the solution. 
According to Ehrlich and Rohn (1994) and Noyes et al. (1996), user involvement is most 
effective in the early stages of development, as the cost of making changes increases 
significantly in later phases. As development progresses, iteratively and continuously refining 
the understanding of the user's situation and interaction with the solution plays a crucial role 
(Wikberg Nilsson et al., 2015). This repeated process is essential for creating a well-rounded 
and effective final product. However, the degree of user participation in the development 
process can vary significantly. In many cases, users are only involved late in the process to 
evaluate a proposed solution and confirm whether it meets the specified requirements 
(Bødker & Buur, 2002; Henninger et al., 2005). Some companies design for users, while 
others design with or by users (Eason, 1995).  

    5 



2.1.3. Future Technology Trends in User Evaluation 

The future of user evaluations is yet to be determined, but technology and user experience 
have always been intertwined; when one advances, it influences the other, and vice versa 
(Experience Haus, 2024). The key emerging technology trends predicted for future user 
evaluations include AI, VR, emotion-sensing technologies, biometrics and eye tracking, as 
identified by Patel (2024) and Experience Haus (2024). Additionally, there is an increase in 
interest for performing user evaluations remotely, particularly accelerated by the Covid-19 
pandemic (Larsen et al., 2021).  

2.2. Automotive Human-Machine Interaction (HMI) 

Many products involve HMI, ranging from household appliances and computers to industrial 
equipment. As a result, the concept of HMI is broad, highlighting the need to clearly define 
its meaning in the automotive context. According to Gong (2024), automotive HMI can be 
described as a system that enables the exchange of dynamic information and emotions 
between the human user and the vehicle, excluding core driving tasks. Its primary function is 
to facilitate information transfer, such as user commands and vehicle feedback.  

Beyond informational exchanges, emotional interaction is emerging as an important aspect of 
automotive HMI (Gong, 2024). This involves evoking more abstract feelings, such as a sense 
of luxury or comfort, through features like dynamic ambient lighting or animated visual 
effects. Once overlooked, these emotional interactions are now receiving greater attention in 
the design of in-vehicle experiences.  

However, it is important to note that automotive HMI is exclusively concerned with dynamic 
interactions (Gong, 2024), meaning static elements like door sill text or seat stitching do not 
constitute HMI, as they lack active communication. Furthermore, primary driving activities, 
such as steering wheel positioning, pedal feel, or gear shifting in manual vehicles, are 
generally not considered part of HMI.  

2.3. Introduction to Emerging Technology 

The term emerging technology covers a broad spectrum of innovations and there has not 
always been a clear consensus on its definition. According to Halaweh (2013), technology is 
considered emerging if it is expected to radically impact businesses, industries or society. 
Similarly, Rotolo et al. (2015) define emerging technologies as radically novel, relatively 
fast-growing, coherent, prominently impactful, uncertain and ambiguous. Examples of 
technology trends regarded as significant and emerging are AI, machine learning, 
connectivity, immersive-reality technologies, quantum technologies, future of mobility and 
future of bioengineering (McKinsey Technology Council, 2024). As such, technology labeled 
as emerging does not necessarily need to be new, but can be regarded as emerging in a 
specific context even if well-established elsewhere.  

6 



The importance of leveraging the opportunities of emerging technology is highlighted in the 
European Innovation Council Tech Report (2024) and similarly by Vinnova (2024). It 
underscores the necessity of acknowledging the potential of emerging technology, both 
within tech-focused industries and on a global scale. However, when identifying the potential 
and leveraging the opportunities of emerging technology, it is crucial to assess whether its 
claims are viable and capable of delivering results to avoid falling for unrealistic 
“technological hype” (Gartner, n.d.). There are also additional challenges regarding emerging 
technology, due to its high uncertainty (Brey, 2017). Such challenges can include predicting 
future use patterns, impacts and consequences - both on an ethical, societal and 
environmental scale. Thus, the novelty of emerging technology presents both promising 
opportunities as well as considerable risks.  

2.4. Emerging Technologies with Potential for User 
Evaluation 

Emerging technologies can contribute to enhanced user evaluation by potentially providing 
new and deeper user-insights. When examining the relevance of different emerging 
technologies in user evaluation, key aspects to consider include technical foundations, current 
applications, challenges and future opportunities. This chapter presents a theoretical 
background for the key aspects of the examined technologies, with an overview presented in 
Figure 1. 

 

Figure 1. Examined technologies with potential for user evaluation. 

2.4.1. Extended Reality (XR) Technology 

XR is an umbrella term that encompasses several technologies such as virtual reality (VR), 
augmented reality (AR) and mixed reality (MR) (Interaction Design Foundation - IxDF, 
2022). These technologies enhance reality by blending the physical and digital worlds or 
creating fully digital environments to deliver immersive experiences. However, XR is not 
limited to these specific technologies, it includes any existing or future innovations that 
enable the seamless fusion of physical and digital realities. Since XR covers the entire 
spectrum from fully real to fully virtual experiences, there are significant differences between 
VR, MR and AR. Virtual reality immerses users entirely in a digital environment, thereby 

    7 



disconnecting them from the physical world. Augmented reality enhances the real world by 
overlaying digital content without replacing the existing environment. Mixed reality 
combines elements of both AR and VR, allowing real and virtual components to interact 
seamlessly in real-time. The distinctions between XR technologies are illustrated in Figure 2. 

 

Figure 2. Distinction between XR technologies. 

Available XR technology devices  

The most common device when wanting an immersive XR experience is a wearable XR 
headset (Yord Studio, 2024). The different types of XR headsets vary from lightweight AR 
glasses to larger VR headsets, with each designed to meet specific use case requirements. 
Some headsets prioritize high resolution and exceptional quality, while others are designed 
with comfort as the main focus. Modern VR headsets typically fall into two categories: 
tethered and standalone. Tethered headsets are physically connected to a PC or PlayStation, 
while standalone headsets provide greater physical freedom by eliminating cables and not 
requiring an external device for processing. 

Currently, some of the most common XR headsets on the market include the Meta Quest 3, 
Varjo and HTC Vive. Another XR device is The HoloLens, a pair of lightweight AR glasses 
that can project AR content in the real world. They have a limited field of view and are best 
used indoors (Skarredghost, 2021). The Meta Quest 3 supports both VR and MR experiences 
and is one of the most affordable standalone headsets, offering a resolution of 2064 x 2208 
pixels per eye and the best overall VR experience for those new to VR and with no access to 
a tethered connection or PC (Greenwald, 2025). On the other hand, the HTC Vive Pro 2, a 
tethered headset, targets both enthusiasts and professionals with a sharp VR display and 
resolution of 2,448 x 2,448 pixels per eye. Varjo offers some of the most immersive virtual 
and mixed reality products, aimed at advanced VR users who require cutting-edge technology 
and resolutions of up to 3840 x 3744 pixels per eye (Varjo Technologies, 2025).  

8 



Current applications of XR technology  

XR technology has its roots as far back as the 1800s, when scientist Sir Charles Wheatstone 
introduced the concept of stereopsis in 1838, explaining how the brain merges two images to 
create a single 3D perception (Marr, n.d.). This discovery led to the development of early 
stereoscopic devices that produced 3D images with a sense of depth. However, modern XR as 
we know it began to take shape in 1956, when cinematographer Morton Heilig created 
Sensorama, the first VR machine. Since then, XR technology has evolved rapidly and 
continues to grow in terms of both importance and application.  

Key applications of XR today include product design, interior design, architectural 
representation and education (Interaction Design Foundation - IxDF, 2022). By enabling the 
visualization and interaction with 3D models, XR enhances prototyping and user evaluations, 
ultimately offering more effective design tools. Similarly, in interior design and architectural 
representation, designers and architects can create spatial layouts and immersive 
walk-throughs of rooms and buildings. This allows for better-informed decisions regarding 
layout, materials, and structure, which can reduce errors before construction or production. In 
education, XR provides immersive and engaging learning experiences, making complex 
concepts more tangible and interactive. However, the impact of XR goes far beyond these 
applications, extending into a wide range of industries. Today, XR technologies are also being 
adopted across diverse sectors, including manufacturing, healthcare, construction and even 
law enforcement (Marr, n.d.). 

Challenges with using XR technology in a corporate setting 

While there are many opportunities with XR technology, there are still underlying challenges 
that must be considered. In a study on virtual office work, Berlin and Babapour Chafi (2024) 
highlight some issues. One significant concern when using XR in a corporate setting, such as 
with Meta Quest headsets, is confidentiality. Using commercial XR headsets and associated 
softwares in organizational environments often results in data being processed on local 
servers, with files and other content uploaded to the cloud. This raises the risk of sensitive 
data being transmitted to unintended recipients. Additionally, confidential conversations, such 
as discussions of sensitive business information or details of personal health, could 
inadvertently compromise confidentiality and violate GDPR regulations. This poses a 
challenge of handling NDAs and sensitive information in XR environments. 

Another issue discussed by Berlin and Babapour Chafi (2024) is the amount of required user 
accounts. The XR setup is described as a complex IT ecosystem, which may become 
inefficient for IT departments and create operational inefficiencies within organizations. 
Additionally, there are concerns about the reliance on external suppliers for essential 
applications. If a supplier decides to discontinue an XR application that a company has 
integrated into its workflow, this can lead to significant disruptions. It is vital for businesses 
to be able to maintain some control over the applications and devices used and ensure they 
can manage what is being used and displayed within their XR environments. Lastly, Berlin 
and Babapour Chafi (2024) emphasize the importance of understanding the specific value XR 

    9 



technology can bring to a project. Before integrating XR, organizations should critically 
assess how it will enhance the work being done and what specific contributions they want the 
technology to make.  

Future potential of XR technology 

XR is rapidly growing and although its origins trace back many years, it is only in recent 
times that the technology has become commercially viable. This shift has led to a significant 
increase in usage and broader accessibility. An area associated with XR technology that has 
been frequently discussed in recent years and is expected to gain significant traction is the 
metaverse (Program-Ace, 2024). The metaverse is a vision of an interconnected extended 
reality made up of digital worlds, where users can seamlessly transition between experiences 
using virtual reality and augmented reality devices to interact, collaborate, and connect in 
real-time (Walsh, 2023). Although the concept was first introduced in 1992 by author Neal 
Stephenson in his novel Snow Crash, where he described a shared, immersive 3D virtual 
world, it is only in recent years that the metaverse has begun to emerge as a serious 
technological frontier. Major tech companies like Microsoft, Apple and Meta are now 
investing heavily to bring this digital future to life. 

Other upcoming trends in XR include its increasing use for immersive educational 
experiences (Program-Ace, 2024). XR could potentially serve as a virtual classroom that 
allows users to learn and explore concepts that would be impossible or unsafe in the real 
world. Additionally, hyper-realism is gaining increased attention. Hyper-realism involves 
incorporating more senses, such as smell and touch, into the XR experience. This 
development blurs the line between virtual and physical realities as advancements in 
technology enable XR to replicate real-world sensations more convincingly.  

2.4.2. Electroencephalography (EEG) Sensor Technology 

An electroencephalography (EEG) sensor, also called neurosensor, is a device used to 
measure the brain’s electrical activity by detecting small fluctuations in electrical current 
between the skin and sensor electrodes (Soufineyestani et al., 2020). The electrodes consist of 
small metal discs connected by thin wires, which are attached or adhered to the user’s scalp 
(Johns Hopkins Medicine, n.d.). The signals are then amplified and displayed as a graph, an 
electroencephalogram, on a computer screen.  

Available EEG sensor devices  

EEG headsets are available in both wired (medical-grade) and wireless (consumer-grade) 
versions (Soufineyestani et al., 2020). In both cases, the recorded data is transmitted to a 
computer, either through a physical cable or via wireless methods such as Bluetooth. For 
medical-grade EEG devices, electrodes can be either saline-based or gel-based (Sabio et al., 
2024). Gel-based electrodes, while more effective in terms of accuracy, require more time for 
application and may cause discomfort due to the gel sticking to users’ hair and causing 

10 



inconvenience. Saline-based electrodes are therefore frequently preferred due to ease of use 
and quick setup time. 

The decision between using medical-grade or consumer-grade EEG devices largely depends 
on factors such as budget, resources, and specific requirements. The consumer-grade devices 
are more lightweight, cheap and provide greater freedom of movement and ease to set up. 
The medical-grade options often involve a higher number of electrodes and a more precise 
placement in comparison. However, medical-grade EEG devices typically offer more stable 
connections and higher data transfer rates. As a result, medical-grade devices are more 
suitable for clinical use, particularly in diagnosis and treatment (Sabio et al., 2024). In 
contrast, consumer-grade devices are especially appealing to novice researchers or those 
seeking to gather data beyond the confines of a traditional lab environment.  

Even though medical-grade EEG devices have traditionally been used, consumer-grade EEG 
devices have become more widely available and increasingly popular over the past decade 
(Sabio et al., 2024). According to Soufineyestani et al., commonly used EEG devices include 
medical-grade devices from Enobio and SMARTFONES as well as consumer-grade devices 
from Emotive, Imec, OpenBCI and Neurosky (Soufineyestani et al., 2020).  

Current applications of EEG sensors 

EEG technology has become widely used in neuroscience due to its ability to provide 
valuable insights into mental states, cognitive processes and even imagination 
(Soufineyestani et al., 2020). Researchers in various fields have thus taken advantage of this 
leading to EEG technology being used in many applications.  

One of the most common applications of EEG technology is for brain-computer interfaces 
(BCIs), which use real-time EEG data to control and interact with mechanical or electronic 
devices (Soufineyestani et al., 2020). BCIs process brain activity and translate it into signals 
that can be used by external systems, enabling users to control devices using only their 
thoughts. This technology offers valuable support for individuals with mild to severe motor 
disabilities. Another primary application area for EEG technology is the medical field, where 
it (mostly through wired devices) is used to evaluate various brain disorders and diagnose 
conditions that affect brain activity, such as epilepsy seizures, Parkinson’s disease, 
Alzheimer’s disease, schizophrenia, anxiety, dyslexia and autism (Soufineyestani et al., 
2020). Beyond diagnosis, EEG technology can be used to assess the brain’s overall electrical 
activity and consequently provide valuable insights into the effects of trauma, drug 
intoxication or the extent of brain damage in individuals who are in a coma (Johns Hopkins 
Medicine, n.d.). 

Another application of EEG technology is in neuroscience, encompassing both cognitive and 
behavioral areas (Soufineyestani et al., 2020). In cognitive neuroscience, EEG can be used to 
measure cognitive load, analyze brain activity during decision-making and task performance, 
detect sleep patterns, and explore how the brain responds to different scenarios. In behavioral 
neuroscience, EEG technology helps assess brain alertness during various situations and can 

    
11 



also measure levels of drowsiness, mental workload, and stress. A third area of neuroscience 
where EEG technology is applied is neurophysiology, particularly in studying fatigue and its 
impact on brain function. Other application areas are sport, fitness and mediation as well as 
for educational purposes (Soufineyestani et al., 2020). 
 
Sabio et al. suggest that affordable and accessible neuroscientific solutions are becoming 
increasingly available to those outside the research community (Sabio et al., 2024). This trend 
is particularly relevant as technology becomes more and more integrated into daily life. 
According to the authors, it is in fact entirely possible that EEG devices could become a 
common tool in everyday life within the next few decades. 

2.4.3. Emotion Interpretation Technology 

Emotions can influence users’ assessments of interfaces. The emotional state of users can 
impact their perceptions of an interface, their willingness to use it, and their overall 
satisfaction with the experience (Brave & Nass, 2008). Recognizing this influence, the need 
for emotion interpretation is becoming an emerging area of interest, particularly in 
understanding user behavior and decision-making.  

Available emotion interpretation tools 

Today there are various methods available for assessing emotions. However, most of the 
methods rely on self-assessment tools, where users rate their own emotional states. Examples 
of such tools include the Geneva Emotion Wheel, PrEmo and similar instruments (Güiza 
Caicedo & van Beuzekom, 2006). While there are few technologies claiming to objectively 
track emotions, two emerging approaches claim to interpret feelings and emotions based on 
specific measurements: one relying on sensors and the other utilizing artificial intelligence 
(AI). 

One example of a company that offers sensor-based technology for emotion interpretation is 
Merkle (Merkle, 2019). Their system interprets emotions by analyzing the physiological 
responses they provoke. During testing, various sensors are placed directly on the 
participant’s body to collect psychophysiological data of the following key indicators: 

1.​ Breathing Depth: Indicates the level of active interest. Measured using a clamp sensor 
attached to the index finger. 

2.​ Electrodermal Activity: Reflects emotional arousal, either positive excitement or 
negative stress, by measuring changes in hand perspiration. Recorded through 
electrodes placed on the palm. 

3.​ Heart Rate: Provides insight into mental stress or relaxation by analyzing heart rate 
variability. Measured using electrodes placed on the left side of the chest. 

12 



4.​ Brain Waves: Reveal levels of mental stress and emotional engagement. Higher 
activity may suggest a stronger likelihood of negative emotional responses. Measured 
via electrodes attached to the head. 

5.​ Facial Muscle Activity: Indicates emotional valence (whether responses are positive 
or negative) by detecting micro-movements in facial muscles. Measured using sensors 
at three points on the face, capturing subtle changes invisible to the naked eye. 

In addition to these indicators, Merkle evaluates the participant's level of awareness, 
including interest and attention, to assess how appealing a product is perceived to be. To 
account for individual differences, baseline readings are taken before each session. 

One example of a program that offers AI-based technology for emotion interpretation is 
Noldus FaceReader, an automated system for analyzing facial expressions (Noldus 
Information Technology, n.d.). It provides insights into how various stimuli affect emotional 
responses and identifies six recognized facial expressions: happiness, sadness, anger, 
surprise, fear, and disgust - along with contempt and a neutral state. Noldus FaceReader can 
capture dynamic, unconscious responses to stimuli or objects while the person is engaged 
with them. The program processes video recordings of participants to interpret emotional 
states based on analysis of facial muscle movements. It does this by employing a deep 
learning-based facial recognition system and constructing an artificial face model by 
analyzing approximately 500 facial key points. Facial expressions are then classified using 
neural networks trained on a dataset of over 100,000 labeled facial images.  

Current applications of emotion interpretation 

When it comes to application areas, emotion interpretation has found increasing relevance 
across various fields due to the significance of human emotional responses (Khare et al., 
2024). Human emotions are dynamic cognitive and physiological states that arise in response 
to experiences, thoughts, or social interactions. They involve subjective feelings, cognitive 
processing, behavioral responses, physiological changes and communication. As a result, 
emotion interpretation is widely used in fields such as marketing, human-robot interaction, 
healthcare, mental health monitoring and security.  

In the field of market research, customers cannot always accurately explain why they choose 
one product or design over another according to Pichardo (Merkle, 2019). While they may 
attempt to justify their decisions, these explanations could be unreliable due to underlying 
reasons not being consciously known. Emotion interpretation could offer a valuable lens in 
such cases, helping to uncover unconscious triggers of behavior. Pichardo further argues that 
emotion interpretation provides more objective insights than verbal responses alone, since 
emotions influence our actions but cannot be consciously controlled or fabricated. 

 

 

    
13 



Furthermore, emotions can provide valuable insights into conditions like fatigue, drowsiness, 
depression and pain (Khare et al., 2024). With advances in technology and the growing use of 
electronic devices, emotion interpretation is likely to expand across areas such as 
brain-computer interfaces, robotics, medical diagnostics, driving assistance, recruitment and 
patient care. 

Challenges of emotion interpretation technology 

Even though emotion interpretation technologies promise many benefits, psychologist and 
neuroscientist Lisa Feldman Barrett criticizes their scientific foundations (Barrett, 2017). She 
argues that the assumption that facial expressions reliably indicate emotions, such as smiling 
when happy or frowning when sad, is a myth. Barrett's research shows that only 30% of 
adults in urban cultures scowled when they were angry, with others displaying various 
reactions like crying, smiling, or showing no expression (Center for Law, Brain & Behavior, 
2020). This low reliability suggests that facial expressions are not universally tied to specific 
emotions (Barrett, 2017).  

Additionally, the same facial expression, such as a scowl, can indicate different emotions 
(e.g. confusion or concentration) highlighting the low specificity of such expressions. Despite 
this variability, people still categorize facial expressions as indicators of specific emotions, 
but these can be cultural stereotypes rather than universal truths (Barrett, 2017). Barrett 
emphasizes that the same emotion can manifest in radically different ways across individuals 
and contexts, and that identical facial movements can be seen during different emotional 
experiences. She also argues that individuals respond differently to emotions, and even the 
same person can exhibit varying physiological responses to the same emotion on different 
occasions. This does not imply that the body reacts randomly; rather that variation is the 
norm. Physical reactions have no inherent emotional meaning but instead, the brain interprets 
these changes in the context of a situation and constructs them into an emotional experience. 

2.4.4. Additional Relevant Technologies  

In addition to the previously described technologies, several other technologies that could be 
considered emerging in certain contexts, show potential for user evaluation. The following 
sections briefly outline some of these technologies: eye tracking, driver-monitoring systems 
(DMS), wearable technology, digital twins and AI. 

Eye tracking  

Eye tracking technology monitors the user's eye movements to identify where they are 
looking, the duration of their gaze and the path their eyes take. By analyzing the user's eye 
location, movement and pupil size at any given moment, the system can pinpoint areas of 
interest (Novák et al., 2023). While eye tracking may appear to be a recent innovation, the 
concept has existed since the 1800s (Leggett, 2010). However, the technology has undergone 
significant advancements and continues to evolve, making it increasingly popular for 
assessing the usability and user experience of digital interfaces and products.  

14 



A wide range of eye tracking devices are available on the market, including glasses and 
specialized bars that can be mounted on monitors or other surfaces (Novák et al., 2023). 
These devices typically operate using an infrared light beam that is invisible to the human 
eye, directed at the subject's face and eyes. The technology relies on two key reference 
points: the reflection of light from the retina and the reflection of light from the cornea. Eye 
tracking technology is costly and demands specialized expertise to operate effectively. 
Therefore, it is crucial to weigh the potential benefits against the investment required before 
deciding to implement it.  

According to Smart Eye, eye tracking technology is continuing to grow and evolve (Smart 
Eye, 2025). It has expanded from being solely a research tool used in controlled laboratory 
settings to being applied in complex, real-world environments to study a wide range of 
behaviors. As demands change, so too does eye tracking technology, creating significant 
potential for future developments. Smart Eye identifies several key trends, including the 
integration of eye tracking as part of a multi-sensor approach, the move toward tracking in 
dynamic and unpredictable environments outside the lab, and its increasing use in safety and 
health monitoring.  

Driver monitoring systems (DMS) 

Driver-monitoring systems (DMS) are technologies designed to detect driver impairment and 
enable appropriate interventions (Koniakowsky et al., 2025). They monitor behaviors such as 
distraction and issue warnings when necessary. DMS operates by recording drivers' eye 
movements in real time, identifying signs of fatigue or inattention. The system determines the 
driver's glance location relative to predefined areas of interest using distraction detection 
algorithms. Glances toward the forward roadway are classified as attentive, while glances 
toward other areas, such as in-vehicle displays, are classified as distracted. When distraction 
is detected the DMS can, for instance, issue a warning to prompt the driver to refocus on the 
road.  

Eye tracking is a key component in driver-monitoring systems, significantly expanding the 
range of functions these systems can perform (Anyverse, 2025). For instance, a DMS 
equipped with eye tracking can monitor driver drowsiness and fatigue, assess whether the 
driver is scanning their surroundings effectively and detect interactions with pedestrians. 
Additionally, it can evaluate passenger emotions, identify the presence of children in the 
vehicle to enhance their safety and even support in-cabin health monitoring. By analyzing eye 
behavior and interaction patterns, the system may also help detect early symptoms of medical 
conditions. 

DMS is currently used in millions of vehicles, and looking ahead, the goal is to expand its 
capabilities to detect impaired or intoxicated drivers, with studies on this already underway 
(Lyrheden, 2022). From July 7th, 2024, all new vehicles are required to be equipped with a 
Driver Drowsiness and Attention Warning (DDAW) system, which is designed to detect signs 
of driver fatigue and inattention (European Commission, 2022). However, starting in 2026, 
all new and existing vehicles, both passenger cars and commercial vehicles, must be 

    
15 



equipped with an Advanced Driver Distraction Warning (ADDW) system. To comply with 
this regulation, all new cars will be required to incorporate a camera-based DMS (Lyrheden, 
2024).  

Wearable technology 

Wearable technology refers to electronic devices that are comfortably worn on the body or 
attached to clothing (Shen et al., 2017). This type of technology goes beyond simply counting 
steps, measuring heart rate, or tracking sleep patterns (KMS Healthcare, 2024). Wearable 
devices today offer far more advanced capabilities. As one of the fastest-growing segments in 
the tech market, they are widely recognized as an emerging technology (Burmaoglu et al., 
2018). These devices encompass a broad range of applications that collect and display 
real-time data related to health, movement and other sensory inputs. 

Wearable technology has had a significant influence on the healthcare sector, where they play 
a key role in enabling continuous monitoring of vital health metrics (KMS Healthcare, 2024). 
Equipped with integrated health sensors, these devices allow for remote observation and early 
detection of potential medical issues, features that are particularly useful in the management 
of chronic conditions. As a result, they help facilitate more responsive care, improve access 
to personal health data and have the potential to lower healthcare costs. Beyond healthcare, 
wearable technology is also becoming increasingly popular in the commercial sector. Fitness 
trackers and smartwatches, for example, are now widely used by consumers seeking to 
monitor and improve their overall well-being through real-time feedback (Burmaoglu et al., 
2018). 

Other industries adopting wearable technology include aviation and space research. In 
aviation, wearables are used to monitor pilots' vital metrics, helping to identify and prevent 
abnormal physiological conditions such as impaired blood flow, oxygen deficiency or 
fatigue-related symptoms, which can affect performance across different types of aircraft 
(Bellamy III, 2020). In the space sector, wearable technology is widely used to monitor 
parameters for health research on astronauts. On the International Space Station, for instance, 
astronauts have worn specialized devices such as watches, headbands, and vests to help 
scientists study sleep quality, exercise effectiveness, cardiovascular health, and more (Gaskill, 
2024). 

Digital twins 

The foundational idea of digital twins dates back to NASA’s use of mirrored systems during 
the Apollo program (Glaessgen & Stargel, 2012). The term "digital twin" was officially 
introduced in 2002 as a virtual model of a physical system that is updated with real-time data. 
It can be a virtual representation of a physical object, system, person or process, intended for 
real-time monitoring, simulation, and analysis (McKinsey & Company, 2024). A digital twin 
works by a physical object being paired with sensors that monitor key aspects like energy 
output, temperature and weather conditions (IBM, 2021). These sensors transmit data to a 
processing system, which then updates the digital replica of the object in real-time. With this 

16 



data, the digital twin can run simulations, identify performance issues and suggest 
improvements. The goal is to leverage insights from the digital model to optimize the 
performance of the physical object. 

Digital twins are used extensively in various industries such as manufacturing, automotive, 
aviation, energy, healthcare and logistics (Willige, 2022). Applications span across areas like 
establishing time frames, improving designs, tracking health indicators and improving 
product performance (IBM, 2021). As stated by RISE (RISE Research Institutes of Sweden, 
n.d.), digital twins have the potential to be the next major advancement in the development of 
a digitalized society. This is further emphasised by IBM (2021) that states the future of digital 
twins to be nearly limitless in a rapidly expanding market driven by increased demand. 
However, it is important to note that not all projects would benefit from digital twin 
technology, as it may place a high demand on resources and not be suitable for less complex 
products.  

Artificial intelligence (AI) 

AI can function independently but also serve as part of a solution, for example, in 
technologies such as eye tracking, emotion interpretation, XR and more. AI is expected to 
significantly shape the future of such technologies, for example by improving data processing 
and enhancing overall capabilities. 

While artificial intelligence may seem like a relatively recent innovation, the concept dates 
back to early work by pioneers like Alan Turing and John McCarthy in the 1950s (Lawrence 
Livermore National Laboratory, 2021). The term artificial intelligence was chosen as a name 
for a project aimed at developing ideas on thinking machines. Defining AI at the moment is 
no easy task and there are numerous different definitions used (Sheikh et al., 2023). A 
common and broad definition of AI is as a technology that allows machines to replicate and 
simulate complex human abilities. Other definitions delve deeper into what such “complex 
human abilities” entails with suggestions like the capacity for environmental awareness, 
goal-directed behavior, action initiation and learning through feedback. The High-Level 
Expert Group on Artificial Intelligence (AI HLEG) of the European Commission defines AI 
as: Systems that display intelligent behaviour by analysing their environment and taking 
actions - with some degree of autonomy - to achieve specific goals.  

AI operates by processing large volumes of data through algorithms, often based on machine 
learning or neural networks, to identify patterns and make predictions or decisions without 
being explicitly programmed for each task (Dwivedi et al., 2021). It is already widely used 
across various industries including healthcare, finance and consumer technology, and it is 
further enhancing the performance of already established technologies. Dwivedi states that as 
AI continues to evolve rapidly, with new opportunities constantly emerging, its significance 
and potential to impact multiple industries and society as a whole become increasingly 
evident. Stige et al. (2023) suggest that AI can be leveraged in key areas such as gaining 
insight into the context of use, identifying user needs, informing solution design, evaluating 
usability and supporting the development process. 

    
17 



Although AI presents vast opportunities as described above, the way forward is not clear and 
its implementation requires careful consideration. Challenges include high data and 
computing requirements, a lack of transparency in decision-making (the “black box” 
problem) and ethical concerns around privacy, fairness and accountability (Floridi et al., 
2018).  

18 



3 
Study Methodology and Process 

 
The study was carried out in two main phases: Phase one - Technology Scouting and Phase 
two - User Evaluation Testing. The phases overlapped iteratively as research of the 
technologies and their potential applications continued throughout the study. The process is 
illustrated in Figure 3. 

 

Figure 3. The iterative project process of Phase one and Phase two. 

Phase one served as the foundation of the project, with phase two building upon its insights 
and findings. The first phase began with an initial research analysis that included background 
research and meetings with the team at Volvo, as well as relevant contacts at Chalmers. This 
work helped define the project scope, identify available resources, determine what could 
realistically be achieved during the semester and clarify the definition of emerging 
technologies.  

Following this, the study entered a deeper research phase, focusing on a more detailed 
exploration of emerging technologies and their potential role in shaping future user 
evaluation methods. This also included a series of expert interviews with both internal Volvo 
stakeholders and external experts. The aim was to gain insight into current and upcoming 
technologies, their potential applications for Volvo and the challenges they may pose. Key 

    
19 



outcomes from phase one included a benchmarking of technologies, identification of 
opportunities and barriers as well as refined focus for the continued work. 

Building on the findings from phase one, phase two explored how both new and existing 
technologies could be included in user evaluations. A central task was developing user 
evaluation methods that incorporated emerging technologies to uncover the types of insights 
they could generate. Two types of user evaluations were conducted: an XR workshop 
simulating early-stage product development and a driving simulator study incorporating 
emotion interpretation technology. The outcomes from phase two included concepts for user 
evaluation methods and insights from testing in practice.  

The combined findings from phases one and phase two resulted in a set of recommendations 
to guide Volvo Group’s future efforts and adoption in this area.  

20 



4 
Phase One - Technology Scouting: Method 

 
The aim of phase one was to mainly address the first research question: How can emerging 
technology be used in future user evaluation methods? Accordingly, this phase focused on 
exploring which technologies could be considered emerging, which have recently seen 
notable advancements and which hold significant potential for user evaluation. Throughout 
the entire phase, the perspective of Volvo Group and the implementation for user evaluations 
was kept in mind. The phase involved two main parts: data collection and data analysis. 
Iterations between the two parts took place continuously. The data collection involved expert 
meetings and extensive technology literature research, while the data analysis involved 
mindmapping, KJ-analysis and internal group discussions.  

4.1. Technology Literature Research 

The data collection included an initial research stage that developed into a targeted 
technology literature research on relevant technologies. In parallel, existing literature on user 
evaluation methods and prior studies in related fields was reviewed. The initial research stage 
involved a broad literature scan to identify emerging technologies relevant to the study. Based 
on insights from this scan, along with input from expert meetings conducted in parallel, a 
selection of technologies for further exploration was made. This led to the more targeted 
technology literature research, in which the selected technologies were examined in greater 
depth. This part of the literature research involved online searches using keywords such as 
“emerging technology in user evaluation”, “emerging technology in HMI evaluation”, “XR 
user evaluation” and “future applications for eye tracking”. The searches aimed to identify 
relevant information about the technologies and their potential applications and selected 
articles along with references were reviewed in greater depth to inform the study. 

4.2. Meetings with Experts for Technology Knowledge 

In parallel with the literature research, numerous expert meetings were conducted to gain 
deeper insights into practical usage of emerging technologies. Meetings were held with both 
internal Volvo stakeholders and representatives from external companies. The overall 
objective with the expert meetings was to assess how each technology is currently used and 
to explore its potential applications in future user evaluations, including associated 
challenges, opportunities and considerations. Internal meetings were conducted to gather 
insights into Volvo’s current practices regarding user evaluations and the integration of 
emerging technologies. External meetings with stakeholders from other organizations were 
carried out to provide benchmarking input and inspiration for potential opportunities and best 

    
21 



practices. Table 1 shows the distribution of meetings by technology, with parentheses 
indicating the number of internal meetings out of the total number of meetings.  

Table 1. Overview of meetings in each technology area. 

Technologies Total # of meetings 

XR 11 (5) 

Emotion interpretation 2 

EEG-sensors 2 

Eye tracking 1 

AI 1 

User evaluations 4 (3) 

Other meetings 2 (2) 

The internal meetings were held with experts from the departments Volvo GTT Complete 
Vehicle, Volvo Group Design Advanced Design, Volvo Group Digital & IT Visualization & 
Global Support, Volvo Penta Digital Strategy & Business Office, Volvo Group Digital & IT 
Mobile & XR, Volvo Group Digital & IT Mobile, 3D and XR as well as Volvo GTT Human 
Behaviours & Usability. The meetings were conducted with senior roles specialized in areas 
such as design, strategy, management and engineering. 

External meetings included employees and researchers from Chalmers University of 
Technology, specifically from the Design and Human Factors department and FUSE 
makerspace. Additionally, external meetings were conducted with the companies Smart Eye, 
Imitera, AI Sweden, RISE, VTI, Innobrain, Noldus and Oulu University in Finland. The 
meetings were conducted with roles such as tech leads, system engineers, chief officers, 
researchers and co-founders. Beyond the meetings, participation in an XR-related event 
hosted by XR Sweden was undertaken.  

4.2.1. Recruitment of Experts for Technology Meetings 

The expert recruitment aimed to include a diverse range of knowledge and expertise, 
covering different roles, assignments, and work experience. They represented different 
genders, ages, and nationalities. Some had general knowledge of XR, eye tracking, or user 
evaluations, while others specialized in areas such as Volvo’s use of XR, emotion 
interpretation applications, or EEG sensors. 

A total of 19 experts or teams were consulted, with several participating in recurring 
meetings that either focused on the same technology or area or showcased expertise across 
more than one area. Of these, 10 were internal experts or teams from various Volvo Group 

22 



departments. Experts were selected based on their knowledge in areas such as user evaluation 
methods, specific emerging technologies or Volvo’s internal processes. Experts were 
identified using a combination of sampling strategies. In some cases, experts available for 
meetings responded to inquiries sent to selected companies’ general information addresses. 
This approach combined purposive and convenience sampling. In other cases, referral 
sampling was applied, drawing on recommendations from earlier meetings and input from the 
SGO team at Volvo. At other times, specific experts were deliberately selected based on their 
area of expertise, representing purposive sampling. 

4.2.2. Technology Meeting Structure 

An interview guide was tailored for each expert meeting, based on the interviewee’s area of 
expertise and the specific objectives of the meeting. While the guides were adjusted slightly 
between participants, they all had a similar structure and followed a semi-structured 
approach, allowing for flexibility to explore interesting topics or perspectives during the 
conversations. In the initial stages of the study, the questions were broad and exploratory, 
aiming to gather foundational insights. Over time, as the understanding of each explored 
technologies deepened and knowledge gaps became clearer, the interview guides were 
refined accordingly to include more targeted and specific questions aligned with the evolving 
focus of the study. See Appendix A for an example of the full interview guide. 

While several meetings were conducted in person, others were held via Microsoft Teams due 
to geographical distance or similar constraints. In-person meetings often included guided 
tours of relevant facilities and opportunities to interact with, or test, the technologies 
discussed. Meeting durations ranged from 30 minutes to two hours, with one or more experts 
participating. One member of the study’s project team facilitated the discussion, while the 
other was responsible for taking notes. The meetings were primarily conducted in Swedish, 
although English was used occasionally when necessary. No audio or video recordings were 
made during these sessions. 

4.3. Analysis of Technology Scouting 

Given the wide range of technologies identified during the initial stage of data collection, an 
iterative analysis process was adopted. This involved a screening and prioritization process, 
ensuring that attention remained focused on the most promising and applicable technologies. 
The screening and prioritization process was conducted by continuous discussions within the 
project team. Technologies that, based on research or insights, were deemed no longer 
relevant to the study's objectives were excluded from further exploration. Conversely, 
technologies that demonstrated potential were selected for more in-depth analysis.  

During the analysis, it became apparent that certain information was lacking, preventing 
informed and accurate decisions regarding the potential of some technologies. As a result, a 
mind map was created to highlight the corresponding knowledge gaps. The mindmap 
visualization led to the initiation of an additional technology scouting aimed at addressing 
those gaps. 

    
23 



Following the different research phases, the insights collected were analyzed using the KJ 
method. Insights and reflections from both the expert meetings and the literature review were 
transcribed onto color-coded post-it notes (representing internal or external insights) and 
grouped into thematic categories. The categories included the technologies themselves, user 
evaluation, opportunities, needs, challenges and other relevant themes (see Figure 4), 
facilitating the identification of patterns and emerging insights. 

Figure 4. Overview of the KJ-analysis structure.  

24 



5 
Phase One - Technology Scouting: Results 

 
This chapter presents the results of the study’s first phase, the technology scouting. The 
technology scouting consisted of technology literature research as well as expert meetings 
and covered a wide range of technologies across various industries. The presented result 
provides insights into the value emerging technologies could provide for user evaluations at 
Volvo Group. 

5.1. Findings from Technology Scouting  

The following technologies were initially identified as relevant in the study’s context: 
extended reality (XR) technology, electroencephalography (EEG) sensors, emotion 
interpretation, eye tracking, artificial intelligence (AI), driver monitoring systems (DMS), 
wearable technology and digital twins.  

In phase one, DMS, wearable technology and digital twins were excluded for further 
exploration despite their emerging nature and future potential in the automotive industry. This 
decision was made since DMS and wearable technology were considered less suitable for 
short-term user experience or HMI evaluations, but show greater potential for long-term 
driver monitoring, especially in health-related contexts. Similarly, digital twins were deemed 
as less suitable for short-term user evaluations and additionally classified as better suited as 
complements to user evaluations rather than a stand-alone solution. Moreover, there are two 
main technologies for emotion interpretation: sensor-based and AI-based. Given that 
sensor-based techniques (e.g., heart rate monitoring) are less emerging in this context and that 
AI-based emotion interpretation software is more easily accessible, the decision was made to 
proceed with the latter. Consequently, sensor-based emotion interpretation was excluded from 
further exploration. 

5.1.1. Key Aspects and Potential Applications of the Identified 
Technologies 

Following the exclusion of DMS, wearable technology, digital twins and sensor-based 
emotion interpretation, the remaining technologies were deemed relevant for further 
exploration in the next stages of phase one. Table 2 presents these technologies in terms of 
the key aspects measurements, current use, challenges and possible combinations. Table 3 
outlines potential future applications of each technology across diverse areas related to user 
involvement in the automotive industry.  

    
25 



Table 2. The identified technologies: metrics, applications, challenges and combinations. 

Technology Key aspects 

XR 

Measures: Mixed. 
 
Current use: User evaluation in immersive controlled settings for contextual insight. 
Also used in training, teambuilding, onboarding, collaboration, entertainment, 
monitoring, marketing, sales and as a virtual shared meeting space. Factory layout 
design. Reviews of technical product aspects and design concepts. Quick prototyping.  
 
Challenges: Potential struggles with stability and drifting of virtual objects (especially 
with “heavy” models). Precision of tools (eg. pointing and selecting). Not always 
seamless or quick enough to go from virtual 3D model into review in XR. Distracting 
due to low-fidelity (resolution). Lacking in haptic feedback. Influenced by the 
suppliers’ interest (eg. software updates removing certain tools). Costly to create 
detailed and high-quality 3D-models. Requires resources such as time and money. 
Potentially complicated and time-consuming setup, especially with multiple users. 
Initial learning-curve for tools and applications. Gamified feeling could lead to the 
experience being too engaging or non-professional. Experienced nausea. Ergonomic 
issues due to bulky hardware. Issues regarding hygiene when sharing headsets. 
Confidentiality issues if external application is used, GDPR. 
 
Possible combinations: Eye tracking, EEG sensors, emotion interpretation 
technology, wearable technology, AI. 

EEG sensors 

Measures: Brain imaging technique that uses scalp electrodes to measure brain 
activity. 
 
Current use: Primarily used in medical fields for diagnosis, but also applied in 
research on user experience and human-machine interaction for identifying cognitive 
load, fatigue, interest etc. 
 
Challenges: Involves lengthy setup, complex result interpretation and potentially 
bulky equipment. Almost impossible to interpret the results as a non-professional. 
Limited in detecting factors such as emotions and user preferences/experience. Need a 
lot of data for accurate measurements. Many factors can disturb the signals and affect 
results (e.g. blinking and clenching jaw). Some wireless devices are not accurate 
enough for user evaluation. May cause discomfort or distrust due to the feeling of 
being observed. Costly equipment.  
 
Possible combinations: Eye tracking, emotion interpretation technology, XR, 
wearable technology, AI, digital twins. 

26 



Emotion 
interpretation  

(using AI) 

Measures: Muscle movements in facial expressions to infer emotions. 
 
Current use: To observe how individuals behave in interactions with others and to 
understand the subconscious feelings influencing their choices. 
 
Challenges: Users react differently to emotional stimuli (some may laugh when 
scared while others may appear neutral) which highlights a limitation in using AI, 
such as facial recognition, to interpret emotions. Needs precise camera set-up, 
dependent on good lighting. Extensive data output and complex result interpretation.  
 
Possible combinations: Eye tracking, wearable technology, AI, EEG sensors.  

Eye tracking 

Measures: Eye-movements, which can expose human behaviours often expressed 
subconsciously. 
 
Current use: To understand where users focus their attention, how they process 
information, make decisions and more. Supports the improvement of digital 
interfaces and the analysis of user and consumer behavior. Used in cars (DMS), 
simulators for tests and training, Face-ID, Face-Authenticator and in usability testing, 
among others. 
 
Challenges: Relies on the assumption that there is a correlation between eye fixations 
and cognitive processes which may not hold true for all users or products. Extensive 
data output and complex result interpretation.  
 
Possible combinations: Emotion interpretation technology, AI, XR. 

AI 

Measures: Mixed (can be part of several technologies and methods).​
 
Current use: Supports coding and testing, makes sense of large amounts of data, 
chatbots, data analysis, ideation feedback, inspiration. ​
 
Challenges: AI can “hallucinate” by generating false information based on learned 
statistical patterns. If AI has been trained “wrongly” it can make misinterpretations. 
Bias can occur if only trained on a certain type of data.​
 
Possible combinations: Mixed (can be part of several technologies and methods). 

 

 

 

 

 

 

    
27 



Table 3. The identified technologies: potential future applications. 

Technology Potential future applications 

XR 

●​ Review early-stage designs. 
●​ Enable quick concept comparison. 
●​ Experience “pre-feasible” and future-oriented designs. 
●​ Design in XR, eg. use AR to overlay sketches on physical objects. 
●​ Evaluate non-static digital models responding to what users do. 
●​ Evaluate anthropometry in a digital truck. 
●​ Co-simulate interaction between truck and other road users to better 

understand traffic safety. Combine with motion tracking (suit, stripes or 
full-body estimation). 

●​ Conduct global digital user evaluation. 
●​ Conduct design workshops. 
●​ Conduct virtual focus group discussions. 
●​ Share physical environments (immersive feeling) with participants at different 

locations. 
●​ Create XR experiences without headsets, via projections. 
●​ Create “3D photographs” with Gaussian splatting camera technique 

(photorealism). 
●​ Replace screens (in-vehicle) with lightweight interactive AR glasses. 
●​ Provide customer support and service guidance. 
●​ Create interactive marketing experiences. 

EEG sensors 

●​ Evaluate and compare different design concepts. 
●​ Observe user reaction to simulated scenarios in an XR environment. 
●​ Evaluate cognitive load, mental workload, frustration. 
●​ Evaluate audio or visual alerts in terms of user attention. 
●​ Evaluate different ways of altering other road users (eg. pedestrians). 
●​ Detect low driver engagement and trigger real-time alerts through headbands. 
●​ Evaluate fatigue with lightweight in-ear devices. 
●​ Assess pre- and post-drive performance. 

Emotion 
interpretation  

(using AI) 

●​ Evaluate and compare different design concepts. 
●​ Observe user reaction to simulated scenarios in an XR environment. 
●​ Gather unconscious responses in user experience evaluations.  
●​ Gather unconscious responses when a person is exposed to multiple stimuli. 
●​ Replace surveys with camera data using emotion interpretation. 
●​ Use as a basis for discussion in evaluation (complementary), after a test. 

28 



Eye tracking 

●​ Iris recognition in vehicles for secure access and unlocking. 
●​ Adjust settings according to driver preference through driver recognition. 
●​ Track the entire cabin and all passengers by improved monitoring systems. 
●​ Analyze in-vehicle behavior and adjust settings as well as issue alerts needed. 
●​ Detect driver fatigue and use an engaging AI companion to provide support. 
●​ Detect user focus and automatically adjusts settings accordingly, eg. different 

parts of interface lighting up during driving in the dark. 
●​ Provide interactive HMI feedback to guide driver attention and prevent false 

alerts or wrongful driving behaviour. 
●​ Evaluate and compare different design concepts. 
●​ Observe user reaction to simulated scenarios in an XR environment. 

AI 

●​ Suggest relevant questions and identify gaps in user evaluations execution. 
●​ Analyze and interpret evaluation results. 
●​ Analyze data based on driver performance in real driver contexts. 
●​ Detect health issues by face-analysis. 
●​ Predict how users will interact with a product, “participate” in evaluations. 
●​ Allow personalized user experiences. 

​
5.1.2. Emerging Trends and Considerations in XR 

XR is becoming more broadly accessible, moving away from its previous status as an 
exclusive or high-end tool and several aspects of the technology could currently be 
considered emerging. This includes improved avatars, increasingly realistic virtual 
environments, improved wireless headsets, the ability to simulate walking long distances 
within a confined physical space and the growing use of natural hand gestures instead of 
hand-held controllers. Opinions on hand gestures vs. hand-held controllers vary. While some 
value the immersive and intuitive feel of using their own hands, others prefer hand-held 
controllers due to their precision, lower learning curve and haptic feedback. A promising 
development in the area of haptic feedback is the emergence of tactile gloves, which aim to 
deliver a more natural interaction experience without sacrificing feedback quality. Another 
critical, yet often overlooked, aspect of immersive feel in XR is the development of realistic 
sound design. Some argue that high-fidelity audio plays a significant role in enhancing the 
sense of presence and overall realism in XR experiences. 

While XR holds great promise, it is also important to acknowledge its current limitations, as 
considerations that can guide more purposeful and value-driven use. Thus, XR should be 
used selectively when it truly adds value. The value mostly lies in enabling immersive 
collaboration with virtual representations of products or spaces. This type of interaction can 
lead to faster decisions and clearer communication than traditional tools such as PowerPoint 
presentations, Teams meetings or Excel-based analysis. Full immersion, where all 
participants use headsets, is essential for realizing these benefits and partial participation 
through desktop interfaces often diminishes the collaborative and spatial advantages of XR. 
As such, participants in an XR experience joining in via computer should be an exception.  

    
29 



5.1.3. Practical Considerations in Applying EEG Technology 

EEG technology is advancing, but its application remains complex, especially outside 
professional research. Both industry and academic voices agree that EEG signals are difficult 
to interpret without specialized expertise. Despite this, EEG technology is becoming more 
accessible, with wireless devices and systems that simplify data collection and interpretation 
entering the market. Using such innovations should be done with caution since some argue 
that they are sometimes marketed with exaggerated claims of resolution needed for serious 
research. On the contrary, there are arguments made that EEG intended for industrial use does 
not need as high resolution or wide range measurements of brain activity as medical research. 

Since industry-focused research typically wants tools that are fast, reliable, and easy to 
deploy, lengthy and complex setups of EEG sensors remain a challenge. High-quality signals 
still depend on wet active electrodes, which provide the best data but require time-consuming 
preparation and are sensitive to variables like hair products or skin conditions. Semi-wet 
alternatives such as saltwater-dipped sensors offer a middle ground, reducing setup time but 
still requiring careful handling. While dry electrodes are the most convenient in these aspects, 
their signal quality is as mentioned often less sufficient for detailed analysis. 

Another constraint of EEG technology is comfort, since wearing EEG sensors for over an 
hour can become uncomfortable, leading to reduced participant engagement and data quality. 
This creates a trade-off where longer sessions may provide more data (necessary for reliable 
testing), but only if participants remain attentive and at ease. There is also a risk of 
misconceptions among participants, on things such as that EEG technology can "read 
thoughts." These misunderstandings can cause discomfort or hesitation so to build trust, 
researchers can demonstrate how brain activity is measured by showing the signals generated 
from simple actions like blinking or jaw clenching. In most cases, this helps participants feel 
more at ease and often sparks fascination and a positive testing experience. 

A promising but still-developing area is the integration of EEG with VR. While some 
manufacturers are exploring this, usability challenges remain. For the setups to work, the VR 
gear must be comfortable during extended use and avoid interfering with the EEG signals. 

5.1.4. Balancing Emotion Input from User and System 

When evaluating HMI by emotion interpretation, it is important to consider both user 
expression and the system (machine) context, since tracking facial expressions alone is not 
enough. Capturing what the machine is doing at any given moment, for example by using an 
additional camera, enables more accurate correlation between system behavior and user 
response. 

Asking participants about their experience also adds depth and context that technology alone 
might miss. But, it is also worth noting that users might forget or misremember how they felt 
at some stages of an evaluated interaction and that their final emotional impression can 
overwrite initial reactions. For this reason, combining direct user input with objective data 

30 



(video analysis) can provide a more nuanced understanding. Thus, these methods are 
complementary and can together possibly offer a more complete picture of emotional 
engagement. 

5.1.5. Opportunities and Concerns when Implementing AI 

As AI technologies become more accessible and affordable, their use has expanded beyond 
traditional tech-centric fields. This has opened up new possibilities of AI but also raised 
important concerns around data transparency and ethics. Automated systems should not be 
implemented blindly, especially if it is unclear how data is being processed or analyzed. 

In the context of user evaluation, there is some hesitation around using talking or guiding AI 
assistants. While they can streamline tasks, there is a risk of over-reliance and there are points 
made that AI could not replace the value of independent human thought (also applicable in 
AI-contexts outside of user evaluations). As one person put it: "Nothing beats having to think 
for yourself". 

One of the most promising applications of AI is the growing field of digital assistants, both 
for professional and personal applications. There is also untapped potential in personalizing 
experiences. However, some highlight that automated AI-personalization is often skipped by 
companies due to high development costs and limited perceived value. Looking ahead, AI 
tools that can collect and analyze large volumes of user data may fundamentally reshape how 
industries operate. By enabling deeper and faster insights, such tools could revolutionize how 
we understand behavior and develop products – if used responsibly. 

5.1.6. Combining Eye tracking with Complementary Methods 

While eye tracking is a well-established technology, it could still be considered emerging in 
certain contexts. The technology has only recently become more affordable, enabling its 
adoption across various industries. Furthermore, continuous advancements in eye tracking 
technology contribute to its ongoing relevance and potential. However, eye tracking on its 
own has its limitations. While it provides a visual mapping, such as heat maps showing where 
users look, it does not reveal the reason behind the gaze pattern. To truly understand user 
intent or emotion, eye tracking should be combined with other data sources, technologies or 
follow-up questions. Eye tracking is as such best utilized as part of a triangulation approach, 
complementing other user testing and observational methods rather than replacing them.  

Some companies are focusing on non-intrusive eye tracking technology to observe user 
behavior. By designing tools that do not make users feel actively monitored, they can capture 
more natural and authentic interactions. There is also future potential in combining eye 
tracking with other non-intrusive methods, such as radar-based sensing, to gather data from a 
distance without the need of wearable equipment.  

    
31 



5.1.7. Considerations for Comprehensive User Understanding 

In user research, quantitative methods like eye tracking, EEG, stress measurements and heart 
rate monitoring offer precise measurements, but they need to be combined with qualitative 
methods such as interviews and observations to create meaningful insights. Due to this, there 
is a need to triangulate methods to capture both objective and subjective dimensions. When 
doing so, starting with open-ended methods often helps identify the core challenges and make 
sure that additional measurements provide the sought insights. Without clarity on what is to 
be understood, even the most reliable data risks being irrelevant. 

A major industry challenge regarding user evaluation is that large amounts of data that are 
collected risks remaining underused. This is highlighting the argument that there is potential 
in utilizing the data that is already being collected to a large extent, rather than collecting 
more. If the analysis of large amounts of data (eg. data collected from users driving their own 
vehicles) could be streamlined and made more efficient, it could have a transformative impact 
on the human factors field. 

Another area of concern is the tendency to avoid real end-user testing due to confidentiality, 
logistical challenges, scheduling constraints, limited budgets and lack of expertise. As a 
result, global business decisions may be based on insights from a small sample or rely too 
heavily on assumptions rather than honest feedback. Additionally, the individuals conducting 
user evaluations are in some cases not the ones analyzing the results or making product 
design decisions. This disconnect can lead to important insights being lost in translation. To 
avoid this, designers should ideally be closely involved in user evaluations, observing 
behaviors and interpreting feedback in context. Ultimately, deep user understanding and 
strong early-stage research create the foundation for taking bold design risks later in the 
process. However, there is an added challenge in interpreting user reactions since truly 
innovative design can feel unfamiliar at first, but appreciated over time.  

5.2. Phase One’s Implications for Future User Evaluation 

The analysis in phase one revealed several gaps in user evaluations relevant to this study. 
These insights are categorized below by identified needs and areas for improvement. The 
findings have important implications for future user evaluations and directly informed the 
ideation activities carried out in phase two of the project. The identified needs are: 

●​ In-depth understanding of users 

●​ Understanding of driver interaction with other road users 

●​ A high feeling of reality in lab-simulations 

●​ Effective and efficient collection and analysis of data 

There is in some cases a lack of conducted user studies with end users, sometimes due to 
several resource issues. Important questions to address are: How many users are targeted in 

32 



evaluations? Who are the users targeted in evaluations? When in the process are user 
evaluations performed? Furthermore, the results of user studies are not always taken into 
account in design decisions and designers would prefer to be present in evaluations more 
often. Important questions to address are: What are the reliability of user evaluations? Is 
there a lack of trust in the results from user evaluations, why? Are user evaluations 
representative of the targeted user group?  

Take-aways from Phase One: Technology Scouting  

●​ Each emerging technology has the potential to offer significant value in future 
applications. 

●​ Some emerging technologies still face significant challenges and require further 
refinement, while others are more mature and ready for immediate implementation. 

●​ Combining and complementing different methods and emerging technologies can 
unlock benefits for user evaluation. 

●​ Current uses of XR in user evaluation and various industries offer diverse benefits, 
from training to product design, but challenges include technical limitations, cost, 
setup complexity, and potential user discomfort, emphasizing the need for careful 
implementation. 

●​ EEG technology is valuable for diagnosing medical conditions and researching user 
experience. Depending on the chosen device, the complexity, signal interference and 
potential discomfort pose challenges for effective and reliable use. 

●​ Emotion interpretation technology is used to observe and understand subconscious 
emotional responses, but its limitations, such as varied user reactions and complex 
data interpretation, highlight the challenges in accurately interpreting emotions. 

●​ Eye tracking helps analyze user behavior but faces challenges in data interpretation 
and the assumption that eye fixations always correlate with cognitive processes. For 
best results, it may be combined with other methods. 

●​ AI supports coding, testing, and data analysis and has the potential to transform user 
research. Issues like "hallucination", misinterpretations and bias require careful 
implementation.  

    
33 



6 
Phase Two - User Evaluation Testing: Method 

 
The second phase of the project built upon the insights and outcomes from Phase One - 
Technology Scouting, with the primary aim of addressing the second research question: What 
types of insights and results can be obtained from user evaluation methods that incorporate 
emerging technologies? To explore this, the phase focused on ideating and testing emerging 
technologies in practice as part of user evaluation of human-machine interactions. This was 
done in order to better understand the types of results such evaluations could generate. 
Several types of user evaluations were designed, conducted and analyzed to assess their 
potential and effectiveness. An overview of the tests in practice are presented in Figure 5.  

 

Figure 5. Overview of the user evaluations tested in practice. 

6.1. Aim of Testing User Evaluation for HMI in Practice 

Building on the gaps identified from the result in phase one, a more refined aim and objective 
for this specific phase of user evaluation testing were established. The aim of the user 
evaluation testing was to explore how emerging technologies can enhance user evaluation by 
enabling earlier testing in the design process, increasing accessibility for designers and 
allowing for broader user participation. Additionally, the study sought to investigate how 
these technologies can capture more nuanced user insights to ensure that valuable aspects are 
not overlooked. This was summarized in the overarching objective of assessing:  

●​ How can the understanding of truck drivers as end user's needs and preferences be 
increased? 

34 



This aim and objective served as a guiding framework throughout the development of the 
user evaluation testing phase. To enable effective comparison between the tested evaluation 
methods, a clear and specific focus was needed. While the instrument panel was initially 
chosen, it was further refined by focusing on driver interaction in terms of comparison 
between physical (haptic) and digital (touchscreen) buttons.  

6.2. Several Considered User Groups  

This study considered several user groups relevant to the user evaluation process. The 
primary focus was on the truck driver, as the end user, whose interaction with the interface is 
central in evaluation and product development. However, other participants, such as internal 
truck drivers, experienced drivers, customers, Volvo employees, managers or truck driver 
students were also considered as potential test participants. 

Additionally, designers as well as those responsible for conducting and overseeing the user 
evaluations also play a crucial role as end users of the study’s developed user evaluation 
methods. The individuals involved could be internal teams at Volvo, tech experts or external 
partners. Furthermore, the actors analyzing the insights and results from the user evaluations 
play a key role. Similarly, this group may include designers, tech experts, Volvo teams or 
external partners. Tech scouts are also relevant, as they bring fresh insights and can assist in 
improving user evaluation methods that incorporate emerging technologies. 

6.3. Exploratory Ideation of User Evaluation Use Cases  

The ideation process occurred throughout the study at various stages. The focus of the 
ideation at the initial stage remained exploratory and ideas on how emerging technology 
could be implemented in user evaluations were intentionally kept broad, visionary and not 
constrained by concerns on feasibility. To facilitate this exploratory idea generation, various 
techniques were employed including brainwriting, brainstorming, the Dark Horse method and 
collaborative discussions within the project team (see Figure 6). 
 

 

Figure 6. Selected pictures from ideating sessions. 

    
35 



As more insights and information emerged, ideation sessions continued and evolved. A 
second round of ideation was conducted with a stronger emphasis on feasibility, potential 
value and emerging technology relevance of the use cases within the context of Volvo Group. 
As a result, the earlier ideas underwent a filtering process that eliminated irrelevant concepts, 
refined and developed promising ones and in some cases broke them down into sub-ideas or 
entirely new concepts. This process was informed by the Technology Scouting Matrix and the 
potential application areas identified in phase one. The methods employed during this round 
of ideation included brainstorming, braindrawing, brainwriting and mindmapping. 

Two ideas were selected based on their potential, relevance, feasibility for immediate testing 
as well as alignment with the project's objectives. The first type of user evaluation consisted 
of an XR workshop simulating remote collaboration during early-stage product development, 
while the second type of user evaluation was a driving simulator study incorporating emotion 
interpretation technology. These evaluations were designed to have similarities between 
them, facilitating easier comparison. As a result, the focus of each evaluation was on 
assessing the user experience of interacting with both digital screens and physical buttons, 
aligning with the earlier goal of comparing low-tech and high-tech interfaces in the user 
evaluation process. 

Moreover, since the project team was less familiar with emotion interpretation technology, 
this technology underwent preliminary testing to assess its reliability. In contrast, Volvo was 
already using XR technology, so no such uncertainty existed around its application. 

6.4. User Evaluation Testing in Practice - XR Technology 

The first type of user evaluation chosen for practical testing consisted of an XR workshop. 
The goal with the test was to observe user responses, determine whether the evaluation would 
generate new insights and evaluate its overall effectiveness. The following section outlines 
the process of employing and testing this type of evaluation using XR technology. 

6.4.1. Development of the XR Workshop  

The user evaluation employed XR technology in a workshop setting with end users to explore 
the potential for generating new insights. This method required the use of an XR-software, 
3D files compatible with XR, separate rooms, XR-equipment and a detailed manuscript. 
Several different XR collaboration programs were considered before the choice was made on 
using Campfire 3D, an XR collaboration platform designed to facilitate intuitive, spatial 
communication around 3D models. It enables teams to interact with and review complex 
designs in a shared virtual space, regardless of their physical location or device type. The 
program features collaborative elements, enabling interaction through commenting, 
sketching, laser pointing, measuring and more. In Campfire, each participant had their own 
account and were represented as avatars with names displayed below. The 3D-file used 
consisted of a car instrument panel which was sourced from an Open Source 3D-model 
website (see Figure 7).  

36 



 

Figure 7. 3D-model of car instrument panel used in XR workshop (Urveshk2623, 2022). 
CC BY 4.0 

The workshop was conducted at FUSE, Chalmers, using standalone Meta Quest 3 headsets. 
The chosen technology for the workshop was mainly MR with some quick scenes showcased 
in VR.  

6.4.2. User Recruitment for the XR Workshop 

A pilot test was conducted prior to the XR workshop to assess its functionality and identify 
any potential technical issues. The pilot test involved two male Industrial Design Engineering 
students, both 25 years old, experienced drivers but with no experience with similar user 
evaluations. One had some prior experience with XR through gaming, while the other was 
unfamiliar with the technology. They were recruited through convenience sampling, with a 
preference for participants with an educational background in design, as their perspective 
could provide valuable insights into how designers - who may also be considered end users of 
the developed user evaluation meth