Master’s thesis 2022 Knowledge transfer within complex systems Mapping the usability and transferability of in-house training in a large corporation Alice Sjöqvist & Julia Hammare Department of Communication and Learning in Sciences Division of Language and Communication Chalmers University of Technology Gothenburg, Sweden 2022 Knowledge transfer within complex systems Mapping the usability and transferability of in-house training in a large corporation Alice Sjöqvist & Julia Hammare © ALICE SJÖQVIST & JULIA HAMMARE, 2022. Supervisors: Maria Cervin-Ellqvist, Division of Language and Communication, Department of Communication and Learning in Science, Chalmers Johan Fleischer, UNICO Consulting AB Examiner: Philip Gerlee, Mathematic Science, Chalmers Master’s Thesis 2022 Department of Communication and Learning in Siences Division of Language and Communication Chalmers University of Technology SE-412 96 Gothenburg Telephone +46 31 772 1000 Typeset in LATEX, template by Kyriaki Antoniadou-Plytaria Printed by Chalmers Reproservice Gothenburg, Sweden 2022 ii Knowledge transfer within complex systems Mapping the usability and transferability of in-house training in a large corporation ALICE SJÖQVIST & JULIA HAMMARE Department of Communication and Learning in Sciences Chalmers University of Technology Abstract In today’s society, organizations strive to develop knowledge that is useful within the work place. In a large corporation, with a lot of employees, standards and complex system in play, there is a need to structure the information that builds up the organization. In light of this, the organization needs inclusion for updates and work methods. The information needs to be spread through out the organization and in order to achieve that transfer, training sessions could be established. This master’s thesis was made in partnership with UNICO Consulting and investi- gates an in-house course implemented at Volvo Cars Corporation. The main focus was to look at the transferability of the course content and how useful the mate- rial is for the participants taking the course. The research questions posed for this master’s thesis objective included what could be done to optimize the training to benefit the needs of Volvo Cars. Moreover, if the literature on learning theories, more specifically transfer of training, corresponds to the way that the training takes place in the organization. Through surveys and observations data was collected from participants who had taken the course and this thesis examined if the answers, and the participants perception of the course, correlated with what the literature states on how organizational development should take place through internal training ini- tiatives. The results show that participants have a good overall experience of the course. The evidence presented in this thesis concludes that a clear objective, definition of the activities and usability are primary factors for engaged learning. In order to provide the organization with training that should create effective knowledge transfer, it is important to have updated material and to receive feedback from both the participants and the organization. What that entails differs form case to case and therefore this thesis suggests how to increase transfer in the organisation, with specific suggestions for the observed course. Keywords: knowledge transfer, learning strategies, course evaluating, transfer of training iii Acknowledgements There are many individuals who we would like to thank for contributing with their time and knowledge to this master thesis. First of all, Maria Cervin-Ellqvist, our supervisor, for the continuous support and guidance from start to finish. Johan Fleischer, and UNICO Consulting for making this thesis possible. Furthermore, we would like to thank everyone involved from the Volvo Cars Corporation training team, who have helped us on a daily basis by answering all of our questions and who have made it possible for us to become a part of the organization. Lastly we would like to thank all the respondents from Volvo Cars Corporation who have answered our survey and the participants we have had the chance to present the training to. Thank you! Alice Sjöqvist & Julia Hammare, Gothenburg, May 2022 v Contents 1 Introduction 1 1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.1 Specification of issues under investigation . . . . . . . . . . . . 2 1.1.2 Volvo Cars as an organization . . . . . . . . . . . . . . . . . . 2 1.1.3 The course: Teamcenter V5 for Catia users . . . . . . . . . . . 3 1.1.4 The hybrid classroom . . . . . . . . . . . . . . . . . . . . . . . 5 2 Learning theories 7 2.1 Overall learning theories . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 Transfer of Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.2.1 Learning within an organization . . . . . . . . . . . . . . . . . 8 2.2.2 Models of transfer of training . . . . . . . . . . . . . . . . . . 9 2.2.3 The decision to transfer . . . . . . . . . . . . . . . . . . . . . 12 2.3 Evaluating training . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3 Methods 15 3.1 Research approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.2 Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.2.1 Course evaluation from Volvo Cars . . . . . . . . . . . . . . . 16 3.2.2 Survey study methodology . . . . . . . . . . . . . . . . . . . . 16 3.2.2.1 Survey questions . . . . . . . . . . . . . . . . . . . . 17 3.2.2.2 Survey respondents . . . . . . . . . . . . . . . . . . . 19 3.2.3 Literature study methodology . . . . . . . . . . . . . . . . . . 19 3.2.4 Observation study methodology . . . . . . . . . . . . . . . . . 20 3.3 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 4 Results 23 4.1 The course evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 4.2 The survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 4.2.1 Open questions from the survey . . . . . . . . . . . . . . . . . 30 4.3 Direct observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 4.3.1 Specific course situations . . . . . . . . . . . . . . . . . . . . . 33 5 Discussion 35 5.1 Optimizing the course . . . . . . . . . . . . . . . . . . . . . . . . . . 35 5.2 Agreement between the course and learning theories . . . . . . . . . . 37 5.2.1 Enabling transfer for the Participants . . . . . . . . . . . . . . 37 vii Contents 5.2.1.1 Determination of the current situation . . . . . . . . 38 5.2.1.2 Maintenance to ensure transfer . . . . . . . . . . . . 38 5.2.2 Enabling transfer through the Training design . . . . . . . . . 39 5.2.2.1 Determination of the current situation . . . . . . . . 39 5.2.2.2 Maintenance to ensure transfer . . . . . . . . . . . . 40 5.2.3 Enabling transfer in the Organization . . . . . . . . . . . . . . 41 5.2.3.1 Determination of the current situation . . . . . . . . 41 5.2.3.2 Maintenance to ensure transfer . . . . . . . . . . . . 42 5.3 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 6 Conclusion 45 Bibliography 47 A Appendix A I B Appendix B III C Appendix C IX viii 1 Introduction Billions of dollars are today spent on training within organizations to develop and create a knowledge transfer among employees (Grossman and Salas, 2011). This is necessary to maintain a competitive advantage as companies grow and change. The investment in training is significantly related to firm profit growth via the impact of increased knowledge (Birdi et al., 2008; Kim and Ployhart, 2014), which means that companies continue to spend much money on training. An increasing amount of research indicates that investments in training to gain knowledge in workplaces are related to improved results for the organization in several aspects (Ford et al., 2018). This is about managing and structure the or- ganizational memory. It could for example be, but is not limited to, updates on information, learning new application and/or development of working methods. Al- though research shows that individual training as a form of learning is needed, there can still be great difficulties in applying newly gained knowledge in an effective way in the workplace. It is costly, both in time and money, to have employees away do- ing training as they cannot perform their daily work within the organization while attending courses. Therefore, the most useful and effective knowledge transfer is the goal to strive for. Transfer was originally defined as “the extent to which learning of a response in one task or situation influences the response in another task or situation” (Blume et al., 2010). Nowadays it is rather defined in two dimensions; generalization and maintenance. Generalization refers to the new knowledge that can be used in a work context, and maintenance refers to the process about the knowledge and behavior that persists over time (Baldwin and Ford, 1988; Blume et al., 2010). This chapter contains an introduction and a background to the objectives of this master’s thesis. The research questions will be introduced together with a broad description of the starting point for this study. 1.1 Background The thesis will review, evaluate and propose improvements regarding an in-house course at Volvo Cars. It is a four-day course that focuses on Volvo Cars computer system and their Product Lifecycle Management (PLM) methods. More about the course will be described further down. Transfer of training should be at the center of all courses and training, in all or- 1 1. Introduction ganizations, including Volvo Cars, in order to use newly acquired knowledge. This is because the knowledge is not useful until it is set into practice, hence that the extent of transfer is as important as the information itself. The thesis is based on a specific course at Volvo Cars. However, the literature and conclusions that are used can be addressed in several situations. 1.1.1 Specification of issues under investigation The following research questions will form the basis of the thesis: • 1. What factors could optimize Volvo Cars PLM course so that the participants gain the right tools to perform their day-to-day work? • What parts of the course are vital and what parts could be eliminated? • To which extent do the participants experience knowledge transfer after participation in the course? 2. Is the workplace learning at Volvo Cars representative and relevant according to what the literature states about transfer of training? 1.1.2 Volvo Cars as an organization Volvo Cars Corporation has produced cars since 1927 and its global headquarters has remained in Gothenburg, Sweden since the start. Volvo Cars has about 23,000 employees, which makes them one of Sweden’s largest employers (Volvo Cars, 2022). With this in mind, is it understandable that Volvo Cars is a complex workplace and thus uses complex systems and methods. In order to achieve an equal way of working, a comprehensive introduction is given for all employees. For example of the systems and methods used within the company, a course is required to ensure the quality of future work. Today, it has become more common for companies to outsource training assignments. In general, companies are more willing to acquire expert knowledge in areas in which they themselves do not have much knowledge. This is especially done in the case of competence development among staff (Hansson, 2016). Volvo Cars also follows the pattern of outsourcing, and thus outsourced the training assignments. Volvo Cars has commissioned, among others, UNICO Consulting AB to be responsible for their product lifecycle management system training. Product lifecycle management, PLM, is the process of managing the entire lifecycle of a product from its incep- tion through the engineering, designing and manufacturing stages required. The system integrates data, process, people, business systems and provides a product information backbone (Bokinge and Malmqvist, 2012). One of the courses conducted within the PLM introduction is a four-day course pri- marily intended for design engineers within construction. The purpose of the course is for each employee to gain a knowledge base about what Volvo Cars standards look like. After completing the course, the participants gain access to different computer systems and applications used within the organization. This thesis will investigate and be based on this course, which is called Teamcenter for Catia V5 users, with 2 1. Introduction a focus on the transferability, relevance and usability of the course content for the participants in their day-to-day work. 1.1.3 The course: Teamcenter V5 for Catia users The course that this thesis is focused on is a four-day training in Volvo Cars PLM- system; Teamcenter. The participants are mainly newly hired Volvo Cars designers, but also consultants and individuals who want an update of the different methods used at Volvo Cars. There are between 5-8 participants attending each course, and it is mainly performed on site but also remotely. The course is based around a PowerPoint presentation and an exercise book created by the training team. The reason for the course’s existence is to organize information in a structured way. The documentation needed to get a project from start to finish within Volvo Cars is extensive and there are a lot of different procedures for each employee. Parts of this documentation are stored in the application Teamcenter and due to the fact that one individual’s work is used and reviewed by many others, there is a strong need for synchronicity and alignment in the work methods and result. The approach during training sessions is generally that a part of the course content is reviewed and then the participants do exercises in the related area. In the beginning of the course, all exercises are done together with the teacher on a more basic level. As the course proceeds the participants work more independently with the help of the exercise book based on what has been discussed together with the trainer in the classroom. The exercise book is laid out in a similar way, where the first tasks are very well described, step by step. Later on, the steps are explained less meticulously, with the intention that the participants must try to recall what has been done prior (or scroll back to previous tasks). The course extends over four days and thus contains a lot of information about different applications. Table 1.1 describes the different parts that the course contains and that this thesis has focused on. Table 1.1: Applications included in the course. Applications: Resume: Teamcenter Is a system for structuring and storing data. Teamcen- ter handles processes; including 3D designs, electronics, embedded software, documentation, and the bill of ma- terials (BOM). The system can also share essential in- formation between multiple domains and departments, such as manufacturing, quality, cost technology, compli- ance, service and supply chain. Volvo Cars is using it to structure the design models for virtual constructions, furthermore for calculations, requirements and prepara- tion. 3 1. Introduction Catia Is the CAD system that is used within Volvo Cars Co- operation. Here the master data files are created. The content of these files can be 3D models, 2D drawings and other supportive documents and annotations. TCVis Is a tool for analyzing and visualizing the geometries for the virtual car. TCVis stands for Teamcenter Visualiza- tion. KDP Is an engineering database, the database consists of sub- systems and includes information describing products from Volvo Cars, everything from a single part to a com- plete car. In this system the physical car is handled. Document Portal Is a workbench in Teamcenter which ensure the connec- tion between the Teamcenter documents and the KDP’s part numbers. NGD Editor Is an online application to apply material to the virtual car. The application is used to visualize the materials of the car for down stream users and ensure that the car looks visually pleasing. BOM/CAD alignment Is an online application where the data in Teamcenter and the data in KDP is compared within a certain car or part of a car. It is used to see if there are clashes or if there are mismatches in the two different databases. The application will show a pie chart together with regular "how to’s" if there are parts that are not aligned and describe the proper solution. DMU Garage Is a web based garage for the virtual car. The purpose of the DMU garage is to provide easy navigation to the different car variants and to ensure a fast load for the car variants to other applications/programs. It withholds the information from Teamcenter which is the master, the garage is a way to present the information in a user friendly and structured way. All the car variants that can be found in Teamcenter can also be located in the DMU-garage. MIP Is an application designed to see how the mechanical parts integrate with other parts of the car. MIP stands for Mechanical Integration Process. Exter Is a tool to encrypt data and a way to distribute that data in a safe and easy manner. It is an internal appli- cation with the purpose to send and receive files towards suppliers. The information about this course was obtained by the authors with observations, training sessions and conversations with the course trainers. All this took place during an on-boarding period which led to the authors themselves starting to teach 4 1. Introduction the material and work as trainers at Volvo Cars. As a result, perspectives both as participants and trainers have been combined with an in-depth understanding of the system, structure and purpose of the course. Ten courses have been observed and performed during the time of this study. 1.1.4 The hybrid classroom As Volvo Cars is an international corporation and due to the global pandemic, Covid19, the course has recently been offered both face-to-face and online for the participants. The classroom is set up with training equipment, both hardware and software, and a smart board where the content can be presented in an educational way. The individuals who participate online may take over control of an exercise computer in the classroom via a second meeting tool. This enables them to be in the virtual meeting and still have a course computer to work from. Moreover, this set up enables the trainer to see what all the participants are working on and can easily step in and help if so is needed. The hybrid classroom enables a learning experience similar to the one which is just intended for the students attending on-site. There are several things that can be done to facilitate the learning process for students who attend online and in the classroom. This can for example be accessibility to the learning material, a well- functioning professional communication tool and reworking of the learning sessions’ time span (Poskitt et al., 2021). Poskitt et al. (2021) also states that there are some difficulties with the hybrid classroom, such as the teachers knowledge of whether the students are overwhelmed or other aspects that may go amiss without the visual clues that can be obtained if all students attend on-site. 5 1. Introduction 6 2 Learning theories In the upcoming sections, theories will be presented with the objective to understand what can be done to provide the best conditions for learning elements. The chapter begins with a general background on learning theories, followed by a section more in depth on transfer of training where several models are discussed and used in the interpretation and analysis of results. 2.1 Overall learning theories There are many theories about learning that agree with each other but also have different approaches. In the next section, several well-known learning theories are presented. These will then be linked to more adapted transfer of training models. The philosophy of cognitivism states the importance in how the mind receives, or- ganizes, stores and retrieves the information. It is an internal process and it looks beyond what is observable (Ally, 2004). Learning is therefore an internal mental process. The definition of cognitivism, is that learning takes place in a process and that the information is conceptualized in models for understanding (Philips D. C., 2014). For a participant in a classroom to be able to acquire what is presented, the information needs to be organized, sequenced, meaningful and understandable for the learner. The teacher’s role is to provide the learner with the right tools so that the learner’s brain can process what is presented. Another key aspect from cognitivism is repetition, which secures the schematic struc- ture in the brain (Philips D. C., 2014). Cognitivism is a schemata defined theory, meaning that every learning opportunity will either update a schema and construct a new schema. This happens every time a person reads, listens to, observes or tries out something new. These schemas are used as reference points and will determine how the new information and knowledge will fit in to what is already known (Philips D. C., 2014). Constructivism on the other hand is based on active learning such as observation, processing and interpretation. The theory is about textualizing knowledge to person- alize the information with the aim of immediate application and personal meaning (Ally, 2004). In other words, the theory states that the reality is based on the learner’s own experiences which indicates that every learning experience is unique to each individuals. According to constructivism, learners should construct their own knowledge. Learn- 7 2. Learning theories ing is not about memorizing facts and theories; it is to create an understanding that has a meaning to the learner’s previous knowledge (Ally, 2004). The learner is always in the centre of learning. The trainer is taking an advisory and facilitating role, where they give the learner the opportunity to construct their own knowledge instead of issuing instructions (Ally, 2004). Moreover, learning is constructed in the mind and therefore hands-on activity is necessary but not the only part. Successful learning will take place when the mind is engaged and able to retain knowledge (Ally, 2004). In order to design an effective and favorable training session with a successful learn- ing experience, the neuroscientific prospective should also play a key role (Dumont et al., 2010). As the knowledge about the brain has increased with technological and methodological methods, research have also obtained a better understanding of how we learn and the factors that can characterise our differences as regards aptitude. The parts of the brain that are linked to learning can broadly be split in to three parts, the recognition, strategic and affective networks (Rose et al., 2002). The ’use it or lose it’ is a principle stating the importance of repetition (Dumont et al., 2010) is strengthened by how the neurons creates inhibiting connections be- tween each other. The ones that are active will be invigorated and grow while the ones that are relatively less active will diminish. With the strengthening the thresh- old will lower and the task of learning will become easier. The repetitive activity eventually leads to long-term changes for the synaptic connections. (Dumont et al., 2010). These learning theories with different approaches but also with many similarities, takes place on a daily basis knowingly and unknowingly. If they are valued and taken into account, significant progress can be made due to conscious choices that benefits the learner. This results in transfer of training models. 2.2 Transfer of Training The following chapter presents learning theories on the phenomenon transfer of training which propagates that knowledge becomes of value only when it is used (Weinbauer-Heidel, 2018). This refers to aspects on how to achieve an effective knowledge transfer in organizations. 2.2.1 Learning within an organization Adapted individual learning is not to be despised; it is a basic requirement for learn- ing. However, this is not crucial for the knowledge to be applied in real situations. More things are important in the concept of generalization. For knowledge to have a meaning, a situation-based learning must take place, where the context depicts real- ity (Ford et al., 2018). If the knowledge can not be used in the daily work situation, the knowledge will never be transferred to the workplace. Moreover, there are factors impacting generalization as measured by trained knowl- edge and the effectiveness of the use, for example work climate, support and work 8 2. Learning theories constraints (Blume et al., 2010). Additionally, there are cases where the newly acquired knowledge is used extremely rarely. The skills deteriorate in these cases which therefore can lead to a lack of motivation to use the knowledge, even fewer opportunities to practice the skills, constraints or lack of rewards (Ford et al., 2018). This can also be linked to the principle ’use it or lose it’ which was mentioned earlier. From a review of 13 studies done by the US Army Research Institute, where skill- based tasks were performed in an operational environment, some key factors were found (Hagman and Rose, 1983). The four most important factors were: • Increasing the amount of task repetitions • Testing during training • Spaced practice • Implementing different working methods and equipment during training In order to maintain the learned knowledge, two decisive factors were shown. The first factor is about the number of steps to complete the task and the second factor is to be present during the training (Hagman and Rose, 1983). Another meta-analysis found that the effect from overlearning (task repetition) is crucial for learning. The degree of overlearning does matter, more repetition is better (Driskell et al., 1992). The type of task and the length of retention interval is also crucial for retention of knowledge. These statements are in good alignment with cognitivism arguments regarding the importance of the learning process and cognitive schemas. 2.2.2 Models of transfer of training A model of transfer process was presented in 1988 in Baldwin’s and Ford’s review of transfer of training (Baldwin and Ford, 1988), see figure 2.1. This model has become the most commonly cited model for transfer of training (Blume et al., 2010). The model includes training inputs, training outputs and conditions of transfer. Train- ing inputs are divided into three main categories; trainee characteristics (cognitive ability, self-efficacy, motivation and perceived utility of training), training design (behavioral modeling, error management and realistic training environments) and work environment (transfer climate, support, opportunity to perform and follow- up). Figure 2.1 shows that trainee characteristics and work environment have both direct and indirect effects on the transfer. For example motivation from a trainee’s characteristics can have a direct effect to enable transfer (link d) and similarly, su- pervisor support from work environment has a direct effect on transfer (link e). Furthermore, these mentioned categories (link d and link e) also have an indirect effect to the extent that they affect the level of knowledge learned (link b and c), for example through perceived utility of training and transfer climate. In a meta-summary of how to transfer knowledge created by Ford et al. (2018), they have summarized the most important attributes about training transfer, see figure 2.2. Based on the different stages, figure 2.2 explains factors that impact transfer and points to implications to achieve better teaching. In the stage personal 9 2. Learning theories Figure 2.1: Model of transfer process based on Ford et al. (2018) model. characteristics, factors like personality and ability, learning states, motivation and efficacy are included. Furthermore, in the factor personality and ability, it is written about the implication cognitive ability and conscientiousness that have the greatest relationships to transfer. The summary from this stage is that individual differences should be expected and design the learning based on that fact. The next stage is training design and implementation, which includes, among other things, learning strategies and demonstration. The overall outcome for this stage is to incorporate and use well-known learning principles for the training design, for example multiple learning strategies. In addition there are two more stages; work environment and transfer measurement. Moreover, it is important to provide opportunities to apply trained skills immediately on the job. In other words, the preferred way to achieve transferred knowledge is to invest a lot in facilitating transfers and continuously work with follow-ups and improvement work within the organization. Figure 2.2: Summary of how to transfer knowledge based on Ford et al. (2018). 10 2. Learning theories Sometimes it can be difficult to use the theoretical inventions in practice. Several models have been developed about how the interface between transfer research and transfer practice should be linked. Weinbauer-Heidel (2018) has developed twelve practical tips for transferring the learned knowledge to creating behavioral changes, as shown below: Enabling transfer for the Participants 1. Transfer mission: the extent to which the participants have the desire to im- plement what they have learned. 2. Self-efficacy: to which extent the course participants are convinced that one can master acquired skills in practice. 3. Transfer volition: the extent to which the participants show their ability and willingness to devote attention and energy to the implementation of the be- havioral change, even when there are obstacles and difficulties. Enabling transfer due to the Training design 4. Clarity of expectations: how well course participants already before the ed- ucation know what to expect from the training, before, during and after the education. 5. Content relevance: the extent to which course participants experience the educational content as well matched with the tasks and requirements in their work. 6. Active practice: the extent to which the education provides opportunities to experience and practice new behaviors that are desirable in work contexts. 7. Transfer planning: the extent to which the transfer is prepared during the training. Enabling transfer in the Organization 8. Opportunities for application: the extent to which situations are made possible where newly acquired behavioral change can be applied. 9. Personal transfer capacity: the extent to which course participants have the capacity, in terms of time and workload, to successfully apply the newly learned skills. 10. Support from supervisors: the extent to which participants’ managers / su- pervisors actively demand, monitor and support transfer. 11. Support from peers: the extent to which colleagues assist course participants with transfer. 12. Transfer expectations in the organization: the extent to which course partic- ipants expect positive consequences from applying what they have learned or negative consequences as a result of non-application. The levels above have a clear connection to the model from Ford et al. (2018), where similar divisions of categories are made. In addition, theory of the formative assess- ment process has many similarities to these mentioned models. Wiliam’s (2011) describe formative assessment as an iterative learning process that is constantly on- going. The goal is to create a culture where the student takes responsibility for their 11 2. Learning theories own learning, which is in correlation to Ally (2004). Formative assessment can be achieved through five strategies according to Wiliam (2011), see below. 1. Clarify and understand the expected learning objects. 2. Create activities that elicit evidence of learning 3. Provide feedback that benefits learning 4. Let the students become peer teachers for each other 5. Let students become owners of their own learning Strategies 1 and 2 correspond to Enabling transfer due to the Training Design from Weinbauer-Heidel (2018) model. Strategies 3 and 4 correspond to Enabling transfer in the organization and the last one strategy 5 corresponds to Enabling transfer for the Participants. All the mentioned steps are important to achieve an effective and favorable learning. From the social perspective on learning, Wiliam (2011) indicates that strategy 4 - peer teacher, is one of the most effective way to increase learning. This aspect is supported by several researchers who state that learning increases through collaboration (Dewey, 1916; Kyndt et al., 2013; Puzio and Colby, 2013; Capar and Tarım, 2015). 2.2.3 The decision to transfer In the discussion of transfer of training, it is important to keep in mind that to which extent transfer occurs depends on each individual trainee. When transferring training, each individual decides what knowledge to transfer and what they leave behind (Baldwin et al., 2009). In other words, it means that each individual trainee personalizes the training process to fit their own conception of needs in their context. The knowledge must be relevant to each individual for transfer to happen. The decision to transfer is a key factor in the research of transfer. The choices that the learner makes go on throughout the learning process and play a significant role, if not the most significant of them all. Furthermore, research shows that initial attempts to transfer had a decisive impact on the subsequent learning process (Huang et al., 2017). The conclusion is that education needs to be adapted so that each individual chooses to learn exactly the things for which the training is created. 2.3 Evaluating training As mentioned earlier, there are many theories and models for how knowledge transfer should take place. In order to actually know whether a training has been successful or not, some form of evaluation is required. Kirkpatrick and Kirkpatrick (2016) created a four-level training evaluation. This is the most commonly used training evaluation model and includes the following levels: reaction, learning, behavior, and results (Kirkpatrick and Kirkpatrick, 2016). Level one is about the participant’s attitude and reaction to the training, which has become standard in training. Level two is examining whether learning has taken place during the training and if the knowledge is used afterwards. Level three and four are used much less nowadays. Level three is about how new knowledge and behavioral changes are transferred to 12 2. Learning theories the workplace, and level four is about how the training affects the organization’s results (Bates, 2004). The Kirkpatrick model has been used in for-profit companies for over 30 years, although it has been criticized by several researchers in the field (Bates, 2004). According to Bates (2004), there are 3 reasons why Kirkpatrick’s model has received such attention and impact in organizations. First of all, the model created a way to understand training evaluation in a systematic way (Shelton and Alliger, 1993). It created a system for talking about training benefits and outcomes. Furthermore, Kirkpatrick’s fourth level has created an opportunity to discuss training in business terms, which is needed for training to have an impact and be taken seriously in the organization. Finally, the model simplifies a very complex issue into manageable partial solutions. To use the model, no advanced tools are required, a few variables that are collected after the training are enough to evaluate the effectiveness of the training. 13 2. Learning theories 14 3 Methods This chapter contains the strategies and explains the methods used for this master thesis. Initially described is the research approach followed by the data collection and the analysis. 3.1 Research approach The research questions posed for this master’s thesis have been considered against several different methods to be able to achieve the desired result. The thesis will therefore consist of four parts; a literature review, a course evaluation, a survey study and an observation study for collection of both qualitative and quantitative data. When choosing methods, Kirkpatrick’s evaluation model has been taken into account (Kirkpatrick and Kirkpatrick, 2016). Parts of the findings from the survey will be presented statistically as diagrams and graphs of the participants’ perception of the course together with a qualitative mapping of their perception. To further complement the entirety of this study’s objective, direct observations were obtained in the classroom. According to Bryman (2016), direct observations as a research method enables a greater understanding and generation of ideas, theory and conclusions. Moreover Lavrakas (2008) states that a mixed method is suitable when the goal is to obtain both quantitative and qualitative answers for the data analysis. The mixed method survey design also enables the ability to address and uncover the state and possible problems within a key group of interest (Lavrakas, 2008), like the survey examined for this thesis. To ensure that the answers from the survey were reliable, it was important to pro- duce accurate data with caution and carefully selected questions. In order for the questions to have a fair outcome, no interpretative aspects were given, i.e. it should not be possible to misinterpret the questions. The validity measures the relevance of the survey and a high validity corresponds to a high relevance (Esaiasson et al., 2007). To ensure a high validity, the surveys respondents were limited to the em- ployees who worked within the organization. Moreover the credibility was taken into account. It can be explained as the sincerity of the research findings as for the qualitative part of the survey of a study (Bryman, 2016). This refers to open questions in the survey. Bryman (2016) also states that the questions in the survey should be transferable. To be transferable means that if the questions that have been stated in the survey 15 3. Methods were to be used in another context they should still be able to constitute the same objective. The reader should therefore be provided with a broad description to get an overall picture where the background and aim for the survey and study are clearly stated. 3.2 Data Collection Both primary and secondary data collection have been carried out in this study. Primary data refers to the data that has been generated for this research while the secondary data is data that was collected for another purpose earlier (Kempf- Leonard, 2004). As the base, secondary data from a course evaluation from Volvo Cars was used. The evaluation is standardized and distributed to courses held within the organization. The course evaluation is sent to the participants after completing a course with the aim of gaining an idea of the course experience, get feedback and obtain knowledge for all the different courses that is provided within the organization. This includes the environment, the teacher, and the overall impression. The primary data was generated from a new survey that was constructed for this thesis and distributed to the course participants that took the Teamcenter for Catia V5 users course. With the knowledge gained from the course evaluation, additional questions were included in the survey to ensure that the research questions for this study could be answered and discussed. The survey was constructed to investigate the usability of the course content and determine what the course participants had learned. As another base the aspects of transfer of training were considered in order to obtain answers which possibly could correlate with the learning theories presented in Chapter 2. The aim and questions are further explained in the upcoming sections. 3.2.1 Course evaluation from Volvo Cars The course evaluation from Volvo Cars was created to ensure a good standard for the courses provided within the organization. This course evaluation is shown in Appendix A. After every completed course, the evaluation was sent out to the par- ticipants. For this thesis the course evaluations between year 2013 - 2021 were used, with 358 answers. The frequency of respondents was not available. The evaluation included six closed questions with a ranking scale from 1-6 about the teachers per- formance, how well the training met the expectations and how useful the activities included in the training would be in the participants work. Included in the course evaluation there was also one open question where the participants were supposed to explain what part of the course they thought would be the most useful in their work. 3.2.2 Survey study methodology The upcoming section will present the questions from the survey that were con- structed for this thesis and motivate why they were included. 16 3. Methods During the process of creating the survey several factors were taken into account to improve the chances for a higher response rate. According to Esaiasson et al. (2007), it is of importance to have a clear distinction between the different questions. More- over, the structure of the survey and the order of the questions should have a natural flow, to ensure that the survey is easy to understand and decrease the threshold for the respondents to complete the survey. To further motivate the respondents to take the survey, the design of the questions was optimized to shorten the time estimated for the respondents to complete the survey. Lastly, it was important to ensure that the survey looked aesthetically pleasing, which in this case meant that the survey gave a professional and simple impression. Esaiasson et al. (2007) advocates that this is a vital part and required when setting up the best conditions for increasing the number of respondents. The survey consisted of nine questions where the respondents answered the questions with either an ordinal scale or with a short self-written text. The survey was sent out to individuals who had participated in the course in the last two years, 2020- 2022. Since some of the participants took the course a long time ago, the survey focused on the participants’ current use of the applications included in the course. To minimize the risk of statements of the course experience itself which could be modified by the participants over time (Persson, 2016). In a survey like the one designed for this thesis it is important that the results from the questions represent and describe how common different answers are in a certain population of people, for example different employees at Volvo Cars with different assignments and work titles. Therefore, the questions must be unambiguous and contain relevant answer options (Persson, 2016). The survey was an online survey on the platform Microsoft forms. The participants filled in the survey on their own and submitted it themselves. The survey was distributed via e-mail at one specific time with one reminder. This type of survey was thus cost-effective, especially concerning the time commitment for the participants (Lavrakas, 2008). Furthermore this method gave the opportunity to reach a large number of employees who had the possibility to voluntarily contribute with their opinions, compared to sending an interview request to a few course participants. 3.2.2.1 Survey questions The survey that was established contained nine questions of different types to gain both quantitative and qualitative answers, shown in Appendix B. For the ranking questions, a scale from 1-4 was given to force the participants to fold in one direction or another i.e. not give the option to respond with a neutral statement. This differs from the course evaluation which has a scale between 1-6. The answer alternatives must be in a logical order according to Persson (2016). The ranking question also required the respondent to focus on the question itself and it is therefore of importance that the intention of what is ranked is clear. Furthermore, the answer options had to be comprehensive and mutually exclusive. This was taken into account and thus, four response options were considered appropriate. 17 3. Methods The first question in the survey asked about the respondents work title, and aimed to distinguish if it was possible to see a connection between the participants’ expe- rience of the course and its degree of difficulty, compared to what the individual is working with. The second question asks the participants when they took the course. Esaiasson et al. (2007) means that it is relevant to determine how the perception of the course might change over time and to take time span in to consideration due to outside factors. Question three and four gave the respondent the chance to answer how much prior knowledge they had in the applications CATIA and Teamcenter. This to determine what information is needed to be presented, especially in the beginning of the course, and to be able to shape the course according to the participants’ starting point. The next two questions, question five and six, are ranking questions and the respon- dent answered how often they use the different applications that is introduced in the Teamcenter for CATIA V5 users course. Question five had four different choices Never, Monthly, Weekly and Every day, which corresponded to how often the appli- cations took part their daily work. The next question asked about the participants perceived needs regarding the different applications introduced during the course. Question five together with question six aimed to determine if the participants found the various parts of the course helpful and usable. For the questions that were either multiple choice or rank, the participants would show their thinking and opinion by ticking one (or more than one) box (Peterson, 2000). This allowed the respondents to choose from a set of prepared answers, combined with a last alternative "Other" for further self-established options. In order to narrow down and give the respondents the opportunity to answer in their own words, question seven was given as an opened ended question. In the survey, including an additional question, with a qualitative answer is favored, where the respondent is given the opportunity to think for themselves in addition to given alternatives (Creswell, 2017). Moreover, the answers were not limited to just appli- cations, and the goal for the questions was to see if the course contributed in other means than the use of applications, such as methodologies, strategies and other ways of thinking to be able to transact the different task assignments. As is stated in a report by Statistics Sweden (Persson, 2016) a survey benefits from a structure that has broader and easier questions in the beginning and narrowing it down as the survey is progressing. Question eight asks the respondents if they feel that the course is needed, has contributed to their work and if it gave the right tools. It can be perceived as a more challenging question. To place this question near to the end of the survey also assumes that the respondents have thought through the course during previous questions and can thus give a more qualified answer than if it had been one of the first questions. The last question for the survey is also an opened ended question and gave the respondents the chance to comment what they would like to learn if they were to take the course today, for example something that already is included but maybe could be enhanced or modified, something to add that is relevant for their work 18 3. Methods tasks, or something else. 3.2.2.2 Survey respondents The survey was sent to 20 participants from year 2022, 80 participants from year 2021 and 64 participants from year 2020. See figure 3.1 for the distribution over years. Figure 3.1: The distribution over years of people who received the survey. In total the survey was sent to 192 recipients, but certain factors had to be taken into account regarding the response rate. Among the people that the survey was sent to some might have changed jobs and thus their answers were not as relevant. Furthermore, this is mainly a course for new employees and in some cases the par- ticipants have not had the time to get the right access before the course starts. When this occurs the participant’s manager might be the one on the sign-up list and therefore the one who got the survey and not the intended participant who took the course. As a result, there might be incorrect information among the employees in question. With this in mind, the list of participants was reviewed and revised as closely as possible to minimize possible errors and the final number was thus 164 individuals. The survey was e-mailed to the participants’ work e-mail, Volvo mail, to ensure the anonymity of the data collection. All data is only stored on Volvo’s servers. The participants received a reminder after about one week and after another week the survey was closed. To get the participants motivated to contribute with their knowledge and experiences, an introductory text was written in the email and for the survey. The text explained the purpose of the survey and the value of the participants’ responses. 3.2.3 Literature study methodology The literature study was a qualitative text analysis. The search for relevant lit- erature started off broad to get an overview and was then limited with relevant 19 3. Methods keywords to direct the subsequent research (D O’Gorman and MacIntosh, 2014). To obtain relevant literature and ensure answers to the research questions, different databases such as Google Scholar, Scopus, Sage Research Methods and Business Source ultimate were used. When searching for literature, it is of the utmost importance to be source-critical and consider aspects like authenticity, time, dependency and tendency (Leth and Thurén, 2000). This was done throughout this study and the literature used has been reviewed carefully. Key words that were used includes but are not limited to: knowledge management, transfer of training, organizational learning and organiza- tional development. 3.2.4 Observation study methodology Due to the possibility to participate in the four-day course, direct observations was also a method for this master’s thesis. Direct observation as a method aims to examine the processes and structures that in a set context may be difficult to explain (Esaiasson et al., 2007). The method aimed to seek knowledge of how people acted within the specific environment and social setting, and to observe levers of transfer. The direct observations were a form of action research, which is a comprehensive research where the effects of practical and theoretical strategies in learning sessions are observed (Clark et al., 2020). This type of methodology is suitable when the researcher has a strong corporation with the social setting and learning experience that is observed. To make the observations, assorted notes were taken during the course. The ob- servation took place under approximately ten course occasions, where observations were made both as attendees and trainers. This resulted in an understanding of both perspectives and was used as information regarding specific situations but also to identify tendencies. During the time that the observations were done, the authors for this thesis were both present. This enabled an assurance in the interpretation of the observation as it could be discussed at once. The participants’ engagement and comprehension could then be compared with the different assignments in the course, which became the main focus for the observation methodology. In order to ensure relevant information from the observation, Kirkpatricks’ evalu- ation model, presented in 2.3, was used as a inspiration. What was looked at was the participants’ attitude to the course in its entirety, what questions were asked and how the participants commented on what was presented. Furthermore, whether they were satisfied or not at the end of the course. If there was a different experience for those who attended on site or remotely and what factors could contribute their experience. 3.3 Analysis After completing the data collection, an analysis was started to find explanations and conclusions related to the research questions. The analysis started with organizing 20 3. Methods and categorizing the raw data from the survey to make it easier to perform tests. The tests were chosen based on the type of data that was collected and based on this thesis research questions. A categorization was done on one of the survey questions, professional role, where the division was made in consultation with a Volvo Cars employee, who had a lot experience from the corporation and is a PLM-trainer, as a guide. The participants who had answered the survey were divided into five occupational groups: engineer/developer, product owner, analyst, consultant and other. The descriptive statistics were performed mainly in MS Excel. Here, both the primary and secondary data were used as a basis. The primary data was used mainly to see the usefulness of the course’s different applications and how the perception was from different work roles together with what level of transfer the participants experienced. The secondary data was used to get an overall impression of the course over the years. To investigate whether different groups responded differently, Pearson’s χ2-tests were used with a confidence interval ≤ 0,05. Pearson’s χ2-test is a statistical hypothesis test (Devore, 2012). The choice to use Pearson’s χ2-tests was based on what was sought, a comparison between different groups instead of producing statistics based on normal distribution and mean value. Pearson’s chi-square tests are used to assess goodness of fit, homogeneity and inde- pendence (Wikipedia contributors, 2022). The first establishes whether an observed distribution differs from a theoretical distribution. Homogeneity compares whether the different groups using the same categorical variable, and the independence check whether the variables are independent of each other. From these three assessments, a hypothesis is created about how the data should be distributed based on the theoretical distribution. The tests gathered for this thesis showed if there were any statistical bias between a certain group and a certain answer. χ2-tests are based on the comparison between observed frequencies and expected frequencies. If the hypothesis is true, the observed and expected frequency should be very similar. This in turn means that if we square the difference of observed and expected frequency, we get a small number (p ≤ 0,05). χ2-test is created for data with ordinal scale or nominal scale. The purpose of the tests were not to show exactly where the differences are, but rather to show that correlations exist between different group constellations and how it interacts. There were two open questions in the primary survey: • Question 7: In addition to the mentioned applications in question 6 did you learn something else that was valuable to your work? • Question 9: If you were to take part in a Teamcenter course today, what would you like to learn? The answers to these questions were examined and divided into different clusters based on relevance to the thesis questions and the literature study. Recurring an- swers and answers strongly linked to the literature were given the highest priority. Thereafter the answers were rewritten to useful points that could easily be presented as a result. 21 3. Methods Regarding the direct observations, they did not follow any specific analysis method but were rather used as a supplement to the survey, this to observe and evaluate difficulties in the course. Notes were taken during the course sessions and a summary was made after each occasion of the course, approximately ten occasions. This summary was then compared with the literature and the data collected for the survey and the course evaluation. 22 4 Results This chapter presents the results gathered during this master’s thesis. Data from the course evaluation and the survey will be presented and interpreted together with the observations from the classrom. 4.1 The course evaluation The course evaluation was sent out by Volvo Cars included three relevant questions. The questions have a six-point scale, where the scale starts at one which corresponds to "Not good", and ends with number six which corresponds to "Very good". In other words; the higher the number, the more satisfied the participant was with the course. In the course evaluation there was a question about the overall impression of the course, see figure 4.1. It is a good overall impression and hardly anyone answered a one or a two for this question. 7% are a little dissatisfied and answered a three. 25%, 42% and 25% correspond to the answers 4, 5 and 6. This means that 92% rated the course above the average grade. Overall, the participants are satisfied with the course, where the majority answered the second best grade, number 5. This indicates that the course is appreciated and needed within the organization. However, an result like this one could possibly give an indication that some might have given a rating based the course itself rather than the information aimed to help them perform their work. If the participants do not know the intentions and main goal of the course, they can not rate the course based on the right parameters. There is a clear pattern that participants over the years have responded in a similar way, see figure 4.2. There is a lot of data from the year 2014, hence it has a higher peak than the others. The evaluation has been sent out sporadically to the participants over the years. For this reason, there is a significant difference in the numbers of answers from the different years, even though the course has been run in an equivalent way. However, the positive response pattern is similar. Most answers are found on the right-hand side of the scale, with a majority on the answer option 5. Since the answers are similar over the years, the following results will contain data from all the years between 2013-2021. 23 4. Results Figure 4.1: Overall assessment of the course. Figure 4.2: Overall assessment of the course between year 2013 and 2021. The next question is about how well the activity met the participants’ expectations. The result is very similar to the question above. Around 10% answered a three or lower. This in turn means that 90% chose alternatives 4,5 or 6, where the division is approximately equal. This is also a good result for the course with satisfied participants. However, this is a question that can be a bit difficult to interpret as the expectations that the participants have, coming in to the course, can vary. Moreover, how prepared and motivated the participants are for the course also has an impact. If they have read the purpose and content of the course, they probably have completely different expectations compared to someone who is not prepared. The same logic can be used when looking at the time frame, e.g. how long the 24 4. Results participants have been working within the organization and how familiarized they are with the standards and procedures. Figure 4.3: How well the course meet the participant’s expectations. The last question of interest from the course evaluation, is about how useful the outcome of the activity will be for the daily work, see figure 4.4. Here, 44% answered a six, which corresponds to "Very useful". The answer options 5 and 4 received 48% of the answers and option 1, 2 and 3 have together received 9 % of the answers. This question is very important for Volvo Cars’ business as it is costly to have its staff away on training, especially a four-day course. In summary, the result is good with 91% of the answers on the upper grades. However, a discussion can be held about why not all answered the highest rating, which should be the end goal in the design process of a course. Moreover, it can be difficult to know what is useful when the participants are new to their professional role. Figure 4.4: How useful the outcome of the activity was in the participants work. 25 4. Results 4.2 The survey The following results are based on the survey that was created and sent out to col- lect data for this thesis. Figure 4.5 refers to the first question in the survey and explains what the participants were working with, their professional title. Most of the respondents are developers, more exactly 29 respondents, which correspond to 56 % of the answers. Developers commonly have an engineering background and could work in all the stages of a car project. 6 people are managers, for example Lead Engineers, who is responsible for a team. Three employees are working with calculation and analysis and four respondents are consultants from other organiza- tions. Lastly is a group of ten respondents, other. Within this group are professional titles such as thesis worker, technician and supplier. The result regarding the professional title is not very surprising. This question was asked as an open question where the division has been done manually afterwards. On the other hand, it would probably have been an advantage to have set alternatives that were more specified, in order to be able to use these answers for further analysis with the questions about applications. From the results that followed, it is difficult to draw conclusions based on work title as the majority of respondents ended up in the same group. Figure 4.5: Professional title of the participants The results from figure 4.6 show that the vast majority of the participants have previous experience of Catia. The answers for this question show that 15,4% of the participants had no earlier experience of Catia before taking this Teamcenter course. The course does not focus on Catia and does not require advanced Catia knowledge. However, Catia is included and used to a relatively large extent throughout the course in order for the course content to have meaning and relevance. This means that the participants’ prior knowledge still influences the outcome of the course. 26 4. Results Furthermore, the result based on prior experience of Teamcenter as an application shows that the participants have a mixed level of knowledge, see figure 4.7. Some of the participants, 17%, have a lot of experience and 37% have none. The rest of the respondents have either a little or some experience. These results are important in deciding what level the course should be at. Figure 4.6: Participants’ previous experience of a Catia course. Figure 4.7: Participants’ prior knowledge of Teamcenter For question number five the participants had to rank how often they are using each application in their daily work, figure 4.8. The result shows that Teamcenter, Catia and TCVis are the most common ones among the respondents. About 80% use all of these weekly. The DMU Garage, Document portal and KDP are used to some extent, around 50% are using them weekly. The least used applications are EXTER, BOM/CAD alignment, MIP and NGD Editor, where the latter is highly specialized and is never used more often than monthly. 27 4. Results Figure 4.9 shows the result of the question about relevance of each application included in the course. The result agrees quite well with the previous figure, how often the application is used in daily work. The applications that deviate slightly are EXTER, MIP and NGD, which many consider unnecessary to include in the course (more than 80 %). As for KDP and BOM / CAD alignment, the participants are divided. There are about as many who think that the applications are relevant as those who do not. The other applications have a majority of respondents who believe that they are important to include in the course. Figure 4.8: Description of how often the participants use each application. Figure 4.9: Participants’ attitude to the relevance of different applications. 28 4. Results Figure 4.10 shows the result from the last question in the survey. The question concerning if the participants would be able to perform their daily work without an introduction course in Teamcenter. The largest proportion, 44% answered Maybe, 34% answered that they would not be able to do their job without the course and 22% had not needed the course. From the results it can be interpreted that many do not need an introductory course to perform their daily work. This gives an even greater reason to update and adapt the course content to the participants needs. Figure 4.10: The participants’ answers to whether they could perform the daily work without an introductory course in Teamcenter. In table 4.1 the results from the χ2-tests are stated. There is a bias between pro- fessional title and how often the participants use Teamcenter (p = 0.004). Further- more, the professional title also has a decisive factor in terms of how relevant the participants think it is to introduce Document Portal (p = 0.032) and BOM/CAD alignment (p = 0.043) during the training. The result shows that calculation and analysis engineers almost never use Teamcenter and they do not think BOM/CAD alignment or Document portal are relevant to include in the course. In addition, a bias were discovered between how often Teamcenter is used and how often Catia, TCVis and DMU Garage is used. For the first two, Teamcenter and Catia, the χ2-tests show a p <0.001 and for for DMU Garage it is p=0.007. Figure 4.11 shows a bias between answers of how often the participants use Teamcenter and how often they use Catia. From the figure it is clear that those who use Teamcenter daily also use Catia daily. Similarly, it is the people who rarely use Teamcenter who also rarely use Catia. A similar outcome is shown in Appendix C, where the participants that use Teamcenter a lot, also use TCVis and DMU Garage more frequently. The tests show that there exist a bias in how often the participants use DMU Garage and how often they use TCVis (p = 0.047) and BOM/CAD alignment (p = 0.008). The result turns out to be that the more one uses DMU Garage, the more TCVis and BOM/CAD alignment are also used. Besides that, the use of the DMU Garage 29 4. Results is also linked to the outcome of the relevance of Document Portal (p = 0.036) and BOM/CAD alignment (p = 0.027). In Appendix C this is presented to its full extent with figures. To conclude, these results should be used in the creation of the course content. The results state that there exists a bias between what applications that are used and the professional title. If the participants do not use the applications, it is not essential to spend time on them. In the same way, if the applications are used a lot together, for example applications such as Teamcenter and TCVis, these should have a greater focus during the course. Table 4.1: Test for significance between professional title and different application. Test for significance between Chi-square test forsignificance professional title and how often one use Teamcenter p=0.004 professional title and relevance of Document Portal in the course p=0.032 professional title and relevance of BOM/CAD alignment in the course p=0.043 Figure 4.11: How often Teamcenter is used in bias with how often Catia is used. (p <0.001) 4.2.1 Open questions from the survey The results of the two open questions are presented below. The answers have been analyzed and categorized, and it is these categories that constitute the results. The first question with an open answer was question seven and asked if the participants learned something else that was valuable to their work. After compilation, two categories emerged; Answers about deeper understanding of everything around and answers about tips and tricks in different applications, as shown in table 4.2. 30 4. Results The next question, question nine, that required an open answer was about what the participant would want a Teamcenter course to include, if the participants were to take the course today. After compiling the answers, it turned out that the following categories are most sought after; More about TCVis, advanced functions in TC, tips and tricks and a recap opportunity. Examples are shown in table 4.3. Table 4.2: Categorization of survey response from the question; In addition to the mentioned applications in question 6 did you learn something else that was valuable to your work?. Grouping Survey answer Deeper understanding of everything around App Laucher CADPDM Tools, like Rescue Save and how the Options under CATIA V5/Product/Electrical affects CATIA environment. I got a deeper understanding of HOW teamcenter is working and why we do some things. Agile WOW/VIRA, Pecca Procost, Lotus Notes, etc. I have learned lots of different things that I will use in the future. Tips and tricks in different applications Tips and tricks in TcVis, very useful. How to handle mirrored parts (Left hand and Right hand) in Teamcenter and CATIA. 31 4. Results Table 4.3: Categorization of survey response from the question; If you were to take part in a Teamcenter course today, what would you like to learn?. Grouping Survey answer More about TCVis. More capability on TCVis Would like to learn more about TCVis and how to manipulate 3D-models in that software Advanced functions in TC Structure of the CAD data for all components. I think what I would have dive deeper into is MCI, apply material and etc. and especially for me that are supplier Regarding flexible relatives how connect more than one file... Tips and tricks Tricks to create stuff faster. Optimizing layouts with shortcuts. Functions that you do not know of but is really good for work. How to find stuff easy. More detailed detail level and fix and tricks. A recap opportunity A quick recap for a couple of hours about everything thought in the training will be useful. From my experience was this very good to repeat how Volvo Cars works that is different from CEVT, and repeating is a way to learn. 4.3 Direct observations What became clear during the time that the course was observed was that the participants have an overall good experience. Examples of what those observations were included, but was not limited to, the participants reactions when finishing an assignment, the questions asked during the training and the direct feedback at the end of each course day. Another clear factor that was observed was that the participants professional title had impact on their perceived workload in the course. Moreover, the reactions when the trainers presented a new section of information from the course content and the participants acknowledged parts that was related to their work, a greater motivation and inspiration immediately appeared. The course design is customized to fit individuals with some to no experience of the applications that are included during the four days of training. The course starts on a basic level with exercises together with the trainer to then progress and have the participants working more on their own closer to the end of the course. The exercise book works as the foundation and provides an explanation of what needs to be included, both by being presented by the trainer, talked about in the classroom and used in practical exercises in the training environment. This aims to create a situation similar to the participants’ upcoming day-to-day work. There are specific characteristics of the exercises that works well for the participants and the same occurs for those tasks that are more challenging to execute. The design off the exercise book starts off with clear step by step actions and by the last exercises 32 4. Results just gives clues on all the steps in the procedure. What has been observed is that the participants rely on the book to different degrees. There are some that clearly rather ask the trainer and some who prefer to go back and forth in the book to find an explanation on what to do next. The distinction was mostly defined during the last two exercises called The scenario (and is accounted for two paragraphs further down). Due to the fact that there are different trainers responsible, there is a natural ex- planation to a slight variation in the outcome and what the participants will com- prehend after the training. This may possibly be related to how long the trainers have been involved with the course, but also who have taught the trainer during the on-boarding process and lastly, the trainers’ interests and prior knowledge. 4.3.1 Specific course situations The Scenario exercises are the participants’ chance to test their newly acquired knowledge. The two exercises include all the crucial steps learned during the course. It is important that the participants are working independently since it works roughly like an exam element for the course even though the trainers are there to help where help is needed. What was observed here is that the participants joining online are more hesitant to ask questions when they encounter problems. There is one part of the training where the participants are supposed to Change the ownership with each other. This exercise has a good intention, but due to the fact that the observed courses have been offered as a hybrid classroom, the option to attend remotely has created some difficulties. When the participants are supposed to talk with a peer, the observation was that it became difficult to distinguish who was saying what with participants who were attending online. Another factor here is that the participants rarely know each other and with some people attending the course remotely with only a short presentation on day one, they can be reluctant to take the initiative to start a conversation. With all participants in the classroom, and a better introduction of each other, the collaboration between the participants increases. Another situation that was observed in the classroom was that the participants might not have read the whole email sent out before the course. As mentioned before, the participants need to apply for access and if they have not done this before the start of the course, other logins need to be used. If the course participants are not prepared for the course, it takes away unnecessary time for all participants and sometimes the content must be adjusted. 33 4. Results 34 5 Discussion The results for this master’s thesis will be discussed in this chapter. The discussion will be divided into two sections, the first discussing the optimization opportunities and the second comparing how well what has been stated in the results corresponding to the learning theories discussed above. Conclusions and possible further measures will be presented. 5.1 Optimizing the course The first research question, ’What factors could optimize Volvo Cars PLM course so that the participants gain the right tools to perform their day-to-day work?’ could incorporate many different aspects, such as personal prioritization of what generates most profit, what is considered the most important and the meaning of optimization. Therefore two sub-questions were posed; the first asking which parts of the course were vital and what could be revised and the second asked to what extent the participants experience knowledge transfer after completing a course. The results from course evaluation show that the overall impression of the course is good and that it meets the participants’ expectations. Due to the fact that a majority of the people taking the course are new to the organization, a good aspect to keep in mind is that some might not know what they will find useful in their work. When the survey for this thesis was constructed, questions were not asking for the participants experience of the course but rather focusing on what applications are used in their line of work i.e. the relevance and applicability of the course content. From the survey it can be deduced that parts of the course on which a lot of time is currently spent are not used in the capacity expected outside the course, see figure 4.8. One application that this is true for is Document portal. The result showed that only a little over 10% use it every day and 25% have never used the application. This while the course talks about, and has exercises, for this application as a main focus, both during the second and third day of the training. It is also the other way around where the participants have answered that some applications that are briefly mentioned in the course are used more often than expected. Here the DMU Garage is one example. A lot of the participants are using this application but in the content of the course it plays a small part. It can be deducted from the observations and the course evaluation that the participants are satisfied with that approach, however, there is no answer in this thesis to what the participants think when they have evolved in their professional role. 35 5. Discussion Important to keep in mind when looking at the results, chapter 4 figure 4.8, is that the actions that the application provides are not taken into account. A good expla- nation to this phenomenon is again the DMU Garage. The application provides a virtual garage, with an overview and timeline, and enables the virtual car projects to be opened in other applications (i.e. Teamcenter and TCVis) where work and progress can be made in the project. The correlation of the usability in the appli- cations is shown in Appendix C. To optimize the course after what applications are used the most is therefore a good way to structure the content creation of the course. However, applications are intended for different purposes and attributes, and can therefore be fully explained and elaborated with more or less time. The result stated that the participants wanted an overview to be able to see con- nection between the applications, even though some applications are rarely used according to the survey, see table 4.2. If the applications that rarely are used would be eliminated from the course content the overview would be compromised and the objective would change. As a result from the observations it was confirmed that participants came in to the course with different backgrounds and knowledge, which resulted in different starting points. The conclusion can thus be that for some everything will be new information and for others the same information will be repetition. With the overall objective for the course, that the participants should learn as much as possible, it is arguable that the participants prior knowledge should be taken into account. A possible solution could therefore be that participants can enter a course on a fair level to them. This is possible if the trainers provided the course on, for example, two levels, one where there is no prior knowledge needed and one where the participants have worked in these applications before. This would provide a more effective training where the trainers could prepare relevant information and at the same time keep the group of participants aligned with each other during the exercises. Moreover, a division can be made with the argument that different professional roles have different usage of the applications. This also points to an easier transfer with knowledge that will be used and incorporated in their daily work. One enabling factor that Weinbauer- Heidel (2018) presents is to ensure opportunities for application, which means that there should occur situations where the participant can apply their newly gained knowledge to make it stick. If the course would have a varying content depending on the participants professional title the threshold for transfer would be lowered. Moreover, the hybrid classroom can be an aggravating factor. Having some partic- ipants in the classroom and some attending online creates a lot of extra work for the trainer, which was observed in the classroom. First of all, both the participants and the trainer will get a better introduction in the classroom compared to the ones attending online. Since the group will spend four days together, they start to get to know one another already on the first day. This seems to affect the participants online as they ask fewer questions and will from time to time seem like they are away from their computer, which Poskitt et al. (2021) also states. Secondly, the hardware equipment in the classroom is of great help with both speakers and screen sharing during the predominant part of the course, but during some exercises that 36 5. Discussion the participants take on in the classroom the sound level increases. The participants help each other and ask the trainers for help as well. However, when the trainer is about to help the ones online, it is often pointed out that it is difficult to hear what the trainer is saying due to the noise in the background. Despite the fact that the hybrid classroom has clear disadvantages, the course has evolved and entered another phase where the trainers do not have to travel and present the course in different parts of the world and can now do it from the same classroom regardless of where the participants are located. The suggestion is thus not to eliminate the course online but to provide the choice for the participants to either participate remotely or in the physical classroom and exclude the combination of the two. This would benefit both the trainer and the participants, whether it is a course remotely or on-site. The meaning of optimizing could include several aspects depending on professional title and prior knowledge. The goal should be that the participants feel that they have the right tools to be able to perform their work after attending the course. De- pending on the professional role, the content of the knowledge transfer will vary due to applicability. However, in order for the course to fit the majority of participants from Volvo Cars, the course needs to have an overview and be kept on a fair level to create a knowledge base. This results in some smaller parts of the course being included to provide an overview, even if the parts are used less frequently. 5.2 Agreement between the course and learning theories The upcoming section is about the research question that refers to if the course is relevant and representing what the literature states about transfer of training. It will include a discussion based on theories presented in chapters 2. The the- ory chapter has also inspired the structure of the discussion, which corresponds to transfer of training research with a main focus on Weinbauer-Heidel (2018) and the twelve points that enables transfer. The first section is about enabling transfer for participants, followed by the enabling of transfer due to the training design and lastly enabling transfer for the organization, similar to Weinbauer-Heidel (2018), as described in 2.2.2. Every section is divided into two subsections; determination of the current situation and maintenance. Determination of the current situation gives a general picture of the course based on the results of the course evaluation, survey and direct observations. Furthermore, suggestions for improvement based on theory are discussed under the subsection maintenance. 5.2.1 Enabling transfer for the Participants The following section focuses on the participant and how the individual can affect the transfer of training themselves. Enabling transfer for the participants focuses on the participants motivation, self-efficacy and transfer volition (Weinbauer-Heidel, 2018). 37 5. Discussion 5.2.1.1 Determination of the current situation The answers from the survey can be interpreted as the overall impression of the course being very good, see figure 4.1. The course has looked similar for many years and the impression does not seem to have changed remarkably, as presented in figure 4.2. In terms of the usefulness of the course, the results vary slightly, but with a continued positive attitude. Due to the participants having backgrounds that differ from each other, the learning process also differs. Looking at the course from the perspective of cognitivism, the participants will be able to take on the presented knowledge if the information fits into their cognitive schemas. In other words, the more puzzle pieces they have, the more easily they will take in the course content (Ally, 2004). With that in mind, previous experiences will play a vital part if the information in the course does not have a previous connection in the schema for the participants. The observations generated, among other things, the participants’ individual per- spective, and resulted in that prior knowledge seem to have a key impact in what is perceived as easy and what is considered more difficult. Some of the participants ask a question rather than search for the answer on their own. The survey showed that the professional role is not as important but rather the time that the partic- ipant has been working within the organization and also what characteristics the participants have. Notable is also that the respondents from the survey did not have a remarkable difference in their professional titles, see figure 4.5. There is one group which stands for 56% of the respondents while the rest are divided in four groups. This can result in conclusions that do not correspond to reality. To further strengthen the argument made previously, the direct observations show the same tendencies, i.e. that the participant’s previous experience plays a role the course contents relevance. During the first day of the course, a part of the introduction is that the participants get to answer a question regarding their expectations for the course, as mentioned in the result under section 4.3. This is a great way for the trainer to get a forecast on what the participants know and need to obtain during the training. Furthermore, the trainer will be able to adapt the course to the participants’ needs and motivate them by drawing references to the participants’ own needs, corresponding to Weinbauer- Heidel (2018) aspects of transfer of training. 5.2.1.2 Maintenance to ensure transfer To achieve efficient transfer of training, motivation, attitude and ability for each participant are the key elements to focus on (Weinbauer-Heidel, 2018). After being in a transition period after the pandemic, the direct observations show that the participants who are in the classroom generally have a different attitude than the participants who are attending remotely. The participants in the classroom are not as hesitant to ask questions, neither directed at the trainer nor at the participant sitting at the computer next to him or her. The collaborative exercises work well when all participants attend on-site and are preferred by Wiliam (2011) 38 5. Discussion among others, in order to gain new skills and knowledge. This aspect forms the conclusion that this training should be held on-site as much as possible. Baldwin et al. (2009) states the importance of motivation, and that it is up to the individual in question to make the choice to transfer. In order to increase motivation and thus also the transferability, there must be a clear purpose for the course. An assurance that the content is useful and necessary in order for the participants to be able to carry out the work required for their role. This could be obtained with more in-depth information about the course that the participants can attain before applying for a greater objective. Moreover, acquiring background information about the participants beforehand will enable the trainer to tweak the information to fit the crowd. Adapting the course content according to prior participants’ suggestions enhances the chances of a course that is both transferable and useful. As presented in table 4.3, participants expressed a desire that the course include more repetition and an overview of how all the different actions and applications are connected and interdependent. Wiliam (2011) stated that students should become owners of their own learning, and that is a reason to allow the participants to influence the course and its content, and adapt the course in accordance with these wishes. To conclude, the participants’ attitude can possibly play a vital role for their ability to attain the knowledge and transfer the information to their professional role. A way to accomplish that is to make sure of the objective for each learning session and ensure a relevance to each individual attending the course. 5.2.2 Enabling transfer through the Training design The overall objective of the course is that the participants should be given the right tools to be able to perform their work in a correct and efficient way according to Volvo Cars standards. In order to achieve the enabling of transfer the training design should be considered. To attain transfer for the training design is for example clarity of expectations, content relevance and active practice (Weinbauer-Heidel, 2018). 5.2.2.1 Determination of the current situation As mentioned, the participants have to present their expectations of the course. They also update the trainer on their previous experiences in Teamcenter, which makes it easier for the trainer to set the starting level. In connection to this, a short introduction is held about what the next four days will include. The purpose of this introduction is to clarify what the participants can except of each day and the result afterwards, which is one of the aspects that Weinbauer-Heidel (2018) states. As Baldwin et al. (2009) pointed out, the participants choose to learn themselves, which means that the trainer needs to be aware of what the participants expect and what they want to learn. From the results and figure 4.3, it can be deduced that the expectations are quite clearly in line with the outcome of the course. The goal for the course is that all participants should feel that all content in the 39 5. Discussion course is relevant and useful. In the current situation, 44% answered the highest grade on the question of the usefulness of the course material in their daily work. The majority of the participants gave a good rating on the question. However, the goal have be to make everyone attending think that the usability is maximized as much as possible, even though it is hard for a course with different attendees each week. Shown in the results is that there are some applications that are used more often than other, for example Teamcenter and Catia in the top for most used and MIP and NGD Editor at the bottom, see 4.8. This outcome was along the lines of the hypothesis. The applications that are used the least can be declared as "supportive" applications for the primarily applications. The training design could therefore be refocused with the basic idea of emphasizing the active learning for the applications that are used the most, since active learning increases transferability (Wiliam, 2011). With this in mind it is easier to justify the relevance of the content, which is one of the enabling factors Weinbauer-Heidel (2018) points out. The course already contains relatively many different learning methods, such as presentation, exercises with the trainer and exercises carried through on their own. Wiliam (2011) states that formative assessment can be channeled through activities that elicit evidence of learning, in this case can resemble exercises where the partici- pant test their own knowledge. However, this can be expanded further where active learning should take a greater place. In order to keep the different areas of focus separated during the course it is nec- essary to to ensure a clear objective for each part of the course, in line with Ally (2004). What can be perceived as confusing is that there are similarities between the applications but some functions differs between them. To minimize the confu- sion, the purpose and usefulness must be crystal clear for each application. The trainers try to separate and explain these as much as they can, but with a mixed result shown from the observations in the classroom. The trainers have different ex- periences and backgrounds, which means that some have more knowledge in certain areas. In addition, participants need to understand that the content will become more clear as they precede with the course, and will be able to understand where the differences are for the different applications. 5.2.2.2 Maintenance to ensure transfer To achieve efficient transfer through the training design, the expectations, content relevance, active practice and transfer planning are the key elements to focus on (Weinbauer-Heidel, 2018). To be able to construct the optimal course with content that Volvo Cars’ organi- zation needs, transfer planning has to take place. There must be a high clarity of expectations, both from the participants, the supervisors and the trainers (Wiliam, 2011). With similar expectations from the organizations divisions, a course can be created based on the right starting level and essential content. Another aspect for the training design is to continuously update the material so that 40 5. Discussion exercise examples and slides correspond to reality and therefore ensure relevance. Volvo Cars is constantly working on development, it therefore requires that the training material matches the updates and give the content a correct relevance. There should be an active learning where both the participants and the trainers can be involved and influence the outcome of the training session. As it is a four- day course, there is a great amount of information to be processed, which means that the participants must have time to learn in their own way. This is best done through many repetitions, variation in the working method and getting time to reflect (Hagman and Rose, 1983). Furthermore, overlearning is critcial for transfer, which means that the participants should do the same thing over and over until the knowledge has been transferred (Driskell et al., 1992). Nevertheless, different kinds of activities can be created to practice a certain type of task in order to create strong referent points (Philips D. C., 2014), i.e. repetition but with a slight variation. Both Wienbauer-Heidel’s (2018) and Wiliam’s (2011) models suggest that feedback from both the trainers and other participants improves the learning. For this reason, more collaborative exercises should be added to the course material. It could be even better if pair exercises were preformed at the beginning of the course, were they can get to know each other, with the intention that the participants could help each other throughout the whole course. The training design can therefore have a vital impact on the level of transfer that the participants experience. If the content can be tweaked, without changing the entirety, the outcome could increase motivation for the participants and the relevance of the material. Moreover, one of the stronger benefits of the course is active learning, which will enable the participants to get a good indication of the standard way of working and purpose of the applications. 5.2.3 Enabling transfer in the Organization Enabling transfer in the organization contains, among other things, opportunities for the participants to use the new knowledge, support from their supervisor and transfer expectations from the organization (Weinbauer-Heidel, 2018). 5.2.3.1 Determination of the current situation The time frame given for the course is four full days. Within the time frame, mate- rials have been added and removed to get a good common thread and a favorable result for the participants. The four days are always in a row, which can be seen as both positive and negative. Driskell et al. (1992) state the importance of task recep- tions as the same time as Hagman and Rose (1983) point to the benefits of spaced practice. In a four-day course like this, transfer learning becomes very concentrated, with no opportunity to have space between the learning sessions. The course is mainly developed for design engineers at Volvo Cars, which means that the participants will have many opportunities to apply the new knowledge gained from the course and also ensure the personal transfer capacity. Other professional 41 5. Discussion roles may however not get that same opportunity. For example the minor usage of Teamcenter, TCVis and DMU Garage for calculation and analysis engineers stated in section 4.2. Not having the opportunity to use their new knowledge can make the participants forget the standard way of working within Volvo Cars. This is a common position of many researchers, such as Hagman and Rose (1983) and Ford et al. (2018). Repetition is important to maintain the knowledge gained during the training session. The course is mandatory to gain access to the system, which means that supervisors must ensure that their employees take the course. For this reason, there is an obvious support from the manager for the participant to show up at the course and learn the material, which Weinbauer-Heidel (2018) states as one enabling factor for successful transfer. However, the feedback and support afterwards is more unclear. It probably differs between the departments and who is asked. An argument for continuing the feedback all though the organization is that all actions will be noticed, as the organization is affected and a lot of people work within the same project. 5.2.3.2 Maintenance to ensure transfer To achieve efficient transfer of training in the organization, opportunities for ap- plication, transfer capacity, support from supervisors and peers and lastly transfer expectations in the organization are the key elements to focus on (Weinbauer-Heidel, 2018). According to Weinbauer-Heidel (2018), Ford et al. (2018) and Wiliam (2011), feed- back is a crucial aspect of learning. Of course, the trainer must be responsible for the feedback and give advice for learning the new knowledge during the course, in order to constantly take the participants to the next level of training. However, feedback from colleagues and managers is equally important for retaining knowledge long term. Similarly, more parameters need to be achieved for knowledge to be maintained in the long term. From the results in table 4.3, it could be deduced that several partic- ipants had wanted a rehearsal opportunity later in time, to rehearse the knowledge. For this, the organization must ensure that there are opportunities for rehearsal for both the participants and the trainers, and that there is a functioning administra- tion. This opportunity does not have to be that long, but it must be encouraged and accepted by the managers. After all, it is time that disappears for the partic- ipants’ usual tasks. On the other hand, the long-term result should be favored for the organization and the individual that can execute their work in a effective and correct manner. Fi