Predicting Deviation in Supplier Lead Time and Truck Arrival Time Using Machine Learning A Data Mining Project at Volvo Group Master’s thesis in Supply Chain Management Supply and Operations Management MENG HUANG MASOOD BAGHERI Department of Technology, Management and Economy Division of Supply and Operations Management CHALMERS UNIVERSITY OF TECHNOLOGY Gothenburg, Sweden 2019 Report No.E 2019:033 Master’s thesis 2019:033 Predicting Deviation in Supplier Lead Time and Truck Arrival Time Using Machine Learning A Data Mining Project at Volvo Group MENG HUANG MASOOD BAGHERI Supervisor, Chalmers: Joakim Andersson Supervisors, Company: Anton Ottosson & Martin Granic Examiner, Chalmers: Patrik Jonsson Department of Technology Management and Economics Division of Supply and Operations Management Chalmers University of Technology Gothenburg, Sweden 2019 Predicting Deviation in Supplier Lead Time and Truck Arrival Time Using Machine Learning A Data Mining Project at Volvo Group MENG HUANG MASOOD BAGHERI © MENG HUANG, MASOOD BAGHERI, 2019. Master’s Thesis 2019:033 Department of Technology Management and Economics Division of Supply and Operations Management Chalmers University of Technology SE-412 96 Gothenburg Telephone +46 31 772 1000 Printed by Chalmers Reproservice Gothenburg, Sweden 2019 iv Predicting Deviation in Supplier Lead Time and Truck Arrival Time Using Machine Learning A Data Mining Project at Volvo Group MENG HUANG MASOOD BAGHERI Department of Technology Management and Economics Division of Supply and Operations Managemen Chalmers University of Technology Abstract The deviation in delivery performance from a company’s suppliers directly affects the company’s performance, causing availability loss for the customer orders and large costs for the rush transportation. If the deviation can be predicted in advance and used as deviation alerts, actions can be taken in advance either to prevent the deviation or decrease the impact of the deviation. To predict the deviation in the supplier delivery performance from a buying com- pany’s point of view, this thesis work specifically focuses on the first two phases of a supply chain, namely supplier lead time from material suppliers and truck arrival time from logistics service providers (LSP). In order to examine the possible imple- mentation of machine learning, a data mining project has been conducted at Volvo Group Service Market Logistics. The factors associated with deviation of supplier lead time and truck arrival time are identified, while the corresponding features are prepared under the constraint of the case company’s data availability. For pre- dicting deviation in the two phases, two machine learning models are constructed accordingly based on the characteristics of output and input features. The opportu- nities and obstacles along the data mining process in the case company are identified. The results show currently in the case company, both generated machine learning models do not have enough predictive power in lead time deviation. This could be caused by the absence of some key features that have strong associations with deviation. However, the performance of the prediction model for truck arrival time is regarded to be improved to a deployable level when the desired features are con- structed into the model by the case company. Future recommendations regarding constructing the desired features and improving the model performance are pro- posed. In comparison, predicting deviation in material suppliers’ lead time could be practical when the buying company get more information sharing from material suppliers. Keywords: Lead time deviation, Estimated time of arrival (ETA), Prediction, De- livery precision, Machine learning, Supplier evaluation, Spare parts, Automotive. v Acknowledgements This master thesis was conducted during the spring of 2019 as part of the Supply Chain Management program at Chalmers University of Technology. This thesis collaborates with Volvo Group Service Market Logistics, where this data mining project was conducted. First, we would like to thank our supervisors at Volvo, Anton Ottosson and Martin Granic. Their great support and experience guided us through the entire project. Their generous encouragement and compliment also made us confident towards the completion of this project. We would also like to thank all the involved people at Volvo for their time and insights contributed to this project. Then, we would like to express great gratitude to Joakim Andersson. As our su- pervisor at Chalmers, his profound interests in our finding motivated our work and his feedback inspired us at every key point in this project. Further, we want to appreciate the insightful inputs from our examiner, Patrik Jonsson at Chalmers. Finally, we thank our friends and family for their immense support. Meng would like to give a special thank to her girlfriend, Yannan Shen, who supported her ded- icatedly from home. Her knowledge of statistics and academic writing also helped trigger a lot of thinking during this project. Meng Huang and Masood Bagheri Gothenburg, June 2019 vii Abbreviations CART Classification and Regression Trees Catboost Categorical Boosting CDC Central Distribution Center CMP Continental Material Planners CRISP-DM Cross Industry Standard Process for Data Mining DDT Door to Door DIP Demand Inventory Planners EMEA Europe, Middle East and Asia ETA Estimated Time of Arrival FN False Negative FP False Positive FTL Full Truck Load KPI Key Performance Indicator LSP Logistics Service Providers LTL Limited Truck Load MMOG/LE Materials Management Operational Guidelines / Logistics Evaluation QPM Quality Performance Measurement RDC Regional Distribution Center SDC Support Distribution Center SEM Supplier Evaluation Measurement SLT Supplier Lead Time SM Supplier Managers SML Service Market Logistic SRM Supplier Relationship Managers TLT Transportation Lead Time TMC Transport Material Coordinators TN True Negatives TP True Positives PPM Parts Per Million x Contents List of Figures xiii List of Tables xv 1 Introduction 1 1.1 Theoretical Background . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Company Background . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.4 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2 Literature Review 7 2.1 Frame of Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.1.1 Spare Part Logistics Context . . . . . . . . . . . . . . . . . . . 7 2.1.2 Supplier Evaluation . . . . . . . . . . . . . . . . . . . . . . . . 8 2.1.3 ETA/Lead Time Prediction . . . . . . . . . . . . . . . . . . . 8 2.1.4 Conclusion from Frame of Reference . . . . . . . . . . . . . . 10 2.2 Machine Learning Tool and Terminology . . . . . . . . . . . . . . . . 10 2.2.1 Fundamental Machine Learning Definition . . . . . . . . . . . 10 2.2.2 Algorithms and Feature Selection . . . . . . . . . . . . . . . . 11 2.2.3 Classification and Regression models . . . . . . . . . . . . . . 12 2.2.4 Random Forest . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.2.5 Gradient Boosting and Categorical Boosting . . . . . . . . . . 13 2.2.6 Handling Class Imbalance . . . . . . . . . . . . . . . . . . . . 14 2.3 Evaluation Metrics for the Prediction Models . . . . . . . . . . . . . . 15 2.3.1 Confusion Matrix . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.3.2 Feature Importance . . . . . . . . . . . . . . . . . . . . . . . . 16 3 Methods 19 3.1 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.2 General Strategy and Process . . . . . . . . . . . . . . . . . . . . . . 19 3.3 Business Understanding . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.3.1 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.3.2 Internal Documents . . . . . . . . . . . . . . . . . . . . . . . . 22 3.4 Data Collection and Understanding . . . . . . . . . . . . . . . . . . . 22 3.4.1 Delimitation in the Data Collection . . . . . . . . . . . . . . . 23 3.5 Data Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.5.1 Transferring Categorical Variable . . . . . . . . . . . . . . . . 24 xi Contents 3.5.2 Integrate and Link Data . . . . . . . . . . . . . . . . . . . . . 25 3.5.3 Delimitation in the Data Preparation . . . . . . . . . . . . . . 25 3.5.4 Feature Selection . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.5.5 Handling Missing Data . . . . . . . . . . . . . . . . . . . . . . 28 3.6 Machine Learning Modelling and Evaluation . . . . . . . . . . . . . . 29 3.7 Validity and Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4 Results: Business Understanding 33 4.1 The Set-up of Lead Time in Volvo . . . . . . . . . . . . . . . . . . . . 33 4.2 The Process and Roles Involved in Dealing with Lead Time Deviation 34 4.2.1 Process Overview . . . . . . . . . . . . . . . . . . . . . . . . . 34 4.2.2 Continental Material Planner . . . . . . . . . . . . . . . . . . 36 4.2.3 Supplier Relationship Manager . . . . . . . . . . . . . . . . . 37 4.2.4 Transport Material Coordinator . . . . . . . . . . . . . . . . . 37 4.2.5 Supplier Manager . . . . . . . . . . . . . . . . . . . . . . . . . 38 4.3 Situation of Deviation . . . . . . . . . . . . . . . . . . . . . . . . . . 39 4.4 Impacts of Lead Time Deviation . . . . . . . . . . . . . . . . . . . . . 40 4.5 Business Goal of the Project . . . . . . . . . . . . . . . . . . . . . . . 41 4.6 Factors Related to Lead Time Deviation and the Availability of Data 42 4.6.1 Factors of Material Suppliers’ Lead Time Deviation . . . . . . 42 4.6.2 Factors of Inbound Transportation Lead Time . . . . . . . . . 45 5 Results: Data Understanding and Preparation 49 5.1 Data Mining Goal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 5.2 Features Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 6 Results: Models and Evaluation 55 6.1 Classification Report . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 6.2 Feature Importance . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 7 Discussion 59 7.1 Implication on literature . . . . . . . . . . . . . . . . . . . . . . . . . 59 7.1.1 Predicting Material Supplier Delivery Precision . . . . . . . . 59 7.1.2 Predicting Deviation of Truck Arrival Time . . . . . . . . . . 60 7.2 Implication on the Case Company . . . . . . . . . . . . . . . . . . . . 61 7.2.1 Monitoring on Material Suppliers . . . . . . . . . . . . . . . . 61 7.2.2 Monitoring Process on LSP . . . . . . . . . . . . . . . . . . . 62 7.3 Underlying Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . 64 7.4 Limitation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 8 Conclusion 67 8.1 Research Finding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 8.2 Practical Contribution and Future Work for the Case Company . . . 69 8.3 Theoretical Contribution and Future Research within Academia . . . 70 Bibliography 71 A Appendix: Interview Questions I xii List of Figures 3.1 Illustration of the data mining process based on CRISP-DM (1999) . 20 3.2 Transferring categorical variables into dummy variables . . . . . . . . 24 3.3 Integrate and link data for the phases of material supply and trans- portation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.4 The data linkage in different transportation modes (A) Door to door; (B) Cross dock; (C) Milk run . . . . . . . . . . . . . . . . . . . . . . 27 3.5 The relationship between processes and research questions . . . . . . 30 4.1 The set up of lead time in Volvo SML . . . . . . . . . . . . . . . . . . 34 4.2 The roles involved in dealing with lead time deviation . . . . . . . . . 35 4.3 The working procedure of CMP . . . . . . . . . . . . . . . . . . . . . 36 4.4 The working procedure of TMC . . . . . . . . . . . . . . . . . . . . . 38 4.5 Average SLT deviation deviation for 2017-2018 . . . . . . . . . . . . . 39 4.6 Delivery precision of material suppliers for 2017 and 2018 . . . . . . . 40 4.7 Delivery precision of LSP for past one year from 2019 . . . . . . . . . 40 7.1 Percentage of deviated supplier delivery by price . . . . . . . . . . . . 62 7.2 Percentage of deviated transportation by countries . . . . . . . . . . . 62 7.3 Generating a deviation alert in the process of monitoring LSP . . . . 63 7.4 Recommendation for linkage of prediction model for truck arrival time deviation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 xiii List of Figures xiv List of Tables 2.1 Review of predicting ETA/machine learning with machine learning . 9 2.2 Confusion matrix defines four possible scenarios when classifying class “C” (Chawla et al., 2003) . . . . . . . . . . . . . . . . . . . . . . . . 15 3.1 A table for interviewees list . . . . . . . . . . . . . . . . . . . . . . . 21 3.2 Main data source from the case company’s database . . . . . . . . . 23 3.3 Missing value for supplier lead time phase . . . . . . . . . . . . . . . 28 4.1 The key roles and KPIs for lead time performance . . . . . . . . . . . 35 4.2 Factors related to lead time deviation in Volvo . . . . . . . . . . . . . 45 5.1 Features selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 5.2 Current available features for SLT model . . . . . . . . . . . . . . . . 52 5.3 Current available features of truck arrival time model . . . . . . . . . 53 6.1 Confusion matrix for SLT models (columns being predicted classes and rows being actual classes) . . . . . . . . . . . . . . . . . . . . . . 55 6.2 Confusion matrix for truck arrival time models (columns being pre- dicted classes and rows being actual classes) . . . . . . . . . . . . . . 56 6.3 Classification report for SLT models . . . . . . . . . . . . . . . . . . . 56 6.4 Classification report for truck arrival time models . . . . . . . . . . . 56 xv List of Tables xvi 1 Introduction In this chapter, the theoretical background and company background of this thesis project is introduced, following by the aim of the project. The research questions are thereby formulated and the scope of the project is presented. 1.1 Theoretical Background Spare part supply chain is a high-margin business bringing in high profits for the company. However, delivering spare parts is more complex than manufacturing the products, since a spare part supply chain has to cover the aftermarket service for all the products sold by the company. Customers also expect their things to be fixed quickly when they break down, while their demands are intermittent because the breakdown happens unexpectedly. These difficulties make only companies that pro- vide the spare part efficiently can make revenues from aftermarkets (Cohen, Agrawal and Agrawal, 2006). The supply chain management in a company should match the demand and supply (Jonsson, 2008). Forecasting the demand in order to mitigate the risks of uncertainty and availability loss of spare parts has received lots of research attention (Dekkeret al., 2013). The uncertainties also come from supply sides (Heydari et al., 2009), where deviation in lead time impacts the delivery precision and raises uncertainty on the supply. According to Ioannou and Dimitriou (2012), lead time has direct impacts on inventory and supply availability, and therefore the issue of managing lead time has also been consistently discussed in the literature since the late 1960s. To be specific, when a deviation occurs to the lead time, it results in the estimated time of arrival (ETA) being not accurate and further disturbing inventory planning. The inventory of spare parts is, therefore, going to fluctuate, causing stockouts when spare parts arrive late or inventory holding costs when they arrive early (Heydari et al., 2009). In particular, spare parts belong to maintenance inventories and the stockout costs of the spare parts could be significantly high (Kennedy, Patterson and Fredendall., 2002). Inspired by preventive and corrective maintenance (Mobley, 2002; Schmidt and Wang, 2018), if the deviation of lead time can be predicted be- forehand, preventive actions can be adopted to minimize deviation, improving the accuracy of ETA and secure delivery precision. Corrective actions can also be sched- uled to mitigate the impacts of the deviation. For instance, to diminish deviation, 1 1. Introduction more attention can be put on monitoring the supply process where it is predicted to have deviated time of arrival and therefore the company can proactively take ac- tions to avoid the deviation. To mitigate the impacts of deviated arrival time that could bring fluctuated stock level, inventory planning can be updated considering the deviation of ETA to ensure the availability of stock. Overall, the successful prediction of the deviation on lead time can, firstly lead to a lower total cost, because the right information of arrival time contributes to having the right amount of spare parts in the inventory at the right time, saving both in- ventory holding costs and inventory shortage costs (Carbonneau, Laframboise and Vahidov, 2008). Secondly, it can improve customer satisfaction by securing their vehicle up-time with the availability of spare parts needed in the warehouse (Car- bonneau et al., 2008). Therefore, costs saving and capability of fulfilling customer orders on time are the outputs of an accurate prediction of lead time deviation. Since there are various companies cooperating in the supply chains, the performance from supplier companies is going to affect buying companies’ performance. This is the case especially for manufacturing industries including automotive, who relies heavily on component suppliers (Krause, Handfield and Tyler, 2007). Therefore, it is beneficial to predict the delivery performance from the buying companies’ per- spective to secure their business operation. Machine learning models are emerging to be used to predict suppliers’ performance and predict the lead time or ETA in different transport modes, due to its ability to capture the pattern from complex relationship between input features and output performance (Witten et al., 2017). For example, predicting arrival time of truck in distribution are discussed (van der Spoel, Amrit and van Hillegersberg, 2017). Delay in passenger airplanes and freight trains (later than ETA) have also been predicted using machine learning from transport handlers’ perspective (Belcastro et al., 2016, Takacs 2014, Barbour et al, 2018). However, for material supplier per- formance, existing literature only predicts supplier overall performance rather than specifically focusing on delivery precision (Jiang et al., 2013; Khaldi et al., 2017). For the transportation, the performance of prediction models varies with different input features. So far, we have not found literature that is based on input variables of organisation and human to predict truck arrival time with machine learning. 1.2 Company Background Volvo Group (Volvo) Service Market Logistics (SML), as one of the departments in the case company where this project is performed, is responsible for the develop- ment and optimization of the spare part supply chain which strives for securing the availability of spare parts at the lowest possible costs. 2 1. Introduction To achieve this goal, the target of delivery precision performance from logistics ser- vice providers (LSP) in SML is 97%. It means 97% of transportation delivery shall not arrive late on each node. However, due to the fact that lead times are negotiated with their suppliers and set in the planning system for a longer period of time since the cooperation starts and there are various uncertainties in supply process, the de- viation occurs frequently in lead time. For the spare parts of Volvo truck in Europe in 2018, around 37% delivery does not meet the ETA at their central distribution center (CDC) according to predefined transportation lead time (TLT). Among the deviation, 27% of them arrived earlier and 10% of them arrived later than ETA. Pre- vious than that transportation delivery goal, the target for the material suppliers’ delivery precision is 95%, which means 95% of the orders from material suppliers shall not be ready later than scheduled. However, for the previous performance in 2017 and 2018, merely 77% of them does not have deviation in supplier lead time (SLT) and was dispatched on time, with 9% of them dispatched earlier than sched- uled, and the remaining 14% dispatched later than estimated. This big share of deviation could directly bring fluctuation in inventories. Spare parts arriving earlier than estimated are bringing extra tied-up capital, inventory costs and disturbing the work schedule in warehouses, while late-arrived spare parts could either cause extra delivery costs in recovering the back-orders by expediting logistics using air transport, or become excess inventory and end up being scrapped because of missing out to supply the demand. As it is important for Volvo to fulfil customers’ demand at a lower cost, there is a need for predicting lead time deviation for monitoring the delivery precision performance on their material suppliers and logistics service providers (LSP) in order to proactively checking ETA of spare parts and take actions. In Volvo, the importance of big data is increasingly raising attention. More and more data are collected and analyzed. These new data resources combined with advanced analytic methods are creating new opportunities to reap the fruits of data mining to benefit business. Volvo has realized the power of machine learning models in prediction and has been initiating data mining projects to explore its possible us- ages and potential benefits. Therefore, this study targets on predicting deviation of lead time on its suppliers of material and transportation by implementing machine learning. 1.3 Aim The aim of this thesis is to evaluate whether and how machine learning modelling can be implemented to predict lead time deviation from buying companies’ suppliers of material and logistics, under the consideration of achieving benefits of a predic- tion model in the current stage of the case company Volvo SML. 3 1. Introduction To achieve the aim of implementing machine learning models to predict lead time deviation, the first research question is to investigate how the company can utilize the lead time prediction. This question sets the business goal and answers potential benefits of this data mining project. RQ 1: What are the benefits of predicting lead time deviation for buying com- panies? The second research question is to investigate the factors that are associated with deviation from the buying company’s perspective. These factors are the basis for features construction for machine learning modelling. RQ 2: What are the factors that could be associated with lead time deviation perceived by buying companies? However, only the factors that can be represented with available data in the com- pany’s database can be analyzed and constructed into the prediction model. This research question reflects the limitation existing in the case company for the con- struction of the model and contributes to set the data mining goal of this project. RQ 3: Which data are available to be used as features when building the pre- diction model of lead time deviation at Volvo SML? The fourth question is to develop a prediction model by testing different machine learning strategies and algorithms. The modelling process is based on Volvo’s situa- tion considering the benefits that the company can practically achieve in the current stage. The results of modelling will be also examined and interpreted regarding their usability. RQ 4: How should the prediction model be built using machine learning con- sidering the practicality of use in the current stage at Volvo SML? 1.4 Scope In order to fulfil the aim of this thesis project, a certain scope is needed. The scope of the thesis is focusing on the spare parts that belong to Volvo truck in European region. Further, for the scope of lead time, the chosen phase will be examined from the moment that Volvo places orders to its material suppliers and shipped by LSP until they arrive at the CDC in Ghent, Belgium. The reason for choosing this in- bound flow is because it currently suffers from the largest deviation and this flow is at the beginning of the supply chain which has cascade effects on later processes. In this project, this lead time is named inbound flow lead time and it consists of two phases which are supplier lead time (SLT), and inbound transportation lead time (TLT). The SLT is the time taken by the material suppliers to get ready for 4 1. Introduction ordered parts. The TLT is the time taken by LSP from consignors (material sup- pliers) to consignee (CDC Ghent). The deviation in TLT results in the deviation in arrival time. In line with the company’s measurement system, the deviation for SLT is measured in the weekly basis, while the deviation for TLT is measured in daily basis. This means deviation of SLT beyond one week and deviation of TLT beyond one day is counted as ‘Late’ or ‘Early’. In our scope, the transportation mode is regarded as road transportation with trucks, since the delivery within Europe is mainly adopt trucks only with the exemption of the cross-docking shipments from Sweden to CDC Ghent which are transported via sea. This sea flow is not considered in the prediction model. The information used in the project is limited within the case company. The data related to deviation in scope are not including the suppliers’ solely owned informa- tion such as production information in material suppliers, and fleet management information in LSP. No external data is used. 5 1. Introduction 6 2 Literature Review In order to support the analysis and discussion by providing theoretical resources and domain knowledge for machine learning, a literature review is conducted in this chapter. It is divided into two parts with the first part reviewing spare parts supply chain, previous work and current state of predicting lead time deviation, while the second part including the last two sections is introducing machine learning. 2.1 Frame of Reference This section introduces the frame of reference which helps to present the context of spare part logistics and the application of machine learning in the area of sup- plier evaluation and ETA prediction. They are corresponding to the subjects of this project. 2.1.1 Spare Part Logistics Context The requirements for planning spare parts logistics are different from the logistics of other material from several aspects (Huiskonen, 2001). Firstly, the service re- quirement of logistics is high due to the remarkable costs and penalties for spare parts shortage. However, the demand for spare parts is sporadic and hard to predict which bring high risks of late delivery. Secondly, due to the decrease of the buffers of time and material in the supply chain and production systems, streamlining the spare parts logistics is under the pressure (Huiskonen, 2001). Most papers are addressing these requirements by focusing on the inventory manage- ment of spare part locally rather than considering the whole supply chain (Zanjani & Nourelfath, 2014). However, inventory optimization often has strict assumptions and difficult to apply. There is a need to increase the collaboration between different actors to plan spare parts logistics to deal with the special requirements of spare part logistics (Huiskonen, 2001). 7 2. Literature Review 2.1.2 Supplier Evaluation One aspect of collaboration for today’s supply chain management is to maintain a long term relationship with suppliers by having a fewer number of suppliers with reliable performance. Hence, it is important to evaluate the suppliers’ performance effectively in order to maintain the right suppliers (Ho, Xu and Dey, 2010). Since automotive companies are especially dependant on their sub-component suppliers, their performance is much affected by their supplier performance in delivery time, reliability and flexibility, according to Krause, Handfield and Tyler (2007). It means if a supplier improves its production time then its industrial customers could get their order faster as a consequence. Therefore, to evaluate the performance of their suppliers is very important for buying companies’ performance. As a multiple criteria decision-making problem, supplier evaluation can have several quantitative and qualitative criteria. The relationship between these criteria and supplier performance could be complex (Rezaei, Fahim, and Tavasszy, 2014). While existing papers mainly discuss supplier evaluation for the purpose of choosing the right supplier, which belongs to a pre-evaluation at a strategy level, very few pa- pers are focusing on adopting post-evaluation at an operational level ( (Khaldi et al., 2017). Only Khaldi et al. (2017) adopt artificial neural network algorithm to evaluate and predict the hospital’s suppliers performance from their transactional contracts and paperwork of delivery articles including delivery delays, the number of partial deliveries, turnovers, amount of orders. The output of the prediction model is the efficiency score of suppliers. Jiang et al. (2013) conduct an experiment to forecast new suppliers’ classification in terms of their performance and efficiency. They train the support vector machine model with the input of cost reduction per- formance, price, delivery, quality. For predicting supplier’s lead time deviation, in essentials, it is a supplier evaluation task which focuses specifically on suppliers’ delivery precision performance. Delivery precision or delivery reliability refers to the ability to delivery according to schedules or promises (Sarmiento et al., 2007). The higher the delivery precision, the lower the deviation of lead time. This research has not been performed previously to our best knowledge. 2.1.3 ETA/Lead Time Prediction For TLT prediction, there are literatures developed in each transportation scenario, such as train, road and flight. However, according to a literature review conducted by Van der Spoel, Amrit, and Hillegersberg (2017), there is very few literature pre- dicting arrival time focusing on trucks. Therefore, this study considers to learn from the practice from each mode of transportation, one up-to-date paper is chosen and described for a review and summarized into Table 2.1. 8 2. Literature Review Van der Spoel, Amrit, and Hillegersberg (2017) state that unlike the travel time which may be well predicted by using weather and traffic information, the truck arrival time could be much affected by human and organizational factors such as planning departure time. That means there is the difference between predicting lead time and arrival time. The result of lead time prediction cannot be directly applied to arrival time prediction without considering planning departure time. They test it by predicting arrival time only using those weather and traffic information. The response output is classified by the tardiness of trucks arriving at the distribution center. The classes are roughly from very early and slightly early to very late and slightly late. They test a set of algorithms such as random forest. Finally, the result is as estimated. The prediction power of the developed models for arrival time is not satisfying since human and organization factors are not included as features. Belcastro et al. (2016) predict flight delays by focusing on weather condition since the weather is the cause of delay for more than 1/3 of the flights. They have high precision and recall score up to 86% for a large delay threshold to be 60 minutes. The threshold means when a flight arrives more than one hour later than the ETA, this flight is counted as ‘late’. Barbour et al. (2018) predict the travel time of a freight train in real time in order to generate ETA. A full network state information from transportation handler in- cluding physical train characteristics and train crew information are the input for having regression results. Compared to the current analytical method calculating the travel time only considering the network topology and traffic through particular routes, they manage to improve the performance by over 60% using random forest. Table 2.1: Review of predicting ETA/machine learning with machine learning Author(s) Subject Classifi- cation/ Regression Input data Model Remark Van der Spoel et al., (2017) Truck ar- rival time at Distribution center Classifi- cation Traffic informa- tion, Weather information M1 ensemble, Random Forest. . . Low prediction power 72% ac- curacy Belcastro et al.(2016) Flight delays Classifi- cation Weather Condition Flight information MapReduce Accuracy 85.8% Recall 86.9% Barbour et al. (2018) Freight Train Arrival Time (travel time) Regression A full network state including physical train character- istics, train crew information Random forest, Support vector re- gression, Neural network maximum predictive im- provements of over 60% using random forest compared to the current method 9 2. Literature Review 2.1.4 Conclusion from Frame of Reference From the frame of reference, we can conclude that implementing machine learning model on predicting suppliers’ delivery precision is an unexplored topic. Existing literature only implements machine learning to predict the overall performance of suppliers based on multi-criteria. Therefore, it remains to explore whether supplier delivery precision can be predicted with machine learning models from the buying companies. Similarly, plenty of work has been done on predicting ETA for various transporta- tion modes but few of them focuses on truck. For flight delay prediction, since the weather is one of the major causes for the delay, only considering weather and flight information could generate a good prediction result with machine learning. However, for predicting ETA of the truck, only considering weather and traffic information is not enough to have good prediction power since organization and human factors could frequently cause deviation in arrival time. When a full network state infor- mation including human and organization factors is used for predicting ETA of the freight train, a significant improvement of prediction is made compared to the pre- vious prediction model where only traffic and route information is used. Therefore, our work will try to consider organization and human factors into the prediction model for ETA of trucks, since it is unexplored which information could be effective to be used as input features for machine learning models to predict delivery precision of LSP. 2.2 Machine Learning Tool and Terminology This section is going to introduce machine learning and its relevant terminology such as input and output, algorithm selection, classification and regression models, boosting and bagging, random forest, catboost and gradient boosting, handling class imbalance. 2.2.1 Fundamental Machine Learning Definition Machine learning is a field covering the main techniques used for data mining which is finding the patterns in the substantial amount of data. The discovered patterns must be insightful which can assist decision making (Witten et al., 2017). There are two extremes about a pattern, from a black box whose mechanisms are incom- prehensible to a transparent box whose construction reflects the formation of the pattern. The difference between them is whether the patterns can be explained and interpreted. Both of them could lead to good predictions and knowing the inputs and outputs are way more important than understand the mechanisms in between (Witten et al., 2017). 10 2. Literature Review There are some fundamental machine learning definitions. Input is including con- cepts, instances and feature. Concept is the thing to be learned. The input to a machine learning model is a set of instances that needs to be classified, associated or clustered. Each instance is an independent example of the concept used for learning or evaluation. There are features which is another set of predefined attributes that are measuring various aspects of the instance (Witten et al., 2017). Dimension of features measures the number of features. There are typically two types of features for machine learning, namely categorical and continuous one. According to Prokhorenkova et al. (2018), categorical features refer to a discrete set of values that are incomparable to each other in a numeri- cal way. The measurement scale of the categorical features consists of a different set of categories (Agresti, 2018). Categorising the features can be implemented in three different ways. The simplest one is regarded to the situations of having bi- nary features when the values could be categorised in “0” or “1” or “YES” or “NO” segments. Furthermore, the categorical features could be mapped on an ordinal scale. For instance, they could be classified such as: “very late”, “late”, “on time”, “early” and “very early”. These features are also called “ordinal variables”. Nominal features are the final segment according to Agresti (2014). Nominal features have no numeric values and are independent of each other. These features are normally used to identify something (e.g. countries) and have not any kind of natural order. In contrast, continuous features are referred to as the variables that have an infinite number of possible values. Label is the values or categories belonging to instances (Mohri, Rostamizadeh and Talwalkar, 2012). The input instances are divided into training set and test set. Training set is used to train a machine learning model, while the test set is used to evaluate the perfor- mance of the model. The test set is separated with the training set and not available at the training phase. The output of the model is the form of prediction on new instances (Mohri et al., 2012). 2.2.2 Algorithms and Feature Selection Knowing which algorithm is likely to deliver a good performance for the investigated problem is known as an algorithm selection problem (Rice, 1976). There is no uni- versally best algorithm for solving a vast problem domain (Wolpert and Macready, 1999). Identifying the most suitable machine learning algorithms which can discover the relationship between the output and the relevant features is a challenging issue (Lingitz et al., 2018). It is necessary to well understand the characteristics of the problem in order to choose the suitable algorithms (Smith-Miles, 2009). There is the ensemble method which can adopt multiple machine learning algo- rithm to achieve better predictive performance. Based on the different strategy, it is categorized into boosting and bagging. García-Pedrajas et al. (2012) describe the function of boosting by saying that it builds an ensemble in a step-wise manner 11 2. Literature Review by making a new classifier and add it to the ensemble. The logic of this process is that the new classifier would be trained towards the biased samples. If any sample has been misclassified during the boosting process they will be assigned by a higher weighted value (García-Pedrajas et al, 2012). Boosting is a general method to use in order to improve learning algorithms since it is capable to reduce the errors of weak learning algorithm (Freund and Schapire, 1996). In terms of the bagging method, it is a set of predictors based on bootstrapped aggregated samples in order to achieve an aggregated performance (Breiman, 1996). For predicting specific classes, the majority of the votes from multiple predictors for one class would be selected. For the prediction of a numerical output, the average value of the output from the aggregated predictors would be considered. When adopted machine learning, the first decision is to choose between supervised machine learning which assumes that training examples are labelled, unsupervised machine learning which has focused on the analysis of unclassified examples, or other techniques such as semi-supervised machine learning or reinforcement learning (Lin- gitz et al., 2018). Semi-supervised learning consider both labeled and unlabelled data which is commonly used when some labeled data are expensive to obtain but unlabeled data could also help achieve better model performance. Reinforcement learning is intermixing the training and test phase, for each move receive immediate rewards to help prediction(Mohri et al., 2012). According to Öztürk et al (2006) supervised learning is considering the relationship between the output and the in- dependent or explanatory features in a model. It aims to predict output based on input features with a prerequisite of a known training set (Pfeiffer et al., 2015). Feature selection is another key process in machine learning. There are many possi- ble benefits with feature selection: decreasing dimensions for improving prediction performance, providing faster and effective predictors with lost cost, assisting to understand the underlying process of data generation (Guyon and Elisseeff, 2003). According to Dash and Liu (1997), in real word practice, most classification prob- lems require the supervised learning with each instance associated with a class label. Since the relevant features could not be known beforehand, the candidate features are often selected for their representativeness for the domain. Unfortunately, many of these candidate features are often irrelevant or redundant to the output concept and not affecting the output result. However, as soon as the size of features or dataset are up to hundreds to thousands, reducing them could significantly increase the speed of machine learning (Dash and Liu, 1997; Guyon and Elisseeff, 2003) 2.2.3 Classification and Regression models Classification and regression are two important data mining missions for supervised machine learning. Both of them contribute to building a data-driven model to learn an unknown underlying function that illustrates the relationship between several input features and one target variable as the output of the function (Cortez and 12 2. Literature Review Embrechts, 2013; Lingitz et al, 2018). To compare the regression and classification model, this selection should be based on predictive capability, computational re- quirements and explanatory power (Cortez & Embrechts, 2012). The difference between these two types is made by the existence of categorical and continuous features in a model. When the output in a predictive model is set to be categorical variables then the classification techniques would be used. In the case of having a output in the form of a continuous value, the regression techniques would be applied (James et al., 2013). 2.2.4 Random Forest Random forest has combined two powerful algorithms namely bagging and ran- dom feature selection (Breiman,2001; González et al., 2014). According to Breiman (2001), random forest is an ensemble Classification and Regression Trees (CART) classifiers, that each decision tree is created without any pruning and bagging algo- rithm is applied in order to create a “forest” of classifiers voting for specific labels. Each tree is considered as a predictor. Random forest could be used for both clas- sifications and regression problems. Pfeiffer et al. (2015) adopt the random forest regression to estimate the lead time as a continuous output variable. They argue the random forest model has better performance than the decision tree model and multiple linear regression model. According to González et al (2014), random forest is capable to capture the complex interactions with different data structure and it is also robust to over-fitting problems. 2.2.5 Gradient Boosting and Categorical Boosting Gradient boosting has been used as an advanced machine learning technique for many years, which can handle complex data sets in an effective way. According to Zhang & Haghani (2015), gradient boosting is a regression tree based algorithm that builds a model in a stage-wise fashion and updates it by minimizing the expected value of certain loss function. Gradient boosting basically applies gradient descent in a functional space to build ensemble predictors. Friedman (2001) describe gradient boosting as an algorithm that is highly robust and explainable for both regression and classification problems. According to Prokhorenkova et al. (2018), categorical boosting(Catboost) is the execution of gradient boosting that uses binary decision trees as base predictors. In Catboost, the decision trees have the same split criterion along with the entire level of the trees. These trees are less prone to over-fitting and have a higher speed of processing time for the testing data set. Prokhorenkova et al. (2018) claim that Catboost outperformed the other advanced gradient boosting algorithms, XGBoost and LightGBM on plenty of different machine learning tasks. Dorogush, Ershov and Gulin (2018) introduce Catboost as an algorithm that has been successful in 13 2. Literature Review dealing with categorical features which are in practice very hard to deal with. The authors also mention that Catboost algorithms can handle the over-fitting problem in a convenient manner. 2.2.6 Handling Class Imbalance Handling class imbalance distribution is a significant topic happening frequently in practice. Class imbalance arises when classes are represented unequally. Namely, most of instances are labelled as one class, while the rare instances are labelled as the other class which might be of more interest or importance. It is crucial that a classification model should be able to achieve higher identification capability on the rare occurrences in datasets. Many traditional classifiers are not compatible with the learning task with imbalanced classes (Kotsiantis, Kanellopoulos and Pintelas, 2006). According to Ali, Shamsuddin and Ralescu (2015), there are two problems in handling class imbalance. One of the main concerns is that data mining performers could be accuracy driven. The traditional way of examining a model performance focus on accurate performance. Classification algorithms selected for their high accuracy performance are likely to group all the data into the majority class to min- imize overall error. This is often at a cost of misclassifying the rare instances. In a class imbalance dataset, classification accuracy tells very little about the minor- ity class and may give a misleading evaluation of classifier performance. Another issue in learning with class imbalance distribution is that standard classification al- gorithms are based on the assumption of the evenly distributed underlying training set. Failing to consider the skewed distribution of data is most likely to hinder the classification performance (Ali, Shamsuddin and Ralescu, 2015). The classification performance for imbalanced data is also subjective to the size of the dataset (Kotsiantis, Kanellopoulos and Pintelas, 2006). It may be even worse for an small imbalanced dataset compared to the larger one, due to the insufficient sample size of instances representing minority class for learning. On the contrary, the effects are relatively less severe with larger datasets, as the minority class is bet- ter represented by a larger size of examples (Kotsiantis, Kanellopoulos and Pintelas, 2006). To handle class imbalance classification, sampling techniques and cost-sensitive learning are commonly applied. Sampling techniques are used to either remove a small number of examples from majority class or over-sample minority class or both. By introducing this sampling step, the discrepancy between the two classes is minimized so that traditional classification algorithms can work well. For example, Balanced Random Forest, incorporating under-sampling majority class technique and the ensemble learning, artificially re-balances the class distribution to ensure that classes are equally represented in each tree (Chen and Breiman, 2004). 14 2. Literature Review Cost-sensitive learning approaches, on the other hand, impose an expensive cost on a classifier when a misclassification happens in order to emphasize any correct classification or misclassification regarding the minority class (Kotsiantis, Kanel- lopoulos and Pintelas, 2006). For instance, in Boosting algorithms, different weights are placed on the training distribution in each iteration. In order to emphasize misclassified examples in the next iteration, boosting increases the weights on the misclassified examples and decreases the weights on the correctly classified examples after each iteration. Since minority classes are more likely to be improperly clas- sified in comparison with majority classes, boosting may improve the classification performance through increasing the weights of the examples from rare classes. Also, as boosting effectively rebalance the distribution of the training data, it can also be considered as an advanced sampling technique (Kotsiantis, Kanellopoulos and Pintelas, 2006). 2.3 Evaluation Metrics for the Prediction Models Since the overall accuracy could insufficiently or even misleadingly evaluate a clas- sifier performance (Visa, 2006; Japkowicz and Stephen, 2002; Wang and Mendel, 1992), the confusion matrix and its derivations are introduced as a more proper way to summaries the performance results. Feature importance is also introduced as another measurement for the input features. 2.3.1 Confusion Matrix A confusion matrix shown in Table 2.2. is typical for evaluating the machine learning models’ performance with imbalanced classes. Class “C” is regarded as the minority class which is in the focus, while “NC” is a combination of all the other classes. There could be four kinds of results when detecting class “C” (Chawla et al., 2003). The first one is true positives which correctly recognized focused class examples. True negatives are those correctly identified examples that do not belong to the focused class. The third factor, false positives, considers the examples that were incorrectly assigned to the focused class and finally the last one is false negatives which were not successfully recognized as focused class examples. These four factors constitute a confusion matrix (Chawla et al., 2003). Table 2.2: Confusion matrix defines four possible scenarios when classifying class “C” (Chawla et al., 2003) Predicted Class “C” Predicted Class “NC” Actual class “C” True Positives (TP) False Negatives (FN) Actual class “NC” False Positives (FP) True Negatives (TN) 15 2. Literature Review From Table 2.2., recall, precision and F-value are defined as follows: Precision = TP/(TP + FP ) (2.1) Recall = TP/(TP + FN) (2.2) F − value = (1 + β2) ∗Recall ∗ Precision β2 ∗Recall + Precision (2.3) The performance metrics derived from the confusion matrix are including precision, recall, F1 score which comprise of a classification report for the modelling result. Precision measures the exactness, which is the proportion of correctly predicting classes. It shows the ability of a classifier for avoiding misclassifying negative classes into the positive class. Recall measures the completeness, which represents a clas- sifier’s ability to learn positive class. It is calculated by the proportion of correct detection of positive example out of all positive example in the data. F-score is a way of balancing the measurement between precision and recall. As the β is com- monly set to 1, therefore F1 score is used for classification (Sokolova and Lapalme, 2009). The common pursue of all learning model is to improve the recall while not to sac- rifice the precision. However, there is often the conflicts between them and it may be difficult to improve both of them at the same time. This situation is especially true when one or more classes are rare (Chawla et al., 2003). 2.3.2 Feature Importance The increasing popularity of machine learning models is largely credited to their capability to handle high-dimension data with large number of predictors and other advantages including relatively good accuracy, robustness, ease of use (Breiman, 2001). However, it is common that not all the features are important and some of input features can be relative irrelevant or redundant in data mining. Identifying the most important features is beneficial because it indicates which features have the highest predictive power for the model and may help the domain users to have a better understanding of the problem. It can also help to develop recommendations for the future, and it may lead to changing the role of the underestimated features more seriously (Petkovic et al, 2016). To identify the features with the most signif- icant impacts on predictions, feature importance is one of the most commonly used measurements, which facilitates feature selection and model interpretation. The most widely used feature importance measures are the impurity importance and the permutation importance (Breiman, 2001). The impurity importance, also known as Gini importance, is based on the mechanism of mean decrease of impu- rity. It is the default feature importance measure embedded in some most popular implementation platform such as R and scikit-learn in Python. In the impurity importance, a feature is considered as important if it is effective at diminishing uncertainty for classifiers or variance for regressors. The impurity importance for 16 2. Literature Review a feature in random forests, for example, is computed by adding up all impurity decrease measures of all nodes in the forest where a split on this feature has been made, normalized by the number of trees. Another type of importance measure, the permutation importance is also known as mean decrease of accuracy. Under this mechanism, the important features are those positively contributing to reduce the prediction error. Despite its popularity, for years, the impurity importance is acknowledged to be biased. The impurity importance is likely to inflate the importance of categorical variables with many categories and continuous variables (Breiman et al., 1984; Strobl et al., 2007), also in favor of variables with high category frequencies (Nicodemus, 2011). The permutation importance, on the other hand, is safe from these concerns (Nicodemus et al., 2010; Szymczak et al., 2016; Ziegler and Konig, 2014). However, the permutation importance can be extremely computationally intensive when en- countering high dimensional data. Also, Calle and Urrea (2011) argued that feature importance rankings based on the impurity importance can be more robust over those obtained with the permutation importance. 17 2. Literature Review 18 3 Methods In this chapter, the methods that were used to conduct this project are described. First, the literature review was then conducted and also throughout the entire process of the project. Then the CRISP-DM (Cross Industry Standard Process for Data Mining) methodology and the reasons for choosing it are introduced. The correlation between methods and research questions is also demonstrated. Finally, the reliability and validity issues are described in the end section. 3.1 Literature Review There are several reasons for conducting a literature review at the beginning of and throughout the project. Bryman and Bell (2015) describe the first thing is to be aware of and understand what has been already discussed in the research area. Sec- ondly, it also gives a way for authors to develop an argument about demonstrating the significance of the project and where it contributes. Beyond that, a literature review with an interpretation from reliable sources in the research field could also increase the credibility of the project. Based on the above reasons, a literature review was conducted with the purpose of providing information for four research questions and assisting the data mining process for realizing the aim of the project. We searched literature from electronic database including Scopus, Google scholar and Chalmers Library. The keywords used in the search including the combination of lead time deviation, estimated time of arrival (ETA), prediction, delivery precision, machine learning, supplier evaluation, automotive. Peer reviewed articles and books were examined and used in the literature review.The result of the literature review is compiled in the chapter 2. 3.2 General Strategy and Process The most commonly used process for data mining projects is CRISP-DM (Marban, Mariscal and Segovia, 2009) It is process model being developed by a group of data mining leaders for carrying out data mining projects. The purpose of this process model is to make these projects more reliable and replicable with less money and time spent (Wirth and Hipp, 2000). Wirth and Hipp (2000) discuss that the pro- cess can not only be performed by experts, but the novices with less experience and 19 3. Methods technical skills can benefit in a limited time. This is due to the characteristics of CRISP-DM being both structural and flexible depending on whether it is generic or specialized process. For less experienced people such as master students, we can get guidance and structure of the project, as well as advice for each process. The processes of CRISP-DM from generic to specific are described as Phases, Generic Tasks, Specialized Tasks and Process Instances (CRISP-DM, 1999). For the top level, the phases of the model include business understanding, data understanding, data preparation, modelling, evaluation and deployment representing the life cycle of a data mining project. The second level is generic tasks with its intention to cover all data mining situations. The third level aims to describe what actions should be taken within the general tasks. The fourth level is a requirement of recording the actions, decisions and results during the process. We adapted the generic CRISP-DM process based on our data modelling project, and the process is summarized in Figure 3.1. There are six phases in the CRISP-DM process that are described in the following sections. Business Understanding Data Understanding Evaluation Modelling Data Preparation Deployment Determine Business Objectives Assess Situation Determine Data Mining Goals Select Data Integrate Data Clean Data Construct Data Generate Test Design Select Modelling Technique Build Model Assess Model Evaluate Result Review Process Determine next steps Collect Initial Data Describe Data Verify Data Quality Figure 3.1: Illustration of the data mining process based on CRISP-DM (1999) 3.3 Business Understanding The first phase is about understanding the business. Business understanding in- volves figuring out the feasible goals based on the situation and requirements from the business perspective to achieve potential benefits. Therefore, qualitative data about business were collected by means of conducting interviews and examining internal documents in the company in order to set a feasible goal. 20 3. Methods 3.3.1 Interview As Yin (2018) says, one of the most important sources of information is interviews since they can help with explanations of key events and providing insights from the interviewees’ point of view. The interviewees were selected based on the scope and aim of this project, including representatives such as Continental Material Planners (CMP) and Transport Material Coordinators (TMC). The interviewee list is seen as Table 3.1. Both semi-structured and unstructured interviews were adopted since interviews in the case study are like guided conversations (Yin, 2018). Unstructured interviews were held throughout the project when there was a need to get clarifica- tion of any concepts and questions. Semi-structured interviews were adopted to gain an initial understanding of the researched topic. Questions about the performance of lead time and the relevant factors that are associated with lead time deviation were asked. During the semi-structured interviews, audio recordings were collected for the purpose of capturing all the information from the answers, by enabling au- thors to revisit the answers from interviews. The transcription was generated by one author in the interview with the aid of audio recordings and the interview re- sults were then examined by the other author attending the meeting. The interview templates used in the semi-structured interviews are presented in Appendix A. Table 3.1: A table for interviewees list Title Interview Topic Date Operational Resource Planner CDC management 2019-02-08 Refill Analyst Outbound Logistics 2019-02-11 Demand and Inventory Planner Demand Forecast 2019-02-11 Continental Material Planner Monitor Material Suppliers performance 2019-02-18 Transport Developer Transportation Lead time set up 2019-02-26 Supplier Relationship Manager Material Suppliers evaluation 2019-02-27 2019-03-05 Manager Supplier Management Evaluation of Logistics Service provider 2019-03-01 2019-03-07 Logistics developer & Business Analyst Material Planning Data extraction for supplier lead time 2019-03-08 Data Scientist Modelling 2019-03-15 Transport Material Coordinator Monitor Logistics service provider 2019-03-25 Project Manager Lead time strategy 2019-04-02 21 3. Methods 3.3.2 Internal Documents For a case study research, the most crucial value of internal documents is to au- thenticate and argue the evidence from other sources (Yin, 2018). With the inter- nal documents, this project gained up-to-date knowledge about the case company’s structures and business processes. The collected information also became evidence to support arguments from interviews. The internal documents used in this project were found in the internal database of the case company, including company pre- sentation, process description system and the team places. These documents could exist in the form of PowerPoints, word documents and other informative data from databases. After this stage, the goal of the business was defined to respond to the research ques- tion 1. To answer the research question 2 about the factors of lead time deviation, a pile of factors were compiled after conducting the literature review, interviews and examining internal documents. A list of preliminary potential features was also identified in this process. 3.4 Data Collection and Understanding For the data understanding process, one investigated aspect was to collect the histor- ical data of lead time deviation performance, which was used as the output variable for modelling. Another aspect was gathering those available data that could asso- ciate with factors of the deviation of lead time identified in the first stage. These quantitative data were extracted from different databases in the case company as archival records, as Table 3.2 shows. Historical lead time performance data of sup- plier lead time was extracted from the Business Intelligence where the previous two years data (2017 and 2018) were included. The data related to features of the first model were also extracted from business intelligence and the reports generated from supplier management portal VSIB. For the second model, most of the data were extracted from the logistics management portal Atlas. These data were limited to the previous one rolling year as the maximum amount of data the system held at the time the project was conducted. Noted from the transportation delivery precision report, there is up to 30% of delivery where goods were not delivered according to planned deliveries. These missing deliveries were deleted and not considered into the calculation of delivery performance since they are not generated the output of delivery whether they are on time or deviated. Then data understanding was to get to know the data about its variability and availability, including the quality and quantity of the data. Since the business goal needs to be translated into the goal of data mining, the availability of the data in the company was under consideration. Hence the data mining goal was developed. 22 3. Methods Table 3.2: Main data source from the case company’s database Phase Data Sources Content Supplier Lead Time Historical lead time performance (Output Variable) Business Intelligence – Parts- DWH ver1.5 – For Std Report Developer 2017&2018: 402,708 pieces of records Features Parts: Business Intelligence – PartsDWH ver1.5 Suppliers: VSIB – supplier man- agement portal Segmentation, Sales level spend, delivery precision, ... Inbound Trans- portation Lead Time Historical lead time performance (Output Variable) LSPs portal Atlas Filter: all volvo truck parts were ended in CDC Ghent 2018.04-2019.03: 49,948 pieces of records Features Parts: Business Intelligence – PartsDWH ver1.5 Suppliers: LSPs portal Atlas Consignors and Volvo logistics scheduling: Atlas Weight, volume, country ... 3.4.1 Delimitation in the Data Collection There were a few limitations in the data collection phase. Firstly, when sampling data from the data warehouse, the period was limited to what the data warehouses hold. For the transportation phase, the data are recorded for one rolling year. Therefore the amount of data for training were limited to one year period, which could bring problems of bias and robustness. The evaluation results of material suppliers were extracted from the supplier man- agement system VISB. The options for evaluation period are from past three months to past one year, the granularity of the evaluation results such as delivery precision is limited by being made as average value for that chosen period. There were data related to factors that were scattered in lots of separate reports but not integrated into the data warehouses. In this sense, these data were not able to be gathered and used as features for modelling. For example, the logistics audit results of LSP exist in individual excel files for each LSP, then these data were not utilized as a potential feature. There were factors that relate to deviation but suffering from the data quality in the system and not being used as a feature. For example, the departure time of truck could have effects on deviation since it affects the arrival time of a truck to a warehouse which could cease operation during the night and the late arrival truck need to wait for one night to be processed. However, the departure time is not precisely recorded in the system and therefore not suitable to be used. There was the data transparency issue that the names of some items in the databases were confusing without further explanation. In order to make sure the right data 23 3. Methods was used, it also took time for us the data practitioners to find who can explain the data in the company. Sensitive information such as the relationship between suppliers delivery perfor- mance and evaluators in the buying company was also not gathered and examined. 3.5 Data Preparation The data preparation phrase is including all the actions that creating a final data set which were fed into the modelling from the raw data including selecting data, cleaning data, constructing data, integrating and formatting data. 3.5.1 Transferring Categorical Variable There were many categorical variables in the feature list, in order to quantify them and feed them into modelling, a function called dummies in the commonly used python package Pandas was used to turn a categorical variable into a series of zeros and one. One example is illustrated below, the feature of categorical variable ‘stack- able’ is divided into two columns with ‘1’ represent of the characteristics being true, and ‘0’ for not being true. Stackable Yes No ... Stackable 1 0 ... Non-stackable 0 1 ... Get dummies Figure 3.2: Transferring categorical variables into dummy variables However, some categorical variables have a lot of classes such as 59 kinds of seg- mentation of spare parts. When directly getting dummies for these variables, the input data will get lots of columns with each one having little weight. Therefore, these categorical variables were reduced into a reasonable amount of columns by reconstructing and combining them based on some criteria. Segmentation of Volvo spare parts is a comprehensive measurement defined in terms of criticality, life cycle, cost and order frequency. For segmentation result, there are five different initial let- ters from ‘A’ to ‘E’ as main catalogues. From ‘A’ to ‘D’, they represent four kinds of criticality code, and ‘E’ represents non-critical parts. The criticality of a part depends on specific function groups and vital codes. Under each letter, there is the second letter starting from ‘A’ to ‘L’ for the sub catalogues representing the cost, life cycle and order frequency information. Vital code, cost, order frequency are available as independent features, while using function groups directly may result too many categories, and life cycle phase is not directly available. Segmentation was 24 3. Methods adapted to present information of function groups by keeping the main catalogues, and clustered the second catalogues into ‘fast’ and ‘slow’ to roughly represent the life cycle phase. After the modification, the segmentation was simplified into ‘A-fast’, ‘A-slow’, ‘B-fast’, ‘B-slow’ and so on to roughly reflect the function groups and life cycle phases. 3.5.2 Integrate and Link Data After the previous phases of understanding, we realized there was the need to build two prediction models for the two phases, since the deviation could happen in each phase and the detection of deviation is necessary to take actions in each phase. For the modelling of supplier lead time deviation, , the information of parts and suppliers were integrated into the records of delivery precision performance Then, for building the models of transportation lead time, to consider the previous delivery precision performance from material suppliers could also be beneficial. However, the data of two phases in the company are independent. They are separated into two systems, managed by different departments and not linked with each other. In consequence, there is no information about which parts are carried in the shipments from the transportation booking. We manually linked the instances from these two phases, using event time (Dispatch week in material suppliers records, Prove of collection date in LSP records) and companies (supplier ID in material suppliers records, con- signor ID in LSP records) as linking keys. When these two keys were in line with each other in two instances, these two instances were integrated and regarded as the same ordered flow as Figure 3.3 illustrate. This linkage can help the prediction of TLT to have more potential features including relevant parts and material suppliers information. Another issue is that one transportation booking could contain several ordered parts, therefore, when left joining parts information into the transportation book- ing records, several transportation booking instances were duplicated with the only difference of part information between them. Then, in order to integrate these du- plicated instances into one independent instance, the information for those parts in the same transportation booking was used their average value in this project. 3.5.3 Delimitation in the Data Preparation For data preparation in modelling supplier lead time deviation, normally there are existing several orders for a spare part with one supplier in two years duration. Even though the differences between these orders and further integrated features could be only the event time, the deviation could differ from one order to another order. Therefore, all the orders kept for input instance for the benefits of repre- senting the real case, although this might sacrifice the variance of each feature in each instance and affect the model performance and the result of feature importance. 25 3. Methods Supplier Lead Time Deviation Performance Transportation Delivery Precision # Supplier ID # Dispatch week # Consignor ID # Prove of collection date Integrated Inbound Delivery Flow Figure 3.3: Integrate and link data for the phases of material supply and trans- portation There are three ways of transportation, namely Door to Door (DDT), Cross-docking and Milk run. For the transportation mode using cross-docking, the transporta- tion booking reservation is separated into two independent transportation booking records. The previous cross-docking is from material supplier to cross-dock point, while the later cross-docking process starts from cross-docking point to CDC. The consignor for the second transportation booking records, therefore, becomes the cross-dock point. In this way, the second phase of cross-docking transportation failed to be linked with previous corresponding records of material suppliers due to the key of supplier ID and consignor was not to be matched. Only the previous leg of cross-docking were linked. Another limitation happened for the milk run transportation. Even though one milk run generates one transportation booking, with the two keys can be in line with the first material supplier in the milk run, the information of the remaining suppliers and parts information failed to be considered into the input instance for the milk run transportation. As Figure 3.4 shows. 3.5.4 Feature Selection To represent previous identified factors into candidate features for modelling, there were a few cases occurred in this process. Firstly, there are data which can directly represent the factors such as the demand, value, stackable, hazardous, custom, eval- uation results for material suppliers. Secondly, there were data representing the fac- tors at an aggregated level, such as TB weight and volume data for the total weight and volume in one shipment, segmentation data for integrating function groups and life cycles, country for traffic and weather. Thirdly, some factors that were not recorded in the data form, such as the prioritization. Some factors’ information is 26 3. Methods not available in the buying company due to that information is owned by material sup- pliers such as material suppliers’ production information. These factors were tried to be indirectly reflected by other available data, such as sales spend level data on suppliers for representing the prioritization, quality and environment certificate for representing the production capacity of suppliers. However, some data currently are not integrated into the database, and we could not either find other suitable data for the indirect representation of their corresponding factors, such as historical delivery precision performance and evaluation results of LSP. Since the dimension of input features in this project is limited, all potential candidate features were kept as input for the modelling. No further feature selection is needed for the benefits of dimension reduction which is not the case with limited feature dimension. CDC CDC CDC Material Supplier 1 Material Supplier 1 Cross-docking points Material Supplier 2 Material Supplier 3 Material Supplier 1 Material Supplier 2 Material Supplier 2 Material Supplier 3 Material Supplier 3 (A) (B) (C) Figure 3.4: The data linkage in different transportation modes (A) Door to door; (B) Cross dock; (C) Milk run 27 3. Methods 3.5.5 Handling Missing Data Missing data imputation is a method for filling the missing values with some prob- able and possible values before the process of learning algorithm begins (Lepping, 2018). Replacing each missing value for a variable by using the average observed values for that variable is a common method that may accurately predict the value of the missing data but, also leads to poor estimation of variances and correlations (Schafer and Graham, 2002). There was a proportion of missing value when we examined the extracting result. For supplier phase, these missing data particularly exist in the evaluation information for suppliers, including the Supplier Evaluation Measurement (SEM) result, logistics audit result and historical delivery precision. There could be several reasons for the missing value. For example, no evaluation has been performed or no more cooperation with those material suppliers. The degree of missing data for supplier phase was presented in Table 3.3. In comparison, for the transportation model in the data preparation stage, only successful linked and integrated records were kept, and therefore there is no missing value. The missing data were filled in with mean value in this project. Table 3.3: Missing value for supplier lead time phase Variables Number of instances Missing rate (%) Dispatch Week 400641 0.00 Part No 400641 0.00 Supplier No 400641 0.00 Lead time deviation 400641 0.00 Parameter reference 388011 3.15 SEM result 288761 27.93 QPM score 399906 0.18 Quality Certificate 329809 17.68 Purchase agreement 400641 0.00 Sales level Spend 399906 0.18 Vital 400641 0.00 Hazardous Code 400641 0.00 Prepacking Type 400641 0.00 Country 400154 0.12 Registration Date 398617 0.51 Stand Price 398617 0.51 Order Hits Roll 13 Period 398617 0.51 Delivery Precision 362068 9.63 Logistics Audit Result 262179 34.56 28 3. Methods So far, a list of features has been constructed as input data for modelling and re- search question two were covered. 3.6 Machine Learning Modelling and Evaluation Different machine learning modelling and techniques can be chosen and tested in the modelling phase. The parameters are required tuned into the optimal values. Noted the modelling is also closely linked to its previous phase of data preparation since the new problems of data set could not be unveiled until modelling or new ideas are generated for collecting new data. The first choice in the modelling is to choose from supervised learning, semi-supervised learning and unsupervised learning (Lingitz et al., 2018). Since the purpose of the project work is to predict the lead time deviation as the output with labelled input data from databases, supervised machine learning was used for this situation. Based on the previous understanding of the business goal and data mining goal (Smith-Miles, 2009), the output variable is made into three classes, namely ‘On time’, ‘Early’, ‘Late’. This is an imbalanced data set with the majority of the observation falling into the ‘On time’ class. Balanced Random forest (Chen and Breiman, 2004) and boosting algorithms (Kotsiantis, Kanellopoulos and Pintelas, 2006) could be two approaches to deal with imbalanced data set. In addition, based on the knowledge from the data scientist in the case company, several classifica- tion machine learning algorithms were selected to build the models for each phase, including Balanced random forest, Catboost and Gradient boosting. Balanced ran- dom forest has been selected as the algorithm is combining the bagging method and under-sampling technique for the majority class (Chen and Breiman, 2004). The rea- son for selecting the Catboost and Gradient Boosting is that both of them are using the boosting method which can give high penalty to missing classified minority class as a cost-sensitive learning technique (Kotsiantis, Kanellopoulos and Pintelas, 2006). Finally, an evaluation process was conducted. The performances of the above- con- structed models were compared and recorded using confusion matrix. The results were analyzed from a data analysis point of view. Furthermore, the improvement and deployment of the models were examined considering the fulfillment the business goal. The process of the CRISP-DM model was reviewed. Future possible actions were proposed. Until this point, the research question 4 was answered. The rela- tionship between processes and research questions are illustrated below in Figure 3.5. 29 3. Methods Business Understanding RQ3: Which data are available to be used as features when building the prediction model of lead time deviation at Volvo SML? RQ4: How should the prediction model be built using machine learning considering the practicality of use in the current stage at Volvo SML? Data Understanding & Preparation  Modelling RQ 1: What are the benefits of predicting lead  time deviation for buying companies? RQ 2: What are the factors that could be  associated with lead time deviation perceived  by  buying companies? Figure 3.5: The relationship between processes and research questions 3.7 Validity and Reliability According to Bryman and Bell (2011), there are two important aspects regarding the evaluation of the quality of research, namely reliability and validity. Reliability is about the consistency of measures, whereas validity refers to whether a measure of a concept actually manages to measure it (Bryman and Bell, 2011). In the qualitative part of this thesis, reliability will be increased by contemplating inter-observer consistency. According to Bryman and Bell (2011), inter-observer consistency is an issue of inconsistent declaration that could happen when there are several observer-constellations judging information subjectively. All the interpreta- tion from interviews were analysed and agreed upon by the presented interviewers. Validity in the qualitative data of research would increase through internal validity, it means that the findings from observations should fit into the theoretical frame- work developed (Bryman and Bell, 2011). This subject was considered during the thesis process in order to verify the findings from interviews with actual modelling further on. During the quantitative data of the thesis, face and convergent validity were con- sidered. According to Bryman and Bell (2011), face validity is about the process of evaluation of a model by an outside expert to see if it is reasonable. Based on this factor, a machine learning expert from the department where the thesis project is conducting evaluated the scientific aspect of machine learning algorithms in the 30 3. Methods context of this project. Convergent validity, according to Bryman and Bell (2011), considers the result of a method and compare it to the outcome of other methods from the same category. Based on this definition, convergent validity was considered for the new prediction model and the generated models were evaluated by consid- ering some measurements such as the goodness of the fit of the model. Reliability of the quantitative data is gained by examining the correlation between different factors of lead time and the lead time deviation by using correlation analysis. In addition, the quality of the data was examined and modified. 31 3. Methods 32 4 Results: Business Understanding The first section in this chapter is going to describe the company’s operation around lead time including involved processes and roles. Then, the current performance of lead time deviation and its impact are presented. Furthermore, the business goal for this data mining project is set. Finally, the factors related to lead time deviation are described. 4.1 The Set-up of Lead Time in Volvo At Volvo SML, most of the lead times are negotiated with material suppliers and LSP. As agreed, these lead times will be set as predefined parameters in the planning systems. The supply process in Volvo SML could be categorized into five processes, as Figure 4.1 shows. Inbound supply phase starts from Continental Material Planner (CMP) placing orders to material suppliers and ends till the orders are received and registered at Central Distribution Center (CDC), including supplier lead time, in- bound transportation lead time and internal receiving lead time. Outbound supply phase begins after CDC have received and registered the orders until customers get their requested spare parts including outbound transportation lead time and order lead time. The shipments are carried by LSP. Since the set up lead time between Volvo and suppliers by negotiation is an esti- mation of lead time, together with other causes of disruption alongside the delivery process, the deviation in lead time is inevitable. There are also cascading effects along the supply chain. For example, when the material supplier does not dispatch the orders on agreed time, that is going to affect LSP on picking up the orders and further affect later process of transportation. The affected trucks may further arrive at CDC later than the schedule and may need to wait to be unloaded since the capacity of CDC is limited. Most importantly, currently there is no existing process or tool to predict the deviation of lead time in the company. 33 4. Results: Business Understanding Supplier A Supplier B Supplier C Supplier D Supplier X CDC RDC SDC \ Dealer A Dealer B Dealer C Dealer D Dealer X Supplier lead time Inbound transportation lead time Internal receiving lead time Outbound transportation lead time Order lead time Inbound flow lead time LSP1 LSP2 LSP3 Inbound Outbound Figure 4.1: The set up of lead time in Volvo SML 4.2 The Process and Roles Involved in Dealing with Lead Time Deviation The process and roles involved in dealing with lead time prediction are introduced in the below sections. These results lead to the setting of business goal. 4.2.1 Process Overview The inbound delivery process behind SLT starts from Demand and Inventory Plan- ners (DIP) generate demand forecast for CDC Ghent. The demand forecast contains information about at what time and how much of which spare part is needed in the CDC. These demand forecasts pass through the planning system. Based on the forecast information, CMP place orders to corresponding material suppliers. When material suppliers are ready to dispatch the order, they book the shipments from LSP through Volvo’s transportation management portal ‘Atlas’. The transportation booking (TB) contains information such as pick up and shipping address, volume, weight of spare parts. LSP will ship the order to the CDC based on transporta- tion booking information scheduled by Atlas. Atlas portal also incorporates the transportation orders from several material suppliers by arranging different ways of delivery including DDT, cross docking and milk run. 34 4. Results: Business Understanding Inbound logistics lead time Supplier lead time Suppliers Continental Material Planners Atlas Specialist Logistics Service Providers Manager Supplier Management Transport Material Coordinator Supplier Relationship Managers Place Order Lead Time Monitor and Review EscalateLead Time Follow Up Transportation Booking Transportation Booking Lead Time Monitor Lead Time Review EscalateLead Time Follow Up Demand and Inventory Planners Planning Demand Transport Developers Lead Time Optimize Figure 4.2: The roles involved in dealing with lead time deviation Noted the role description is in line with current responsibilities, which could be changed from time to time. The following section is going to describe in detail the responsibility of the most relevant roles, that are the monitors and evaluators of lead time deviation, including Continental Material Planner (CMP) , Supplier Re- lationship Manager (SRM), Supplier Manager (SM). For managing material suppli- ers, Volvo has CMP for monitoring the individual level of performance on material suppliers and SRM perform a higher integrated level of management. While for transportation, TMC are responsible for managing the individual level of LSP and SM are for a higher level of measurement. Delivery precision measures whether the suppliers dispatch requested order on the scheduled time and this key performance indicator (KPI) directly links to the degree of deviation on SLT. Similarly, there is also delivery precision measuring the transportation lead time deviation from LSP representing the accuracy of ETA. The information about the key roles and KPIs for lead time performance is summarized in Table 4.1. Table 4.1: The key roles and KPIs for lead time performance Suppliers KPI of lead time performance Key Roles Material suppliers Delivery precision Continental Material Planner (Monitor) Supplier Relationship Manager (Evaluator) Logistic service providers Delivery precision Transport Material Coordinator (Monitor) Supplier Manager (Evaluator) 35 4. Results: Business Understanding 4.2.2 Continental Material Planner CMP are responsible for the inbound material supply process for spare parts. Their mission is to ensure the availability of spare parts at the central warehouse and provide a sharp ETA to the customers. Their first responsibility is to set up SLT with material suppliers when the part is first sourced to them and then to review the lead time after a certain period of time. The guideline is to propose 2 weeks of lead time for high running spare parts which are frequently ordered, 4 weeks for the middle runner, and best possible lead time for low runners. If proposal for SLT is not accepted by material suppliers, then CMP will take what material suppliers answer to them. Lead time review is done once or twice with two material suppliers per year for each CMP. The purpose of lead time review is to shorten lead time and have lead time information alignment with suppliers. SLT is important since it determines the amount of safety stock. Besides, during the period of SLT, CMP cannot change the order from suppliers unless the change is agreed by suppliers. Create & Send Delivery SchedulePurchase Order Logistic Preparation Parameters Follow up Supplier Dispatch Analyze & Decide Corrective Action Solve Delivery Deviation No No No Yes Yes Yes Delivery Schedule Covers Demand? Supply According to Plan? Escalation Needed? Delivery Schedule Closed Delivery Schedule Closed Continental Material Planner Figure 4.3: The working procedure of CMP Another important responsibility of CMP is to place the order to material suppliers based on purchase orders from DIP and logistic preparation parameter set in the system. After placing the order, CMP then monitor suppliers’ delivery precision by having frequent contacts with them. If suppliers confirm the order information, CMP send the information of ETA to the following process. If there is deviation 36 4. Results: Business Understanding happened in the material suppliers, CMP are responsible to figure out the reasons for the deviation and take actions for dealing with deviation. For example, if the order is dispatched later than schedules, CMP can arrange extra transport with the rush option in order to ensure the availability of spare parts. Since the rush trans- port causes high costs, only with critical spare parts and backorder from customers, CMP shall use this option. CMP can also decide to escalate the problematic suppli- ers to SRM where re-examinations of the suppliers will be performed. In contrast, if one supplier’s performance is above a certain percentage for a certain period of time, CMP tend to trust this supplier and may send out the ETA information very soon without confirmation from material suppliers. The process is illustrated above in Figure 4.3. 4.2.3 Supplier Relationship Manager SRM take responsibility for supporting and developing material supplier in the field of logistics by evaluating supplier delivery performance. SRM are also in charge of conducting Materials Management Operational Guidelines / Logistics Evaluation (MMOG/LE) audit. The purpose of this audit is to evaluate the logistics maturity of material supplier and initiate an action plan for identified gaps. This audit has three levels namely supplier self-assessment, desk verification of a self-assessment and on-site verification. Specifically, in the audit, there is a document of evaluating suppliers performance purely on logistics including lead time agreement, value, ma- terial handling, organization, production, communication, planning of all logistics. Material suppliers fill in the report and SRM have a site visit to evaluate these performances when necessary. SRM are also managing low performing suppliers, if these suppliers performance are not improved for an agreed period of time, SRM should escalate them to supplier purchasing department and these material suppliers may end up losing contract from Volvo. Another task of SRM is prioritizing deliveries between Volvo manufacturing sites and CDC when there is crisis such as lack of capacity in material suppliers. Critical spare parts are among the first priority, and then the manufacturing sites get their capacity, finally, the non-critical spare parts get the rest of capacity. 4.2.4 Transport Material Coordinator Similar to the responsibility of CMP on material suppliers, TMC is responsible for monitoring the performance of LSP in terms of agreed procedure and targets. For their appointed distribution flow including DDT and milk run, they are following up the performance indicators agreed upon with LSP while cross-docking transports are managed by another specialist. 37 4. Results: Business Understanding If deviations happen, TMC also need to analyze the cause of deviations and take corrective actions within their responsibility area or propose corrective actions out of their responsible area. For example, if material suppliers cause the deviation, they should be escalated by TMC. If the deviation is caused by LSP, TMC could take corrective plan or escalate them to SM. This process is demonstrated below as Figure 4.4. Transport Coordinator Material Supplier Manager Specialist Transport Performed Group Analyze & Log Deviations Logistics Service Provider Material Supplier Deviation cause? Perform Logistics Service Provider Root Cause Analysis Perform Material Supplier Root Cause Analysis Performance followed up Performance followed up Monitor & React on Transport Transport type? Multileg Direct flow or delivery to ultimate consignee Monitor & React on Arrival Transport Collection Cross Dock Support Specialist End Supplier Manager Figure 4.4: The working procedure of TMC 4.2.5 Supplier Manager One of the responsibilities that SM have is the quality assurance for LSP. This means that SM have to make sure that every appointed LSP will deliver the agreed level of delivery performance based on their contract. There are some predefined targets related to the service levels for the LSP, such as pickup and delivery precision, their communication performance regarding reporting deviation in time. Following up these targets, making improvements and reporting them in terms of different weekly and monthly KPI are SM’s tasks. It means that they follow up the performance of LSP in terms of delivery precision. 38 4. Results: Business Understanding For those delivery deviations, SM are required to perform root cause analysis and take correction plan accordingly, in order to avoid or limit the consequence of de- viation. For example, due to the dynamic character of the business environments, there would be disruptions such as harbour strike, storms, which would affect the planning. Efficient crisis management for them is a must to solve the problem in a short time and be sure that the planning schedule would not be affected too much. One of the solutions SM are using is to arrange meetings with LSP. The objective of these arrangements is to analyse the new situation and agree upon the standards and performance expectations based on new conditions in an open, straightforward and easily understood way to finally reach the target. 4.3 Situation of Deviation Figure 4.5 shows the average SLT deviation of all spare parts for Volvo truck during the period of 2017 and 2018. The negative value represents the length of early dispatched orders in week (s) while the positive value represents the late ones. As the figure shows, there is one fluctuation in performance happened at the end of 2017, where large deviation occurred. The reason for this fluctuation is because this period corresponds to the Christmas break when the material suppliers cease production and operation. Otherwise, the delivery precision for truck spare parts has no seasonal trend. -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 20 17 03 20 17 08 20 17 13 20 17 18 20 17 23 20 17 28 20 17 33 20 17 38 20 17 43 20 17 48 20 18 01 20 18 06 20 18 11 20 18 16 20 18 21 20 18 26 20 18 31 20 18 36 20 18 41 20 18 46 Average Supplier Lead Time Deviation Total Figure 4.5: Average SLT deviation deviation for 2017-2018 The goal of delivery precision for material suppliers in Volvo is 95%, that contains all the dispatches not being late (including early and on time). Figure 4.6 shows that for the past two years, this actual delivery precision of not being late is 86%. Besides, among this 86%, up to 9% of the order dispatched earlier than scheduled. There is a significant gap between the goal and current deviation of both late and early delivery. 39 4. Results: Business Understanding 9% 14% 77% Delivery Precision of Material Suppliers Early Late On Time Figure 4.6: Delivery precision of material suppliers for 2017 and 2018 The goal of delivery precision for LSP in Volvo is 97%. However, for the trans- portation of the spare parts to Ghent CDC for past rolling one year, only 90% of them was not delivered late as Figure 4.7 shows. Further, 27% out of 90% actually delivered earlier than expected. The deviation of transportation is even larger than the previous delivery performance of material suppliers. 27% 10% 63% Delivery Precision of LSP EARLY LATE ON TIME Figure 4.7: Delivery precision of LSP for past one year from 2019 4.4 Impacts of Lead Time Deviation The deviation of lead time could bring various side effects and deteriorate the com- pany’s performance. These potential effects can be closely examined when the de- viation occurs in material suppliers and LSP in terms of late and early delivery respectively. When the spare parts cannot be dispatched on time according to the schedule from material suppliers, the immediate consequence could be the waste of transportation when LSP go to material suppliers based on TB information but end up failing to pick up the requested order. Even if the material suppliers communicate well about the delay information and change the new transport booking, the parts still arrive late at CDC Ghent. This could result in loss of availability when there is a demand for those parts, which means the company will fail to deliver what is requested due to lack of inventory. Likewise, the late delivery of LSP directly affects the stock 40 4. Results: Business Understanding level in CDC, and could further impact availability of stock possibly. This conse- quence could also cascade till the rest of the supply chain including the availability in regional distribution center (RDC) and dealers. Finally, it impacts customer sat- isfaction. In order to maintain the availability of spare parts, the cost is to adopt rush transportation such as air which is bringing in the high cost of transporting freight. The cost of rush air is huge for Volvo SML. The spare parts could also be dispatched earlier than ordered from material suppli- ers. This is because on some occasion when they finish producing the orders earl