Forecasting Initial Phase Spare Part Demand Using Installed Base Data A Case Study at Volvo Cars Master’s thesis in Supply Chain Management AXEL LUNDH MARTIN MARKLUND DEPARTMENT OF TECHNOLOGY MANAGEMENT AND ECONOMICS DIVISION OF SUPPLY AND OPERATIONS MANAGEMENT CHALMERS UNIVERSITY OF TECHNOLOGY Gothenburg, Sweden 2020 www.chalmers.se Report No. E2020:080 Master’s thesis NO. E2020:080 Forecasting Initial Phase Spare Part Demand Using Installed Base Data A Case Study at Volvo Cars AXEL LUNDH MARTIN MARKLUND DF Department of Technology Management and Economics Division of Supply and Operations Management Chalmers University of Technology Gothenburg, Sweden 2020 Forecasting Initial Phase Spare Part Demand Using Installed Base Data A Case Study of Volvo Cars AXEL LUNDH, MARTIN MARKLUND Examiner: Patrik Jonsson, Chalmers University of Technology Supervisors: Patrik Jonsson, Chalmers University of Technology Rickard Holm, Volvo Cars Advisors: Niklas Österlund, Volvo Cars Claes Reje, Volvo Cars Joakim Andersson, Chalmers University of Technology Master’s thesis NO. E2020:080 Department of Technology Management and Economics Division of Supply and Operations Management Chalmers University of Technology SE-412 96 Gothenburg Telephone +46 (0)31-772 1000 Gothenburg, Sweden © Axel Lundh, Martin Marklund, 2020. iii Abstract The often inconsistent and irregular demand patterns of spare parts make them hard to forecast using traditional time series methods. This is especially true throughout the initial phase following market introduction, when no sales data exists. The advances in Big Data has given rise to a way of combating these difficulties; installed base data, which is mainly derived from the population and age of products on the market. Together with the development of new machine learning algorithms doors are opening for demand planners to utilize previously untouched data to improve forecast accuracy. In this thesis, a machine learning model is developed to utilize installed base data for initial phase spare part forecasting to test, analyze and evaluate its performance, compared to a traditional exponential smoothing model. The spare parts involved are categorized according to their demand characteristics, and model performance for each demand patterns. This exploratory case study is performed in collaboration with the Demand Planning & Optimization team at Volvo Cars. The findings of this thesis clearly show the potential of utilizing installed base data when performing forecasts in the early phase of spare part life-cycles. It is also shown that a machine learning model using installed base data can provide improved results when predicting the first sale of a given part, when no previous sales data exists. The decision to stock or not is crucial for many spare parts, and the classification analysis performed by this model significantly outperforms Volvo’s results for a comparable period. Further, the combination of an installed base model with a traditional time series model shows significant promise, outperforming both pure models in all stages of the initial phase. This is especially true for parts with high demand variability. Although the data used in this study is limited to that collected from Volvo Cars, the challenges faced and practices implemented by the case company are likely similar to those of many other manufacturing companies, and the authors argue for the transferability of its findings. Further, the methodology and analysis performed should be applicable to similar studies and projects elsewhere. This study presents a way of producing demand forecasts before any sales data is available which outperforms current judgmental practices. The combination of the two models performs impressively, and suggests the combination of time series forecasting with machine learning models holds promise for both Volvo Cars and other similar companies. The main contributions of this thesis lie in the production of demand forecasts without sales data, as well as the shown potential in combining explicit installed base models with more traditional time-series forecasting. Keywords: Big Data, Spare Parts, Initial Phase, Spare Part Life-cycle, Demand Forecasting, Installed Base Data iv Acknowledgements The greatest lesson in life is to know that even fools are right sometimes. Winston Churchill As Winston Churchill said, even a fool can be right sometimes. We started this project as fools ourselves, during the spring semester 2020. Luckily we had brave, steadfast and competent advice along the way. In a funny way, this also relates to forecasting, where a lucky guess may be right once or twice, but the name of the game is being right most of time. First off, we would like to thank our solid supervisor at Chalmers, Professor Patrik Jonsson, for sharing his vast knowledge of the domain of forecasting and supply chain experience. His guidance helped keep us on the right track during some trying months, and to course-correct when we were veering off the path. You helped remind and inform us of key features of the study, and this aided us immensely. Another massive thank you goes out to Joakim Andersson, Industrial PhD Student at Chalmers, for taking an interest in our project, despite not being under any obligation to do so. The time and effort you spent on us is admirable, and we are very happy for it! Your academic insights, advice on sources and methodology as well as experience was a great support to lean on throughout the project. Further, a shout out to Rickard Holm, our supervisor at Volvo Cars and in charge of the DP&O team. We have felt and received your support from the very start, even when you’ve had your plate full with other stuff. Thanks for being a kind, helpful and sympathetic person, always keen to assist. We still hope to test-drive new Volvo’s with you in the future! Lastly, Niklas Österlund and Claes Reje... Where do we even begin? Thanks to the both of you for being our lovely and slightly nerdy advisors at Volvo! Our frequent contact has been vital, and your willingness to go out of your way to be part of the project has been much appreciated. Thank you for lending your expertise, hosting table tennis tournaments during lunch breaks, showing off impressive medieval weaponry and also emphasizing the importance of Sabaton for correct motivation! Hopefully our work can aid you in the future. All that said, we’d also like to extend our gratitude to friends, family and other dearly beloved for, in one way or another, taking part in our journey. Axel Lundh & Martin Marklund Gothenburg, June, 2020 v Glossary Big Data A term describing the increasing volume and variety of available data mainly due to digitalization. CatBoost An advanced open-source gradient boosting decision tree algorithm. It can be used for Classification and Regression problems. Compared to Random Forrest it provides faster training time, higher accuracy and a number of aids to simplify its implementation. Causal-based Forecasting Forecasting that uses other data than historic sales to predict future demand. Classification The process of classifying something based on its features. In machine learning a classification problem is the process of classifying a set of independent variables to a set of pre-defined categories. Combined Model A model evaluated in this thesis. The model implements the CatBoost algorithm and takes spare part features, installed base data and the outcome of the exponential smoothing model as its inputs and use these to predict future demand. Confusion Matrix A way to visualize and evaluate the performance of a binary classification algorithm. The matrix shows the four possible outcomes of a binary prediction: 1) The actual outcome is YES and the model prediction is NO, 2) The actual outcome is YES and the model prediction is YES, 3) The actual outcome is NO and the model prediction is YES, 4) The actual outcome is NO and the model prediction is NO. This allows evaluation of the model’s performance on both true positives and true negatives. Cross Validation A robust way to reduce overfitting in an algorithm. Instead of splitting data in a training and test set just once, Cross Validation does this multiple times. The number of iterations is often denoted ’K’-times. The model with the highest performance is deemed to be the one with the highest average performance over the K iterations. Demand Pattern How the demand for a particular product behaves. Generally defined by the mean time between demand occurrences and the variability of demand size. Dependent Variable The variable that an algorithm tries to predict with by using independent variables. This is generally the prediction output of an algorithm. DP&O The Demand Planning & Optimization team at Volvo Cars. Exponential Smoothing Model A model evaluated in this thesis. The model applies an exponential smoothing Time Series forecasting method to historic sales data. vii Glossary Feature Another name for independent variable. In this thesis ’Feature’ refers to the inherent attributes exhibited by individual spare parts such as what function group it belongs to or its weight. Grid Search A method to aid in parameter tuning. The method takes a set of values for a various number of parameters, a so-called grid. The model is then trained with each possible combination of these parameter values in order to find the combination with the best performance. Independent Variable A variable used to describe another. Independent variables are used as input for many Classification and Regression Machine Learning algorithms. Initial Phase The initial part of a product’s life-cycle. In this thesis defined as the first 24 months after market introduction. Installed Base Data Information about the units of a product currently utilized on the market. Installed Base Model A model evaluated in this thesis. This model implements the CatBoost algorithm and takes spare part features and installed base data as its independent variables. Machine Learning Advanced computer algorithms that take data as input in order to learn and improve, while finding patterns and predicting outcomes when presented with new sets of data. OEM Original Equipment Manufacturer. Open-Source Computer software openly available and freely distributed, often with several individuals and organizations cooperating and contributing to its development. Overfitting An issue where a machine learning algorithm becomes too fitted to the data it is trained on and performs worse when presented with previously unseen data. Parameter Tuning The process of changing and evaluating parameter values in order to increase the predictive power of an algorithm. Parameters A set of values which modify the behavior of an algorithm. The values impact algorithm performance. Planning Frequency Determines the time between which forecasts are updated. Planning Horizon The time covered by the current predictions of future demand. Python A computer programming language commonly used in data science and many other fields. viii Glossary Random Forest A kind of machine learning algorithm which can be used for Classification and Regression problems. Regression A regression problem tries to predict a quantity or other outcome based on a set of independent variables. SCM Supply Chain Management. SKU Stock Keeping Unit, a unit for which a stock record is kept. Test Data Data used to evaluate a machine learning algorithm previously trained on training data. This is done to ensure the algorithm has not only learned to make predictions about the data it has previously been trained on. Time-Series Forecasting Forecasting based on how demand varies over time. Generally uses historic sales data as its input. Training Data Data used to train a machine learning algorithm. The algorithm tries to find patterns in the data and in the case of Classification or Regression the algorithm searches for the relation between the independent variables and the dependent one. ix Contents 1 Introduction 1 1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.1 Automotive Spare Part Demand . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.2 Installed Base Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1.3 Demand Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1.4 Case Company . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1.5 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.2 Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.3 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2 Theory 9 2.1 The Automotive Aftermarket . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.1.1 Spare Part Demand Drivers . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.1.2 Product Life-Cycles and the Installed Base . . . . . . . . . . . . . . . . . 10 2.1.3 Demand Patterns and Item Categorization . . . . . . . . . . . . . . . . . 11 2.2 Forecasting Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.2.1 Time-Series Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.2.2 Causal-based Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.2.3 Forecasting Methods Utilizing Installed Base Data . . . . . . . . . . . . . 16 2.2.4 Forecasting Errors and Accuracy Measures . . . . . . . . . . . . . . . . . 16 2.3 Big Data Analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.3.1 Dependent and Independent Variables . . . . . . . . . . . . . . . . . . . 18 2.3.2 Training Data, Test Data and Overfitting . . . . . . . . . . . . . . . . . . 18 2.3.3 Random Forest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.4 Conceptual Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3 Methodology 21 3.1 Research design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.2 Workflow and Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.2.1 Phase 1: Orientation and Data Collection . . . . . . . . . . . . . . . . . . 21 3.2.2 Phase 2: Data Analysis and Evaluation . . . . . . . . . . . . . . . . . . . 21 3.2.3 Phase 3: Results and Findings . . . . . . . . . . . . . . . . . . . . . . . . 22 3.3 Choice of Accuracy Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.4 Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.4.1 Quantitative Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.4.2 Qualitative Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.5 Data Pre-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.6 Validity and Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.7 Overview of Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 x Contents 4 Analysis 29 4.1 Planning Horizons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 4.2 Forecasting Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.2.1 The Installed Base Model . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.2.2 The Exponential Smoothing Model . . . . . . . . . . . . . . . . . . . . . 32 4.2.3 The Combined Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 4.3 Forecast Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 4.3.1 Stage 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 4.3.2 Stage 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 4.3.3 Stage 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 4.4 Feature Importance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 4.5 Demand Pattern Categorization . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 4.6 Model Performance per Demand Pattern . . . . . . . . . . . . . . . . . . . . . . 38 4.6.1 Lumpy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 4.6.2 Smooth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 4.6.3 Intermittent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 4.6.4 Erratic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 5 Discussion 41 5.1 The Implications of Model Performance . . . . . . . . . . . . . . . . . . . . . . . 41 5.1.1 Stage 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 5.1.2 Stage 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 5.1.3 Stage 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 5.2 The Impact of Model Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 5.3 The Imposition of Demand Pattern Categorization . . . . . . . . . . . . . . . . 46 6 Conclusion 47 6.1 Recommendation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 6.2 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 6.3 Future Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 Bibliography 51 A Appendix I A.1 Interview Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I A.2 CatBoost Classifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . III A.3 GridSearch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV A.4 Implementation of Exponential Smoothing . . . . . . . . . . . . . . . . . . . . . V A.5 Feature importance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VI xi 1 | Introduction This thesis is performed in collaboration with Volvo Cars Group, and their Demand Planning & Optimization (DP&O) team within the department of Supply Chain Management (SCM). This team in particular is tasked with demand planning for Volvo’s global aftermarket. In order to carry out their objective they require accurate forecasts of future demand for spare parts. The demand for spare parts has unique characteristics which make inventory management challenging (Bacchetti and Saccani 2012). There are thus financial and customer service interests in evaluating and implementing more accurate forecasts during the initial two years of a spare part’s life-cycle. One opportunity is to use installed base data, the effectiveness of which is examined in this thesis. 1.1 Background This section provides a background to the thesis, as well as presenting an introduction to the topics covered, aspects relevant to forecasting using installed base data and the literature laying the foundation for the thesis in general. This is followed by a presentation of the case company in relation to spare part forecasting. 1.1.1 Automotive Spare Part Demand The aftermarket for a product is the potential extended customer-buyer relationship after the initial sale of a product. According to Cohen and Agrawal (2006), the aftermarket includes services such as maintenance and repairs as well as the sale and provision of spare parts. Further, Cohen and Agrawal (2006) explain the ongoing shift towards a service centered economy and the aftermarket’s role as an important part of this transition for traditional manufacturing companies, such as automotive manufacturers. Koudal (2006) estimates that for producing companies, service offerings can represent as much as 25% of total revenue and have a 75% higher profitability than the sale of finished products. The life-cycles of physical products can, according to Dekker et al. (2013), be divided into three phases: initial, mature and end-of-life. The demand for spare parts generally follows the demand for their associated product rather closely, albeit with a slight time-delay. This time-lag has been shown by and dubbed the ’Life Cycle Mismatch’ by Solomon et al. (2000). The demand patterns throughout product life-cycles obviously vary depending on the type of part, but in general this correlation can be said to hold true (Solomon et al. 2000). However, Dekker et al. (2013) argue that the size and age of the installed base and the distribution of related spare parts provides a more comprehensive explanation of spare part demand than relying on demand derived from the life-cycle phase of the product itself. Installed base information is during the initial phase, also known as ’design/purchase’, mainly available from design specifications on expected product and spare part life and the initial purchase data as well as projected sales (Dekker et al. 2013). Combined with the relatively small size of said installed base and the lack of actual sales history, 1 1. Introduction this makes the initial phase especially challenging in terms of forecasting spare part demand. Effective utilization of the installed base data may still be used to improve demand forecasting in the initial phase (Dekker et al. 2013), and is discussed closer in the following section. In addition to the issues mentioned above, Van der Auweraer et al. (2019) state that the majority of spare parts exhibit intermittent demand patterns. This means the demand for said parts is characterized by sequences of zero demand periods interspersed with non-zero demands. These demand patterns are known to make the approximation of demand difficult using traditional forecasting methods (Boylan et al. 2008). Further, when demand does occur the size of demand may vary widely. Such patterns are called erratic, while patterns which are both erratic and intermittent are called lumpy (Boylan et al. 2008). All these patterns pose challenges when estimating demand for spare parts compared to the smoother, more stable demands of finished products (Van der Auweraer et al. 2019). Several methods and forecasting techniques have been proposed to forecast intermittent spare part demand, and Boylan et al. (2008) suggest classifying parts according to their individual demand patterns and applying the appropriate methods separately. 1.1.2 Installed Base Data The digitalization of industries has given way to what is today referred to as Big Data. McAfee and Brynjolfsson (2012) explain Big Data with the help of three V’s, Volume, Variety and Velocity. According to Brinch (2018) this general definition of big data also holds true for Supply Chain Management. Today, companies have access to and have started collecting an ever-growing set of data from an increasing number of sources and at and increased speed, often close to real time. A major enabler for this kind of data is the increased connectivity of products which allow real-time and continuous reporting of various data. In the case of the automotive industry, the increased number of internet-enabled vehicles has created the opportunity to collect large amounts of information from and about each individual vehicle. This includes data from various sensors in the car, mileage, fault codes as well as geo-location (Andersson 2019). An information source emerging along with and as part of Big Data is installed base data. The installed base is, in short, the number of sold products that can generate spare part demand (Dekker et al. 2013). Installed base data has potential to be used for spare part demand forecasting, since the demand for said parts is highly dependent on the age and size of the product population and originates from the replacement of existing parts either from preventative or corrective maintenance (Van der Auweraer et al. 2019). Van der Auweraer et al. (2019) summarize previous research in the field, concluding that correct knowledge of the characteristics of these products (such as age and and usage) is highly important for correct inventory levels and can lead to stock reductions of up to 25%. Van der Auweraer et al. (2019) define installed base information as data derived from three main sources: i) the size and status of the spare part itself, ii) the chosen maintenance policy, and iii) environmental factors impacting part reliability. Further, Van der Auweraer et al. (2019) summarize other findings from existing literature. An important aspect for forecasting in general, and the aim of this thesis in particular, is the connection between the size and status of the installed base and the replacement probability of the included parts to the actual demand for parts. This is also related to the life-cycle of the product. As mentioned above, demand for spare parts is related to the life-cycle of a product 2 1. Introduction but installed base data might be more useful for demand forecasting, especially in the initial phase of said life-cycle (Dekker et al. 2013). 1.1.3 Demand Forecasting Forecasts are used on various levels of an organization in order to plan production and product availability more efficiently (Sanders and Ritzman 2004). More accurate forecasts allow for higher levels of service and in the case of products a lower need for stock keeping. According to Jonsson and Mattson (2009), forecasting methods can be divided into two major groups: qualitative and quantitative. Qualitative forecasts are based on human decisions and experience. Quantitative forecasts are instead based on data, generally sales history and past forecasts. Intrinsic quantitative methods, generally used for time series forecasting, look at data of past sales and try to predict future demand. Extrinsic quantitative methods instead utilize additional data besides historical sales to produce forecasts. Sanders and Ritzman (2004) explains that intrinsic methods are a good fit when demand is relatively stable, historical sales data is available and there is a need to generate forecasts for large amount of products. A major drawback of these methods is the need for historical data which make the forecasts hard to apply in situations with little historical data such as during the initial phase of a product or in situations with volatile and intermittent demand (Jonsson and Mattson 2009). Extrinsic methods on the other hand have little or no need for historical data (Jonsson and Mattson 2009). Traditionally, access to and availability of alternative data has been limited but advances in analytic technology and the trend of Big Data opens up for new possibilities for extrinsic methods in demand forecasting (Andersson 2019). Relevant for this thesis are the possibilities to use extrinsic predictive models on data from the existing and near-future installed base to improve forecast accuracy. This mainly means the size and age of cars and their components sold, combined with numbers of sold spare parts to establish a reliability-based forecasting model. 1.1.4 Case Company Volvo Car Group is a global automotive manufacturer and traces its roots back to 1927, with the founding of AB Volvo in Gothenburg, Sweden. In 1999, Volvo Cars was split from Volvo Group and sold to Ford Motor Company. Today, the company is owned by the Chinese company Zhejiang Geely Holding Group. Volvo Car Group also includes the electric performance car brand Polestar as well as stakes in the Chinese car brand Lynk & Co and Zenuity, a software company. In 2018, Volvo Cars sold 642,000 cars, had 43,000 employees on average and worked with approximately 2,300 dealers globally, with a net revenue of around 250 Billion SEK (Volvo Cars 2018). More specifically, the thesis has been written in collaboration with the department for Demand Planning & Sourcing Optimization. This department focuses largely on the optimization and planning of global spare part logistics between suppliers, warehouses and dealers, assuring satisfactory customer service and spare part availability. DP& O have a heavy operational emphasis on the initial phase of the launch of a new automotive model and thereby its associated spare parts. The in-use fleet of Volvo Cars consists of around 6.5 Million cars, where 4 Million were produced and sold within the last 10 years (Volvo Cars 2018). Supporting this fleet, Volvo 3 1. Introduction provides a range of more than 130,000 spare parts, supplied through 54 warehouses of which 21 are Local Distribution Centers (LDCs). The after market and spare part business has an annual net revenue of 20 Billion SEK, made up of approximately 85% parts and 15% accessories. The service profits are Volvo’s largest EBIT factor, and contributes to 60-70% of total dealer profit (Volvo Cars 2018). Today, Volvo Car Group guarantees the availability of spare parts for up to 15 years after the car model goes out of production. With an average lifespan for most car models of 6-7 years, this means that spare part availability is required for up to 22 years after the car model launches. For Volvo DP& O, the initial phase is mainly considered to be the first two years after the launch of a product. This phase is where they experience most service and availability issues, due to the lack of sales data to base forecasting on. This phase is important for Volvo, since customers are well within their service guarantees and almost exclusively use Volvo’s dealers and spare parts. Expectations on fast and effective service are also high1. The mature phase concerns the period after the first two years until the car model goes out of production. According to the above, this generally means years 2-7. During this phase, forecast accuracy is generally considered acceptable. The large installed base and existing sales history means that the historic and time-series based forecast are relatively accurate. Lastly, the end-of-life phase starts when the model goes out of production and lasts until the guarantee of spare part availability expires, meaning years 7-22. At this stage in the life-cycle, inventories are usually relatively well-established and the available data is considerable. However, customers generally start shifting from OEM parts to third-party dealers, while also repairing and servicing their car less frequently since the service guarantee offered by Volvo on new cars has expired. A further challenge during this stage is that suppliers of parts to Volvo may wish to discontinue production and/or charge higher prices due to less demand and worsened production economies. Lastly, since spare part demand is difficult to predict at the very tail-end of the cars’ life-cycle, there are risks for Volvo and their dealers of being stuck with obsolete stock and associated costs2. The empirical focus of this thesis is Volvo’s US market. The US market is an established and large market for Volvo Cars. In 2018, it was the company’s third largest market in terms of retail sales, making up 15% of the companies total sales. The automotive market in the US experienced a small growth of 0.3%, while Volvo Cars recorded a much larger growth of 21%. In 2018, Volvo also opened their first car manufacturing plant in the US, located in Charleston, South Carolina (Volvo Cars 2018). Regarding the aftermarket set-up in the USA, Volvo has five distribution centers throughout the country, see Figure 1.1. Each dealer is assigned to the DC within the closest geographical proximity, the dealer-facing DC, and is supplied from there. When a dealer requests a part, the request is first sent to this DC. However, all DC’s do not permanently stock all spare parts. This might be for financial reasons, or physical space limitations. So, if the closest DC does not have the requested part, the request is sent on to one of the other four, creating a so-called ’referral’. Given the size of the US market, the distances to the DC stocking the part may be longer than what can be covered by truck within the required lead time. In these cases, the part 1Personal Communication, Rickard Holm Volvo Car Group 2Personal Communication, Rickard Holm Volvo Car Group 4 1. Introduction is sent by express flight to meet the service requirement. This drives costs, and is a challenge Volvo is facing that is likely to increase along with the projected growth of the installed base3. Figure 1.1: Overview of Volvo’s US distribution centers and part shipments. 1.1.5 Problem Internal metrics at Volvo DP&O show a lack of forecast accuracy during the initial phase of new car models and the associated spare parts, leading to availability issues and expensive express shipments. Studies confirm that this is a common challenge in the spare parts market (Van der Auweraer et al. 2019), since the installed base is generally small and sales data limited. Traditional time series forecasting and intrinsic methods struggle to provide reliable forecasts when demand is slow or intermittent, demand patterns are shifting throughout product life-cycles and short delivery times from stock are important, all of which are common attributes for spare parts (Dekker et al. 2013). As mentioned previously, these issues are exacerbated in the initial phase of a products life-cycle, and utilizing installed base data is a potential way of improving forecast accuracy and mitigating the effects of having little or no historic sales data. In the specific case of Volvo Cars, the lack of historical sales data renders the company’s current forecasting methods ineffective during this period. The DP&O team is thus investigating ways to improve their forecasting methods, using both new models and new data sources. The company has access to increased amounts of installed base data, in terms of cars sold, planned and spare part usage, especially from the US market. This opens up possibilities to use this installed base data with new forecasting models. As Dekker et al. (2013) show, there are significant savings to be made in inventory and obsolescence costs by collecting and utilizing installed base data. Thus, the potential to use installed base data in spare part forecasting is relevant not only 3Personal Communication, Rickard Holm Volvo Car Group 5 1. Introduction to Volvo Cars, but to producers and providers of spare parts generally, who often struggle to maintain high part availability while balancing stock-out and obsolescence costs. 1.2 Aim The aim of this thesis is to examine the possibilities of using installed base data in the forecasting of spare part demand. This means developing a forecasting model using methods from forecasting literature, and evaluating it using actual demand and sales data from Volvo Cars. The project will be performed in collaboration with the DP&O team at the Supply Chain Management division at Volvo Cars, and data from their American market will be used as a case study to develop, test and evaluate findings. The targeted life-cycle phase is the initial launch phase of new spare parts, often correlating with the phase-in period for new car models. For the purposes of this report, and the organizational structure of the case company, this is considered to be during months 0-24 (year 0-2) after the launch of a new spare part. The initial phase is further divided into three stages; (i) before any sale of the part has occurred, (ii) the part has sold in three demand periods and (iii) after the end of stage (ii) until the part has been on the market for 24 months. These stages have different demand characteristics and challenges, and will all be treated in this thesis. See Table 1.1 for details on each stage. Stage Description Planning Horizon Problem What to predict 1 No Sales 6 m Classification (Yes or No) Sale in 6 m 2 First sale - 3 m sales 3 m Regression Demand next 3 m 3 3 m with sales - 24 m 1 m Regression Demand next 1 m Table 1.1: Overview of the three planning stages during initial phase, the period from market introduction until 24 months (m) after market introduction. During the analysis, a model will be constructed based on methods and theories from forecasting literature, as well as practical experience from the domain experts at Volvo Cars. This model will be designed with installed base data in mind, and its theoretical suitability to the data provided by Volvo. The model will be iterated upon and developed using more data points, parameters and aspects from other relevant forecasting methods as the analysis progresses. The analysis is performed in order to evaluate the forecast accuracy of the model, and will be discussed and evaluated through collaboration with said domain experts, as well as forecasting accuracy measures. In addition, the models performance on spare parts with various demand patterns is evaluated. This involves categorizing parts according to their demand patterns, and testing the developed model for each category. Lastly, the final stage of the thesis is evaluation of the results and potential impact. This results in a discussion of its usefulness and applicability to situations similar to that of Volvo Cars. In addition to the final forecasting model and its application to the selected group of parts, this 6 1. Introduction includes recommendations to the case company in terms of how to approach and conduct initial phase forecasting in the future, using this or similar models. In conclusion, the research questions to be analyzed and answered in this thesis are the following: RQ1: What forecasting model can be applied to effectively predict initial phase spare part demand for Volvo Cars using installed base data? RQ2: How would the applied model impact forecast accuracy for products with different demand patterns? Research Question 1 is answered through a combination of literature review on the nature of spare part demand and relevant forecasting models and the development and evaluation of a model with potential to outperform traditional methods. It also serves to examine the usefulness of installed base data for initial phase forecasting, while showing its theoretical and practical implementation. Research Question 2 is mainly an extension of the forecasting models used to answer RQ1, with an added emphasis on the demand patterns of the spare parts involved and the impact said patterns have on model performance. This is to provide a framework and context within which the performance of the developed model and similar forecasting methods can be evaluated and practically implemented. The results of the analysis are discussed in Chapter 5, Discussion, from various perspectives. The paper concludes with Chapter 6.1, Recommendation, and Chapter 6.3, Future Studies, where the implications of the study are summarized, both in regards to Volvo Cars and the field more generally. 7 1. Introduction 1.3 Scope The scope of this thesis has been set to facilitate the answering of the research questions, considering the current organization and situation of Volvo Cars. The thesis has been written with the DP&O team, and the knowledge and expertise within that team has been the source of qualitative data. Other actors within the SCM department have also been consulted, but to a lesser extent. To keep the analysis focused on the issue at hand, quantitative data has only been collected through the DP&O team. No data from external sources has been collected for the quantitative analysis. In order to achieve applicability to other contexts with similar challenges, extensive theory and literature on spare parts has been studied, mainly in terms of demand planning and forecasting as well as aspects of spare part demand that complicate the issue. This means that the thesis focuses on the life-cycles of spare parts, mainly the initial phase, and the various demand patterns they exhibit. This has been examined and connected to the development of the installed base over time, in order to both achieve practical use in the case of Volvo Cars and allow for transferability to other situations in similar contexts. To facilitate the thesis’ main focus on the development and analysis of an initial phase forecasting model, both installed base and sales data has been collected from Volvo’s US market. This data was both plentiful and easily accessible through the DP&O team, and meant time and resources otherwise spent on the gathering and comparing of multi-national and varied forms of data could be used to further the aims of the thesis. Lastly, during the writing of this thesis the world suffered through the COVID-19 pandemic, and Volvo and many other companies limited their staff’s working hours and disallowed meetings in person. This limited the possibilities of conducting interviews with various parties, which meant that qualitative aspects from other business areas which where initially considered were excluded from the scope of the thesis. 8 2 | Theory In the following sections, existing literature concerning the topics of the spare part market and forecasting methods are presented. 2.1 The Automotive Aftermarket In the automotive industry, spare parts and accessories, make up a vast part of the revenue associated with a certain car model (Souza et al. 2011). Furthermore, after the initial sale, the cars’ long term reliability and life-time is dependant on replacement and maintenance of its component parts. The aftermarket and sales of spare parts is thus a crucial part of an automotive manufacturers business and profitability, as well as customer satisfaction and value proposition (Khajavi et al. 2014). McKinsey&Company (2017) estimate that the aftermarket generated around 20% of the total revenues of the automotive industry in 2015, which meant approximately $ 760 billion. Service and maintenance makes up 45% of that number, while the remaining 55% is constituted of the retail and wholesale of vehicle parts (McKinsey&Company 2017). In general, the aftermarket can be divided between the networks surrounding and controlled by the OEM’s, such as Volvo, and independent actors. As mentioned above, the market is very lucrative for most actor’s, but in recent years several OEM’s have strived to increase their influ- ence and market share through various service programs as well as utilizing their greater network power, access to end customers, and data derived from the cars’ usage (McKinsey&Company 2017). Customers tend to be especially loyal to the OEM during the early phases of a cars life-cycle, while the warranty is still valid and the value of the car remains high1. Leveraging these inherent strengths successfully will be necessary in a market with stiff competition and several potential disruptive factors on the horizon. One way of gaining or increasing one’s advantage is improved forecasting, making sure that related costs are kept low and service levels high. 2.1.1 Spare Part Demand Drivers The need to replace a specific part could come from a number of factors. In their work on maintenance scheduling, Yang et al. (2008) identify and explore four different maintenance strategies; Preventive, Predictive, Condition-based and Corrective maintenance. These strategies are also considered the main underlying demand drivers for spare parts, as stated by Andersson (2019). The first three approaches are all different variations of pro-active and preventive maintenance, while corrective maintenance is reactive and applied after part failures (Yang et al. 2008). 1Personal Communication, Rickard Holm Volvo Car Group 9 2. Theory Preventive, or time-based,maintenance is a traditional and commonly used maintenance approach usually based on various service intervals, such as mileage or time. The aim of such policies is to reduce the need for corrective maintenance due to breakdowns, by following regular service plans and pre-empting part failures (Kennedy et al. 2002). In terms of forecasting, preventive maintenance theoretically makes the planning of item demand easier since the maintenance is regularly scheduled and also allows for the logging and utilization of part usage data. It does however require high degrees of customer loyalty and adherence to the service plan in order to be effective, and can also result in increased maintenance costs due to over maintenance and pre-mature replacement of parts (Andersson 2019). An extension of the of the preventive maintenance explained above is called predictive mainte- nance, and utilizes on-board sensor data of monitored parts to predict part failures in advance. Forecasting based on such data will be discussed later in this section, but basically this approach replaces static service intervals with dynamic maintenance schedules based on the monitored status of involved parts (Yang et al. 2008). Condition-based maintenance, CBM, is a similar approach to predictive maintenance, but utilizes real-time data from the on-board sensors. Andersson (2019) mentions methods utilizing CBM to optimize the total cost of maintenance, balancing costs of under- and over-maintenance. Such CBM approaches make use of data such as performance and physical degradation, vibrations and other usage-based factors. This approach is the analytically most advanced of these maintenance methods, and is becoming increasingly feasible in practice with technological advances (Fritzsche et al. 2014). Lastly, corrective maintenance is usually defined as an operation done after the part has broken down or failed. As such, it is obviously a reactive approach compared to the pro-activeness of those above mentioned. This approach focused on repairing malfunctions usually leads to higher costs and longer down-times than preventive methods, and it is generally recommended to move towards said preventive and predictive approaches (Andersson 2019). 2.1.2 Product Life-Cycles and the Installed Base The classic product life-cycle, as presented by Levitt (1965) consists of four steps and describes how demand and sales volume change over time for a product, from market introduction until it is phased out and no further sales are made. The first stage of a product’s life-cycle, according to Levitt (1965), is the The Development Stage. This stage is characterised by lack of information and low market knowledge. Since the market has not been exposed to the product before it is hard to estimate the market’s reaction and associated sales volumes. The introduction of new products is generally seen as risky, and failed introductions are often costly (Levitt 1965). Should a product introduction be successful, this stage is followed by the Growth Phase. During this phase sales ramp up, competitors may take notice and try to copy the new products and market awareness of the product increases (Levitt 1965). Eventually, the growth rate of sales starts to flatten out and the product reaches the Maturity Stage. During this stage, the product has generally fulfilled its market potential and substitutes start to catch up and price-based competition increases (Levitt 1965). The fourth and final stage is Market Decline, which the product enters when sales start to slow down and decline. During this stage, customers move away from the product, often looking for and finding substitutes or other replacing products. 10 2. Theory Lastly, the product reaches its end-of-life and is discontinued (Levitt 1965). According to Dekker et al. (2013), the installed base of a product closely follows the product’s life-cycle, as described above. Further, the authors state that demand for spare parts is connected to sales of the actual product and growth of the installed base. Generally, demand for spare parts will follow product demand, albeit with a slight time delay. This ’Life-cycle Mismatch’ as presented by Solomon et al. (2000) is the time and volume gap between product and spare part demand. Dekker et al. (2013) expand on this concept and the product life-cycles explained by Levitt (1965) to present a modified version of the product life-cycle. This was mentioned in Section 1.1.1, and focuses on the size of the installed base over time and corresponding spare part demand. Their visualization is shown, in Figure 2.1. In this case, the Initial phase can be said to replace the first two stages of Levitt (1965)’s model, Market Development and Growth. The Mature Phase is generally the same in both models, while Market Decline is replaced by the End-of-life Phase. Based on this simplified model, Dekker et al. (2013) argue and show that the size and age of the installed base provides a comprehensive explanation of spare part demand, and emphasize the potential to improve demand planning if these factors can be included in spare part forecasting measures. Time Installed base size Spare Part demand New product sales Qty Initial Phase Mature Phase End of Life Figure 2.1: Graphical representation of the installed base size, new product sales and spare part demand across the Product Life-cycle, adapted from Dekker et al. (2013). The initial phase is highlighted in grey. 2.1.3 Demand Patterns and Item Categorization As mentioned in the introduction, there are several factors and characteristics that make demand for spare parts different compared to that of the original products. Bacchetti and Saccani (2012) mention several such aspects that combine to complicate the forecasting of said demand, the presence of so-called lumpy and/or intermittent demand being one of them. The authors go on to discuss and highlight the importance of item categorization, through either quantitative or qualitative approaches, in order to determine appropriate stock-keeping strategies and forecasting methods for various parts, rather than using a uniform approach for all. Commonly used methods include ABC-classification into item categories, often based on 11 2. Theory demand volume and part criticality (Bacchetti and Saccani 2012). However, the traditional ABC-approach struggles to take into account the intermittency of demand, the variations in order frequency which is common to many spare parts. Another approach that includes demand frequency in addition to demand variability is presented by Boylan et al. (2008). The authors suggest classifying the demand patterns of all products in terms of their mean inter-demand period, p, and the variability of demand sizes, measured through the squared coefficient of variation of demand size, CV 2. Mean inter-demand period is quite simply the average number of periods between each period with demand. The squared coefficient of variation of demand size is the coefficient of variation, (σ/µ)2, is the standard deviation of demand divided by the mean demand. Important for CV 2 calculations is to note that zero demand periods are ignored, in order to achieve the variation of demand when it actually occurs (Boylan and Syntetos 2007). This framework is conceptual, since high and low values are not generally quantified and thus need to be specified for the specific environment it is to be used in. The authors suggest statistical evaluation of the performance of various forecasting methods, and adjusting the cut-off values according to the superior performance of the evaluated methods. The authors do however suggest specific cut-off values, based on their comparative study of the performance of the (Croston 1972) and (Syntetos and Boylan 2005) methods. The values recommended are CV 2 = 0.49 and p = 1.32. Regardless of chosen cut-off values, they divide the classification matrix between high and low along both axes, as is shown clearly in Figure 2.2. For the purposes of this thesis, these cut-off values are used for demand pattern categorization. The adaptation of this framework and its application to the examined data sets will be discussed closer in Chapter 4, Analysis. Since infrequent demand and irregular demand sizes make normal distribution unsuitable for representing the demand pattern, Syntetos et al. (2005) present several alternative patterns which are used in the categorization mentioned above: An intermittent demand pattern means that the item shows a demand pattern with infrequent demand occurrences but with rather stable demand volumes. Intermittent demand, according to the framework, thus exhibits high mean inter-demand intervals with low variability of demand (Boylan et al. 2008). An erratic demand item is an item whose demand size is highly variable, while the inter-demand periods are rather short. Erratic is considered the opposite of intermittent, since the variability lies in the size of the demand rather than varying inter-demand periods (Boylan et al. 2008). A lumpy demand item is an intermittent item for which demand, when it occurs, is highly variable. Lumpy demand pattern exhibit the high inter-demand periods of intermittent demand, while the demand itself varies highly as in erratic demand patterns (Boylan et al. 2008). Lastly, a smooth demand pattern means a more stable demand, with low variability as well as short inter-demand periods. For items with smooth demand patterns, traditional methods such as Croston (1972) are more likely to be effective (Boylan et al. 2008). A two-dimensional framework with cut-off values for low and high then theoretically determine the appropriate forecasting model to be applied to items with each demand pattern (see Figure 2.2). As mentioned above, the framework provides a useful way of classifying spare parts 12 2. Theory Variability of demand size Mean inter- demand period Lumpy Smooth Intermittent Erratic High High Low Low p = 1.32 CV2 = 0.49 Figure 2.2: Demand-based categorization for forecasting, adapted from Boylan et al. (2008) according to their underlying demand patterns, allowing selection and analysis of the applied forecasting model on an aggregated level. 2.2 Forecasting Methods For the purposes of this report and its theoretical framework, forecasting and demand planning will mainly be described in the context of the aftermarket and spare parts. This bears obvious similarities with demand planning for manufacturing, but has several characteristics that differ. According to Cohen and Agrawal (2006), these characteristics make the aftermarket planning more complex and include more unpredictable demand, an increased number of SKUs and shorter expected response times. In the following sections, forecasting methods are presented as they relate to spare part demand with an emphasis on time-series and causal-based forecasting. According to Jonsson and Mattson (2009), forecasting is a way for companies to make assessments of future external factors not fully under their own control, such as future market conditions. The authors divide traditional forecasting methods into quantitative and qualitative methods. Quantitative methods rely on information such as time series of sales and other historical data to calculate estimated demand, while qualitative methods instead make use of the expertise, experience and subjective judgement of individuals or groups. This includes methods such as market research, focus groups and Delphi methods, which might be especially useful in combination with more quantitative approaches (Andersson 2019). Quantitative methods are the main focus of this report and are therefor elaborated upon below. 13 2. Theory 2.2.1 Time-Series Forecasting Quantitative methods use various calculations based on available data to estimate future demand. Boylan and Syntetos (2007) state that quantitative methods can be further divided into two different approaches; time-series based and causal-based. Time-based methods are entirely dependent on the history of demand to calculate future demand, while causal-based methods depend on certain explanatory variables to predict future outcomes. Such causal methods are especially useful in the initial phase of a parts life-cycle since there is no (or very little) demand history, making time-series difficult to apply successfully. Time-series methods are, on the other hand, commonly used for fast moving parts with varying demand characteristics (Boylan and Syntetos 2007). Demand characteristics will be discussed later in this chapter. Similarly, Jonsson and Mattson (2009) discuss the differences between intrinsic and extrinsic forecasting methods. Intrinsic methods exclusively analyze data inherent to the variable being forecast, while extrinsic methods attempt to create a model connecting the forecasted variable to one or more explanatory variables. Hence, intrinsic as a forecasting concept is very similar to time-series while extrinsic closely resembles causal-based methods. A commonly used quantitative and time-based method is the moving average method (Jonsson and Mattson 2009). This method calculates the demand for a given period by including the average demand from several previous periods. This evens out fluctuations due to random demand variations, smoothing out the demand curve. The formula for moving average is presented below, in Equation 2.1, as described by Jonsson and Mattson (2009). F(t + 1) = (D(t) +D(t− 1) + ...+D(t− n+ 1)) n (2.1) Where: F(t + 1): Forecast demand for period t + 1 D(t): Actual demand during period t n: number of periods in the moving average forecast It is clear from the formula above that the moving average method puts equal importance on every period included in the calculations, which isn’t always desirable. The exponential smoothing method, as mentioned by Jonsson and Mattson (2009), is a way of using a simple time-series based method but introduce a weighting of the most recent period compared to previous forecasts. This is done by introducing a smoothing factor, called α, with a value between 0 and 1. A smoothing factor closer to 1 will make the calculation more responsive to systematic changes of the demand, while also becoming more exposed to random variations (Jonsson and Mattson 2009). The formula, as the authors state it, is presented below in Equation 2.2. F(t + 1) = α ∗D(t) + (1− α) ∗ F (t) (2.2) Where: F(t + 1): Forecast demand for period t + 1 D(t): Actual demand during period t α: Smoothing factor 14 2. Theory The main benefits of the two traditional methods mentioned above are their ease of use and applicability to fast-moving parts. However, Andersson (2019) states that research in both theory and practice shows their insufficiency when forecasting intermittent and/or slow-moving demand. The author brings up several methods that attempt to bridge the performance gap, such as the most commonly used Croston method (Croston 1972) and a variant thereof developed by Syntetos and Boylan (2005). Other methods include ones based on the ARIMA methodology (Hyndman and Athanasopoulos 2018) and the use of demand aggregation (Bartezzaghi and Kalchschmidt 2011). Even with such improved and expanded methods, the issue of time-series methods not considering the actual underlying demand remains (Andersson 2019). This fact combined with the increasing availability of various product-in-use and installed base data, as well as increased analytical capabilities, means the case for developing and using causal-based methods is gaining academic traction and practical feasibility (Andersson 2019). 2.2.2 Causal-based Forecasting Various and plentiful alternative methods are suggested for predicting spare part demand, in cases where more traditional time-series methods are unsatisfactory. Andersson (2019) mentions causal-based forecasting, judgemental forecasting and mixed method forecasting. Causal-based methods will be elaborated upon below. Causal-based methods differ from time series methods in the sense that they do not depend on historical demand data to predict future demand. Instead these methods aim at finding relations between extrinsic variables and demand and using that connection to predict future demand. Andersson (2019) mentions three categories of causal-based forecasting methods applicable to the automotive aftermarket, which are presented below. Firstly, reliability-based forecasting looks at the expected failure rate or life-time of each spare part and correlates this number with the installed-base of said spare part and can in that way predict future demand. In their work on managing maintenance spare parts, Cavalieri et al. (2008) present a forecasting framework that uses the failure rate of components together with the existing number of products in the installed-base to replace the need for demand history. The authors present and argue for the usefulness of such a reliability-based approach in situations where historical demand doesn’t exist as well as if the demand patterns are lumpy and/or intermittent (Cavalieri et al. 2008). According to Andersson (2019), there are two main methods for retrieving the necessary data when including failure rate in forecasting; collecting the data from databases where the various product failure rates are stored or using dynamic data from monitoring equipment to compile Life Data Analysis. Methods utilizing data from on-board sensors installed in each product, such as vehicles and vehicle parts, are generally known as on-board sensor based (Andersson 2019). The general idea is to find causal connections between sensor values and the need for part replacement and hence a demand for spare parts. This category of methods is commonly referred to as condition-based maintenance, and the main purpose is to improve the planning of maintenance on single vehicles (Andersson 2019). However, the data collected from individual vehicles could be aggregated to produce a forecast for the total population, according to Andersson (2019). The third and final category mentioned by Andersson (2019) consist of regression-analysis based 15 2. Theory methods. As the name implies, these methods use regression analysis including multivariate linear regression, ARIMAX and ARX (Andersson 2019). The main concept here is the attempt to find a causal connection between demand and multiple extrinsic variables. This is sometimes referred to as leading indicator forecasting, and can be combined with machine learning methods to further improve the accuracy of demand forecasts (Wheelwright et al. 1998). 2.2.3 Forecasting Methods Utilizing Installed Base Data From theory and the literature review presented above, a number of promising forecasting methods for utilizing installed base data have been identified. The method selected for further study are presented in the conceptual model section below. Andersson (2019) and Van der Auweraer et al. (2019) suggest reliability based methods have the potential to successfully predict the lumpy and intermittent demand of many spare parts. Further, Liu and Tang (2016) effectively apply a reliability based method on installed base data in order to predict spare part demand. The authors look at the time from the installation of a specific system until it fails and a need for a replacement part arises. Hence Liu and Tang (2016) highlights a way to apply a Reliability-based forecasting method with the help of installed base data and without access to measured or calculated reliability information for individual parts. Steuer et al. (2018) suggest an alternative causal based method for spare part demand. A Demand-similarity Method. Steuer et al. attempts to predict overall life-cycle demand by analysing the demand pattern for spare parts previously sold. This is done in three steps; 1) Cluster or sort previous spare parts based on the shape of their normalized (failure rate) demand pattern. A representative demand pattern is then determined for each of these clusters, 2) Use specific features of products to classify new products which have yet to see a full demand cycle into one of the clusters, 3) Use the normalized demand pattern of the clusters as a predictor by multiplying it with the planned installed base at the particular time of the forecast. 2.2.4 Forecasting Errors and Accuracy Measures Jonsson and Mattson (2009) mention, forecasts are always estimates of the real outcome and are thus more or less accurate in relation to said outcome. A forecast error is the difference between the actual observed value and its forecast. There are several ways of aggregating forecast errors, and thus measuring their accuracy (Papalambros and Wilde 2018). Forecast errors are normally measured on a period-by-period basis, and are in their purest form the difference between the forecast of one period and the actual demand for that period. Over time, and especially for automatic methods of forecasting, the monitoring of these errors is crucial. The aim is to identify both individual random errors as well as systematic errors creating either too low or too high forecast values (Jonsson and Mattson 2009). A common method for continuous monitoring of forecast errors is calculating the mean error, ME, and the average value of the forecast error in absolute terms. This is usually called the mean absolute deviation, MAD, and does not consider whether the forecast error was higher or lower than the actual demand (Jonsson and Mattson 2009). ME and MAD are scale dependent measures, meaning that the value of the errors are on the same scale as the object of the forecast. 16 2. Theory An alternative, but still scale dependent, measure to MAD is the mean square error, MSE, which estimates the variance of demand when the mean demand is 0. In practice, using MSE as your measure penalizes large forecast errors harder than ME, but is naturally suitable for items with specific demand characteristics (Jonsson and Mattson 2009). The mean absolute percentage error, MAPE, is the most commonly used measure of percentage errors. MAPE; and other precentage errors, are scale independent and are thus not on the same scale as the objects being forecasted. The advantage of such measures is that they are unit free, and can thus be used to compare forecast performances between data sets (Hyndman and Athanasopoulos 2018). A percentage error is calculated as the forecast error relative to the observed value. See the standard formula for MAPE calculations in Equation 2.3, as presented by (Jonsson and Mattson 2009). MAPE = 1 n n∑ t=1 |D(t)− F (t) (D(t)) | × 100 (2.3) Where: F(t) : Forecast at time t D(t) : Actual demand at time t MAPE calculates the absolute mean of these errors and is thus susceptible to being infinite or undefined for observed zero values, and having extreme values if the observed value is close to zero (Hyndman and Athanasopoulos 2018). According to Hyndman and Athanasopoulos (2018), these factors, in addition to MAPE penalizing over forecasting more heavily than under forecasting, led to the introduction of the so-called symmetric mean absolute percentage error, sMAPE, which is widely used by actors throughout industry today (Hyndman and Athanasopoulos 2018), Volvo DP&O being among them. The risk for divisions by zero still exists though, especially for products with intermittent demand characteristics. If both the observed value and the forecast is 0, the denominator will be 0. This means other quantitative measures, such as mean square error, or even qualitative judgements may be more suitable for such products. The equation for sMAPE, according to Hyndman and Athanasopoulos (2018), is presented below in Equation 2.4. sMAPE = 1 n n∑ t=1 |F (t−D(t))| (|D(t)|+ |F (t)|)/2 × 100 (2.4) Where: F(t) : Forecast at time t D(t) : Actual demand at time t 2.3 Big Data Analytics The below section introduce and describes a number of concepts within the topics of Big Data analytics and machine learning. In addition specific algorithms applied in this study are described in detail. Hastie et al. (2009) and Rebala et al. (2019) provide an introduction to many of the central topics. 17 2. Theory 2.3.1 Dependent and Independent Variables During regression and classification analysis, the goal of a model is to understand how a set of variables affect the value of another. The variables that are used for the prediction are called independent variables and the value that is to be predicted dependent variable. The idea is that the value of the dependent variable in some way depends on the values of the independent variables and hence a prediction about the dependent variable is possible. 2.3.2 Training Data, Test Data and Overfitting Rebala et al. (2019) explain the construction of training and test data to be an important step in data analytic studies. The general idea is to split your data set in two. One set (training set) is shown to the model and from the training set the model construct correlations between the independent variables and the dependant one. The second set (test set) is excluded from the training set in order to allow fresh data to be used to evaluate the model. This is done to reduce the risk of constructing a model which is too tuned to the specific data it has been trained on and hence would perform poorly when shown new sets of independent variables to make predictions on. This issue is called Overfitting. 2.3.3 Random Forest Random Forest algorithms are a commonly used set of machine learning algorithms applied to regressions and classification problems. The fundamentals and implementation is described by Liaw and Wiener (2002). Rebala et al. (2019) list intuitively, ease of understanding and ability to backtrack what made a model make a particular prediction as benefits of a random forest algorithm. A random forest is based on the concepts of decisions trees and ensemble learning, both are discussed below. A decision is explained by Rebala et al. (2019) as a structure of nodes and edges. The nodes are decisions based on the value of specific variables and the edges links nodes together. This constructs different paths and branches in a tree like structure and the last node in a branch is refereed to as a leaf node and represent a specific value which will be the output of the decision tree. Figure 2.3 shows a simple decision tree. The example tree shown in Figure 2.3 is simple and can without any difficulty be constructed by hand. However once the number of variables and the amount of data increases computers and algorithms are the only viable option. Rebala et al. (2019) explain the general strategy of such an algorithm. The idea is to construct a tree where each node’s decisions adds as much information to the overall model as possible. This can be done in multiple ways and are outlined in detail by Rebala et al. (2019). A random forest is, as the name suggests, a number of random decision trees. A random forest constructs a number of decision trees based on subsets of the input data. The prediction by the model is then performed by averaging the outcome of each individual decision tree. This reduces overfitting by not relying on a single decision tree. The details or the algorithm can be found in Liaw and Wiener (2002) which also explain how a random forest algorithm can be used both for classification and regression problems. 18 2. Theory # of parts 2 months old > 324 Function Group = 4 Function Group = 4 Yes No Prediction = 423 Prediction = 323 Prediction = 276 Prediction = 172 Yes No Yes No Figure 2.3: Visualization of how a decision tree can be used to solve regression problems. In a random forest algorithm multiple trees are created and the outcome of the model is the average of the chosen trees. 2.4 Conceptual Model This section describes which of the topics covered in the literature review that are applied in the analysis section of this thesis.Three main topics, Spare Parts, Forecasting Methods and Big Data, are identified and together they form the conceptual model for this study. Finally the derived forecasting method to be evaluated in this study is presented. The unique characteristics of spare parts are considered when forming the foundation for the analysis. The relation between products and spare parts life-cycle as well as the demand pattern for various types of parts are considered. Time series and Big Data analytics forecasting are deemed necessary for the analysis of this thesis. Installed base data is a kind of Big Data can be evaluated using Big Data analytics. The performance of a Big Data method is to be compared against a time series method and as such time series forecasting is to be applied in the analysis. Big Data poses different challenges compared to traditional intrinsic data and have to be evaluated with different tools. In this study installed base data is evaluated as a source Big Data. The understating of how Big Data differs from traditional data is important for further analysis. Inspiration is drawn from Liu and Tang (2016) when designing the following reliability based method which is to be tested in this study. Neither of the data sets described in Section 3.4 contains reliability data for the various parts. However by combining the data sets in a similar way to Liu and Tang (2016) reliability data for case company spare part can be derived. The derived data is not data over historical sales but instead a kind of extrinsic data for which time series forecasting methods do not work. Instead the suggested method apply a machine learning, more specifically a random forest algorithm to predict demand based on installed base reliability 19 2. Theory information. In order to evaluate this proposed method a Time Series forecasting method is applied to historic sales data. Finally the two methods are combined in a method which utilize both historical sales and installed base information. The forecasting accuracy of each of the three method is evaluated on the different stages of the spare part life cycle and in relation to the demand pattern of the specific parts. 20 3 | Methodology This sections describes the study’s research design as well as how the study was conducted. For an overview of the methodology, see Figure 3.1 3.1 Research design Bryman and Bell (2003) describe the mixed methods research as the combination of quantitative and qualitative methods. This study was built on that principle and both quantitative data as well as qualitative interview data has been utilized. One concept highlighted by Bryman and Bell (2003) is triangulation in which for instance quantitative data was used to verify qualitative findings. During this study qualitative data has been used to determine what quantitative experiments were to be run and quantitative results were analysed with qualitative knowledge accessed through domain experts. 3.2 Workflow and Structure This section describes the approach and how the work during the thesis was structured. The goal of the structure was to allow for ongoing testing and evaluation of forecasting methods while still ensuring a high level of validity and repeatability of findings. The study and thesis work was split into three main phases; 1) Orientation and Data Collection, 2) Data Analysis and Evaluation and 3) Results and Findings. 3.2.1 Phase 1: Orientation and Data Collection The purpose of Phase 1 was to give the authors a broad understanding of the field of study as well as the case company situation and to perform data collection. This was achieved through literature studies as well as the conducting of interviews with domain experts. Interviewees were primarily Volvo Swedish and US managers, domain experts and data scientists. The data gathered was thus both quantitative and qualitative. Finally quantitative data was collected from internal Volvo sales and product databases. This process is outlined in detail in Section 3.4. 3.2.2 Phase 2: Data Analysis and Evaluation The data analysis started with a literature study in order to find forecasting models applicable to installed base data. The methods were to be applied to the data and the outcome evaluated and validated. The purpose of this was to determine if and how well these methods might contribute to a greater understanding and approximation of demand patterns and increased forecast accuracy. The forecasting method selected for study is presented in Section 3.5. 21 3. Methodology The data analysis was conducted in the Python programming language and the models evaluated in this study were implemented with the help of readily and openly available data science Python packages. For instance Scikit Learn (2020), Pandas (2020) and CatBoost, Yandex (2020) were utilized. These packages include tools for data processing, implementations of commonly used machine learning algorithms and tools to evaluate the performance of models. The data analysis was supported by two data scientist experts employed by the case company. Their technical knowledge as well as their understanding of the data to be studied were key for the success of the study. This phase of the study was iterative and structured around the process of plan, develop, test and evaluation of the next step to improve the proposed forecasting model. A phase began with that a potential improvement was identified. The model was modified to reflect the proposed improvement and the performance evaluated. The insight to perform a specific test came both from studying literature and in correspondence with case company domain experts. The performance of each model was discussed in relation to the specific case company situation and the data at hand and was cross-referenced with literature to provide explanations to why each model performed as it did. There is a continuous dialog with Volvo Data Scientists in a number of informal meetings. This allowed the authors to ask questions about the data and discuss and validate findings with what they knew to be true from their day to day job at Volvo. 3.2.3 Phase 3: Results and Findings In the last stage, the results and findings the analysis was further evaluated, and benchmarked against Volvo Cars existing forecasting methods. A summary of the result was produced, based on the findings from the analysis and literature. This was presented as a recommendation of how installed base information can and should be used to increase spare part demand forecast accuracy at the case company and in general. 22 3. Methodology Develop model Construct and modify the forecasting model, based on theory and observed results 1 2 Literature Litterature review of the topics: Spare Parts, Forecasting Methods, Data Analysis,  Demand Patterns Data Collection Collect quantititive and qualitive data through interviews and case company databases Run model Apply model to data and obtain forecast results Evaluate Analyze performance of the model, and consider changes & improvements Construct models Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do 4 Results Final performance of model analyzed, compared and discussed Plan improvements Potential improvements to the model are identified and their implementation planned Analysis Iterative process to create, analyze and evaluate the forecasting model 3 Figure 3.1: Conceptual overview of methodology and workflow 23 3. Methodology 3.3 Choice of Accuracy Measures Firstly, the three accuracy measures chosen for the analysis were mean absolute deviation (MAD), mean square error (MSE) and standard mean absolute percentage error (sMAPE). This was partly because MAD and MSE are easy to use and common in practice, while sMAPE is the KPI used by DP&O at Volvo. Further, MAD and MSE are both scale-dependent, meaning that errors measured by both are on the same scale as the data. Accordingly, these errors are the mean number of units with which the forecasts deviate from actual demand in the case of MAD and the square of said deviation in the case of MSE. sMAPE on the other hand is a scale independent percentage error, potentially allowing for comparisons external to Volvo. For comparisons within the same data set, as performed in this thesis, the essential task of accuracy measures is to identify the model with the best performance. As such, the relative and ranked performance of each model is the key result. However, when comparing to other data sets and situations, the scale-dependent measures of MAD and MSE can be misleading. Since the quantity of actual and predicted units may vary greatly from case to case, using them for comparative purposes can lead to wrong conclusions being drawn. The percentage errors of sMAPE do however allow for comparisons external to both the data and specific situation of this thesis, although the exact parameters and causalities may be hard to replicate. That being said, the differences between accuracy measures are significant and it is important to choose correct and relevant measures for each situation, especially in terms of the magnitude of the errors, to allow for fair and useful comparisons. For the purposes of this thesis and the analysis performed, MAD, MSE and sMAPE were considered to provide a sufficient basis for comparison between the evaluated models. 3.4 Data Collection The data collection was conducted together with representatives from the case company. Installed base data, as well as product and sales data, was collected from a number of the case company’s databases. 3.4.1 Quantitative Data The first data set contains spare parts master data with descriptions of the parts name, volume, weights, product group, function group and a text description of the part. The product group refers to what general type of product the part was, for instance whether it was a spare part or an accessory. The function group was more detailed and describes where in the car the part was installed and what general function the part has. The second set of data was historical invoice data. This data contains all historical sales of spare parts going back to 2007. This means at the time of this study, year 2020 the data contains 13 years of sales history. This data was determined to be accurate as it was based on actual sales of products and has been used for accounting purposes. An overview of the data in Data Set 2 is presented in Table 3.1. 24 3. Methodology Label District Part Year Month Quantity Sold Country Code Description Market Article number Year and month Number of parts replaced Country Example 5413 7846841530 200810 41 SE Table 3.1: Describes the data in Data set 2. The example row shows that 41 units of part 7846841530 were sold in October 2008 on the Swedish market and that Sweden is in District 5413. The third data set contains historical car sales and information of which market it was sold on. This set also include the number of units of each potential spare part that was installed when the car was produced. This data was determined to have high accuracy as it the basis for the Bill of Material for each car that is produced. An overview of the data set is shown in Table 3.2 Label Country Code Part Year Month Quantity car Quantity part Description Country Article number Year and month Number of cars sold Number of parts installed Example SE 78468530 200810 1200 4800 Table 3.2: Describes the data of Data set 3. The example shows that in October 2008, 2000 cars containing article 7846841530 were sold and four were used in each car. In order to curate the data and reduce irregularities data cleaning was conducted. First Data Set 3 was filtered in order to remove parts which have been replaced with another part. Only parts which had never been replaced were kept. This was done in order to keep only spare parts with a full life-cycle in the final data set and avoid parts which have been introduced as a replacement for another in the middle of the life-cycle. Secondly, Data Set 2 and Data Set 3 were combined to filter out parts which are not present in Data Set 1. This means only parts for which historical demand existed were included in the final data set. An overview of the three data sets is presented in Table 3.3. Data Set Source Kind of Data Time range 1 Volvo Internal Spare Part Master Data - 2 Volvo Internal Historical Sales 2007 - 2020 3 Volvo Internal Installed-base 2013 - 2020 Table 3.3: Overview of Data Sets 3.4.2 Qualitative Data In order to gather information and knowledge from experts interviews were planned. However, this part of the study was cut short due to the COVID-19 pandemic and changed priorities for both the authors and Volvo Cars. Instead, only one interview was conducted, following the 25 3. Methodology semi-structured approach described by Bryman and Bell (2003). The questions asked to the interviewee can be seen in Appendix A.1. The interview provided insight into how the logistics of Volvo US is organized. One of the key take-aways was that in the US spare part demand is most often not pre-planned. Instead the customer drives to the repair shop and expects the part to be readily available or at most a couple of days away. This differs compared to Europe, where most repairs are planned and the customer only goes to the shop once the shop was ready to perform the repair. A second point was the longer lead time to the US from CDC in Gothenburg, around two months by boat. The lead time combined with specific tax rules have lead to some vehicles getting their final configuration once they reach the US. This is achieved by ordering more stripped down cars and install customer specific accessories on cars already in US storage. 3.5 Data Pre-processing The three data sets included in this study lacked direct information of spare part reliability and expected life time. This thesis evaluated the use of installed base data and a reliability-based method. Therefore reliability data was derived by combining the three data sets. Data Set 2, containing historical sales of spare parts is combined with Data Set 3 which contained the installed base and expected future installed base. The combinations were done by deriving the age distribution of installed spare parts from the installed base data at specific dates and considering what demand the particular age distribution produced in the following planning period. Demand for a particular spare part was deemed to represent a failure and hence the life time or reliability of that particular spare part. For each part the first date of interest was the first time the part is installed in a car. Beginning with this date and for each of the next 24 months the part number, date and age distribution of the specific part was recorded. Such a record is referred to as an Observation. The date recorded in an observation represented the date for which the age distribution of the installed base was recorded. This is henceforth referred to as Check Date. Thus, for each part the Check Date always lies between the first date the part was installed in a car and 24 months forward. The first observation of a part only included parts with an age of 0 months. The second observation only parts of 0 and 1 months of age, and so forth. For each part 24 observations are generated. Together these 24 observations describe the age distribution of the part for each of the 24 months since the parts first introduction. The population distribution of the installed base for a given date and part are represented by x311 − x3...n in Table 3.4. The demand for the following planning period after the specific date was the dependant variable and was represented by y in Table 3.4. This means that the same part is represented multiple times in the initial data set but with different Check Dates (the date from which the forecast was to be performed). The age distribution of the installed base as well as the spare part features were the independent variables and the dependent variable was the demand. 26 3. Methodology qty month old Obs Check Date Part 1 2 ... n Sales next month 1 x11 x21 x311 x321 x3..1 x3n1 y1 2 x12 x22 x312 x322 x3...2 x3n2 y2 ... x1... x2... x31... x32... x3...... x3n... y... n x1n x2n x31n x32n x3...n x3nn yn Example Feb 1 2020 1681531 432 153 .... 543 287 Table 3.4: Explanation of the initial data set, x represents independent variables and y is the dependent variable, demand. 3.6 Validity and Reliability The overall validity of the data collected was deemed high. The data was collected from internal systems and the data has been used for record keeping and forecasting. Volvo Data Scientists confirm the validity of the data and that it corresponded well to reality. When analyzing the data no prominent inconsistencies were identified. For instance the cleaned data do not contained any negative values for demand and data was present for the whole range that the data sets covered. In addition, findings and results were cross-referenced with previous research and with domain experts at Chalmers and at Volvo Cars. The consensus between this study’s results and literature indicates the data and methods holds high validity. 27 3. Methodology 3.7 Overview of Analysis The analysis was structured around the development of a reliability-based forecasting model, utilizing installed base data. This proposed model was evaluated against a time series forecasting method. The methods were compared both on their overall accuracy and on how well each method performed on spare parts with different demand patterns. An overview of the data included, the methods implemented and their evaluation is shown in Figure 3.2. Each model is explained in detail in the following Analysis Chapter. a Time Series Model  Exponential Smoothing Installed Base Model CatBoost Forecasting Models Installed Base Data Historic Sales Data Data Spare Part Data -Characteristics - Master Data Forecast Accuracy Demand Patterns   Lumpy         Erratic Smooth        Slow Results Accuracy for Each Demand Pattern Combined Model CatBoost Figure 3.2: Overview of data sources, models and results involved in the Analysis 28 4 | Analysis In this chapter, the analysis is presented from the planning of the predictive data study through construction and implementation of the model. The different stages of forecasting during the launch phase of a new spare part are explained. Further, forecasting is performed using an exponential smoothing model representing traditional time series forecasting and a new extrinsic machine learning algorithm. These two models are also combined in a third model, and the results of these three forecasts are compared. The included features are analysed and compared on their impact on model outcome. Lastly, the parts included in the data are split according to their demand patterns, and the results of each model evaluated for each pattern. 4.1 Planning Horizons The aim of this study is to focus on predicting spare part demand in the launch phase of new spare parts, 0-24 months from its market introduction. This period is divided into three sub-phases. The forecast result varies in these phases, since the decisions made based on the forecasts vary and pose different challenges for Volvo. In discussion with domain experts at Volvo the following horizons and corresponding target outputs (summarized in Table 4.1) were established: • During Stage 1, which is before any sale of the part has occurred, the decision Volvo Cars has to make is whether or not the part should be stocked at all. Volvo wants to stock the part if it is likely is to sell in any of the next 6 months, meaning if any demand exists at all. What is to be predicted is thus whether or not the part will sell in the next 6 months. • In Stage 2, once the part has seen initial sales and until the part have seen sales in three periods, the total installed base is still small and total sales low for most parts. The planning horizon is set to three months and what is to be predicted is the total demand for the next three months. This stage lasts until a part has seen sales in a total of three periods, meaning demand has existed in three not necessarily connected months. • The last stage, Stage 3, consists of the months remaining after the end of stage two until 24 months after product launch. During this stage, the demand for the next month is to be forecast. Since the product is slightly more established sales can likely be more accurately predicted. 29 4. Analysis Stage Planning Frequency Planning Horizon Problem What to predict 1 1 month 6 months Classification Sale in next 6 months 2 1 month 3 months Regression Demand next 3 months 3 1 month 1 month Regression Demand next 1 month Table 4.1: Overview of the three planning stages introduced in section 4.1 4.2 Forecasting Models In the following section, the three models used during the analysis are presented. The first, a machine learning model utilizing extrinsic installed base data (referred to as IB from here on). The second is a simpler time series-based model using exponential smoothing (called ES), representing the current method used by Volvo DP & O. This is to facilitate a comparison between the more complex algorithm and the simpler one, evaluating if forecasting accuracy is improved by the added complexity. For Stage 1, only the IB model is run since no historical sales data exists for the ES model to use. The IB model is instead compared with an estimation of Volvo’s actual results. As mentioned above, Stage 1 contains a classification problem, while Stage 2 and 3 consist of regression analysis. Both the IB and ES models are run in the later stages, as well as a model combining both these methods, called the Combined Model (C). 4.2.1 The Installed Base Model The Installed Base model is based on a so called random forest algorithm. Liaw and Wiener (2002) explain how a such an algorithm can be applied as both a classification and a regression method. The details of the random forest algorithm are presented in section 2.3.3. In this study an optimized kind of random forest algorithm is applied to the data set described in table 3.4. The specific implementation is described below. 4.2.1.1 Implementation In this study the random forest algorithm was implemented using the CatBoost, Yandex (2020) python package. CatBoost is based on the concept of a random forest, but is more advanced and applies various optimizations to improve model accuracy and reduce the time needed to train the model. The details of the CatBoost algorithm can be found in Dorogush et al. (2018). Furthermore, the algorithm has built-in support for categorical data such as product groups which further simplifies the implementation. Appendix A.2 shows the Python implementation of the CatBoost algorithm as a classifier. The implementation for Stage 2 and 3 are similar but instead of a classifier a regression model is used. The independent variable in this case is demand for the next month n corresponding to the planning horizon, see Section 4.1. 4.2.1.2 Parameters The CatBoost algorithms takes a number of parameters and a data set as it inputs. It is possible to omit any parameter and if so CatBoost falls back to a default value. The data set should contain the features or independent variables the algorithm should use to predict the dependent 30 4. Analysis variable. The independent variables can be both a continuous numeric value or categorical discrete values. The dependent variable is, in the case of a classification problem, a discrete category and in the case of a regression problem a numeric value. The CatBoost documentation (Yandex 2020) explains the parameters likely to impact the accuracy of the model according to below: • Number of Trees, this refers to the number of trees or estimators that are generated while the algorithm is learning from the data. A high number of trees often results in higher accuracy but increases the risk of overfitting as well as increasing the computational power needed and hence also increases the time it takes to train the algorithm. • Tree Depth is the maximum depth a decision tree can have in the algorithm. For illustration, a tree of depth 2 is shown in Figure 2.3. • Learning Rate defines how aggressively the algorithm will learn from each iteration of the algorithm. A lower value increase training time but tend to yield more accurate results. A high value increases the risk of overfitting. This parameter needs to be tuned in order to reach good results within a reasonable time with regards to available computing resources. 4.2.1.3 Cross Validation The model data is split into a training and a test data set. Though the split is done randomly, there is a chance that the specific split used when training the algorithm is not representative of a new independent data set (Hastie et al. 2009). A common way to make the model more accurately predict new data is to perform cross validation. K-fold Cross validation is one method described in Hastie et al. (2009). It works by splitting data in K parts and uses K-1 parts to train a model. The remaining set is used to evaluate the model. This is repeated K times with each of the sets functioning as the evaluation set. The performance for the model is deemed to be the average of the performance from the K iterations. In Figure 4.1 below, the results of the 5-fold Cross Validation of the Stage 1 classifier. The 5 lines represent the performance of each of 5 folds. The y-axis represents the objective to be minimized during training, while the x-axis represents the number of iterations. The number of iterations corresponds to the number of trees fitted in the model. It is desirable to minimize the variation between the lines, since low variation indicates that the model should perform similar on data it has not previously been trained on. 31 4. Analysis Figure 4.1: Visualisation of 5-fold Cross Validation. The x-axis shows the number of iterations and the y-axis the model error (lower is better). All five lines move downwards and the variation between them is low. This indicates that the model is not too fitted to the test data and that the number of iterations is not too high. 4.2.1.4 Parameter Tuning The performance of forecasting models is dependant on the values of the various parameters defined for the model. In order to optimize the model, parameter values have to be considered and tuned. One way to help find the optimal parameters for a given model and data set is to perform a so called Grid Search. In a Grid Search a set of parameters and related values are defined and a forecast model is trained with each possible combination of the parameters. The idea is that the best of these models should be close to representing the optimal model. The implementation of GridSearch for the Stage 1 classifier can be seen in Appendix A.3. The implementation utilizes 5-fold Cross Validation to determine the performance of each model. 4.2.2 The Exponential Smoothing Model An exponential smoothing model (ES) is chosen to represent general time series forecasting methods since it is a commonly used approach in practice when dealing with intermittent demand (Syntetos and Boylan 2005). It also represents a decent estimation of the methods currently used by Volvo. The main difference between the model implemented here and the one used by Volvo is that the ES model used here applied a dynamic smoothing value, α, whereas Volvo’s is static. Volvo’s model is tailored to their own needs, and the dynamic smoothing 32 4. Analysis value is an attempt to compensate for the general nature of the ES model. The implementation and results of this has been verified with domain experts at Volvo, and is considered a good representative for the purposes of this study. The ES model was implemented in Python, see Appendix A.4 for details. The basis of th