Design of smart orthosis for rehabilita- tion of Achilles tendon ruptures A novel system for monitoring ankle biomechanics during ATR rehabilitation, for a more personalized recovery process. Master’s thesis in Systems, Control & Mechatronics ADAM ANDERSON Department of Electrical Engineering CHALMERS UNIVERSITY OF TECHNOLOGY Gothenburg, Sweden 2024 www.chalmers.se www.chalmers.se Master’s thesis 2024 Design of smart orthosis for rehabilitation of Achilles tendon ruptures A novel system for monitoring ankle biomechanics during ATR rehabilitation, for a more personalized recovery process. ADAM ANDERSON Department of Electrical Engineering Systems and Control Division Bionics research group Chalmers University of Technology Gothenburg, Sweden 2024 Design of smart orthosis for rehabilitation of Achilles tendon ruptures A novel system for monitoring ankle biomechanics during ATR rehabilitation, for a more personalized recovery process. ADAM ANDERSON © ADAM ANDERSON, 2024. Supervisors: Annelie Brorsson, The Department of Orthopaedics, Institute of Clinical Sci- ences at Sahlgrenska Academy, University of Gothenburg Katarina Nilsson Helander, The Department of Orthopaedics, Institute of Clinical Sciences at Sahlgrenska Academy, University of Gothenburg Emmanuel Dean, Department of Electrical Engineering, Chalmers University of Technology Examiner: Emmanuel Dean, Department of Electrical Engineering, Chalmers University of Technology Master’s Thesis 2024 Department of Electrical Engineering Systems and Control Division Bionics research group Chalmers University of Technology SE-412 96 Gothenburg Telephone +46 31 772 1000 Cover: Picture of the sensorized insole (left), data visualization (right) Typeset in LATEX Printed by Chalmers Reproservice Gothenburg, Sweden 2024 iv Design of smart orthosis for rehabilitation of Achilles tendon ruptures A novel system for monitoring ankle biomechanics during ATR rehabilitation, for a more personalized recovery process. ADAM ANDERSON Department of Electrical Engineering Chalmers University of Technology Abstract This thesis presents a compact system for measuring forces under the foot, designed for rehabilitation after an Achilles tendon rupture. The aim is that this system could provide patients and care providers with critical data about the recovery process, for a more personalized and effective treatment. The core component of this system is a newly developed flexible insole that senses forces under the foot. The force is measured in three dimensions (i.e., normal and shear forces) using magnetic-based sensors, placed in a grid of 73 nodes. The large area covered by the sensor, and the flexibility, are improvements over previ- ous magnetic-based force measurement systems. The insole and additional support electronics were mounted on a standard ankle orthosis (also known as Walker). In addition, two IMUs were used to estimate the orientation of the insole. Software was also developed to process and visualize the data. The measurements from the insole are sent to a signal processing chain to calculate relevant biomechan- ics parameters such as the center of pressure and joint torques. The signal processing chain was implemented within ROS2, together with micro-ros for the low-level com- munication with hardware. ROS2 is also used for visualization purposes. The results are promising, showing that magnetic-based sensors are feasible for measuring 3D forces under the foot. The sensors display a nearly linear response to vertical pressure, although there is considerable hysteresis that introduces errors in the measurements. Future work to improve calibration, verify reliability, and improve ease of use is needed before the system can be used in a clinical setting. Keywords: Plantar pressure, shear force, ROS2, micro-ros, Achilles’ Tendon Rup- ture, rehabilitation, Hall-effect. v Acknowledgements First and foremost I want to thank Emmanuel Dean for his enthusiastic support and the many insightful discussions. His guidance has been essential for the success of this project. I would also like to thank Annelie Brorsson, Katarina Nilsson Helander, and the rest of the team at Sahlgrenska, their input and expertise from a medical perspective have been very valuable. Finally, I would like to thank my family and friends for their encouragement through- out the project. Adam Anderson, Gothenburg, September 2024 vii List of Acronyms Below is the list of acronyms that have been used throughout this thesis: ATR Achilles Tendon Rupture AFO Ankle Foot Orthosis CS Chip Select DDS Data Distribution Service EDA Electronic Design Automation FPC Flexible Printed Circuit I2C Inter-Integrated Circuit IC Integrated Circuit IMU Inertial Measurement Unit MISO Main In Sub Out / Master In Slave Out MOSI Main Out Sub In / Master Out Slave In PCB Printed Circuit Board ROS Robot Operating System SMT Surface Mount Technology SPI Serial Peripheral Interface TCP Transmission Control Protocol UDP User Datagram Protocol USB Universal Serial Bus WHO World Health Organization ix Contents List of Acronyms ix List of Figures xv List of Tables xix 1 Introduction 1 1.1 Purpose and scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Literature review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Ethical aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2 Theory 5 2.1 Achilles Tendon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.1.1 Anatomy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.1.2 Rupture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.1.3 Recovery procedure . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 Communication protocols . . . . . . . . . . . . . . . . . . . . . . . . 8 2.2.1 SPI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.2.2 I2C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2.3 USB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.3 Electronics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3.1 Printed circuits . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3.1.1 Structure . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3.1.2 Design Considerations . . . . . . . . . . . . . . . . . 11 2.3.1.3 Flexible Printed Circuits . . . . . . . . . . . . . . . . 12 2.3.2 Hall-Effect Sensors . . . . . . . . . . . . . . . . . . . . . . . . 13 2.3.3 Binary Decoder . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.4 IMU sensor fusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.5 Robot Operating System . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.5.1 Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.5.2 Communication . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.5.3 Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.5.3.1 Header message . . . . . . . . . . . . . . . . . . . . . 16 2.5.4 TF2 package . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.5.5 Micro-ros . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.6 Equivalent force and torque . . . . . . . . . . . . . . . . . . . . . . . 18 xi Contents 3 Methodology & Design 19 3.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.2 System Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.2.1 Working principle of a sensor node . . . . . . . . . . . . . . . 20 3.2.2 Organization of E-skin system: patch, module and node . . . . 21 3.3 Hardware Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.3.1 Insole Flexible Printed Circuit . . . . . . . . . . . . . . . . . . 22 3.3.1.1 Components . . . . . . . . . . . . . . . . . . . . . . . 23 3.3.1.2 Assembly . . . . . . . . . . . . . . . . . . . . . . . . 24 3.3.2 Silicone layer with integrated magnets . . . . . . . . . . . . . 25 3.3.3 Top cover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.3.4 Controller unit design . . . . . . . . . . . . . . . . . . . . . . 27 3.4 Software system design . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.4.1 Micro-ros hardware node . . . . . . . . . . . . . . . . . . . . . 29 3.4.2 Calibration nodes . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.4.3 Center of pressure nodes . . . . . . . . . . . . . . . . . . . . . 32 3.4.4 Ankle joint torque and force node . . . . . . . . . . . . . . . . 33 3.4.5 Rviz visualization . . . . . . . . . . . . . . . . . . . . . . . . . 33 3.4.6 Test bench node . . . . . . . . . . . . . . . . . . . . . . . . . . 34 4 Experimental Validation and Results 37 4.1 Node raw values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 4.1.1 Test setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 4.1.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 4.2 Long-Time Hysteresis . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 4.2.1 Test Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 4.2.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 4.3 Top cover impact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 4.3.1 Test setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 4.3.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 4.4 Walking in boot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 4.4.1 Test setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 4.4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 5 Discussion and Conclusion 45 5.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 5.1.1 Measurements and calibration . . . . . . . . . . . . . . . . . . 46 5.1.2 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 5.1.3 Untested features . . . . . . . . . . . . . . . . . . . . . . . . . 46 5.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 5.2.1 Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 5.2.2 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 5.2.3 ROS2 system . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 5.2.4 End-user experience . . . . . . . . . . . . . . . . . . . . . . . 48 5.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 A ROS2 messages I xii Contents A.1 WrenchStamped . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I A.2 TFMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . II A.3 PointCloud2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . III A.4 TactileSensorData . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV xiii Contents xiv List of Figures 1.1 Concept of the smart orthosis, showing the measured forces and angles. 2 1.2 Structure of magnetic-field-based sensor, showing the effect of apply- ing a force to the silicone. The sensor consists of silicone, a magnet, and a magnetic field sensor (Hall effect sensor). . . . . . . . . . . . . 3 2.1 Anatomy of the Achilles tendon with its connecting muscles in the lower leg. Source: Injurymap [22], CC BY 4.0 . . . . . . . . . . . . . 5 2.2 Illustration showing the plantarflexion and dorsiflexion movements of the foot. Source: OpenStax College [24], CC BY 3.0 . . . . . . . . . . 6 2.3 Illustration showing the rupture of the Achilles tendon. Source: In- jurymap [22], CC BY 4.0 . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.4 Typical SPI topology, with one main and three subs. It can be seen that the MISO, MOSI, and CLK lines are shared between all devices, while there is one CS line per device. . . . . . . . . . . . . . . . . . . 9 2.5 Topology of a typical I2C bus, with one main, two subs, and the mandatory pull-up resistors. Only two signals are needed, and they are connected to all devices. . . . . . . . . . . . . . . . . . . . . . . . 10 2.6 Cutout diagram of a 2-layer PCB, showing the structure of a via, pad, tracks, as well as the stackup. . . . . . . . . . . . . . . . . . . . . . . 11 2.7 Typical stackup of a two-layer FPC with FR4 stiffener. . . . . . . . . 13 2.8 Measurement setup for hall effect sensing. An applied magnetic field Bz will apply a force on the flowing electrons, causing a charge dif- ference between the two sides of the plate. This is measured by the voltage probe VH . Source: Steve Byrnes [36], public domain . . . . . 13 2.9 Schematic symbol and truth table for a typical 2-to-4 decoder. Any combination of inputs Xi results in one of the outputs Yi being acti- vated. If the enable input “EN” is not active, no outputs are active. . 14 2.10 Three TF2 coordinate frames. The relationship between the frames are shown as yellow/pink arrows. . . . . . . . . . . . . . . . . . . . . 17 2.11 Architecture of micro-ros system, the dark-blue components are de- veloped specifically for micro-ros. The leftmost components (over uP) run on the ROS2 computer, and the rest of the components run on the microcontroller. Source: micro-ros [44], CC BY-ND 4.0 . . . . . 18 3.1 Picture of the insole and controller unit. . . . . . . . . . . . . . . . . 20 3.2 System components and the connections between them. . . . . . . . . 20 xv List of Figures 3.3 Structure of a single 3D-force measurement node, showing the mag- net, suspended in silicone, above the hall-effect sensor. . . . . . . . . 21 3.4 Layout of patch, modules, and nodes. The numbers 0 to 72 represent the different nodes of the patch. . . . . . . . . . . . . . . . . . . . . . 21 3.5 Schematic of the electrical system, showing the components and con- nections of the controller unit and insole. . . . . . . . . . . . . . . . . 22 3.6 Pictures of flexible printed circuit . . . . . . . . . . . . . . . . . . . . 23 3.7 Design of the SPI bus traces on the insole FPC, connected in a single chain through all sensor nodes. . . . . . . . . . . . . . . . . . . . . . 24 3.8 Picture of a single node and the connector soldered on the FPC. . . . 24 3.9 Schematic overview of the silicone molding process. . . . . . . . . . . 25 3.10 Lower half, FPC, and upper half of the first mold. . . . . . . . . . . . 25 3.11 Second mold fitted on the silicone, before second pouring. . . . . . . . 26 3.12 Sole with top cover on, placed inside the orthosis . . . . . . . . . . . 26 3.13 Controller unit attached to orthosis. . . . . . . . . . . . . . . . . . . . 27 3.14 Design of controller PCB in ECAD design software. . . . . . . . . . . 28 3.15 Overview of the nodes, topics, and message types in the ROS2 system. 29 3.16 Structure of the micro-ros system, with the micro-ros agent and client. In this diagram, the circle is the ROS2 node, the topics with message types are represented with rectangles, and hexagons represent the services, with the service name and type specified. The micro-ros node (Sole_HW) connects the electrical system of the insole with the ROS2 system on the laptop. . . . . . . . . . . . . . . . . . . . . . . . 29 3.17 Nodes (circles), Topics with message types (rectangles), and services (hexagons). The patch and module nodes communicate with shared memory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.18 ROS2 system for the center of pressure calculations. . . . . . . . . . . 32 3.19 Node and topics for the ankle joint wrench. . . . . . . . . . . . . . . . 33 3.20 Topics that the Rviz node subscribes to. . . . . . . . . . . . . . . . . 33 3.21 Screenshot of the visualization in Rviz. The colored circles show the forces of each sensor node. In addition, COP and joint forces (red arrows), and torques (yellow arrows) are shown. . . . . . . . . . . . . 34 3.22 Structure of the fh-500 node with topics. The node . . . . . . . . . . 34 3.23 Sauter fh-500 test bench with sensor sole. . . . . . . . . . . . . . . . . 35 4.1 Test setup for evaluating single node response. . . . . . . . . . . . . . 38 4.2 Relationship between applied force and hall-effect reading (sensor node reading). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 4.3 Hall-effect reading vs applied force for nine nodes. . . . . . . . . . . . 39 4.4 Estimated and true force during a test run. . . . . . . . . . . . . . . . 40 4.5 Magnetic field measurement before and after a 100 N force has been applied. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 4.6 Total measured force without and with the top cover. . . . . . . . . . 42 4.7 Force measured by each node during one step with the smart walker. The color represents each sensor node and its location on the sole. . . 43 4.8 Maximum force of each node during one step with the smart walker. . 44 xvi List of Figures 4.9 Total force (blue) and joint torque (red) during one step with the smart walker. Negative torque in this case corresponds to plantarflex- ion. The torque starts positive at the start of each step and then goes negative. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 xvii List of Figures xviii List of Tables 4.1 Performance metrics of the calibration models (mean ± standard de- viation). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 xix List of Tables xx 1 Introduction According to the World Health Organization (WHO) [1], rehabilitation is a major unmet need, and it is estimated that globally one in three people have some con- dition that could benefit from rehabilitation. With changes in health and aging populations, this need is expected to increase in the future. Solving this problem requires more resources in the form of technology, trained professionals, and re- search [1]. Parallel to this there is also an initiative in Sweden to provide “good and local care”, where an emphasis is placed on the individual patients’ needs and situation [2]. As part of this transition, there is a recognized need for an increased use of technology, to be able to efficiently provide personalized care [3]. The goal is that more personal technology-assisted healthcare can improve self-efficacy for the patient, which will in turn lead to improved outcomes [4]. This research focuses on the rehabilitation of Achilles Tendon Rupture (ATR), a common musculoskeletal injury. The Achilles tendon is the strongest tendon in the body, and its function is to flex the foot downwards, for example, to stand on the toes [5], [6]. This function is essential for thrusting the body forward during walking. Therefore, if the tendon ruptures it will lead to significant disability [5]. The rupture can happen when the tendon is overextended or overloaded, often during recreational sports activities. The annual incidence rate is estimated to be between 6 to 55 events per 100,000 persons [5], [7]. Recent studies indicate that the biggest contributor to recovery is to encourage effective rehabilitation. However, there is disagreement over how the rehabilitation program should be laid out for the most effective recovery [5], [8]. 1.1 Purpose and scope To support the rehabilitation of ATR, this project proposes a smart ankle orthosis, composed of a conventional ankle orthosis, commonly called Walker, which is fitted with sensors to measure applied forces, as well as estimate the ankle joint movements and loads. These measurements are then processed to estimate potentially useful parameters for ATR rehabilitation, such as plantar pressure, weight distribution, load frequencies, and ankle joint torques. The long-term goal of this project is to support patients with ATR injuries by providing personalized data about the recovery process. In addition, the system 1 1. Introduction could provide researchers with a valuable tool to collect data for clinical studies, analyze further, and improve the rehabilitation process. The force measurement consists of plantar pressure (normal forces) and shear forces under the entire surface of the foot, using a flexible hall-effect e-skin. Multiple inertial measurement units (IMUs) are used to estimate the angles of the ankle joint. The data is collected and processed in real-time in a ROS2 system. Figure 1.1: Concept of the smart orthosis, showing the measured forces and angles. The purpose of this project is to investigate the following research questions: 1. How suitable is the hall-effect e-skin for measuring the forces under the foot, and how should it be designed to optimize the functionality for ankle rehabil- itation? 2. How should the data-processing system be designed to allow for accurate and efficient measurement of parameters that are important for ATR rehabilita- tion? Even though the long-term goal is to develop a commercial product, this master’s thesis will present an initial prototype that will be limited in scope. The most sig- nificant limitation is that the system will only be constructed in one size, for one size of foot. Another major limitation is that the signal processing system will not be designed for an end-user, focusing instead on simplifying the development and evaluation of core systems. In a final product, the user would ideally receive imme- diate feedback from the sensor system through an app, with the medical provider automatically getting data about the progress. This is left for future work. This thesis will also not consider the portability or long-term reliability of the system. 1.2 Literature review Previous research has used a variety of techniques to monitor the rehabilitation of ATR [5], [9], [10]. One simple and common test is the heel-rise test, where the patient stands on the toes of one foot and lifts the heel. The height of the lift and endurance are recorded and compared between the injured side and the non-injured side, to evaluate the rehabilitation progress [9]. Other tests are also used to compare the function of the injured and non-injured Achilles tendon, such as range of motion and jumping height [5], [10]. 2 1. Introduction Another technique to evaluate the function of the calf and Achilles tendon is to analyze the gait during walking [5], [11]. This can be done by placing markers on the body of the patient and recording the patient while walking with multiple cameras, to accurately determine the movement of leg and foot joints [11]. It is also possible to use medical imaging techniques to measure the tendon elongation during the healing process, for example using ultrasound [11]. Patient-reported outcome scores, such as the Achilles Tendon Total Rupture Score (ATRS), are also used to evaluate the recovery [10], [12]. For the ATRS, the patient will answer 14 questions related to the symptoms of their ATR injury, focusing on pain, experienced strength, and limitations in everyday tasks [12]. Some studies have used plantar pressure sensors such as the commercial Tekscan F-Scan system [13]. These sensors have an array of up to 966 points measuring normal force, and since they are very thin and flexible they can easily be put into any shoe or orthosis [14]. Aufwerber et al. [15] used a similar portable plantar pressure sensor measuring normal forces, to monitor ATR recovery and found that these sensors are useful as an objective measure of the recovery process. However, they noted that the critical measurement of load on the Achilles tendon might not be directly inferred from plantar pressure measurements alone. The proposed system in this thesis, in addition to measuring normal force, also measures shear forces and ankle joint angles, our hypothesis is that this additional information will enable a more accurate estimation of the load of the Achilles tendon. Another difference from sensors used in previous studies is that this project aims to develop a sensor that can provide real-time feedback on the recovery process directly to the patient, as opposed to the data only being available to care providers. Figure 1.2: Structure of magnetic-field-based sensor, showing the effect of applying a force to the silicone. The sensor consists of silicone, a magnet, and a magnetic field sensor (Hall effect sensor). The force sensor in this project is based on the principle first proposed by Tomo et al. [16], which is a magnetic 3D force sensor. It constists of a magnet suspended in silicone over a magnetic field sensor (hall-effect sensor). When force is applied to the sensor, the silicone deforms, which makes the magnet move, and the hall-effect sensor measures a change in the magnetic field. From this change, the magnitude and direction of the applied force can be estimated [16]. This principle is shown in Figure 1.2. This technique has been adapted to several applications, with many variations from the original proposal [17], [18]. The contribution of this research 3 1. Introduction is that the silicone is attached to the circuit board in a novel way, multiple sensor elements are placed in a large array with efficient communication, and the array can bend in one direction. There are many other systems for measuring plantar pressure, for various applica- tions [19]. Tang et al. [20] presented an insole with integrated plantar pressure and shear force measurement, similar to the system proposed in this thesis, but in the application of managing foot ulcers common among people with diabetes. In con- trast with the magnet-based sensors in this project, their measurement technology is based on capacitive force sensors, which have a size of 20 mm x 20 mm x 1 mm. Another difference is that their sensor is thinner and more flexible, although it only measures forces at four points under the foot, compared to 73 in this project. 1.3 Ethical aspects This project could result in patients getting much more direct insight into their reha- bilitation process since they could get real-time feedback on their recovery process. The benefit of this is two-fold. Firstly, this can lead to a faster and less painful re- covery for the patient, resulting in an improved quality of life. Secondly, this would free resources from healthcare professionals, reducing the workload per patient. There is a risk that this technology enables better rehabilitation only for people of higher socio-economic status, who can afford to access them. This is dependent on the cost of the technology, as well as the healthcare system of the specific country. However, a future product could also be used in medical research, contributing meaningful data to determine how the rehabilitation process can be improved. In this way, even people who do not have access to the device can benefit from this project. The goal is to be able to adapt the product to the patient so that the system will be effective regardless of patient weight, age, size, or gender. This is however a major challenge, and there is a risk that the system will be a bad fit for some patients. Another goal is that the final system should be reusable, making it less expensive per patient and also reducing environmental effects. In this master’s thesis, there will not be any patient testing. This is because of the additional certifications and ethical considerations needed to ensure patient safety. This is left for future work. This project contributes mainly to the UN sustainable development goal G3: Good health and well-being by improving the quality of care for patients with ATR. In addition, it also contributes to the following goals. G8: Decent work and economic growth, reducing the workload for healthcare providers. G9: Industry, innovation and infrastructure, creating new jobs, markets and fostering innovation. G12: Re- sponsible consumption and production, the modular and reusable components of the proposed technology decrease the need for manufacturing new devices. [21] 4 2 Theory This project is inherently multi-disciplined, combining biology, medicine, electronics, programming, and mechanical engineering. For this reason, overviews of the relevant parts are presented in this chapter. 2.1 Achilles Tendon Even though this is a technical project, knowledge about the Achilles tendon is needed in the design process, to ensure that the final product will be helpful for patients. This section describes the relevant anatomy of the foot. It also provides details about the ATR injury and describes the usual recovery process after an ATR injury. 2.1.1 Anatomy The Achilles tendon is the strongest tendon in the human body and plays a large part in walking and running [5]. It is connected in one end to the heel, onto the calcaneal bone, and in the other end to the gastrocnemius and soleus muscles [6]. The anatomy is shown in Figure 2.1. Figure 2.1: Anatomy of the Achilles tendon with its connecting muscles in the lower leg. Source: Injurymap [22], CC BY 4.0 5 2. Theory The function of the Achilles tendon is to flex the foot downward, plantarflexion, for example, to stand on the toes, and to thrust the body forward while walking [5], [6]. This movement is also shown in Figure 2.2. The foot rotates around the ankle joint, giving the Achilles tendon a moment arm of about 35mm at the neutral position, increasing by around 2 mm at 20 degrees plantarflexion [11], [23]. The force in the tendon can reach the equivalent of around 12.5 times the average body weight [5]. Figure 2.2: Illustration showing the plantarflexion and dorsiflexion movements of the foot. Source: OpenStax College [24], CC BY 3.0 2.1.2 Rupture Rupture of the Achilles tendon typically occurs in healthy individuals during exces- sive loading, often while playing a sport [7], [11]. The incidence rate is reported to be between 6 and 55 injuries per 100,000 persons per year and rising, likely due to increased interest in sporting activities later in life [5], [7]. Typically the patient will experience a sudden painful “snap” behind the ankle as the tendon is torn apart [6]. ATR results in a gap in the tendon and an inability to flex the foot downwards (plantarflexion) [6]. Since plantarflexion is important for walking, running, and jumping, rupture of the Achilles tendon can lead to severe disability [5]. The most common test to diagnose ATR is to squeeze the calf and to see if the foot moves as expected, which is both simple to perform, and has a high sensitivity and specificity [5]. 6 2. Theory Figure 2.3: Illustration showing the rupture of the Achilles tendon. Source: In- jurymap [22], CC BY 4.0 2.1.3 Recovery procedure After the injury has been diagnosed as ATR, there are two alternatives for treat- ment. The first is to perform surgery to re-connect the tendon, and then casting or bracing the foot joint for 6-8 weeks. The second alternative is to skip surgery and immediately put the foot into a cast or brace. Which of these alternatives is the best is still controversial. Most reviews have concluded that the re-rupture rate is lower when treated surgically, but this also introduces additional risks of infection and scarring. [5] Some studies suggest that appropriate rehabilitation is more important to recovery than the choice of surgical or non-surgical treatment [10]. Rehabilitation has gen- erally shown improved outcomes in ATR recovery, but which regimen produces the best result is still controversial [25]. Starting weight-bearing within the first 6 weeks is considered beneficial to the patient, this entails walking on the injured foot with a cast or an orthosis [5]. Typically the heel is lifted up to 30 degrees in the orthosis to minimize the elongation of the Achilles tendon during the healing process. There is evidence that different amounts of heel raise will cause changes in biomechanics when walking, but the effects of these changes on ATR recovery are poorly under- stood [26]. With the help of the system proposed in this thesis, it might be possible to further investigate the effects of heel rasie and other parameters. Between 6 and 11 weeks, the brace can be removed and specific rehabilitation exer- cises are prescribed to the patient. These include heel-rises, balance training, range of motion exercises, and walking. The difficulty is gradually increased according to the recovery progress, and after 3-4 months the patient can return to sporting activities [5]. Long-term follow-ups have shown that differences between the injured and uninjured foot persist years after treatment [27]. 7 2. Theory Even though rehabilitation is considered beneficial, there is still disagreement about the details, such as the optimal type of orthotic used, the timing of when weight- bearing on the injured foot should start, and which exercises should be performed [25], [26], [28]. One of the difficulties of evaluating the effectiveness of any ATR treatment is that there are many different criteria used in different studies, which makes it harder to compare studies and draw conclusions. Commonly used metrics are re-rupture rate, calf muscle strength, tendon elongation, foot kinematics, and time to return to normal activity [11], [28]. 2.2 Communication protocols The proposed system consists of many devices that need to communicate with each other. These all have different requirements for throughput, wiring, and robust- ness. This section describes the various digital communication protocols used in this project, as well as their benefits and drawbacks. 2.2.1 SPI Serial Peripheral Interface (SPI) is a communication standard that is used for low- level communication between integrated circuits (ICs) over short distances. There is no single specification for SPI, leading to many different varieties of SPI depending on the functionality needed [29]. However, SPI devices from different manufacturers are generally cross-compatible. A typical SPI bus consists of a main controller (also called master or host), usually a microcontroller or processor, connected to one or more subnodes (also called slave or target). The electrical interface typically uses four wires, two wires for data in each direction (MISO, MOSI), one wire for clock signal (CLK), and one wire called “chip select” (CS) to select which subnode is active. For each additional subnode added another chip select signal is needed, while the data and clock signals can be shared between all subnodes. This allows the controller to send and receive commands and data from multiple subnodes connected to the same bus [29]. A typical connection is shown in Figure 2.4. 8 2. Theory Figure 2.4: Typical SPI topology, with one main and three subs. It can be seen that the MISO, MOSI, and CLK lines are shared between all devices, while there is one CS line per device. The maximum speed achievable depends on a variety of factors, such as device constraints, the distance between devices, and the layout of the circuit, but typical clock speeds are 1-50 MHz. SPI is simple, with low overhead and relatively fast speed, making it a good choice for communicating with multiple peripherals over short distances. Over long distances, it is susceptible to electromagnetic interference. [30] 2.2.2 I2C I2C, Inter Integrated-Circuit, is another serial communication standard. As the name suggests, this protocol is mainly used for communication between ICs, usually on the same circuit board[31]. An I2C bus consists of a controller and several targets. A significant advantage of I2C is that only two signal lines are needed, no matter how many targets are connected to the bus. One is the clock line (SCL), and one is the data line (SDA). Instead of selecting the active sub by an external electrical signal, as is done in SPI, each I2C target has a specific address. When the controller communicates with a specific target, it first sends the target address, followed by the data, and then the target can send a response [31]. To facilitate communication in multiple directions with only one data wire, the I2C protocol uses what is called “open-drain” signaling [31]. This means that the 9 2. Theory connected devices can either drive the signal to zero volts (drain) to send a zero, or they can disconnect (open/high-Z) to send a one. Pull-up resistors ensure the signal reaches high (VCC) voltage when no devices are driving it low. The topology, with the pull-up resistors, is shown in Figure 2.5. This configuration ensures that there are no short circuits if one device sends a zero at the same time as another device sends a one [31]. Figure 2.5: Topology of a typical I2C bus, with one main, two subs, and the mandatory pull-up resistors. Only two signals are needed, and they are connected to all devices. The main benefit of the I2C protocol is that no matter how many devices are con- nected to the bus, only two wires are needed, provided all the devices have different addresses. However, to achieve this the protocol is relatively complicated, and the speed is limited to 400 or 100 kHz, making it unsuitable for high-throughput devices [31]. 2.2.3 USB USB (Universal Serial Bus) is a widely used communication protocol, generally used for communication between computers and peripherals. The original USB specification from 1996 allows speeds up to 12 Mbit/s, subsequent additions to the specification have introduced speeds of up to 80 Gbit/s, while keeping backward compatibility with devices that only support the old specifications. Electrically, USB uses differential signaling over (at least) one pair of wires. This makes the signal robust against interference and can be carried over larger distances, up to several meters. The USB protocol includes advanced functions such as hot-swapping, device con- figuration, power supply adjustments, error correction, and more. This makes USB good for consumer-facing products with medium to high data throughput needs. However, the complicated protocol also leads to a large overhead and complicated logic, which makes it unsuitable for low-level communication. 10 2. Theory 2.3 Electronics This section provides an introduction to the various electronic components that are used in this project, as described in part 3.3. 2.3.1 Printed circuits Printed circuits are used in virtually all modern electronic devices as a way to connect components electrically, as well as mechanically [32]. Two types of printed circuits are used in this project: the standard printed circuit board (PCB) in section 3.3.4, and a flexible printed circuit (FPC) in section 3.3.1. 2.3.1.1 Structure A PCB consists of a laminate of conducting and isolating layers, with specific pat- terns cut out in each layer to form particular features. PCBs are described by how many conducting copper layers are used, which is usually 2 or 4 for simpler prod- ucts, going to 20+ in advanced PCBs [33]. These copper layers serve as the wiring of the PCB, with traces connecting the different components. Between the copper layers, there are insulating layers, usually made of FR4 fiberglass. To connect traces on different layers, as is often necessary, vias can be used, which are copper-coated through-holes. On the top and bottom of the PCB, there is an insulating layer called solder mask, which covers all of the copper traces. Openings in the solder mask will make pads, which are exposed copper areas that components can be soldered to [33]. Figure 2.6 shows a diagram of these features. Figure 2.6: Cutout diagram of a 2-layer PCB, showing the structure of a via, pad, tracks, as well as the stackup. 2.3.1.2 Design Considerations Electronic Design Automation (EDA) programs are used to design PCBs. The design generally starts with the schematic, where the components and connections are specified. Then the physical layout of the PCB can be designed. The connections from the schematic are automatically used in the layout to ensure that components 11 2. Theory are connected where they should and that there are no short circuits. The program can additionally check for violations of manufacturer design rules, such as too little margin between traces, too small holes, or traces too close to the PCB edge. [33] Ideally, the structure of the PCB would not affect the electrical function of the device, as its purpose is simply to connect the components electrically. This can be assumed to be the case in basic designs. However, when there are high-speed signals or high-power components, more careful PCB design is required, because the physical properties and layout of the traces will have a noticeable impact on the function of the device. These unwanted effects of the traces are called “parasitics”, and can consist of resistance, capacitance, and inductance. [33] For high-current signals, the trace resistance will be a consideration, since it will result in a voltage drop and a temperature rise. To mitigate this, the trace can be made wider, or the length of the trace can be minimized. For high-speed signals, the situation is more complex. Noise can come from “reflections” where there is a change in trace characteristics, cross talk from adjacent traces due to magnetic and electric fields, and many other sources. Generally, the higher the frequency, or faster rise time of the signal, the more consideration needs to be put into the routing of the signal. [33] To minimize these effects, some rules of thumb should be used. The first is to allocate (at least) one entire copper layer to be connected to ground, often called a “ground plane”. This ensures all signal return currents have a short, unobstructed path back to ground, minimizing coupling between signals. Any signals that need to be routed on this ground layer should be kept short, and traces should not be routed above interruptions in the ground plane, if there are any. The next guideline is to put decoupling capacitors as close as possible to any component power-input pins. These should be connected with thick traces to power and ground, to minimize noise and coupling from the supply voltage. Another simple guideline is to keep signal tracks as far away from each other as is practically possible to minimize coupling between signals. [33] 2.3.1.3 Flexible Printed Circuits The structure of a flexible printed circuit is very similar to standard PCBs, the main difference being that the rigid FR4 core is substituted with a thin polyamide film. Figure 2.7 shows a typical FPC stackup, the “Coverlay” serves the same function as the PCB solder mask. This makes the whole circuit flexible and thinner, between 0.1 and 0.3 mm compared with the typical 1.6 mm for a rigid PCB. It also introduces a few extra design considerations that must be taken into account when designing an FPC. Most of these relate to the mechanical stresses introduced when bending, where there is a risk that the copper might crack due to the stress. In any sections where the FPC is expected to bend repeatedly, vias should be avoided, tracks should be perpendicular to the bend, corners should be avoided or at least rounded, and solid planes should be replaced by cross-hatched planes. It is especially important to avoid components where the FPC is expected to bend, otherwise, the solder joints can crack, greatly reducing reliability. Stiffeners are often placed in select locations 12 2. Theory on the FPC, to constrain bending to areas designed to bend. [32], [34] Figure 2.7: Typical stackup of a two-layer FPC with FR4 stiffener. 2.3.2 Hall-Effect Sensors In this project, hall-effect sensors are used in the force measurement, as described in section 3.2.1. The function of Hall-effect sensors is to measure magnetic fields. This is done by utilizing the Lorentz force, which relates magnetic fields to electric currents [35]. A general description of hall-effect sensors is given in this section. Figure 2.8 shows a simplified measurement setup, however, a detailed explanation of the physics will not be given here since it is outside of the scope of this document. Figure 2.8: Measurement setup for hall effect sensing. An applied magnetic field Bz will apply a force on the flowing electrons, causing a charge difference between the two sides of the plate. This is measured by the voltage probe VH . Source: Steve Byrnes [36], public domain Hall-effect sensors are solid-state devices without any moving parts, which means that they are not exposed to the same wear that might affect other types of sen- sors. Another advantage is that they can be manufactured using standard silicon processes, so miniaturization and cost-cutting efforts in the semiconductor industry have also made hall-effect sensors small and cheap. The sensors can also easily be combined with digital circuitry on the same chip, which enables sensing, signal pro- cessing, and communication in one integrated package. It is also possible to arrange multiple sensing elements in the same package so that the magnetic field can be measured in three dimensions. [35] 13 2. Theory One inherent problem with magnetic field measurements is that there are many possible sources of interference, such as from the earth’s magnetic field, stray mag- netic fields from high-current wires, other magnets in consumer products, or nearby ferro-magnetic materials. Another drawback with silicon-based hall-effect sensors is that they can be susceptible to temperature variations, both the gain and offset can be affected by changes in temperature. [35] 2.3.3 Binary Decoder A decoder is a common electrical component that is used to increase the number of controllable outputs. In this project this is used for driving a large number of SPI chip select signals, as described in section 3.3.1.1. It does this by converting a binary-coded input into a specific set of outputs. For a 1-of-n decoder, using x number of inputs will result in one out of x2 outputs being activated. In addition, there is often an “Enable” input that activates or deactivates the outputs, as is common in logic circuits. This type of decoder is also called “address decoder”. [37] As an example, the symbol and truth table of a 2-to-4 decoder is shown in Figure 2.9. 2-to-4 decoder EN EN 0 ? ? 0 0 0 0 1 0 0 0 0 0 1 1 0 1 0 0 1 0 1 1 0 0 1 0 0 1 1 1 1 0 0 0 Inputs Outputs Truth table Figure 2.9: Schematic symbol and truth table for a typical 2-to-4 decoder. Any combination of inputs Xi results in one of the outputs Yi being activated. If the enable input “EN” is not active, no outputs are active. 2.4 IMU sensor fusion An Inertial Measurement Unit (IMU) is a device used for estimating and tracking orientation. It is a combination of two sensors, one 3D accelerometer that mea- sures linear acceleration, and one 3D gyroscope that measures angular velocity. It is also possible to include a 3D magnetometer to measure the Earth’s magnetic field. Without the magnetometer, it is usually referred to as a 6-DoF (Degrees of Freedom) IMU, with the magnetometer it is called 9-DoF.[38] Other solutions for estimating orientation are available [39], but IMU systems are superior for many applications. They are small, low-cost, energy-efficient, and do not need any exter- nal fixed references [39]. In this project, the placement of the IMUs is discussed in section 3.3. In theory, it should be possible to calculate the orientation by simply integrating 14 2. Theory the gyroscope measurements, provided that the initial orientation is known, since the angular velocity is the derivative of the orientation. In practice, this is not feasible since the gyroscope is not perfect. Often, there is a bias in the measurement that would make the orientation drift over time. The accelerometer in turn will, at steady-state, measure the acceleration from the Earth’s gravitation, which provides a reference for the downward direction, that can be used to correct the drift of the gyroscope. The accelerometer is however affected by any movement and also suffers from noise and bias. Similarly, the magnetometer provides a reference to north, which is needed to avoid drift in the horizontal plane. This reading is even more noisy and prone to disturbance, due to the relative weakness of the Earth’s magnetic field. Sensor fusion algorithms can be used to combine the data from the gyroscope, accelerometer, and magnetometer, so the weaknesses of each sensor can be compensated for by the others, and a stable, accurate estimate of the orientation can be obtained. [39] There are many different sensor fusion algorithms, depending on the requirements and limitations of the application. The main tradeoff is between accuracy and computational cost. Different filters also have different performances depending on the movement patterns in the specific application, where some might perform better in fast, large movements, and some in slower more gradual movements. Some common filters used in practice are complementary filters, Mahony filters, Madgwick filters, Kalman filters, and many others. [38] 2.5 Robot Operating System The Robot Operating System (ROS) is a software development kit specifically de- signed for robotics. It provides a set of fully open-source tools and programs, to enable faster development of high-performance systems [40]. In this project ROS2 is used, an upgraded version of ROS which improves reliability, performance, and security by using DDS as a middleware [41]. ROS2 works on top of many different operating systems. For the selected release, ROS2 Iron Irwini, Windows 10 and Ubuntu 22.04 are best supported. A ROS2 system generally consists of a number of “nodes”, which are programs that each execute some specific function, as well as an infrastructure of “topics” and “services” which allow the nodes to communicate with each other [41]. 2.5.1 Node A Node is a standardized architecture for a program in ROS2, generally used to execute one function in the system. The node architecture provides an interface for activating and executing the node, as well as supplying parameters and accessing other ROS2 utilities [41]. Nodes are organized into packages, which are usually entire sub-systems. ROS2 has official support for programming in Python and C++, support in other programming languages is maintained by the ROS community. 15 2. Theory 2.5.2 Communication ROS provides a simple, robust, and efficient way to communicate between nodes, which is very important for a complex modular system. There are two basic com- munication types in ROS2: topics, and services [41]. Topics provide publisher/- subscriber functionality, a form of many-to-many communication where any node can write to the topic (publish) and read from the topic (subscribe). Topics are asynchronous, meaning publishing and subscribing can happen independently at any time. Services are instead direct node-to-node type communications, where a client node sends a request to a server node, which sends a response back to the client. ROS2 also provides flexibility in the Quality of Service (QoS) settings for each communication channel, which allows for tradeoffs between reliability, memory usage, and speed [41]. 2.5.3 Interfaces All of the communication is based on interface definitions, which are specifications for what kind of information is being communicated. These definitions are indepen- dent of programming language and communication implementation, which means that they allow a ROS2 node to communicate with other nodes regardless of how they are implemented or where they are running, as long as they share an inter- face definition.[41] ROS2 comes with many built-in message types, such as integers, coordinate frames, images, various sensor types, and others. Custom messages can easily be defined by combining basic types and previously defined messages into a new message type. As an example, the definition of a std_msgs/Header message is shown in Section 2.5.3.1. It can be seen that the Header message consists of a builtin_interfaces/Time message called stamp, which itself consists of two inte- gers, and a string called frame_id. Appendix A lists the definition of the other messages used in this project, all of which contain a Header message. 2.5.3.1 Header message std_msgs/Header builtin_interfaces/Time stamp int32 sec uint32 nanosec string frame_id 2.5.4 TF2 package The TF2 package handles transforms between coordinate frames, a central concept in ROS2. TF2 frames are used to describe the relative position and orientation of different parts of the robotics system [42]. In this project, frames are used to describe the orientation of the sole and the position of the ankle joint, as described in Section 3.4. TF2 provides functions to track these parts, to calculate transformations of points into different coordinate frames [42]. A visualization of three frames is shown in Figure 2.10. 16 2. Theory Figure 2.10: Three TF2 coordinate frames. The relationship between the frames are shown as yellow/pink arrows. 2.5.5 Micro-ros Since ROS2 runs on a high-level OS like Linux, it is often not suitable for interfacing directly with hardware. This is because high-level OSes struggle with strict timing requirements, and access to low-level electrical peripherals is difficult. Instead, a dedicated microcontroller can be used to interface with the hardware and pass the information onto the computer. Traditionally a specific device driver would be programmed to communicate with the micro-controller device from ROS. Micro- ros instead provides a standard way to communicate between micro-controllers and ROS2 systems [43]. It enables an embedded microcontroller to integrate into the ROS2 system as a normal node, with the ability to communicate with other nodes through the standard ROS2 communication modes: topics, services, and actions [44]. The controller presented in Section 3.3.4 uses micro-ros to seamlessly communicate with the rest of the ROS2 system, as described in Section 3.4.1. Micro-ros is available for a large number of microcontrollers and is also flexible with regards to which RTOS and build system is used. This flexibility makes micro-ros suitable for a large number of applications where hardware needs to be controlled from ROS2 [43]. In order to communicate with the micro-ros device, an additional program called the “agent” needs to run on the ROS2 computer, which acts as a bridge to the micro-ros device [43]. This structure is shown in Figure 2.11. The agent is a standard program provided by the ROS organization and generally does not need to be modified for each use case. There are several protocols available for communication between the agent and the micro-ros device, such as serial, wifi, or ethernet [44]. 17 2. Theory Figure 2.11: Architecture of micro-ros system, the dark-blue components are de- veloped specifically for micro-ros. The leftmost components (over uP) run on the ROS2 computer, and the rest of the components run on the microcontroller. Source: micro-ros [44], CC BY-ND 4.0 2.6 Equivalent force and torque This work involves summing and moving forces to analyze the forces and torques in different frames of reference, such as in the ankle joint as presented in Section 3.4.4. A brief description is given of how to sum and move forces and torques. A number of forces acting on a body can be represented by a single resultant force and torque in one point p ∈ R3×1. Given a set of n forces Fi ∈ R3×1 applied to a body at the points pi ∈ R3×1, and a set of m torques Tj ∈ R3×1 applied to the body, the resultant force is the sum of the individual forces Fi [45]: F = n∑ i=1 Fi (2.1) The resultant torque T is the sum of the individual torques Ti and the torques generated by each individual force Fi [45]: T = m∑ j=1 Tj + n∑ i=1 (pi − p) × Fi (2.2) 18 3 Methodology & Design This chapter will describe the methods used to develop the sensor sole, the steps taken to design and manufacture the system, and their implementation details. 3.1 Methodology The development of this project was done in multiple parts, starting with the de- velopment of the sensor sole, which was then integrated with the controller into the orthosis. Then the software system was completed, based on physical modeling of the system. Tests were done throughout the process, to analyze the performance of each part. As the work progressed, earlier parts were sometimes revised and improved after testing, following an iterative approach. The design of the system was guided through a set of design goals. Primarily, the insole should be able to measure the forces accurately through the entire range of loads that can occur in human walking. Special care needs to be put into the design for the system to be usable as a shoe sole, which means that the system has to be robust to withstand the loads under the foot and to be ergonomic for the user. Additionally, the system should process the data fast enough to allow for meaningful real-time analysis. During the development, emphasis was placed on modularity. This means that it is easy to add and improve sub-systems without reworking all of the system, simplifying development. Modularity is also beneficial in a final product as it simplifies creating customized solutions for different patients in different situations. 19 3. Methodology & Design 3.2 System Overview Figure 3.1: Picture of the insole and controller unit. The system can be split into hardware and software. The hardware system includes the patch with its sensing nodes, a controller unit connected to a computer, and the orthosis where the sensor system is mounted. The software system consists of a ROS2 system with modules, messages, configuration files, as well as micro-ros code running on the controller unit. The system connections are shown in Figure 3.2. Figure 3.2: System components and the connections between them. 3.2.1 Working principle of a sensor node Central to this system is the 3D force measurement, which is based on the research presented by Tomo et al. [16]. The construction of a node is shown in Figure 3.3. It consists of a magnet suspended in silicone above a hall-effect sensor IC. When 20 3. Methodology & Design force is applied, the magnet will move as the silicone deforms. The hall-effect sensor, which measures the magnetic fields in the X, Y, and Z directions, will then register a change in the magnetic field. Once calibrated, these changes in the magnetic field can be used to estimate the magnitude and direction of the applied force in 3D. Figure 3.3: Structure of a single 3D-force measurement node, showing the magnet, suspended in silicone, above the hall-effect sensor. 3.2.2 Organization of E-skin system: patch, module and node To allow for future modular systems where the layout of sensor elements may be different, the insole was conceptually divided into patch, modules, and nodes. Start- ing from the smallest element, a single 3D force measurement point on the insole is called a node. A number of these nodes are arranged into modules, which represent a distinct area. In this case, there are three modules roughly corresponding to the area under the toes, middle of the foot, and heel. These three modules together make up the patch, equivalent to the entire sole. A diagram with the patch layout with nodes and modules is shown in Figure 3.4 Figure 3.4: Layout of patch, modules, and nodes. The numbers 0 to 72 represent the different nodes of the patch. 21 3. Methodology & Design 3.3 Hardware Design The hardware system consists of three main parts: the sensorized sole, the controller unit, and the ankle foot orthosis. The sensor insole measures the forces under the foot and consists of a flexible printed circuit, electronics components, magnets, silicone, and top cover. The insole is connected to the controller unit, which uses a microcontroller to read the insole sensors, process the signals, and communicate with the computer system. An overview of the electrical system is shown in Figure 3.5. Figure 3.5: Schematic of the electrical system, showing the components and con- nections of the controller unit and insole. 3.3.1 Insole Flexible Printed Circuit Inside the sole, the main component is the Flexible Printed Circuit (FPC). It serves as an electrical interconnect for all of the components on the sole, as well as providing mechanical stiffness and flexibility where needed. FPCs are described in detail in Section 2.3.1.3. In this project, the FPC has two copper layers with a thickness of 12 µm (1/3 Oz) each, an inner polyamide layer with a thickness of 25 µm, and 1.2 mm thick FR4 fiberglass stiffeners under the nodes. The stiffeners were placed in strips from side to side so that bending is only possible along one axis, see Figure 3.6b. This ensures that the FPC does not flex where the components are mounted, which could otherwise deteriorate reliability. The FPC was manufactured using a low-cost standard process by a circuit board manufacturer (JLCPCB.com). 22 https://jlcpcb.com/ 3. Methodology & Design (a) FPC with components soldered. (b) Backside of FPC showing stiffeners. Figure 3.6: Pictures of flexible printed circuit 3.3.1.1 Components The sensorized insole includes many individual components, most importantly the hall-effect sensors. Hall-effect sensor theory is explained in Section 2.3.2. The chosen hall-effect sensor is the MLX90393 from Melexis [46], which has a ±50 mT maximum measurement range and up to 500 Hz readout rate in a small 3x3 mm package. Communication with these sensors can either be done through I2C or SPI. SPI is used in this project because of its higher possible data rate, and because a maximum of 4 hall-effect sensors can be connected to each I2C bus. The SPI bus uses four signals to communicate, three of these can be shared by all sensors and are thus connected in a chain. The Chip Select needs to be a separate signal for each of the 73 hall- effect ICs. Since only one chip-select needs to be active at a time, 4-to-16 decoders of the type 74HC154 [47] are used to drive these signals effectively, which reduces the number of input signals for chip-select from 73 to 9. More explanations of SPI, I2C and decoders are given in theory Sections 2.2.1, 2.2.2, and 2.3.3, respectively. The insole also includes a LSM6DSL 6-axis IMU [48]. By measuring the linear acceleration and angular velocity of the sole, the IMU makes it possible to estimate the orientation and motion of the sole, as described in Section 2.4. This IMU uses the I2C protocol to communicate. The FPC has a mounting spot for an additional IMU, however, due to a design error, communication was impossible when both were mounted at the same time. Therefore, only the one at the heel was mounted. In order to connect the sensors on the insole to the controller unit, a 24-pin flat flexible cable (FFC) connector is used, also known as zero-insertion force (ZIF) connector. The signals through the connector include three signals for SPI commu- nication, two pins for I2C communication, nine signals for chip-select decoding, as well as 3.3V power and ground. In addition, four SMT test clips, Harwin S2751-46R [49], are placed around each hall-effect sensor, as shown in Figure 3.3. These clips serve no electrical purpose, instead, they act as anchors for the silicone to attach to. The signals of the SPI bus are high frequency (max 10 MHz) and long. To ensure signal integrity, the design guidelines introduced in Section 2.3.1.2 are followed. The IMU, hall-effect sensors, and encoders all have the recommended capacitors placed close to their power pins to suppress possible disturbances in the power supply. 23 3. Methodology & Design There is a ground plane on the entire top copper layer, where any other necessary traces on this layer are kept as short as possible. The SPI bus is also routed in a long chain to avoid branches, which could otherwise cause reflections in the signal. The full design of the insole PCB can be found in the Hardware folder in the git repository [50]. Figure 3.7 shows the SPI bus traces. Figure 3.7: Design of the SPI bus traces on the insole FPC, connected in a single chain through all sensor nodes. 3.3.1.2 Assembly The components were soldered to the FPC in a solder paste reflow process. First, solder paste was applied to the pads by using a metal stencil. Then all of the components were placed onto the corresponding footprints on the FPC. The FPC with the components and solder paste was then placed in a LPKF ProtoFlow E reflow oven, where the solder melted and bonded all of the components to the FPC. Finally, a manual adjustment of certain pads was required to correct specific solder joints. Figure 3.8: Picture of a single node and the connector soldered on the FPC. 24 3. Methodology & Design 3.3.2 Silicone layer with integrated magnets As described in Section 3.2.1, the silicone layer is integral to the function of force sensing. The elasticity of the silicone allows the embedded magnet to move when force is applied. The amount of force needed to move the magnet depends on the hardness of the silicone. In this project, Smooth-On Smooth-Sil™ 945 was used, which has 45A shore hardness [51]. Figure 3.9: Schematic overview of the silicone molding process. In order to integrate the magnets into the silicone, the silicone was molded in two pourings, the steps are shown in Figure 3.9. The first pouring involved most of the silicone, which fully encloses the FPC. An upper and a lower mold were 3D-printed, the FPC was placed between these pieces, and silicone was poured in. Details were added in the design of the mold to hold the FPC in place, and to leave holes in the silicone for the magnets. When the silicone had cured, it was removed from the molds. Then the magnets were placed in the holes, with their magnetic fields all pointing up (north away from hall-effect IC). The selected magnets are cylindrical, axially magnetized, with a diameter of 1 mm and a height of 1 mm. Figure 3.10 shows the two halves of the first mold, along with the FPC. Figure 3.10: Lower half, FPC, and upper half of the first mold. For the second pouring, a new 3D-printed mold was fitted on top of the existing silicone structure, as shown in Figure 3.11, silicone was poured in, and a piece of glass was placed on top to ensure that the top was flat. This second pouring covered all the magnets so they were fully encased in silicone. 25 3. Methodology & Design Figure 3.11: Second mold fitted on the silicone, before second pouring. 3.3.3 Top cover During testing by standing on the sole, it was found that a substantial part of the force was put between the nodes. This resulted in highly inconsistent measurements. To solve this problem, a 1.5 mm thin sheet of polystyrene was cut into shape and put on top of the sole, as shown in Figure 3.12. This semi-rigid plastic piece acts as a spatial low-pass filter, spreading out the applied force onto multiple sensor nodes, so that any force applied between nodes is distributed and properly measured by adjacent nodes. The effect of the top cover can be seen in Section 4.3. Figure 3.12: Sole with top cover on, placed inside the orthosis 26 3. Methodology & Design 3.3.4 Controller unit design The purpose of the controller unit is to collect readings from all of the hall-effect sensors and IMU, and to forward the data to the ROS2 system running on a com- puter. The controller unit is designed to be mounted just above the ankle on the lower leg, and is held in place with velcro straps, as shown in Figure 3.13. Figure 3.13: Controller unit attached to orthosis. The controller unit consists of a custom PCB, with an IMU, and a ZIF connector identical to the one on the sole. For communication and data processing a commer- cial M5Stamp S3 module is used, which includes a micro-controller with supporting circuitry, an antenna, and a USB-C connector [52]. The micro-controller is a ESP32- S3, which has a dual-core 240MHz processor and a wide array of communication capabilities such as SPI, I2C, USB, Wi-Fi, and Bluetooth low energy [53]. Figure 3.14 shows the design of the PCB. 27 3. Methodology & Design Figure 3.14: Design of controller PCB in ECAD design software. 3.4 Software system design The function of the software is to control the hardware, read the sensors, calculate forces, and estimate other parameters that are relevant for ATR rehabilitation. The code is split into multiple programs across multiple devices. All of this is contained in a single ROS2 system, which allows for easy communication between programs and simplifies development. The ROS2 system consists of several packages and nodes, with the idea of making a modular system where parts are easy to add, modify, or replace, explained in detail in Section 2.5. The basic structure of the system, with nodes, topics, and message types, is shown in Figure 3.15. It can be seen that the data starts in the controller unit, which controls the hardware and reads the low-level data from the insole. This information is sent to the ROS2 system running on the Ubuntu laptop, where a series of nodes calculate the force and estimate parameters that are potentially useful for ATR rehabilitation. An Rviz visualization also subscribes to the data in multiple places in the chain, to visualize the output of each node. A launch file was used to start all of these nodes with their proper configurations, which is described in the project git repository [50]. 28 3. Methodology & Design Figure 3.15: Overview of the nodes, topics, and message types in the ROS2 system. 3.4.1 Micro-ros hardware node The purpose of the insole hardware node is to communicate with the sensorized insole and publish this data into the ROS2 system. This node is special in that it is running on the microcontroller on the controller unit, using micro-ros to communicate with the rest of the ROS2 system. The reason for this is that the low-level communication protocols used in the sole, SPI and I2C, require precise control of timing and electrical signals, which is hard to achieve using a high-level OS like Linux. Micro-ros is described in more detail in the theory section 2.5.5. Figure 3.16: Structure of the micro-ros system, with the micro-ros agent and client. In this diagram, the circle is the ROS2 node, the topics with message types are represented with rectangles, and hexagons represent the services, with the service name and type specified. The micro-ros node (Sole_HW) connects the electrical system of the insole with the ROS2 system on the laptop. 29 3. Methodology & Design As can be seen in Figure 3.16, the insole hardware node publishes a custom Tactile- SensorData message to the /sole_hw_data topic. This topic contains an array of 73x3 16-bit integers that are directly obtained from the hall-effect sensors’ magnetic field measurements, two standard IMU messages, and a standard header containing a timestamp. There is also a /cal_imu service, which can be called when the insole is lying flat to calibrate the IMU offsets. After they are calibrated, the orientation estimate will be more accurate with less drift. The microcontroller itself is programmed with the PlatformIO ecosystem, which consists of an IDE and a set of tools to simplify the development of embedded de- vices. With PlatformIO, a single configuration file is used to specify hardware and manage dependencies, which makes it easy to modify both software and hardware components in the project. Arduino is used as the base framework on the micro- controller since it is simple and has a large collection of third-party open-source libraries. In this project, open-source libraries were used for the IMU [54], sensor fusion [55] and Hall-effect sensors [56]. The hall-effect sensor library was modified to work with decoder addressing. Micro-ros is also available as a PlatformIO package and was easily integrated into the code. 3.4.2 Calibration nodes The calibration package includes the first nodes run on the ROS2 Ubuntu laptop. Its function is to take in the raw data from the hardware node and to output calibrated force data. The calibration package is also responsible for publishing the location of each node in each module, as this information is not sent from the hardware system. This placement is loaded as part of the package configuration file, along with parameters for the calibration model. 30 3. Methodology & Design Figure 3.17: Nodes (circles), Topics with message types (rectangles), and services (hexagons). The patch and module nodes communicate with shared memory. The package is composed of two types of nodes, one is the patch node, the other the module node. When the patch node is started, it subscribes to the hardware messages and starts several module nodes. When a hardware message is received from the /sole_hw_data topic, the patch node will distribute the data to each module node. The module node will then calculate the force in each of its nodes, and publish the location and force of each node in the module in a PointCloud2 message to the /module_n/node_forces topic. This layout is shown in Figure 3.17. The calibration model consists of a simple linear mapping from magnetic field strength to applied force. This is shown in Equation 3.1, where BHE is the mea- sured magnetic field strength in Tesla, K is the calibration constant in N/T , and F̂ is the estimated force in Newton. There is also an offset in the magnetic field measurements that is present even when no force is applied, offsetHE, which can be compensated for by calling the /module_n/calibrate service with no load on the sole. F̂ = K · (BHE − offsetHE) (3.1) 31 3. Methodology & Design 3.4.3 Center of pressure nodes Figure 3.18: ROS2 system for the center of pressure calculations. The center of pressure nodes are used in two levels, as shown in Figure 3.18. First, module-wise centers of pressure are calculated. These centers of pressure are then used to calculate the net center of pressure for the entire patch. The first layer of nodes subscribe to the force measurements from the /module_n/node_forces topics, and publish to the /module_n/cop topic. The second layer node in turn subscribes to all of these /module_n/cop topics and publishes to the /patch/cop topic. Mathematically, the center of pressure consists of a point pCOP located at the mean of the forces. The position of the point pCOP ∈ R3×1 is calculated as an average of all the positions of the nodes, weighted by the magnitude of the force in the z direction (normal force): pCOP = N∑ i=0 pnodei · Fnodei,z Ftot,z (3.2) Ftot,z = N∑ i=0 Fnodei,z (3.3) Where pCOP is the position of the center of pressure, pnodei is the 3D position of node number i, Fnodei,z is the z-axis force measured by node number i, and Ftot,z is the sum of all measured z-axis forces. Once this position is calculated for each module, the equivalent force and torque in this position are then summed from all the node forces from the corresponding module, as described by the theory in Section 2.6. The nodes publish a TransformStamped message containing the position of the CoP, as well as a WrenchStamped message, containing the 3D force, 3D torque, timestamp and reference to the CoP transform frame. 32 3. Methodology & Design 3.4.4 Ankle joint torque and force node Figure 3.19: Node and topics for the ankle joint wrench. The final step of the implemented signal processing chain is to calculate the ankle joint force and torque. This is done by a simple change of frame of the patch COP. The node and relevant topics are shown in Figure 3.19. The force is the same the /patch/cop, the torque can be calculated with the equation presented in Section 2.6. This is then published in a WrenchStamped message to /joint_wrench, together with a TransformStamped which contains the position of the ankle joint. The position is configured as parameters to the node. 3.4.5 Rviz visualization Figure 3.20: Topics that the Rviz node subscribes to. For visualizing the data the standard rviz2 package was used. This is built-in to ROS2 and can be used to visualize many of the built-in message types used in this project, such as Transform, Wrench, and PointCloud2. A screenshot of the visualization is shown in Figure 3.21. 33 3. Methodology & Design Figure 3.21: Screenshot of the visualization in Rviz. The colored circles show the forces of each sensor node. In addition, COP and joint forces (red arrows), and torques (yellow arrows) are shown. 3.4.6 Test bench node Figure 3.22: Structure of the fh-500 node with topics. The node During testing, a force probe model “Sauter fh-500” [57] was used, which is shown in Figure 3.23. This tool can apply and measure up to 500 Newtons [57]. This force probe provides a serial interface to read the sensor measurements. This interface was easily integrated into the ROS2 system with a node that reads the serial port, parses the serial message into Newtons, and publishes the force as a Wrench message, as shown in Figure 3.22. 34 3. Methodology & Design Figure 3.23: Sauter fh-500 test bench with sensor sole. 35 3. Methodology & Design 36 4 Experimental Validation and Results This chapter presents the tests to evaluate the system performance, starting with validating single node measurements, to entire system tests. During many of these tests, the Sauter FH-500 force probe test bench was used, presented in Section 3.4.6. 4.1 Node raw values The objective of the initial test was to evaluate how well the nodes can measure forces. This was done by determining the response of a single node when force is applied, obtaining a model that can estimate the applied force from the hall-effect sensor readings, and evaluating the accuracy of this estimated force. Shear forces were not included in this test, since the force probe can only apply force in the normal direction. 4.1.1 Test setup The insole was placed, without the top sheet, into the Sauter test bench in such a way that the tool only pressed a single node, see Figure 4.1a. The tests were performed for nine nodes in the center of the sole, nodes 1, 2, 3, 6, 7, 8, 11, 12, and 13. All tests followed the same procedure: starting from 0 N, the force was increased and decreased between 0 N and 150 N, with pauses at 50 N and 100 N. This was done to evaluate the response both during loading and during unloading. Since the test bench is manually operated, the test runs are not identical, however, they all follow the trajectory shown in Figure 4.1b. 37 4. Experimental Validation and Results (a) Detail view of the force test setup. 0 50 100 150 200 250 time [s] 0 50 100 150 F o rc e [ N ] (b) Force profile applied during a test run. Figure 4.1: Test setup for evaluating single node response. During the tests, the raw values from the hall-effect sensor were recorded, together with the measurements from the sauter test bench, into a ROS2 bag file. The data was then imported into MATLAB and analyzed. It was found that the Sauter test bench has a built-in low-pass filter, which introduced a delay in the validation data. To compensate for this, the hall-effect measurement data was low-pass filtered in Matlab as well. A 10-sample rolling average filter gave the best match. The hall- effect measurement was then plotted against the pressed force and a curve was fitted, to determine the relationship between applied force and hall-effect reading. 4.1.2 Results Figure 4.2 shows the node reading on the y axis, in milli Tesla, against the applied force in Newton on the x axis. What can be seen is that the response is quasi-linear, there is however a pronounced hysteresis depending on the direction of loading or unloading the sensor nodes. 38 4. Experimental Validation and Results Figure 4.2: Relationship between applied force and hall-effect reading (sensor node reading). Applying the same test to multiple sensor nodes, shown in Figure 4.3, it can be seen that the other nodes respond similarly, with the same shape of the hysteresis curve. There is however variation in the absolute values of the hall-effect readings, for example, node 1 goes from 20 mT to 35 mT, while node 13 varies between 25 mT to 45 mT for the same applied force. Force [N] M a g n e ti c F ie ld [ m T ] 0 50 100 150 20 30 40 Node 1 0 50 100 150 20 30 40 Node 2 0 50 100 150 20 30 40 Node 3 0 50 100 150 20 30 40 Node 6 0 50 100 150 20 30 40 Node 7 0 50 100 150 20 30 40 Node 8 0 50 100 150 20 30 40 Node 11 0 50 100 150 20 30 40 Node 12 0 50 100 150 20 30 40 Node 13 Figure 4.3: Hall-effect reading vs applied force for nine nodes. A straight line was then fit to each node’s data, referred to as the individual calibra- tion, in order to estimate the applied force from the hall-effect reading. Another line was fit to the combined data from all tested nodes, this is referred to as the common calibration. Both of these models are the same as the linear model introduced in 39 4. Experimental Validation and Results 3.4.2, neither model compensates for the hysteresis behavior. The performance of the calibration models is shown in table 4.1, where K is the gain of the model, R2 is the coefficient of determination, and e is the error between the estimates and the true applied force. Fit data K [N/mT] R2 RMS(e) [N] max(e) [N] Individual 7.82 ± 0.64 0.978 ± 0.004 7.7 ± 0.7 19.0 ± 1.7 Common 7.70 0.978 ± 0.004 10.1 ± 1.5 25.3 ± 3.7 Table 4.1: Performance metrics of the calibration models (mean ± standard devi- ation). As expected, individual calibration of each node gives better results, which can be seen in the error performance in Table 4.1. The estimates and true values for one test run are shown in Figure 4.4. It can be seen that the estimates are least accurate at the upper end of the range, and the individual calibration performs slightly better than the common calibration in this case. 0 20 40 60 80 100 120 140 160 180 200 time [s] 0 50 100 150 F o rc e [ N ] Applied force Individual calibration Common calibration Figure 4.4: Estimated and true force during a test run. 4.2 Long-Time Hysteresis The second test aimed to evaluate any time-dependent effects in the force sensing. 4.2.1 Test Setup The physical setup was identical to the single node test in 4.1. The test procedure was to apply 100 N to a single node for two minutes, remove the applied force, and record data for another three minutes. 4.2.2 Results Looking at the measured magnetic field after the force has been released, after 170 seconds in Figure 4.5, it can be seen that the sensor takes some time to settle back to the original zero-level. Even after three minutes (at 350 seconds), the magnetic field is around 150 µT higher than the original level, equivalent to roughly 1 N with 40 4. Experimental Validation and Results the calibration model presented above. This shows that there is a time-dependent effect, which most likely comes from mechanical relaxation of the silicone. The cause for the sudden jump at 310 seconds requires further analysis. 50 100 150 200 250 300 350 Time [s] 19.8 20 20.2 20.4 20.6 20.8 M a g n e ti c f ie ld [ m T ] Force applied Figure 4.5: Magnetic field measurement before and after a 100 N force has been applied. 4.3 Top cover impact As discussed in Section 3.3.3, the goal of the top cover is to distribute the applied force over a larger area. Therefore, any force applied between sensor nodes is also measured accurately. The following test was performed to see the impact of this top cover. 4.3.1 Test setup To evaluate the effect of the top cover, a 160 N force was applied to the center of four nodes using the Sauter test bench. This was done both with and without the top cover. The measured magnetic field strengths were calibrated to force using the common calibration from 4.1.2, and then summed and compared to the applied force. 4.3.2 Results As can be seen in Figure 4.6, the sensor performs much better with the top cover on. Without the top cover, the nodes measure a negative force when pressure is applied between the nodes in this way. This is obviously not the desired behavior. With the cover on, the force is spread out as desired, and the total measured force is close to the correct values. 41 4. Experimental Validation and Results 0 5 10 15 20 25 30 35 40 45 Time [s] -100 -50 0 50 100 150 200 F o rc e [ N ] Applied force Total measured force Individual nodes force (a) Applied and total measured force without the top cover. 0 5 10 15 20 25 30 35 40 45 Time [s] -20 0 20 40 60 80 100 120 140 160 180 F o rc e [ N ] Applied force Total measured force Individual nodes force (b) Applied and total measured force with the top cover. Figure 4.6: Total measured force without and with the top cover. 4.4 Walking in boot This test evaluates the full system in a scenario similar to the specified use case. This test is mostly qualitative, since there is no data available for verification (ground truth). The test subject was not an ATR patient. 4.4.1 Test setup The orthosis with the insole and controller unit mounted, was put on by a volunteer tester. The foot was kept at a neutral angle (0◦ flexion) in the orthosis. The person then walked 20 steps in a straight line, while recording the data from the insole and from the ros system. The data was then post-processed in MATLAB. The data generated from a step can be viewed in the demo video available at: youtube.com/watch?v=23ohwQikxd4 [58]. 42 https://www.youtube.com/watch?v=23ohwQikxd4 4. Experimental Validation and Results 4.4.2 Results 0 0.5 1 1.5 2 2.5 3 3.5 4 Time [s] -10 0 10 20 30 40 50 F o rc e [ N ] Color Legend Figure 4.7: Force measured by each node during one step with the smart walker. The color represents each sensor node and its location on the sole. Figure 4.7 shows the forces measured by each node during a typical step. The step starts with most of the force applied to the heel of the foot (t ≈ 1.3), which corresponds to the red lines in the figure. As the step progresses, more load is put onto the toes and forefoot (t ≈ 2.5), represented by the blue and turquoise lines. It can be seen that the orange and yellow lines, which correspond to the nodes at the midfoot, are never put under much load. Figure 4.8 shows the maximum force put on each node, which also shows that the heel is put under the highest load, then toes and forefoot, while less force is put on the nodes under the midfoot. 43 4. Experimental Validation and Results 0 50 100 150 200 250 X p o s it io n [ m m ] -50 0 50 100 Y position [mm] 5 10 15 20 25 30 35 40 45 Figure 4.8: Maximum force of each node during one step with the smart walker. Figure 4.9 shows the joint torque and total force that is calculated by the ROS2 node from section 3.4.4. A negative torque corresponds to plantarflexion, pushing the toes down, which is the torque contributed by the Achilles tendon. It can be seen that at each step, the joint torque initially goes positive, and then goes negative before returning to zero when the step is finished. This corresponds to the expected response when transferring pressure from the heel to the forefoot during each step. 0 0.5 1 1.5 2 2.5 3 3.5 4 Time [s] -600 -400 -200 0 200 400 600 T o ta l fo rc e [ N ] -60 -40 -20 0 20 40 60 J o in t T o rq u e [ N m ] Step progression Figure 4.9: Total force (blue) and joint torque (red) during one step with the smart walker. Negative torque in this case corresponds to plantarflexion. The torque starts positive at the start of each step and then goes negative. 44 5 Discussion and Conclusion This chapter presents an analysis of the function of the various parts of the system, what can be improved in future work, and a conclusion that answers the research questions based on the achieved results. 5.1 Discussion Starting with the mechanical design and construction, the project was on the whole successful. The developed system shows promising results for ATR analysis, al- though much work is needed before the launch of a commercial product. The hard- ware design involved balancing many trade-offs between competing goals, such as accuracy, cost, durability, and ergonomics. A major tradeoff for a future product is that of cost versus spatial resolution, since every added node means one more hall- effect IC and thus higher cost, as well as a more complex electrical design needed if node density is to be increased. It is likely that the number of nodes in the system can be reduced without a significant decrease in utility, reducing costs and complex- ity. Another tradeoff is that of flexibility versus accuracy since flexing the sensor might introduce unwanted movements in the silicone that can not be distinguished from movements that occur when force is applied. The innovations in the design of the silicone layer were largely successful. Specif- ically, the attachments developed, the clips and the holes in the FPC, allowed for disturbance-free bending of the sole, which was a major challenge. The solid attach- ments to the rigid part of the FPC allowed the silicone to move between nodes when bending the sole, without inducing movements in the silicone around the magnet, avoiding disturbances of the measurements. The electrical system was also mostly successful, showing that it is possible to read a large number of sensor ICs with a single SPI bus, combined with decoders for the chip select signals. No disturbances in the low-level communication were noticed, even with clock speeds up to 10 MHz. Using a flexible printed circuit instead of a rigid PCB proved to be successful, although some concerns about the durability of the FPC remain. Typically an FPC is not subjected to such forces that can come from walking, running, or jumping, so the long-term durability is likely a challenge. 45 5. Discussion and Conclusion 5.1.1 Measurements and calibration A surprise in this work is the apparent linearity of the raw sensor values since both the magnetic field strength and the silicone deformation are highly non-linear. This might be because the changes are relatively small, or that the non-linearities acci- dentally compensate each other. The linear calibration is however not accurate for the hysteresis, most likely an inherent material property of the silicone. To improve this, either the hardware can be changed, by changing the material or geometry, or the calibration model can be improved to handle the hysteresis. Another limitation of the current calibration model is that it is only tested for forces in the Z-direction. To allow for the calibration of shear forces, a different test setup is needed that is able to apply and measure forces in three dimensions. As described in Section 4.1.2, there is quite a large variation in measurement re- sponse between nodes. This could come from a variety of factors, the largest of which is thought to be variations in the silicone layer, coming from variations in the 3D-printed mold and defects such as bubbles or inadequate mixing. The magnet is placed only 1.0 mm from the hall-effect sensor, therefore, even sub-millimeter im- perfections can have a substantial impact. Other possible sources are variations in magnet size and strength, and variations between the individual hall-effect sensor ICs. Either way, if higher accuracy is needed, it would be advantageous to cali- brate each node individually. Therefore, a more sophisticated calibration method is needed. 5.1.2 Software The software system with ROS2 at its core proved to be a good solution. The large ecosystem of ROS2 allowed for rapid development and analysis using the many built-in tools. The modularity facilitated by ROS2 also helps development, since different parts can be improved or replaced independently. Micro-ros fits very well into this system, greatly simplifying interfacing with low-level hardware from ROS2, compared to writing custom communication protocols or OS drivers. Since micro- ros also works together with Arduino, another large ecosystem of libraries and tools can be utilized. ROS2 also allows integration of the insole into other systems in the future. A drawback with ROS2 is that it needs a computer running a full operating system, something that might be too expensive, bulky, or power-consuming for a portable consumer product. A simple microcontroller would likely be enough to collect and process the data, without the need for the ROS2 system. 5.1.3 Untested features The IMU was briefly tested, together with a basic sensor fusion algorithm, and worked well. This is shown in the demo video [58]. However, as mentioned in 3.3.1, there is only one IMU mounted on the sole. Since the sole is flexible, it would be advantageous to have more IMUs mounted on the sole. This would make it possible to estimate how the sole is bent. Using SPI communication instead of I2C would be 46 5. Discussion and Conclusion a beneficial change if more IMUs are added. 5.2 Future work This section gives recommendations on what areas can be improved in future versions of this system. 5.2.1 Calibration One possible improvement that has been discussed above is the calibration. Right now the calibration model does not accommodate shear forces, hysteresis, or non- linearities. These behaviors are quite complicated, therefore a model-based approach might not be feasible. Instead, a data-driven approach could be implemented, which uses a large amount of data to train a machine-learning model. To generate the needed training data, an automated testing system could be set up, that applies force to each node and records the applied force. A testing system like this would allow for easy and accurate individual calibration of each node. Many types of machine-learning models might be appropriate for calibration, such as conventional feed-forward neural networks or convolutional neural networks, that can capture the non-linear behavior of the node. However, due to the hysteresis effects noted in sections 4.1.2 and 4.2.2, it might be beneficial to use a model that can include information from past measurements. An approach directly suited for sequential data processing is a Long short-term memory (LSTM) model, that uses a “memory” of previous time steps to compute the estimate for the current time step. Generally, a more complex model can give a more accurate calibration, which comes at the price of more training data needed, and increased computational demands. 5.2.2 Hardware One easy adjustment that should be made to the system is to move the connector on the FPC. The current placement is close enough to the user’s foot that there is a risk of irritation from rubbing. The current placement also forces the control unit to sit on the inner side of the ankle, where it is possible to accidentally kick it with the other leg. These issues would be solved if the connector was moved to the other side of the foot and extended further from the sole. Another improvement left for future work is to make the hardware more modular- ized. Right now, the sole consists of a single large FPC. This makes it hard to adapt the sole for different users since a new FPC needs to be designed for each foot size. If the sole instead consisted of several smaller modules, it would be easier to customize for different users, which is very important for a medical product. Hav- ing multiple discrete modules would also introduce new challenges, mainly in the connections between modules. Physically, the cables need to be robust and flexi- ble, with connectors that do not require much space. Ideally, they should also be easy to reconnect in such a way that healthcare professionals can reconfigure the 47 5. Discussion and Conclusion sole without consulting a technician. A more modular system would also introduce challenges in communication between the modules. Another potential improvement in the hardware is to integrate the top plastic sheet. Right now, the top cover is simply placed on top of the rest of the sole, but this risks misalignment. The problem with rigidly attaching the top cover to the sole is that it will likely affect the bending of the sole, in turn putting undesired forces on the nodes, resulting in incorrect measurements by the system when the sole is bent. 5.2.3 ROS2 system The ROS2 system has allowed for easy prototyping of software, and the built-in tools, including visualization, are very useful. The connection to micro-ros is also seamless and allows for easy communication with low-level devices from within the ROS2 system. The modular nature of ROS2 enables ergonomic modifications of the system; any node can easily be re-configured, reworked, or replaced as needed. ROS2 has a mature ecosystem, with good documentation and high performance. However, for a portable, low-cost product, which is the long-term goal for this project, ROS2 might have too high a computational cost. Once the algorithms have been implemented and tested in ROS2, they could likely be optimized and fit on a smaller microprocessor, which would make the system smaller, more energy efficient, and cheaper. 5.2.4 End-user experience If this project were to progress to where the system is used directly by patients, several improvements to user-friendliness need to be made. Ideally, the system would be easy to use, portable, and ergonomic. This might include wireless communication between the sole system and an app running on the user’s smartphone, where data that is relevant to the recovery process could be presented in real-time. In terms of ergonomics, safety, and robustness, more testing is needed to evaluate what needs to be improved. 5.3 Conclusion To answer the research questions, they are repeated here for convenience. • 1. How suitable is the hall-effect e-skin for measuring the forces under the foot, and how should it be designed to optimize the functionality for ankle rehabilitation? The presented system shows that the hall-effect e-skin is a feasible technology for measuring forces under the foot. The measurements are consistent even with the bendable FPC, thanks to the structure of the silicone layer and the novel silicone attachment methods. The drawbacks are mainly that it is impossible to change the shape of the proposed hall-effect e-skin without a major rework of the FPC, and the robustness of the system is still not known. 48 5. Discussion and Conclusion • 2. How should the data-processing system be designed to allow for accurate and efficient measurement of parameters that are important for ATR rehabil- itation? The accuracy of the force measurements is relatively poor with the linear calibra- tion model, although it can likely be improved with a more advanced calibration model. However, even with the achieved accuracy, relevant parameters such as the joint torque were successfully estimated. ROS2 is a very effective framework for data processing in the given application since it allows for fast prototyping and vi- sualization of algorithms. The addition of micro-ros into this system is relatively straightforward, enabling easy access to hardware from ROS2. The performance of ROS2 is more than enough for what is needed for this use case, however, in a portable consumer product, ROS2 might not be the best choice since it requires a computer running a high-level operating system. 49 5. Discussion and Conclusion 50 Bibliography [1] World Heath Organisation, “Rehabilitation 2030 initiative,” 2024. [Online]. Available: https://www.who.int/initiatives/rehabili