Determining Maintenance Policies for Partially Observable Multi-Component Systems With Deep Reinforcement Learning

Abstract

In this study, maintenance decisions for partially observable multi-component systems are investigated. Such systems typically operate under conditions where the service provider is remote, and the wear levels of system components cannot be fully monitored with sensors' assistance. Wind turbines provide agood example of these systems. For such systems, besides deciding when the service provider will perform a maintenance intervention, itis also necessary to determine which parts will be taken along to the maintenance point and which components will be replaced after the inspection at the maintenance point. In our study, this complex decision problem is modeled as a partially observable Markov decision process, and related numerical solutions are obtained employing the actor-critic reinforcement learning method. Our numerical studies demonstrate that the policies obtained with the reinforcement learning algorithm outperform several heuristic maintenance policies that are frequently used in practice and wellknown in the relevant literature. In some cases, compared to heuristic policies, these solutions have provided a cost reduction in the range of 10-15% on average. Additionally, it has been observed that the solution obtained with the reinforcement learning algorithm provides more advantages compared to heuristic policies, as the corrective maintenance cost, emergency order cost, and returning cost of excess spare parts increase.