In this blog post we dive into predictive maintenance tutorial from the technical perspective. The post is targeted for developers and data scientists. The focus is in mechanical industrial equipment.
- Machine learning concepts for predictive maintenance
- Introduction for the predictive maintenance tutorial in GitHub
- Conclusion and challenges in the predictive maintenance work
As a background, you can read the previous writing about predictive maintenance for business.
Data scientist needs to understand the phenomenon
To create predictive machine learning models for maintenance purposes, the key is to understand the underlying phenomenon. This is my view at the moment. If you are predicting machine transmission failures, you should have at least basic understanding about the component.
Youtube is a good place to start:
The opposite and more common approach is to take whatever data happens to be available. And then run the training algorithm with the default parameters in the hopes of getting 90% accuracy.
Sometimes you achieve the 90% accuracy and go home happy. Unfortunately you did not realize that the machine learning model is predicting always the same result: No transmission failure. The seemingly great accuracy is explained by the fact that 90% of the time the transmission does not have a failure.
Critical evaluation of the results is one important aspect in the data science and predictive maintenance. But there are even bigger reasons to be familiar with the phenomenon you are exploring.
The problem of infinite options in predictive maintenance
In data science it is more important to think what is needed instead of what is available.
There are pretty much always infinite number of data sources and ways to prepare the data. Chances to hit the jackpot eyes closed is next to zero.
To move under the predictive maintenance umbrella, let’s say you are predicting whether a component is failing during the next week or not. After picking the most promising data sources, you must decide which features indicate the component to breakdown in near future.
Feature is a machine learning term for a variable that is expected to predict the outcome. To predict the speed of a car, most important features might be throttle pedal position, current gear and engine size.
Maybe 3-day continuous increase in horizontal vibration sensor is a signal about upcoming failure? Or maybe high variation for 3 hours in the vertical vibration sensor is more important indicator? You can easily create thousands of similar features even from one sensor. A single connected machine might have tens of similar sensors measuring temperature, pressure and other values.
Yes, there are ways to automatically detect which features most potentially explain the future failures. But it does not help if you are missing the right data sources and features. In that case each of the neural network architectures are equally bad.
If you have even a rough idea what conditions lead to breakdown of a specific type of component, it is relatively easy to create a machine learning model to describe the phenomenon on the code level. So talk with people and explore in the real world as much as possible.
Asking the right question in predictive maintenance
A machine learning model answers to one well defined question. In the predictive maintenance context these could be:
- Does a specific component in specific equipment breakdown in the next 3 days?
- What is the probability of the equipment to breakdown tomorrow?
- How many hours of maintenance a specific equipment needs next month?
The setup can be made more intelligent by combining these narrow scoped models. A car hiring company could build three different models to predict probabilities for engine, brakes and transmission to fail tomorrow. By picking the highest probability, it would be possible to prioritize the maintenance of the whole fleet of cars.
Predictive maintenance tutorial in GitHub
Go to GitHub for the full Predictive Maintenance Tutorial. The example solution is written in Python.
In this predictive maintenance tutorial there are two hand-crafted datasets. They represent equipment data from imaginary Gadget Company Ltd.
The first dataset contains measurements from various sensors of the gadgets.
The seconds dataset is simple. It has only the failure time and the id of the failed gadget.
The assignment is to build a predictive model to predict the failures from the sensor measurement data.
The most of the code lines will be consumed to create meaningful features such as mean of temperature during past 6 hours. The algorithm training and testing will usually be much more straightforward.
Here is my example solution in the GitHub.
Results in the tutorial addressed typical problems
It was educational to develop a machine learning model from self-generated data.
My best model with moderate effort and typical practices was able to capture 95% of the failures. Unfortunately more than 70% predictions were false alarms. From 100 predictions 70 would be false alarms, 30 correct and during that time 2 failures would have been missed.
Precision: Share of failure predictions that were correct.
Recall: Share of failures the model captured.
The model could be balanced for optimal ratio between correct predictions and false alarms for the given business problem. The benefits are dependent on the value of correct prediction versus the cost of a false alarm.
Thoughts about the predictive maintenance tutorial
I expected better results from the dataset where I knew all the tricks. I even cheated by using only those features I knew would be relevant for the model. Apparently the data between the train and test set was somewhat different. The measurement data also included wicked fake patterns and randomness that would be difficult to capture without exact thresholds.
On the other hand the situation is mirroring reality well in a sense that data can be pretty much anything.
Sometimes the service staff have forgets to mark the failure to the maintenance log. Sensors might be dirty and produce faulty values. Outliers can be removed to some extent, but some situation are extremely difficult to interpret.
The time dimension brings an additional level of complexity in predictive maintenance data analysis. How to know if events from the last week have impact to tomorrow’s failures? Or is it only yesterday’s values that matter and only when the other sensor is in the specific range?
Fortunately, the perfection is not required in predictive maintenance. Even modest results might bring significant increase in maintenance efficiency.
Mikael Ahonen
Data Scientist at Unikie
