2
$\begingroup$

The gist is me wanting to separate system faults from sensor faults given some dataset from a wireless sensor network using a machine learning algorithm.

For instance, if I have some temperature sensors in a given area and their corresponding readings from every sort of time interval, I would like to know whether an abnormal value is due to an actual fault, or due to a faulty sensor. Of course, it would be a given that the training set would have such entries tagged with either sensor fault or system fault.

I have thought of just using something like linear regression but I would like it to work even if the system could not be modeled like that. Decision tree seemed to me like a more appropriate algorithm for this.

Lastly, there is also some consideration for the time it takes for training and classification as I wish to see if it can be used for systems which respond really quickly to such anomalies.

Sorry if it's a bit wordy but I wasn't sure how much information I should put since this is my first time posting (I'm not even sure if this is the right stack exchange to post this). Anyway, thanks in advance for the answers!

Improve this question
$\endgroup$
5
  • $\begingroup$ Welcome! To bring more detail, how 's the size of training data? Any requirement of accuracy of prediction? $\endgroup$ Commented Apr 29, 2018 at 8:07
  • $\begingroup$ @Sixiang.Hu I would say on the order of a few thousands, at the most. However, since this is supposed to be a general case thing, in the absolute worst case, I may be able to simulate a dataset by generating my own data using some appropriate mathematical model and then introducing some randomness to simulate noise. For the accuracy, since I will be trying to find out whether this method is feasible, there isn't a hard requirement, though it will be great if it reaches 90% or above accuracy $\endgroup$ Commented Apr 29, 2018 at 8:42
  • $\begingroup$ If your data is generated (so as response) , the tree will come up with the logic you define the response. Hence why not just use the logic you already have? Because the tree will just model your logic anyway. $\endgroup$ Commented Apr 29, 2018 at 8:51
  • $\begingroup$ But I want to see how it will perform if the model is unknown, so if I do generate the training set myself using a model, I'll just feed part of it to the tree as training since I'd like to see how close the model the tree will come up with to my actual one. But again, this is for the worst case only where I can't acquire an appropriate dataset. $\endgroup$ Commented Apr 29, 2018 at 9:05
  • $\begingroup$ To concord with the problem definition answer, it would be helpful to know if this is a steady state or transient system. Steady state could mean measurements are independent by sensor over time. That case would be easy, look at statistical process control methods for failure detection. Since the first guess is often correct (most times things are as they appear), and linear regression leaped to mind (absolute differences), I would test the distribution of the absolute differences by sensor over time (scaled) to see if you get a normal distribution. If so, you may have an easier answer without $\endgroup$ Commented Aug 24, 2018 at 3:20

1 Answer 1

Reset to default
0
$\begingroup$

You need to determine how to formulate your problem. I see it as having two aspects: 1. Detect an abnormal value (in temperature) 2. Determine whether abnormal value is due to sensor or system problem

The first is an anomaly detection problem and there is lots of literature on the topic, including tree-based methods on sensor data. One prominent approach is Isolation Forests, which is implemented in scikit-learn. http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.IsolationForest.html

The second is fault localization. This is an area also well described in literature, and tree-based models were used even before machine learning became a thing. A key here is to only train on anomalous data, then this should become a relatively simple classification problem assuming your have relevant features.

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.