I know that there is no a clear answer for this question, but let's suppose that I have a huge neural network, with a lot of data and I want to add a new feature in input. The "best" way would be to test the network with the new feature and see the results, but is there a method to test if the feature IS UNLIKELY helpful? Like correlation measures etc?
$\begingroup$ $\endgroup$
1 
- 1$\begingroup$ A non-random correlation might be an indicator that the feature is useful. But I'm not so sure about pre-training tests that could rule ideas out. The paper you link makes it clear that non-linear correlations are not well detected by the available tests, but a neural net has a chance of finding and using them. $\endgroup$Neil Slater– Neil Slater2014-07-10 11:28:15 +00:00Commented Jul 10, 2014 at 11:28
Add a comment |
2 Answers
$\begingroup$ $\endgroup$
6 A very strong correlation between the new feature and an existing feature is a fairly good sign that the new feature provides little new information. A low correlation between the new feature and existing features is likely preferable.
A strong linear correlation between the new feature and the predicted variable is an good sign that a new feature will be valuable, but the absence of a high correlation is not necessary a sign of a poor feature, because neural networks are not restricted to linear combinations of variables.
If the new feature was manually constructed from a combination of existing features, consider leaving it out. The beauty of neural networks is that little feature engineering and preprocessing is required -- features are instead learned by intermediate layers. Whenever possible, prefer learning features to engineering them.
- $\begingroup$ I always thought to compare the value to predict with the features, you are talking about correlation between features. Is your answer applicable also to my case? in theory I should add only new features that are correlated to the value to predict, right? $\endgroup$marcodena– marcodena2014-07-10 19:06:57 +00:00Commented Jul 10, 2014 at 19:06
- $\begingroup$ That's also a valuable metric -- just updated my answer to address that as well. $\endgroup$Madison May– Madison May2014-07-10 19:18:34 +00:00Commented Jul 10, 2014 at 19:18
- $\begingroup$ In short, strong correlations with the value to predict is a great sign, but weak correlation with the value to predict is not necessarily a bad sign. $\endgroup$Madison May– Madison May2014-07-10 19:19:17 +00:00Commented Jul 10, 2014 at 19:19
- $\begingroup$ Thanks. I'm writing a report and I wanted to show the linear/non-linear correlations in order to justify the features (even before the results). Does it make any sense? From your answer I could make a matrix of correlations but maybe it's nosense $\endgroup$marcodena– marcodena2014-07-10 19:27:33 +00:00Commented Jul 10, 2014 at 19:27
- 1$\begingroup$ I would use non-linear correlations, but ok thanks $\endgroup$marcodena– marcodena2014-07-10 23:17:47 +00:00Commented Jul 10, 2014 at 23:17
