I started dabbling in neural networks quite recently and encountered a situation which is quite strange (at least with my limited knowledge).

The problem I'm using a NN is a regression problem which tried to predict the sales of a product for a particular kind of promotion in an FMCG.

Although the data is not strictly time series, it still has some time related attributes like a prediction may depend on a similar kind of promotion last year (which I've modelled using feature engineering).

Now to the problem I'm having:

I took the data from 2015-2017,augumented it by adding small amount of noise, shuffled it, and ran it through a neural network(I think the architecture is not important, but let me know if it is, I'll try and post it).

Optimizer: Adam with a decay Loss Function : Volume weighted mape Validation set : Randomly selected 20% of the data

The network trained well and gave me errors as small as 8%. And both training and validation set errors decreased together and did not indicate an overfit.

But the kicker is that when I applied the algorithm to new unseen data(2018, first half), the errors rose to 55%. .

I tried researching the issue on Google but didn't find anything useful. What is happening here? Am I doing something wrong?

I started dabbling in neural networks quite recently and encountered a situation which is quite strange (at least with my limited knowledge).

The problem I'm using a NN is a regression problem which tried to predict the sales of a product for a particular kind of promotion in an FMCG.

Although the data is not strictly time series, it still has some time related attributes like a prediction may depend on a similar kind of promotion last year (which I've modelled using feature engineering).

Now to the problem I'm having:

I took the data from 2015-2017,augumented it by adding small amount of noise, shuffled it, and ran it through a neural network(I think the architecture is not important, but let me know if it is, I'll try and post it).

• Optimizer: Adam with a decay
• Loss Function : Volume weighted mape
• Validation set : Randomly selected 20% of the data

The network trained well and gave me errors as small as 8%. And both training and validation set errors decreased together and did not indicate an overfit.

But the kicker is that when I applied the algorithm to new unseen data(2018, first half), the errors rose to 55%. .

I tried researching the issue on Google but didn't find anything useful. What is happening here? Am I doing something wrong?