Let's say I have a dataset of 2 million. At first, I used only 1 million, trained those and saved the model in h5 format like first.h5. Later I used another 1 million data, trained those using the same algorithm and saved as second.h5. Training requires more than a day , hence I can't use all two million data at once. Is there any way , I can merge those two saved model like first.h5 + second.h5 = merged.h5
Add a comment |
2 Answers
There is no way you can do that (merge models). Let me put it in simple terms. You train a child named first using some 1 million data to identify if an image is a cat or a dog. Then you trained a second child named second using the other 1 million data to identify if an image is a cat or a dog. Now what you are asking for is to combine the first and second.
However, assume that the training data is IID (independent and identically distributed) then what you can do is create an ensemble of both the models for making predictions.
The simple way to ensemble two models is are
- Max Voting
- Averaging
- Weighted Averaging
Follow this link on how to the ensemble.
Or a simple strategy is to average the final score of both the models and use the averaged score to make the predictions.
A more powerful strategy is to use the validation set to find the weights for the classes and then use these weights for making the final predictions on unseen data.
2 Comments
Mithun Sarker
How can I create an ensemble ? Could you kindly give me a hint ?
makeyourownmaker
See How to Develop a Weighted Average Ensemble for Deep Learning Neural Networks The machinelearningmastery site sometimes feels a bit click-baity. Some of the articles are clear and straight-forward but not much of the mathematical background is ever covered. The above article is probably OK as a starting point and covers keras.
You could merge - average weights - but this will not be the same as training with full dataset.
Usually training with more data leads to better results, to better model.
If you don' t want to train with full dataset i would recommend not to average weights but to use both models for inference and average predictions.
1 Comment
Dr. Snoopy
There is no guarantee that taking the weight average will produce valid neural network weights.