2
$\begingroup$

I have a simple question which I cannot find a straight answer to. I am training a neural network to classify some medical images. I initially focused on validation accuracy after each epoch (to determine how the network was generalising) and then after that, test accuracy on an unseen dataset.

But I see that validation loss is also important - and sometimes my validation loss drops even thought for that epoch, the validation accuracy also goes down slightly. Ultimately test accuracy is going to be the gold standard I guess, but thats no use for guiding training and adjusting parameters. Obviously if the the validation loss starts to go up and validation accuracy starts to drop it indicates overfitting. But what about when both validation loss and validation accuracy drop after an epoch?

Improve this question
$\endgroup$
3
  • $\begingroup$ validation loss and validation accuracy are correlated, well, but not perfectly correlated. Perfect correlation is a myth! There could be one off (outlying) sample in your validation set, that shows this. $\endgroup$ Commented Dec 21, 2017 at 14:35
  • $\begingroup$ Ok, but the question is asking how one might interpret these in terms of which might best reflect generalizability of the algorithm. $\endgroup$ Commented Dec 21, 2017 at 14:37
  • $\begingroup$ "loss is in your hand" (that's what you are optimizing!). "accuracy is God's business". $\endgroup$ Commented Dec 21, 2017 at 14:41

1 Answer 1

Reset to default
1
$\begingroup$

Expanding on my comment, The main goal of your training algorithm is to decrease the net loss. So as long as your validation loss is decreasing, your algorithm is doing good, and is generalizing well. But the validation accuracy is not improving or getting worse, consider relooking at the loss function you are optimizing, it indicates that the loss function is not suitable for the task, rather than training algorithm/model being defective. Go for some other loss function, and then select a model/algorithm that is suitable for that loss function.

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.