Skip to main content
Cross Validated

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

2
  • $\begingroup$ you said in your answer that "Also instability can be measured more directly by comparing predictions for the same test case across models from different runs/iterations/repetitions of k-fold cross validation.",this cause another question for me,how can a test set remains same but but the runs/iterations/repetitions number change?i mean in k-fold in each runs/iterations/repetitions we use different part of whole dataset as test and the remaining parts as training dataset , so we could not have a same test set while k change.(i hope i understand your answer correctly otherwise i am sorry.) $\endgroup$ Commented Jun 20, 2016 at 20:20
  • 1
    $\begingroup$ Not the whole test set for the surrogate model is the same: e.g. sample no. 3 may be tested together with samples 1 and 5 in run (i), and with samples 7 and 10 in another (ii). This also means, that samples 7 and 10 are in training set (i) and samples 1 and 5 are in training set (ii). So any difference in the predictin of sample 3 must be due to the difference in the training sets = model instability. $\endgroup$ Commented Jun 24, 2016 at 20:05