1
$\begingroup$

I get some metrics on validation data while training a model , and in my case the they are :

(0.25, 0.31, 0.46, 0.57, 0.65, 0.75, 0.77, 0.78, 0.84, 0.84, 0.85, 0.84, 0.84, 0.84, 0.82, 0.8, 0.8, 0.79, 0.78, 0.77, 0.77, 0.77, 0.75, 0.74, 0.73, 0.73, 0.73, 0.73, 0.73, 0.73)

They can described like this :

enter image description here

In my view , the ideal result should be like : enter image description here

Is it a matter of overfitting ?

Unfortunately , I tried few times to change the regular coefficients to avoid overfitting , and adjust learning rate coefficients to slow down , but it was still "convex" .

How can I achieve the ideal result showed above ?

Much appreciated if anyone would give me some constructive tips ?

Improve this question
$\endgroup$
4
  • $\begingroup$ What model are you training? $\endgroup$ Commented Dec 9, 2016 at 8:34
  • $\begingroup$ General Linear model , such as logic regression $\endgroup$ Commented Dec 9, 2016 at 8:38
  • $\begingroup$ Size of dataset? Type of validation? Still see same effect in k-fold cross-validation? $\endgroup$ Commented Dec 9, 2016 at 9:37
  • $\begingroup$ I used the "toy data" to validate my recommendation system with Spark , so I simplified the whole process . The size of dataset is small (15K) . I tried few tens of times to adjust coefficients , and almost each result came out to be "convex" . $\endgroup$ Commented Dec 9, 2016 at 11:04

2 Answers 2

Reset to default
1
$\begingroup$

Yes, what you are seeing is a classic case of overfitting.

You stated that you use a linear model such as logistic regression. To regularize these types of models, usually L1 and/or L2 regularization is applied. L1 regularization is simply $||W||_1$ and L2 is $||W||_2^2$ usually.

Another method is to alter the labels of the model in a specific way, which is a method of regularization I created (shameless plug). Here is the link to the paper: https://arxiv.org/abs/1609.06693

Hope this helps.

Improve this answer
$\endgroup$
0
0
$\begingroup$

just do cross-validation between test-ds & validation-ds -- to select the moment in iterations, when overfitting starts in order to ignore all further iterations

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.