1
$\begingroup$

I was attending Andrew Ng Machine learning course on youtube Lecture 6.4 He says what a cost function will look like if we used Linear Regression loss function (least squares) for logistic regression Lecture slide

I wanted to see such a graph my self and so I tried to plot cost function J with least square loss for a losgistic regression task.

Here is my code

import matplotlib.pyplot as plt import numpy as np import math x = np.random.rand(10000) # x = np.array([0.1, 0.2, 0.7, 0.4]) y = np.round(np.random.rand(10000)) b = 1 J_list = [] w_list = [] for w in np.arange(-500.5, 500.5, 0.05): J = (1/10000)*np.sum((1/2)*np.square(((1/(1 + np.exp(-1*(w*x + b)))) - y))) J_list.append(J) w_list.append(w) df = pd.DataFrame() df['w'] = w_list df['J'] = J_list import seaborn as sns sns.lineplot(x='w', y='J', data=df)

The output of lineplot is

cost function with least square loss in logistic regression

Note: w in my code in theta in Andrew Ng's lecture

If anyone can help me spot my mistake, would be really appreciated.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

You have taken x,y from random space. So, it should not follow any rule/logic.

Get a real data e.g. Iris(just 2 class) or using sklearn make_classification module
Figure out the approx value of theta for a good model
Then loop before and after this value
Have y_true in probability(not the Class)

Improve this answer
$\endgroup$
3
  • $\begingroup$ Yes you reason is correct. But 'Have y_true in probability(not the Class)' you just meant to convert a Binary label in 0 and 1 right ? $\endgroup$ Commented Jul 10, 2020 at 7:15
  • $\begingroup$ I meant to use the probability output of the model, not the Class. Till probability calculation only, we have any maths functions involved. So, it will depict the underlying variance accurately. Post that we basically do a Floor/Ceiling on the output based on a good Thresold e.g. 0.6 --> 1.0. This should be done only at the end. $\endgroup$ Commented Jul 10, 2020 at 7:32
  • 1
    $\begingroup$ yes. The output of the models are in probability only. $\endgroup$ Commented Jul 10, 2020 at 7:39

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.