2
$\begingroup$

I have trained a text based sentiment analysis model, using SciKit-learn and custom data. I have the model ready and it works fine in predicting a text to a class (Positive or Negative or Neutral). I have achieved over 85% testing accuracy and around 80% cross validation accuracy.

But I want to get the confidence score attached to each of my prediction to a new example data/text I feed to the classifier. This is just an extra parameter I want to show/output apart from just the predicted class.

I have no idea how to achieve this, I shall be really thankful if anyone can provide some helpful insights.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

I was able to solve it myself after some further research. I will be briefly describing my approach here. Cheers!

The idea is to find the confidence interval which was also same as finding the distance from the decision boundary / hyper-plane, in my case.

If you are using the Scikit Learn API, there is a method called predict_proba() for several classification models like Logistic Regression, SVM, Random Forest, etc. If your classifier does not provide one, you can wrap it with the CalibratedClassifierCV which can be found in sklearn.calibration, then use the above method to calculate the distance from the decision boundary.

If you are looking for custom in-depth implementation, here are some papers / references that might help.

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.