2
$\begingroup$

i looked into one of the post about naive bayes calulation of naive part

Predit the class label for instance (A=1,B=2,C=2) using naive Bayes classifcation.

Let C1 be class 1 and C2 be class 2.

enter image description here

For C1, by the assumption of Naive Bayesian Classifier, we have P(A=1,B=2,C=2∣C1)=P(A=1∣C1)⋅P(B=2∣C1)⋅P(C=2∣C1)

Take P(A=1∣C1) as an example. There are 4 training records of C1, among which there are 2 records with A=1. Therefore, P(A=1∣C1)=24. Similarly, you can calculate P(B=2∣C1) and P(C=2∣C1).

It is similar to calculate P(A=1,B=2,C=2∣C2).

My query is how can we calculate the NON Naive part of this ?

P(A=1,B=2,C=2∣C1)= P(A=1∣C1)⋅P(B=2∣C1)⋅P(C=2∣C1) -> Here the events are considered to be independent of each other

What if the events are dependent ? How should we calcualte in that case ?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

I think in practice you would make the assumption of independence in any case. Considering all the dependencies can get quite complicated very quickly and you'd need many observations. For example, in your example A=1, B=2 and C=2 was never observed,but you can't really tell, if that is because P(A=1 , B=2, C=2 | C1) is small and there is too little data, or if P(A=1 , B=2, C=2 | C1)=0. In many practical applications the approximation with independence introduces less error than not having enough observations for all possible events.

There is hybrid approaches, though, where you would model some dependence, e.g. tree-augmented naive bayes. These approaches usually perform better, but are computationally more complicated as well.

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.