Naive Bayes Classifier

Sanjiv R. Das

What is Naive Bayes?

Classification based on the class with the highest posterior probability:

$$ Pr[C_j | x_1,...,x_n] = \frac{Pr[x_1,...,x_n | C_j] \cdot Pr[C_j]}{\sum_i Pr[x_1,...,x_n | C_i] \cdot Pr[C_i]} $$


$$ Pr[x_1,...,x_n | C_j] = f[x_1|C_j] \cdot f[x_2|C_j] \cdots f[x_n|C_j] $$

where the last equation encapsulates "naivety", i.e., $x_1,...,x_n$ are independent and Gaussian with density function $f(x) \sim N(\mu_x, \sigma_x^2)$, computed from the raw data.

NCAA Dataset

Credit Card Dataset