Binary naive Bayesian classifiers for correlated Gaussian features: a theoretical analysis

Abstract

We investigate the use of Naive Bayesian classifiers for correlated Gaussian feature spaces and derive error estimates for these classifiers. The error analysis is done by developing an exact expression for the error performance of a binary classifier with Gaussian features while using any quadratic decision boundary. Therefore, the analysis is not restricted to Naive Bayesian classifiers alone and can, for instance, be used to calculate the Bayes error performance. We compare the analytical error rate to that obtained when Monte-Carlo simulations are performed for a 2 and 12 dimensional binary classification problem. Finally, we illustrate the robust performances obtained with Naive Bayesian classifiers (as opposed to a maximum likelihood classifier) for high dimensional problems when data sparsity becomes an issue.