Naive Bayesian classifiers for multinomial features: a theoretical analysis

Abstract

The authors investigate the use of naive Bayesian classifiers for multinomial feature spaces and derive error estimates for these classifiers. The error analysis is done by developing a mathematical model to estimate the probability density functions for all multinomial likelihood functions describing different classes. They also develop a simplified method to account for the correlation between multinomial variables. With accurate estimates for the distributions of all the likelihood functions, the authors are able to calculate classification error estimates for any such multinomial likelihood classifier. This error estimate can be used for feature selection, since it is easy to predict the effect that different features have on the error rate performance