One-Vs-All (Multi-class classifier)
One Vs All is one of the most famous classification technique, used for multi-class classification. Here, we prepare ‘N’ different binary classifiers, to classify the data having ‘N’ classes. However, if nth class is a weak class (weak in the sense of features/ non-informative/ least informative), then we generally use (N-1) different binary classifiers. In this case, instance(s) negatively classified w.r.t, all the (N-1) classes, can be treated as a member of the nth class. It is highly effective in the case of large values of ‘N’.
NOTE: We generally use additional processing to remove the situations of over-fitting and skew-ness in the case of sufficiently large size of data set, and having a large number of classes.
Preparation of Binary classifier: For the ith classifier, let the positive examples be all the points in the class ‘i’, and let the negative examples be all the points not in the class ‘i’. We use the same procedure to all ‘N’ binary classifiers.
Video Tutorial: In this interactive Video Tutorial on “One-Vs-All (Multi class classifier)”, I have maintained the simplest possible structure for easy explanation. This tutorial demonstrates the entire classification system by using data set available at “UCI Machine Learning repository”, with source-code.
About the Author
Post-doctoral Researcher at Department of Computer Science, University of California (Davis)