Machine learning/Classification algorithms
Jump to navigation Jump to search
Classification is a subcategory of supervised learning problems.
k-nearest neighbor[edit | edit source]
- a simple classification algorithm
- Intuition: Find the majority vote in the training data
- This is a discriminative model, meaning that there is no way to generate the training data points
Algorithm[edit | edit source]
- Define some distance metric or similarity metric. The simplest case is Euclidean distance.
- Given some input point , find the 'th nearest neighbors from the training set.
- Do a majority vote between these nearest neighbor list and classify the input point as the category with highest number of vote.
Probabilistic interpretation[edit | edit source]
Consider the classification output as a random variable . Define probability of given input and training data is
The output of the classification is
Read more about probabilistic interpretation here: