K Nearest Neighbors¶

In [1]:

%pylab inline
import pandas as pd

Populating the interactive namespace from numpy and matplotlib

What is kNN?¶

This is one of the simplest algorithms for classification and grouping.
Simply define a distance metric over a set of observations, each with $M$ characteristics, i.e., $x_1,x_2,...,x_M$ .

Compute the pairwise distance between each pair of observations, using any of the standard metrics. For example, Euclidian distance between data $x$ and $y$ :

$d = \sqrt{\sum_{i=1}^M (x_i - y_i)^2}$

Next, fix $k$ , the number of nearest neighbors in the population to be considered.
Finally, assign the category based on which one has the majority of nearest neighbors to the case we are trying to classify.

	No NAME	GMS	PTS	REB	AST	TO	A/T	STL	BLK	PF	FG	FT	3P	y
0	1. NorthCarolina	6	84.2	41.5	17.8	12.8	1.39	6.7	3.8	16.7	0.514	0.664	0.417	1.0
1	2. Illinois	6	74.5	34.0	19.0	10.2	1.87	8.0	1.7	16.5	0.457	0.753	0.361	1.0
2	3. Louisville	5	77.4	35.4	13.6	11.0	1.24	5.4	4.2	16.6	0.479	0.702	0.376	1.0
3	4. MichiganState	5	80.8	37.8	13.0	12.6	1.03	8.4	2.4	19.8	0.445	0.783	0.329	1.0
4	5. Arizona	4	79.8	35.0	15.8	14.5	1.09	6.0	6.5	13.3	0.542	0.759	0.397	1.0