In [1]:
%pylab inline
import pandas as pd
Overview¶
Grouping individuals, firms, projects, etc.
Cluster analysis comprises a group of techniques that uses distance metrics to bunch data into categories.
Two approaches.
Partitioning or Top-down: In this approach, the entire set of $n$ entities is assumed to be divided into $k$ clusters. Then entities are assigned clusters.
Agglomerative or Hierarchical or Bottom-up: In this case we begin with all entities in the analysis being given their own cluster, so that we start with $n$ clusters. Then, entities are grouped into clusters based on a given distance metric between each pair of entities. In this way a hierarchy of clusters is built up and the researcher can choose which grouping is preferred.
K-means¶
- Form a distance matrix.
- Initialize cluster centroids with evenly spaced items.
- Assign each observation to closest cluster (to centroid, closest or farthest mamber).
- Repeat a few times to re-assign until the scheme stabilizes.
SBA dataset¶
In [2]:
import pickle
SBAdata = pickle.load(open("data/SBAdata.p", "rb"))
X = SBAdata["sba"]
print(X.shape)
X.head()
Out[2]:
In [3]:
#FIT MODEL
#Normalize data, min-max scaling
Xn = (X-X.min())/(X.max()-X.min())
from sklearn.cluster import KMeans
model = KMeans(n_clusters=4).fit(Xn)
In [4]:
#GET CLUSTERS
clusters = model.labels_
X["Cluster"] = clusters
X.groupby(["Cluster"]).mean()
Out[4]:
Clustering on PCA reduced data¶
In [5]:
#FOUR CLUSTERS
from sklearn.decomposition import PCA
reduced_data = PCA(n_components=2).fit_transform(Xn)
kmeans = KMeans(init='k-means++', n_clusters=4, n_init=10)
kmeans.fit(reduced_data)
Out[5]:
In [6]:
#PLOT SET UP
# Step size of the mesh. Decrease to increase the quality of the VQ.
h = .02 # point in the mesh [x_min, x_max]x[y_min, y_max].
# Plot the decision boundary. For that, we will assign a color to each
x_min, x_max = reduced_data[:, 0].min() - 1, reduced_data[:, 0].max() + 1
y_min, y_max = reduced_data[:, 1].min() - 1, reduced_data[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
# Obtain labels for each point in mesh. Use last trained model.
Z = kmeans.predict(np.c_[xx.ravel(), yy.ravel()])
# Put the result into a color plot
Z = Z.reshape(xx.shape)
In [7]:
figure(1)
imshow(Z, interpolation='nearest', extent=(xx.min(), xx.max(), yy.min(),
yy.max()), cmap=cm.Paired, aspect='auto', origin='lower')
plot(reduced_data[:, 0], reduced_data[:, 1], 'k.', markersize=2)
# Plot the centroids as a white X
centroids = kmeans.cluster_centers_
scatter(centroids[:, 0], centroids[:, 1], marker='x', s=169, linewidths=3,
color='w', zorder=10)
xlim(x_min, x_max); ylim(y_min, y_max)
xticks(()); yticks(())
show()
In [8]:
#ELEVEN CLUSTERS
kmeans = KMeans(init='k-means++', n_clusters=11, n_init=10)
kmeans.fit(reduced_data)
#PLOT SET UP
# Step size of the mesh. Decrease to increase the quality of the VQ.
h = .02 # point in the mesh [x_min, x_max]x[y_min, y_max].
# Plot the decision boundary. For that, we will assign a color to each
x_min, x_max = reduced_data[:, 0].min() - 1, reduced_data[:, 0].max() + 1
y_min, y_max = reduced_data[:, 1].min() - 1, reduced_data[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
# Obtain labels for each point in mesh. Use last trained model.
Z = kmeans.predict(np.c_[xx.ravel(), yy.ravel()])
# Put the result into a color plot
Z = Z.reshape(xx.shape)
In [9]:
figure(1)
imshow(Z, interpolation='nearest', extent=(xx.min(), xx.max(), yy.min(),
yy.max()), cmap=cm.Paired, aspect='auto', origin='lower')
plot(reduced_data[:, 0], reduced_data[:, 1], 'k.', markersize=2)
# Plot the centroids as a white X
centroids = kmeans.cluster_centers_
scatter(centroids[:, 0], centroids[:, 1], marker='x', s=169, linewidths=3,
color='w', zorder=10)
xlim(x_min, x_max); ylim(y_min, y_max)
xticks(()); yticks(())
show()
Hierarchical Clustering (bottom up)¶
- Get distance matrix for $n$ observations. Each in its own cluster.
- Club the two closest observations into a cluster. Now we have $(n-1)$ clusters.
- Recalculate centroids.
- Repeat to get hierarchical structure.
NCAA dataset¶
In [10]:
ncaa = pd.read_table("data/ncaa.txt")
yy = append(list(ones(32)), list(zeros(32)))
ncaa["y"] = yy
ncaa.head()
Out[10]:
In [11]:
#CREATE FEATURES
X = ncaa.iloc[:,2:13]
X.head()
Out[11]:
In [12]:
#HIERARCHICAL CLUSTERING
from scipy.cluster.hierarchy import dendrogram, linkage
Z = linkage(X, 'ward')
In [13]:
#DENDROGRAM
figure(figsize=(20, 5))
title('Hierarchical Clustering Dendrogram')
xlabel('sample index')
ylabel('distance')
dendrogram(Z,
leaf_rotation=90., # rotates the x axis labels
leaf_font_size=8., # font size for the x axis labels
)
show()
Redo Hierarchical Clustering in R¶
In [2]:
%load_ext rpy2.ipython
In [3]:
%%R
#CLUSTER
ncaa = read.table("data/ncaa.txt",header=TRUE)
d = dist(ncaa[,3:14], method="euclidian")
fit = hclust(d, method="ward.D")
plot(fit,main="NCAA Teams")
groups = cutree(fit, k=2)
rect.hclust(fit, k=2, border="blue")
In [4]:
%%R
#CLUSTER PLOT
library(cluster)
clusplot(ncaa[,3:14],groups,color=TRUE,shade=TRUE,labels=2,lines=0)
