Clustering Wikipedia

Clustering Wikipedia

Hi, in this article i’ll make a simple clustering example using wikipedia.

You can access full code, here:

# Perform the necessary imports
from sklearn.decomposition import TruncatedSVD
from sklearn.cluster import KMeans
from sklearn.pipeline import make_pipeline

# Create a TruncatedSVD instance: svd
svd = TruncatedSVD(n_components=50)

# Create a KMeans instance: kmeans
kmeans = KMeans(n_clusters=6)

# Create a pipeline: pipeline
pipeline = make_pipeline(svd,kmeans)


import pandas as pd 
from scipy.sparse import csc_matrix
documents = pd.read_csv("wikipedia-vectors.txt")
documents.drop(columns="Unnamed: 0",inplace=True)

titles = documents.columns
articles = csc_matrix(documents.values).T


# Fit the pipeline to articles

# Calculate the cluster labels: labels
labels = pipeline.predict(articles)

# Create a DataFrame aligning labels and titles: df
df = pd.DataFrame({'label': labels, 'article': titles})

# Display df sorted by cluster label


About Deniz Parlak

Hi, i’m Security Data Scientist & Data Engineer at My Security Analytics. I have experienced Advance Python, Machine Learning and Big Data tools. Also i worked Oracle Database Administration, Migration and upgrade projects. For your questions [email protected]

Leave a Reply

Your email address will not be published. Required fields are marked *