t-distributed Stochastic Neighbor Embedding

31. t-distributed Stochastic Neighbor Embedding#

Getting embeddings in lower dimension.

from google.colab import drive
drive.mount('/content/drive')  # Add My Drive/<>

import os
os.chdir('drive/My Drive')
os.chdir('Books_Writings/NLPBook/')

Mounted at /content/drive

%%capture
%pylab inline
import pandas as pd
import os
%load_ext rpy2.ipython

31.1. t-SNE Plots#

Here is the original t-SNE paper: https://www.jmlr.org/papers/volume9/vandermaaten08a/vandermaaten08a.pdf

Properties:

  1. t-SNE is slow as the dataset gets large.

  2. Distances within cluster are correct but not between cluster so it is not a clustering algorithm, it is a visualization algorithm.

  3. You can use it to project down a dataset into 2 or 3 dimensions. If you get separate groups, it suggests that a multiclass classifier may do a good job. See: https://colah.github.io/posts/2014-10-Visualizing-MNIST/

SNE stands for “Stochastic Neighbor Embedding”.

31.2. t-SNE Algorithm#

The basics of the algorithm are as follows, using the equations in the paper:

  1. Get a Gaussian measure of similarity of two points, represented by vectors \(x_i\) and \(x_j\) as follows:

\[ p_{j|i} = \frac{\exp(-\|x_i-x_j\|^2/2\sigma_i^2)}{\sum_{k \neq i}\exp(-\|x_i-x_k\|^2/2\sigma_i^2)}, \quad p_{j|i} \neq p_{i|j} \]

  1. \(x\) is in the high-dimensional space and we are projecting into points \(y_i\) and \(y_j\) in the low-dimension space, so we can define a similarity metric for the low-dimension space as well. We note that the variance chosen for the low-dimension enbedding is \(1/\sqrt{2}\).

\[ q_{j|i} = \frac{\exp(-\|y_i-y_j\|^2)}{\sum_{k \neq i}\exp(-\|y_i-y_k\|^2)}, \quad q_{i|i}=0 \]

  1. We want the distributions \(p\) and \(q\) to be as close as possible, and we define the distance between the two using the Kullback-Leibler (KL) divergence. We call this a “cost” (loss) function and minimize this.

\[ C = \sum_i KL(P_i \| Q_i) = \sum_i \sum_j p_{j|i} \log\frac{p_{j|i}}{q_{j|i}} \]

Here, \(P_i\) represents the conditional probability distribution over all other datapoints given datapoint \(x_i\), and \(Q_i\) represents the conditional probability distribution over all other map points given map point \(y_i\).

  1. How to set the parameter \(\sigma_i\)? This is done by choosing a desired perplexity parameter and setting it equal to

\[ Perp(P_i) = 2^{H(P_i)}, \quad H(P_i) = -\sum_j p_{j|i} log_2 p_{j|i} \]

where \(H(P_i)\) is Shannon entropy.

  1. Finally, minimize the loss function by taking derivatives wrt points \(y\):

\[ \frac{\partial C}{\partial y_i} = 2 \sum_j [p_{j|i}-q_{j|i}+p_{i|j}-q_{i|j}](y_i-y_j) \]

This is the dimension of \(y_i\).

Let’s see the t-SNE plot on the Reuters news dataset.

## Change Python version if needed
# !sudo apt-get install python3.9
# !sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.9 2
!python --version

Python 3.11.11

# Read in text cleaning functions
import nltk
nltk.download('punkt_tab')

# Remove punctuations
import string
def removePuncStr(s):
    for c in string.punctuation:
        s = s.replace(c," ")
    return s

def removePunc(text_array):
    return [removePuncStr(h) for h in text_array]

# Remove numbers
def removeNumbersStr(s):
    for c in range(10):
        n = str(c)
        s = s.replace(n," ")
    return s

def removeNumbers(text_array):
    return [removeNumbersStr(h) for h in text_array]

# Stemming
nltk.download('punkt')
from nltk.stem import PorterStemmer
from nltk.tokenize import sent_tokenize, word_tokenize

def stemText(text_array):
    stemmed_text = []
    for h in text_array:
        words = word_tokenize(h)
        h2 = ''
        for w in words:
            h2 = h2 + ' ' + PorterStemmer().stem(w)
        stemmed_text.append(h2)
    return stemmed_text

# Remove Stopwords
nltk.download('stopwords')
from nltk.corpus import stopwords
from nltk.tokenize import word_tokenize

def stopText(text_array):
    stop_words = set(stopwords.words('english'))
    stopped_text = []
    for h in text_array:
        words = word_tokenize(h)
        h2 = ''
        for w in words:
            if w not in stop_words:
                h2 = h2 + ' ' + w
        stopped_text.append(h2)
    return stopped_text

# Write all docs to separate files
def write2textfile(s,filename):
    text_file = open(filename, "w")
    text_file.write(s)
    text_file.close()

[nltk_data] Downloading package punkt_tab to /root/nltk_data...
[nltk_data]   Unzipping tokenizers/punkt_tab.zip.
[nltk_data] Downloading package punkt to /root/nltk_data...
[nltk_data]   Unzipping tokenizers/punkt.zip.
[nltk_data] Downloading package stopwords to /root/nltk_data...
[nltk_data]   Unzipping corpora/stopwords.zip.

%%time
import nltk
from nltk.corpus import PlaintextCorpusReader
corpus_root = 'NLP_data/reuters/training/'
ctext = PlaintextCorpusReader(corpus_root, '.*')

CPU times: user 120 ms, sys: 20.6 ms, total: 140 ms
Wall time: 25.4 s

%%time
#Convert corpus to text array with a full string for each doc
def merge_arrays(word_lists):
    wordlist = []
    for wl in word_lists:
        wordlist = wordlist + wl
    doc = ' '.join(wordlist)
    return doc

nltk.download('punkt')
#Run this through the corpus to get a word array for each doc
text_array = []
for p in ctext.paras():
    doc = merge_arrays(p)
    text_array.append(doc)

#Clean up the docs using the previous functions
news = text_array
news = removePunc(news)
news = removeNumbers(news)
news = stopText(news)
#news = stemText(news)
news = [j.lower() for j in news]

df = pd.DataFrame(news, columns=["Text"])
df.head()

[nltk_data] Downloading package punkt to /root/nltk_data...
[nltk_data]   Package punkt is already up-to-date!

CPU times: user 12.8 s, sys: 2.08 s, total: 14.9 s
Wall time: 6min 3s
Text
0 bahia cocoa review showers continued througho...
1 computer terminal systems lt cpml completes s...
2 n z trading bank deposit growth rises slightl...
3 national amusements again ups viacom lt via b...
4 rogers lt rog sees st qtr net up significantl... 
#Create plain DTM
from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer

docs = news
vec = TfidfVectorizer(min_df=0.01, max_df=0.9)
X = vec.fit_transform(docs)
dtm = pd.DataFrame(X.toarray(), columns=vec.get_feature_names_out())
print(dtm.shape)
dtm

(8471, 953)
able accepted accord according account accounting accounts acquire acquired acquisition ... working world worth would year years yen yesterday yet york
0 0.0 0.0 0.0 0.0 0.000000 0.0 0.0 0.000000 0.0 0.0 ... 0.0 0.0 0.0 0.026586 0.040825 0.000000 0.0 0.000000 0.0 0.352282
1 0.0 0.0 0.0 0.0 0.000000 0.0 0.0 0.072832 0.0 0.0 ... 0.0 0.0 0.0 0.138419 0.000000 0.066417 0.0 0.000000 0.0 0.000000
2 0.0 0.0 0.0 0.0 0.000000 0.0 0.0 0.000000 0.0 0.0 ... 0.0 0.0 0.0 0.000000 0.218928 0.000000 0.0 0.000000 0.0 0.000000
3 0.0 0.0 0.0 0.0 0.000000 0.0 0.0 0.000000 0.0 0.0 ... 0.0 0.0 0.0 0.000000 0.000000 0.000000 0.0 0.100464 0.0 0.000000
4 0.0 0.0 0.0 0.0 0.000000 0.0 0.0 0.000000 0.0 0.0 ... 0.0 0.0 0.0 0.000000 0.127352 0.000000 0.0 0.000000 0.0 0.000000
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
8466 0.0 0.0 0.0 0.0 0.181656 0.0 0.0 0.000000 0.0 0.0 ... 0.0 0.0 0.0 0.000000 0.000000 0.000000 0.0 0.000000 0.0 0.000000
8467 0.0 0.0 0.0 0.0 0.000000 0.0 0.0 0.000000 0.0 0.0 ... 0.0 0.0 0.0 0.000000 0.000000 0.000000 0.0 0.000000 0.0 0.000000
8468 0.0 0.0 0.0 0.0 0.000000 0.0 0.0 0.000000 0.0 0.0 ... 0.0 0.0 0.0 0.000000 0.000000 0.000000 0.0 0.000000 0.0 0.000000
8469 0.0 0.0 0.0 0.0 0.000000 0.0 0.0 0.000000 0.0 0.0 ... 0.0 0.0 0.0 0.000000 0.055422 0.000000 0.0 0.000000 0.0 0.000000
8470 0.0 0.0 0.0 0.0 0.000000 0.0 0.0 0.000000 0.0 0.0 ... 0.0 0.0 0.0 0.000000 0.126124 0.000000 0.0 0.000000 0.0 0.000000

8471 rows × 953 columns

31.3. Illustrating t-SNE#

We use the texthero library here: https://spacy.io/universe/project/texthero

!pip install -U gensim --quiet

import random
news_sample = random.sample(list(df.Text),100)
news_sample

[' u s weekly soybean crush bushels reporting members national soybean processors association nspa crushed bushels soybeans week ended feb compared bushels previous week year ago week association said it said total crushing capacity members bushels vs last week bushels last year nspa also said u s soybean meal exports week tonnes vs tonnes week ago compared tonnes year ago week nspa said figures include nspa member firms',
 ' claremont tells sec it seeks pct champion products stake two board seats',
 ' uk average gdp rises preliminary pct in up pct in th qtr official',
 ' api says distillate stocks off mln bbls gasoline off mln crude up mln',
 ' norsk hydro saga seek drilling rights in gabon norwegian oil companies norsk hydro a s lt nhy ol saga petroleum a s lt sago ol said applied offshore exploration drilling licenses gabon africa west coast saga petroleum said applied pct share operatorship one block adding finnish oil company neste lt neoy he pct spain hispanoil pct world bank international finance corporation ifs pct joined saga fill remaining shares application saga spokesman roy halvorsen told reuters expected gabonese officials would reply application easter halvorsen said first time saga applied operate opec member gabon continental shelf adding italian oil company agip heading group applicants separate bid license norsk hydro also applied undisclosed share single exploration license u s oil company tenneco already assigned operatorship company spokesman bjoern tretvoll said',
 ' versatile to build polar ice breaker versatile corp shipbuilding subsidiary letter intent build mln dlr polar icebreaker canadian coast guard transport minister john crosbie said in vancouver address crosbie said versatile pacific shipyards inc low bidder build arctic class icebreaker company must meet certain financial engineering conditions contract awarded the government also announced provide mln dlrs loan insurance help versatile prepare construction vessel said contract awarded versatile required offer assurances shipyard technically financially capable performing work crosibie said versatile bid mln dlrs lower competing bidders generate person years direct employment work vessel crosbie said would powerful icebreaker world would begin next year completed the government announced plans build icebreaker last year following controversial passage u s coast guard vessel polar sea disputed northwest passage the u s government seek permission journey claiming area international water way the government said icebreaker needed back country claim sovereignty arctic',
 ' olson industries inc lt olsn th qtr net oper shr cts vs dlrs oper net vs sales mln vs mln year oper shr dlrs vs cts oper net vs sales mln vs mln avg shrs vs note net excludes tax credits dlrs quarter dlrs year net excludes discontinued operations gain dlrs vs loss dlrs quarter gain dlrs vs loss dlrs year',
 ' lionel corp lt lio th qtr jan oper net oper shr dlr vs cts oper net mln vs revs mln vs mln avg shrs mln vs mln year oper shr cts vs cts oper net vs revs mln vs mln avg shrs mln vs note prior year th qtr year oper net excludes mln dlrs mln dlrs respectively income discontinued operations prior year th qtr year oper net also excludes mln dlrs mln dlrs respectively operating loss carryforwards reogranization items prior year th qtr year ended january',
 ' top discount rate u k bill tender falls pct',
 ' may department stores co raises dividend to cts from cts a share',
 ' u k money market forecast revised to show surplus the bank england said revised forecast liquidity position money market today surplus mln stg estimated flat position earlier morning',
 ' taiwan power firm plans more foreign debt cuts state owned taiwan power co boost domestic borrowings cut foreign debt line government policy trim island huge foreign exchange reserves company official said the company foreign debt mainly japanese u s banks cut nearly half billion taiwan dlrs equivalent calendar billion said its domestic borrowings however rose billion taiwan dlrs billion period added taiwan foreign exchange reserves stood billion u s dlrs due record trade surplus billion',
 ' u s energy chief rules out gasoline tax as way to avert pending oil crisis',
 ' dlr falls on fear tokyo won t hike demand miyazawa finance minister kiichi miyazawa said dollar drop today yen partly attributable perception inside outside japan country failed fulfill promise expand domestic demand he told lower house budget committee parliament natural nations think japan enough delay passage budget the budget delayed opposition boycotts parliament protest government plans new sales tax',
 ' reliable life insurance co lt rlifa year net shr dlrs vs dlrs net mln vs note net includes gain dlrs chjange accounting pension plans investment gaions three mln dlrs',
 ' uk average gdp rises preliminary pct in up pct in th qtr official',
 ' japan s lipc to buy beef on april japan livestock industry promotion corp lipc said april import tonnes beef first half fiscal year started april simultaneous buy sell tender system year ago the system calls agency buy beef trading houses simultaneously sell users one tender the tender quota april september pct agency beef import share japan beef import quota set tonnes period',
 ' exxon lt xon may close one french refinery exxon corp world largest oil company said published interview today reviewing worldwide refinery operations might decide close french refineries lee r raymond exxon new president singled possibility closure one exxon refineries france interview an exxon spokeswoman confirmed raymond specifically mentioned refineries france said specific refinery named she also said exxon opertations constant review exxon currently two refineries france fos mediterranean capcity barrels per day port jerome west paris similar capacity petroleum intelligence weekly influential trade journal said current issue understood exxon looking possibility refinery closures antwerp southern france possibly italy paul mlotok oil analyst salomon brothers inc said closures exxon made europe improvement european refining situation future profits good exxon major oil companies closed bunch refineries europe upgraded rest shaken many indepedents market now demand products rising efficient operations exxon show superior earnings mlotok said just royal dutch lt rd seen one highest grade refiners europe added industry sources said oil companies likely feel greater pressure operations southern europe competition opec countries increasing producers move downstream operations piw said refiners mediterranean expect increased shipments saudi arabia opec export refineries piw said sales libya algeria elsewhere expected reclaim markets lost italian european refiners result abundance cheap netback oil last year',
 ' baker int l lt bko hopes to complete merger baker international corp treasurer eric mattson said company hoped iron snags proposed merger hughes tool co lt ht declined say compromises might acceptable the proposed merger would create billion dlr oilfield services company thrown limbo yesterday hughes management balking terms government consent decree offered counter proposal baker earlier today hughes adjourned shareholders meeting called vote proposed merger march said hoped resume negotitations baker hughes chairman w a kistler told reporters hughes want sign department justice consent decree baker submersible pump drilling bit operations sold mattson told reuters baker still believed merger possible two giant oilfield service companies the company position would like see merger completed best interests shareholders shareholders mattson said our goal merger occur because litigation i go said baker filed lawsuit texas state court late yesterday force hughes abide terms proposed consent decree mattson also declined comment whether directors baker hughes scheduled meetings discuss merger',
 ' ec backs new move to encourage unleaded gasoline the ec executive commission backed plan allow member countries ban regular leaded gasoline move designed encourage use unleaded fuel it said statement backed proposal allow ec states ban sales regular grade gasoline containing lead six months notice the proposal needs approval ec ministers would force member state impose ban it option obligation spokesman said but commission said imposing ban would encourage use unleaded fuel well making easier gas stations cutting number types gasoline stock',
 ' horizon corp lt hzn th qtr net oper shr profit dlrs vs loss eight cts oper net profit mln vs loss revs mln vs year oper shr profit dlrs vs loss cts oper net profit vs loss revs mln vs mln avg shrs vs note net includes tax credits dlrs quarter dlrs year net periods excludes dlr loss discontinued operations net periods includes pretax gain mln dlrs sale remaining interest paradise hills n m development',
 ' bfg partial operating profits fall sharply in bank fuer gemeinwirtschaft ag lt bkfg f bfg partial operating profits fell mln marks mln new majority shareholder aachener und muenchener beteiligungs ag lt amvg f amb said but total operating extraordinary profits including earnings currency securities trading bank account earnings sale holdings firms double previous year level amb said bfg accounts included prospectus amb capital increase finance insurance company acquisition pct plus one share bfg despite fall partial operating profits bfg paid unchanged mln marks open reserves transferred unchanged mln marks trade union holding company lt beteiligungsgesellschaft fuer gemeinwirtschaft ag amb acquired majority stake the bank said business last year suffered turbulence around troubled trade union owned housing concern neue heimat amb said mln mark drop bfg business volume billion marks affected interest surplus the interest surplus fell mln marks mln also depressed point fall interest margin pct a rise surplus commission mln marks mln enough compensate the rise total operating profits enabled bfg step risk provisions country risks particularly emphasised continuing difficulties countries disclosed risk provisions west german accounting rules necessarily reflect full amount rose mln marks mln bfg parent credit volume eased billion marks billion foreign debtors accounted pct credit volume latin american debtors accounted pct total lending foreigners bfg posted extraordinary earnings sale pct lt volksfuersorge deutsche lebensversicherung ag pct lt bsv bank fuer sparanlagen und vermoegensbildung ag five pct lt allgemeine hypothekenbank ag the sale linked amb acquisition majority bfg these sales show mln marks profit transfer agreements mln writing back risk provisions',
 ' royal dutch shell u s earnings sharply lower royal dutch shell group lt rd as earnings u s fell sharply difficult market conditions lower crude gas prices also due different accounting methods shell chairman peter holmes said the shell oil dollar net income fell pct year additional effect currency movements reduced contribution group net income pct mln stg the group earlier reported drop net income year billion stg billion previously lower crude prices outweighing effect increased sales volume although figures lower nonetheless top end market forecasts shell transport trading plc lt sc l shares u k arm group rose stg close last night stg analysts noted general collapse exploration production volumes partially offset earnings chemicals rising mln stg mln also windfall tax credit lower expected currency losses added mln stg onto fourth quarter results main reason figures exceeding forecasts industry analyst chris rowland barclays de zoete wedd noted however added could well sharp fall performance first quarter due improbability group would able repeat high refining marketing margins first quarter the impact recovering oil prices would come faster downstream side upstream high proportion upstream activity centred gas typically reacted oil price changes half year lag analysts said holmes said upstream u s sector third quarter worst only two oil majors managed make profit period shell oil one the decrease u s earnings accentuated tax rates group increased share become volume market leader holmes added continued low crude oil prices would continue subdue u s exploration activity exploration currently pretty flat we going go dlrs massive upturn said a renewal exploration high cost areas north sea u s requires prices around dlrs barrel ultimately would lead rise u s imports if exploring going find anything noted u s oil production dropped half mln barrels day bbd would continue fall price stayed dlrs barrel this favored opec attempts stabilise prices lower price likelihood non opec marginal production shutting opec done pretty extraordinarily well everything moving direction added',
 ' u k money market given mln stg early help the bank england said provided mln stg assistance money market response early round bill offers earlier bank estimated shortage system today around billion stg the central bank purchased bills resale market april interest rate pct',
 ' taiwan s export orders fall in february export orders taiwanese products fell pct billion u s dlrs february billion january rose nearly pct billion year earlier economic ministry official said he attributed fall rising taiwan dollar february orders electric electronic goods mln u s dlrs mln january mln february garment orders mln dlrs mln mln footwear orders mln compared mln mln',
 ' video library inc lt vlvl th qtr loss shr loss two cts vs profit three cts net loss vs profit revs vs year shr profit cts vs loss two cts net profit vs loss revs mln vs avg shrs vs',
 ' shamrock holdings unit aquires software company lt shamrock holdings inc said subsidiary shamrock holdings california acquired controlling interest lt dbms inc purchase shares preferred stock the company said stock purchased undisclosed amount unnamed group investors shamrock said raymond nawara former executive vice president dbms privately held software company also granted options voting rights major portion shares common stock',
 ' rowntree mackintosh plc lt rwnt l year to end january shr p vs p div final div p vs p pretax profit mln stg vs mln net tax mln vs mln turnover mln vs mln trading profit mln stg vs mln consisting u k mln vs europe mln vs mln north america mln vs mln australasia mln vs mln rest world mln vs mln',
 ' mitsubishi heavy c itoh to sell triguna stakes mitsubishi heavy industries ltd lt mith t mhi c itoh co ltd lt citt t decided sell combined pct stake indonesia lt pt triguna utama machinery industries lt caterpillar tractor co spokesmen two japanese companies said triguna set owned pct mhi pct c itoh pct indonesian company it makes forklift trucks similar number excavators month technological cooperation mhi the spokesmen said sale results expected restructuring later year caterpillar mhi joint venture japanese company lt caterpillar mitsubishi ltd formed they said venture renamed lt shin caterpillar mitsubishi ltd capitalised billion yen it still owned equally mhi caterpillar set aim centralising mhi excavator business',
 ' quaker oats lt oat sells unit to keystone the quaker oats co said sold vernell fine candies inc unit privately held keystone partners inc undisclosed price the company said vernells sales around mln dlrs year ended august quaker oats acquired vernells august purchased golden grain macaroni co',
 ' australian wheat area to fall forecaster says australian wheat plantings forecast fall mln hectares mln sown australian wheat forecasters pty ltd awf said first preliminary crop forecast but reason expect australian production would less mln tonnes last year private forecaster said crops new south wales queensland suffered poor yields last season most fall plantings expected western australia state average yields would assisted growers sowing wheat fallows rest paddocks said the main reason low western australia estimate poor profit outlook cost credit yield pressures but eastern states wheat area hold provided rainfall june less average awf said although farmers saying intended cut back wheat area pct awf said unlikely since needed cash flow problems alternative crops the lack statutory marketing oilseeds pulses oats cause concern crops comprise high proportion growers income awf said awf state area forecasts mln hectares production mln tonnes follows crop forecasts given new wheat year area crop queensland n s w victoria s australia w australia',
 ' sunshine mining co th qtr shr loss cts vs loss cts',
 ' mar mar',
 ' swiss net capital exports rise in increased activity switzerland banks pushed net capital exports provisional billion francs last year billion national bank said pre publication copy annual report it also said current account surplus swiss balance payments reached provisional billion francs last year billion the national bank currency reserves rose billion francs billion rise however taking accounts effects shift exchange rates reserves actually fell value billion the banks net capital exports climbed billion francs billion capital exports domestic non banks fell billion billion the national bank gave following figures brackets current account billion made goods services factor income transfers capital account billion made direct investment n a portfolio investment n a capital traffic banks other capital traffic included n a traffic included statistical error n a change currency reserves national bank interest income foreign currency foreign currency transactions',
 ' some miners go on strike in south africa some black miners went strike south african gold coal mines national union mineworkers num said a num spokesman said workers began underground sit grootvlei gold mine owned general union mining corp protest transfer colleagues different jobs he said employees anglo american corp new vaal colliery also downed tools reason stoppage immediately clear officials two companies available comment num said trying start negotiations management',
 ' reading lt rdgc to sell philadelphia properties reading co said agreed principle convey properties owns around philadelphia convention center site city philadelphia philadelphia convention center authority it said exchange receive mln dlrs cash rights acquire acres adjacent philadelphia international airport option acquire development rights gallery ii shopping mall market street downtown philadelphia reading said responsible demolition cleanup certain properties portion cash proceeds escrowed the company said continue operate reading terminal market philadelphia the land giving amounts seven acrease includes part reading terminal train shed said',
 ' investor firms have pct of symbion lt symb a group affiliated firms led warburg pincus capital co l p new york venture capital investment partnership said acquired shares symbion inc pct total outstanding in filing securities exchange commission warburg pincus group said bought stake investment purposes although group said considering buying shares symbion said plans seek control company',
 ' gould lt gld division gets financing gould inc systems protection division said selected merrill lynch interfunding inc provide financing purchase division parent company terms disclosed the agreement announced february gould expects transaction completed end april the agreement assures retention jobs northeast philadelphia plant jeapardy gould accepted tentative offer westinghouse inc later terminated',
 ' comprehensive care corp lt cmph quarterly div qtly div nine cts vs nine cts pay may record may',
 ' citgo raises crude postings cts today wti to dlrs bbl',
 ' american aircraft corp buys private firm lt american aircraft corp said acquired pct interest privately held lt hunter helicopter nevada inc the purchase made undisclosed amount american aircraft stock company said it said acquisition increase shareholder equity american aircraft cts per share cts per share hunter helicopter builds two passenger helicopters retail dlrs',
 ' zenith laboratories inc lt zen th qtr loss shr loss cts vs profit cts net loss vs profit sales mln vs mln year shr loss cts vs profit cts net loss vs profit mln sales mln vs mln avg shrs mln vs mln note net includes tax credits dlrs quarter dlrs year',
 ' virateck inc lt vira st qtr feb oper loss oper shr loss cts vs profit cts oper loss vs profit revs vs note oper data include year ago extraordinary gain dlrs nine cts per shr',
 ' kentucky central life lt kenca sets payout kentucky central life insurance co said declared semi annual dividend cts per share payable march shareholders record march the dividend equal company previous semi annual payout',
 ' london ore enquiries fixtures south africa salvador brazil long tons manganese ore end april callao crotone mt minerals spot geraldton tampico mt bulk mineral sands fio itea dunkirk mt bauxite seven days purposes tampico north spain mt ore two days shinc shex one port german north sea usg mt bulk ore',
 ' co operative bancorp lt cobk raises quarterly qtly div cts vs cts prior pay april record march',
 ' u s commerce dept s ortner says yen is or pct undervalued',
 ' chemlawn says it rejects dlrs per share tender offer from waste management',
 ' talking point american express lt axp american express co plan sell stake shearson lehman brothers believed prelude public offering shares brokerage unit analysts said american express earlier said general understanding nippon life insurance co japan sell pct interest shearson mln dlrs the statement triggered rise brokerage stocks investors speculated possibility investment u s brokerage industry japanese concerns the stocks brokerage firms also climbed response relatively high value nippon put stake u s firm analysts said mln dlrs pct represents price times book value perrin long lipper analytical said brokerage stocks depressed somewhat dark cloud u s insider trading scandal closed february market value times book value american express trading today for last several weeks stock strong rumors spinoff part shearson analysts said american express shearson unit benefit doors nippon life open increasingly important tokyo financial markets i think honesty preliminary step said long what see probably future american express selling additional pct public market public vehicle shearson long said normally investors make major capital commitment illiquid situation said nippon some analysts previously speculated pct stake shearson might sold public they said firm like investment banks needs capital expand globally american express said total employees shearson expand pct internationally however number substantially lower growth last year company spokesman said american express officials would comment beyond statement made morning that statement however close possibility public offering option shearson david anthony smith barney analyst said possible shearson would partially sold public but believes firm digest nippon investment first i think figure going money said joan goodman analyst pershing co also said american express could determine sell shares public in statement financial services giant said continuing study various plans shearson addition investment nippon american express said options study range expanding shearson capacity meet international competition broadening access capital american express also said options reflect continuing integral role shearson american express worldwide financial services strategy shearson follows goldman sachs co finding japanese partner goldman last year sold pct stake sumitomo bank exchange mln dlr capital infusion analysts speculated matches those u s brokers companies expertise they money there enough internal capital growth said wertheim analyst james hanbury follows u s brokerage companies hanbury said japanese companies flush cash interested expertise american brokers those japanese companies capital companies capital needs growth opportunities use money that nice marriage said morgan stanley co lt ms rose first boston corp lt fbc one e f hutton group lt efh rose painewebber group lt pwj climbed merrill lynch co recommended today painewebber analyst rose',
 ' davy corp unit wins india smelter contract davy mckee ltd subsidiary u k engineering contruction company davy corp plc awarded contract worth around mln stg design zinc lead smelter india spokesman company said the contract funded britain overseas development administration part total grant mln stg india state owned hindustan zinc ltd the grant development major zinc lead mine associated smelting complex rajastan north west india agency said the agency said project expected account pct india total production zinc a total mln stg grant allocated supply goods services britain the grant comes part package announced march indian mineral development the agency also paying mln stg india coal sector expansion indigenous coal development programme using british designed longwall technology',
 ' twa sold four mln shares of usair believed to institutions wall street sources',
 ' csr bids dlrs a share for pioneer sugar mills csr ltd lt csra s said offer dlrs cash shares already hold lt pioneer sugar mills ltd csr already holds pct pioneer mln issued shares said statement the offer price values entire pioneer sugar share capital mln dlrs compares today closing market level dlrs share csr said announce details offer soon including alternative offer csr shares pioneer sugar stock it said offer generous since give pioneer sugar shareholders price equivalent times pioneer net earnings last financial year premium pct yesterday market price csr said believed already contained element takeover speculation it also gives premium pct pioneer last reported net tangible assets per share csr said csr said generous offer price reflects cost savings flow integrated management csr pioneer raw sugar mills building materials businesses these economies achieved csr control management pioneer sugar added the takeover announcement came soon csr earlier reported statement sell lt delhi petroleum pty ltd unit exxon corp lt xon unit mln dlrs proceed previously announced float part delhi',
 ' usx lt x consolidated natural lt cng end talks usx corp texas oil gas corp subsidiary consolidated natural gas co mutually agreed pursue talks consolidated possible purchase apollo gas co texas oil no details given',
 ' united asset management corp lt uam ups payout qtly div four cts vs three cts prior pay april record march',
 ' conrac lt cax exploring restructuring buyout conrac corp whose board rejected dlr share tender offer mark iv industries inc lt iv said exploring alternatives restructuring leveraged buyout takeover company in filing securities exchange commission conrac said board directors authorized financial legal advisors explore several alternatives mark iv offer said considers inadequate the alternatives explored include restructuring leveraged buyout takeover company another company conrac said conrac said board discussed alternatives special meetings march said decision whether adopt alternatives would hinge future actions mark iv unsolicitied tender offer putting effect alternatives explored could hurt defeat mark iv offer said board members also instructed keep confidential terms transaction might entered becomes final company said the board also took defensive steps including granting severance agreements officers said at board meeting yesterday concac directors deleted provision allowing holder least one third classes voting stock call special shareholders meeting deleted another provision allowing shareholders remove directors without cause company said the defensive moves taken mark iv said planned elect majority designees directors special holders meeting succeeded acquring majority company common stock offer conrac said conrac president vice president treasurer given severance agreements golden parachutes said under agreements conrac president paul graf would get cash payment twice annual salary dlrs last year change control company including acquisition pct company voting stock conrac said treasurer joseph smorada vice president joseph o donnell made dlrs last year would get one half times salary change control smorada salary listed sec filing the executives would also get cash settlements options plans continuation insurance benefits',
 ' oranges import controls farm products he said shintaro abe ruling liberal democratic party executive council chairman would propose unspecified measures response demands ease import controls u s farm products kato declined give details',
 ' bahia cocoa review showers continued throughout week bahia cocoa zone alleviating drought since early january improving prospects coming temporao although normal humidity levels restored comissaria smith said weekly review the dry period means temporao late year arrivals week ended february bags kilos making cumulative total season mln stage last year again seems cocoa delivered earlier consignment included arrivals figures comissaria smith said still doubt much old crop cocoa still available harvesting practically come end with total bahia crop estimates around mln bags sales standing almost mln hundred thousand bags still hands farmers middlemen exporters processors there doubts much cocoa would fit export shippers experiencing dificulties obtaining bahia superior certificates in view lower quality recent weeks farmers sold good part cocoa held consignment comissaria smith said spot bean prices rose cruzados per arroba kilos bean shippers reluctant offer nearby shipment limited sales booked march shipment dlrs per tonne ports named new crop sales also light open ports june july going dlrs dlrs new york july aug sept dlrs per tonne fob routine sales butter made march april sold dlrs april may butter went times new york may june july dlrs aug sept dlrs times new york sept oct dec dlrs times new york dec comissaria smith said destinations u s covertible currency areas uruguay open ports cake sales registered dlrs march april dlrs may dlrs aug times new york dec oct dec buyers u s argentina uruguay convertible currency areas liquor sales limited march april selling dlrs june july dlrs times new york july aug sept dlrs times new york sept oct dec times new york dec comissaria smith said total bahia sales currently estimated mln bags crop mln bags crop final figures period february expected published brazilian cocoa trade commission carnival ends midday february',
 ' united tote inc lt tote st qtr jan net shr profit nil vs loss cts net profit vs loss revs vs',
 ' venezuela moves closer to champlin purchase council ministers approved petroleos de venezuela planned purchase half interest champlin petroleum refinery corpus christi texas government sources said the cabinet authorized energy mines minister arturo hernandez grisanti approve purchase shareholders assembly state oil company petroleos de venezuela pdvsa pdvsa last april signed letter intent buy half interest champlin corpus christi refinery undisclosed sum under terms provisional agreement venezuela would supply barrels day plant new company would jointly owned pdvsa champlin subsidiary union pacific corp lt unp the deal would also allow pdvsa joint share champlin refinery distribution network the purhcase one series overseas joint ventures venezuela managed assure markets barrels approximately mln bpd exports hernandez grisanti told reporters cabinet meeting pdvsa pay mln dlrs half ownership champlin refinery mln dlrs cash mln dlrs crude oil products through deal said venezuela assured sale least bpd crude products hernandez said pdsva entered similar joint ventures veba oel west germany nynas petroelum sweden citgo united states',
 ' the home depot inc lt hd th qtr feb one net shr cts vs cts net vs revs mln vs mln year shr cts vs cts net mln vs revs billion vs mln',
 ' convest energy partners ltd lt cep th qtr loss shr loss dlrs vs loss dlrs net loss mln vs loss mln revs vs year shr loss dlrs vs loss dlrs net loss mln vs loss mln revs mln vs mln note net includes oil natural gas writedowns mln dlrs quarter mln dlrs year',
 ' ftc urges veto of georgia gasoline station bill the federal trade commission said staff urged governor georgia veto bill would prohibit petroleum refiners owning operating retail gasoline stations the proposed legislation aimed preventing large oil refiners marketers using predatory monopolistic practices franchised dealers but ftc said fears refiner owned stations part scheme predatory monopolistic practices unfounded it called bill anticompetitive warned would force higher gasoline prices georgia motorists',
 ' south africa gdp up pct in last quarter the south african reserve bank confirming previous estimates said real gross domestic product fourth quarter grew seasonally adjusted annual rate pct versus pct third quarter pct final quarter the bank latest quarterly review said nominal growth rate year quite reach one pct pct contraction but said strengthening terms trade taken account real gnp advanced pct compared decrease pct gdp total value goods services produced economy omits income abroad gnp includes payments outflows the bank also said indications country economic recovery becoming broadly based with exception commerce major sectors economy contributed fourth quarter rise domestic production the bank said major increases foreign reserves recorded january february first half march reserves rose billion rand january february total reserves declined mln rand billion rand equivalent pct total annual value import payments the current account surplus amounted billion rand versus mln previous year the bank said continuing strength current account allowed foreign reserves rebuilt comfortable levels this strengthened authorities hands lending support exchange rate rand action called bank said',
 ' aequitron lt aqtn sees th qtr charge aequitron medical inc said costs related previously announced plan consolidate life products operations boulder colo expected total dlrs eight cts share fourth quarter ending april it said costs including moving expenses severance pay future lease payments the company said consolidate life products company headquarters manufacturing operations minneapolis',
 ' iwc ups soviet grain import estimate three mln tonnes mln official',
 ' coradian corp lt cdin year net shr profit one cent vs loss cts net profit dlrs vs loss revs mln vs mln note company said net extraordinary items taxes declined provide data items',
 ' mcandrews and forbes are offering dlrs per share for revlon stock',
 ' simmons airlines inc lt simm nd qtr loss period end jan shr loss cts vs loss cts net loss vs loss revs mln vs mln six mths shr nil vs profit cts net loss mln vs profit revs mln vs mln avg shrs vs note prior yr results restated reflect change value aircraft resulting increase five cts shr six mths',
 ' chinese port underused despite congestion nearby the chinese port ningbo working well capacity despite miles shanghai seriously congested port official said jiang feng xiang said port deepest china handled mln tonnes cargo last year mln well potential capacity mln tonnes shanghai handled mln tonnes cargo jiang said ningbo utilised inadequate facilities including single track rail line linking hangzhou joins national network ningbo handles crude oil exports transhipments coal north south china imports include fertiliser soda ash iron ore brazil australia the docks handle ships tonnes oil tankers tonnes load unload harbour most ningbo port infrastructure built since official publications show a container berth two timber three general cargo berths added current five year plan jiang said the rail line hangzhou double tracked by maybe overtake shanghai added',
 ' perception technology lt pcep to take charge perception technology corp said expects take charge dlrs cts per share earnings second quarter ended march due bankruptcy proceeding customer t c new york inc perception said outstanding lease receivables t c dlrs it said exact amnount charge depend extent recovery leased equipment involved arrangements might made bankruptcy court equipment',
 ' manufacturers lt mhc in building foreclosure manufacturers hanover corp said ofp inc unit acquired dallas office building foreclosure the company said foreclosed allied bank tower mln square feet office building one fountain place dallas',
 ' u k merger clearances the secretary state trade industry said decided refer proposed acquisition reed international plc lt reed l lt technical publishing company inc monopolies mergers commission the proposed acquisition lt rosehaugh plc lt the general funds investment trust plc also cleared',
 ' data systems inc lt toot st qtr jan net shr three cts vs four cts net vs revs vs',
 ' general cinema corp lt gcn common stock dividend qtly div cts vs cts prior pay april recrod april nine',
 ' chase manhattan raises prime rate to pct from effective today',
 ' lt chauvco resources ltd year net shr cts vs cts net vs revs vs',
 ' bethlehem steel lt bs sets plate price increases bethlehem steel corp said base price carbon plates high strength low alloy plates increased dlrs dlrs short ton effective july one the company said composite prices alloy plates also increased dlrs per ton date adding publish prices product bethlehem steel said composite prices strip mill plates increased dlrs ton',
 ' itt corp lt itt regular qtrly dividend qtly div cts vs cts prior payable july one record may',
 ' usda asks tight insect rules for grain shipments the u s agriculture department proposing tighter federal standards setting allowable limits insect infestations grain shipments the changes standards would include following establishing equal tolerances number live insects shipments food grains feed grains oilseeds revising definition infested give equal value insects injurious grain establishing lower levels infestations in infested level would set three live insects per representative sample grams two insects final infestattion level would set one live insects per representative sample revising definition sample grade adding limit live dead insects per sample revising definition sample grade wheat adding limit insect damaged kernels per grams wheat the department asked public comments proposals april',
 ' grain ships loading at portland there five grain ships loading three ships waiting load portland according portland merchants exchange',
 ' baker denies change in u s policy on dollar treasury secretary james baker said change u s policy value dollar baker asked policy changed view comments yesterday senior commerce department official said thought japanese yen undervalued dollar pct replied no yesterday robert ortner undersecretary commerce economic affairs said thought yen undervalued said personal view this remark caused dollar drop appeared conflict recent agreement paris united states five industrial nations currency rates right level reflect underlying economic conditions baker spoke earlier national newspaper association declined elaborate statement u s policy',
 ' ico board seen unlikely to set new coffee talks chances international coffee organization ico executive board meeting week agree resume negotiations export quotas soon look remote ico delegates trade sources said ico observers doubted brazil key consuming countries ready give sufficient ground convince side reopening negotiations would worthwhile said ico talks quotas last month broke eight days producers consumers failed reach agreement since seen signs change positions difficult see positive outcome stage brazilian delegate lindenberg sette said but quotas must negotiated sometime said the u s indicated open dialogue quotas brazil must flexible rather refuse lower export share last negotiations delegates said at week march april meeting member ico board scheduled discuss current market situation reintroduction quotas verification stocks administrative matters according draft agenda the fact brazilian coffee institute president jorio dauster assistant u s trade representative jon rosenbaum chief colombian delegate jorge cardenas attending meeting signalled market watchers non event far negotiating quotas concerned i would imagine lot politicking among producers behind closed doors work kind proposal september next scheduled council meeting bronwyn curtis landell mills commodities studies said traders delegates said seen sign date set earlier council meeting if stalemate continues much longer analysts expect coffee agreement end operating without quotas remainder life september when talks broke u s brazil largest coffee consumer producer respectively blamed one another sabotaging negotiations refusing compromise brazil wanted maintain previous export quota shares allocated pct world coffee exports consumers small group producers pressed shares redistributed using objective criteria would threatened brazil share at recent meeting managua latin american producers costa rica honduras said willing put objections members group eight ico dissident producers aside order stem damaging decline prices nicaraguan external trade minister alejandro martinez cuenca told reporters saturday he london brief producers managua meeting however producers said aware move toward producer solidarity london coffee prices closed stg tonne today around beginning march',
 ' ccc accepts export bonus dairy cattle to morocco the commodity credit corporation ccc accepted bonus offer exporter sale head dairy cattle morocco u s agriculture department said the department said bonus awarded dlrs per head made brown swiss enterprises inc it paid form commodities ccc inventories the cattle delivery march september the department said purchase completes export enhancement program initiative dairy cattle morocco announced april',
 ' mitsui lt mitsy security pacific lt spc set pact mitsui co ltd said signed letter intent security pacific corp buy pct japan security pacific finance co ltd undisclosed sum form joint venture japan security pacific assets mln dlrs the joint venture introduce various financial products customer base mitsui group companies company said security pacific provide expertise consumer commercial lending well data processing support japan security pacific finance wholly owned subsidiary security pacific international finance inc owned security pacific corp security pacific said addition originating consumer commercial loans leases joint venture market related financial products services',
 ' wall street stocks tenneco inc lt tgt tenneco inc company long rumored takeover candidate rose sharply today speculation surfaced investor t boone pickens may targeting company acquisition traders analysts said tenneco spokesman joseph macrum said comment make whatsoever pickens available comment traders noted activity stock increased today published report linked pickens tenneco tenneco rose two points paul feretti analyst new orleans based howard weil labouisse friedrichs inc said surprised market rumors tenneco might target takeover attempt it pure market speculation boone pickens group may interested feretti said but tenneco would challenge run sheer size diversity mr pickens man likes challenge pickens made unsuccessful attempt take dallas based diamond shamrock corp lt dia last winter made secret plans acquire additional natural gas reserves belief gas prices stabilize sooner oil prices tenneco holds natural gas reserves estimated excess trillion cubic feet analysts said feretti conservatively estimated tenneco breakup value dlrs share said company generated cash flow dlrs share probably attractive pickens other analysts however estimate breakup value well dlr share range i strongly doubt going hostile takeover drexel burnham lambert inc houston based analyst john olson said putting little credence rumors tenneco gargantuan enterprise seven billion dlrs long term debt preferred stock outstanding he said buyer would also deal intractable problems tenneco faces farm equipment unit energy businesses analysts suggested tenneco could use number defensive strategies fend unwanted buyer spin non energy businesses directly shareholders buy back shares create limited partnership natural gas pipeline interests an analyst asked remain unidentified discouraged pickens rumor pickens plateful already holdings burlington northern amerada hess said i think would willing take tenneco problems case farm equipment unit losing mln dlrs annually worth less billion dlrs market today',
 ' great western financial lt gwf unit buys banks great western financial corp subsidiary great western bank said purchase three retail banking branches south florida total deposits mln dlrs great western said purchase branches deerfield beach hollywood approximately mln dlrs deposits guardian savings loan association one palm beach approximately mln deposits goldome savings bank',
 ' piedmont lt pie agrees to usair lt u buyout usair group inc said piedmont aviation inc agreed acquired dlrs per share the company newspaper advertisement said started tender offer piedmont shares price piedmont board two directors absent unanimously approved bid the offer withdrawal rights expire april three unless extended bid followed merger price usair said piedmont granted irrevocable option buy new shares certain circumstances piedmont mln shares outstanding usair said tender conditioned receipt enough shares give usair least pct interest piedmont fully diluted basis approval u s department transportation voting trust agreement permitting usair buy hold shares pending review application gain control piedmont the company said merger agreement piedmont provides offer amended without piedmont prior written consent way would adverse piedmont shareholders said could cut number shares bought without piedmont consent usair said could reduce number piedmont shares purchased offer less minimum number needed cause voting trust condition bid satisfied in case said minimum number shares tendered would buy shares pro rata basis in february usair offered pay dlrs per share cash pct piedmont stock usair shares remaining piedmont share last week carl c icahn controlled trans world airlines inc lt twa made conditional offer acquire usair dlrs per share bid rejected usair board the transportation department friday rejected twa application acquire usair grounds application failed comply department regulations omitting necessary information twa said would refile today providing information needed on friday twa said already acquired four mln shares pct usair',
 ' swiss national bank says bought dollars yen',
 ' royal bank canada ups u s base rate pct to pct',
 ' conrac says it is mulling restructuring or buyout as alternatives',
 ' mitsui and co buys stake in imatron a mitsui co lt mits t spokesman said subsidiary lt mitsui co usa inc bought two mln dlrs newly issued shares lt imatron inc unlisted california based medical equipment manufacturer mitsui co usa imatron fifth largest shareholder pct firm outstanding shares imatron capitalised mln dlrs spokesman said mitsui co intends import imatron computerised diagnostic equipment japan said',
 ' ec rejects all free market wheat export offers at weekly tender traders',
 ' cocoa council chairman seeks buffer compromise international cocoa organization icco council chairman denis bra kanon said attempt reach compromise buffer stock rules international cocoa agreement bra kanon called bilateral consultations among producers consumers thursday morning resolve outstanding differences much non member cocoa buffer stock purchase differentials fixed different origin cocoa consumer delegates told reporters bra kanon expected meet eight delegations individually attempt iron remaining problems producers consumers indicated support principle draft buffer stock rules package formulated past week small working group consumer delegates said despite remaining differences delegates remained confident buffer stock accord would agreed friday council session ends certain technical points need clarification delegates said certain consumers concerned differentials included draft buffer stock package line market realities consumer delegates said unless modified fears would promote purchases quality cocoas ghana origin normally required manufacturers said restrictions buffer stock purchases non member cocoa might lead supply overhang malaysian cocoa would depress prices added',
 ' u s intervened to buy mln dlrs against yen on january fed says',
 ' standard bred pacers lt stbd yr loss shr loss cts vs loss seven cts net loss vs loss revs vs note full name company standard bred pacers trotters inc',
 ' venezuelan president defends refinancing accord president jaime lusinchi defended billion dlr debt rescheduling accord government recently signed saying open new credit flows bring needed foreign investment venezuela in annual state nation speech lusinchi also said venezuela supports latin american debtors despite reached rescheduling accord in minute speech congress lusinchi summed achievements administration took office february he pointed pct growth non oil gdp moderate pct inflation rate lusinchi said billion dlr debt rescheduling signed feb put end process calvary nation post four years in refinancing accord said venezuela managed achieve inmportant thing reopening financial flows abroad essential financing development financing trade investment once details agreement finalized said venezuela seek new financing imported components new development projects intention continue net exporters capital protect balance payments flow capital towards venezuela said under agreement signed february venezuela extended payments debt years interest rate lowered pct libor at time payments next three years lowered billion dlrs lusinchi defended rescheduling accord critics said merely deferred weight payments future governments',
 ' malaysia expects rubber shortage in april may malaysia said expects natural rubber shortage april may effects current wintering rubber trees the expected shortage stronger demand rubber goods especially condoms surgical gloves consumers likely push prices malaysian rubber exchange licensing board said latest monthly bulletin during annual wintering period february april rubber trees shed leaves latex output low',
 ' apr apr',
 ' comp u card international inc lt cucd th qtr net january end oper shr profit cts vs loss six cts oper net profit vs loss revs mln vs mln avg shrs mln vs mln year oper shr profit cts vs profit cts oper net profit vs profit revs mln vs mln avg shrs mln vs mln note net excludes extraordinary tax charge dlrs vs credit dlrs quarter credits dlrs vs dlrs year',
 ' new world pictures to have gain from sale of five pct of taft broadcasting']

import gensim, logging
logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s', level=logging.INFO)

#Tokenize each document
def textTokenize(text_array):
    textTokens = []
    for h in text_array:
        textTokens.append(h.split(' '))
    return textTokens

sentences = textTokenize(news_sample)

#Train the model on Word2Vec
model = gensim.models.Word2Vec(sentences, min_count=1)

%%time
from sklearn.manifold import TSNE

# X = model.wv[model.wv.key_to_index]
# labels = model.wv.index_to_key

X = model.wv.vectors
labels = model.wv.index_to_key

figure(figsize=(20,10))

tsne = TSNE(n_components=2)
X_tsne = tsne.fit_transform(X)
x = X_tsne[:, 0]
y = X_tsne[:, 1]

scatter(x,y)
for i, txt in enumerate(labels):
    annotate(txt, (x[i], y[i]))
show()
_images/a153c58a0890e5358d718f4463087e9aec380a6ebfd9adababd52143345fac24.png

31.4. UMAP#

Recently, there is an improvement on t-SNE, called UMAP, coming from the Biology and Life Sciences, see: https://arxiv.org/abs/1802.03426

UMAP extends to any dimensional enbedding, replaces the perplexity requirement with a nearest-neighbors one (see the top of page 15 here: https://arxiv.org/pdf/1802.03426.pdf, and is faster in run time.

31.5. Let’s use TextHero#

https://texthero.org/

TextHero’s text cleaning function does the following cleanups:

  1. Fills in empty space for na values

  2. lowercase text

  3. removes digits

  4. removes punctuation

  5. remove stopwords

  6. removes whitespace

  7. removes accents on words (also known as “diacritics”)

# !pip install unidecode
# !pip install gensim==4.2.0
# !pip install texthero==1.0.4

CPU times: user 36.2 s, sys: 390 ms, total: 36.5 s
Wall time: 52.2 s

Sadly, texthero has not been maintained or updated since 2021, and has many conflicts with other active libraries, in particular pandas. The installs above may be useful but need a lot of work to get working, so I have deprecated it for now in these notes.

Contents