Glove Word Vectors

[KarahanS]

The feature, motivation and pitch

As our third algorithm, we have to train word vectors using GloVe. Official implementation can be found here.

We got terrible results after the training - and the iterations take a surprisingly short amount of time. Preprocessing seems ok as it displays the correct statistics in terms of the number of unique words in the vocabulary.
Possible solutions?

• Is there a problem with how we feed our corpus to the script?
• Can the problem be related to the remote server? Should we try another computer?
  • Maybe I can try running the script on Mac.
  • Maybe we can try running it on our Windows machines. It may not be straightforward as it is right now but there is a Windows adaptation: https://github.com/hfxunlp/GloVe-win
• Should we ask? Uras hoca and others had trained Glove for Turkish word vectors - https://github.com/inzva/Turkish-GloVe. Maybe we can ask for some help?
• Using pre-trained word embeddings is always an option but ugly.

Alternatives

No response

Additional context

No response

[KarahanS]

stanfordnlp/GloVe#210

[KarahanS]

Skimmed through the paper and these are the parameters suggested for word vectors with 300 dimensions:

• Iteration count: 100
• Alpha: 0.75
• $x_{max}$ = 100
• Window size = 10 (we should set it to 5 in order to be able to compare it with word2vec and fasttext)

[CahidArda]

Glove embeddings were added.