Word2Vec Skip-Gram in TensorFlow 2

An implementation of the Word2Vec skip-gram algorithm in TensorFlow 2.0 that learns vector representations of words from a slice of Wikipedia (the text8 dataset).

How it works

1. The text8 corpus is downloaded (a helper script fetches it on demand) and processed into a vocabulary, dropping rare words below a frequency threshold.
2. A skip-gram model is trained with stochastic gradient descent and noise-contrastive estimation (NCE) loss.
3. During training the model is evaluated by finding the nearest neighbors of a set of test words using cosine similarity between their embeddings.

The implementation follows the approach described in Mikolov et al., Efficient Estimation of Word Representations in Vector Space (2013), and builds on Aymeric Damien’s TensorFlow examples.

What it demonstrates

Working knowledge of word embeddings, negative-sampling style objectives, and writing training and evaluation loops directly against the TensorFlow 2 API.