[Re] Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigation

Abstract

Despite widespread use in natural language processing (NLP) tasks, word embeddings have been criticized for inheriting unintended gender bias from training corpora. Previous methods highlight that in word2vec embeddings trained on the Google News dataset, “programmer” is more closely associated with “man” and “homemaker” is more closely associated with “woman”. Such gender bias has also been shown to propagate in downstream tasks. Despite plenty of work in this field, with methods ranging from corpus level modification to post-training modifications to embeddings, it remains an unsolved problem. With this work, the authors combine two techniques to reduce gender bias in embeddings. First, they argue that the frequency of words in the corpus adds to the bias. And thus use the work of to remove the frequency component from trained embeddings. Second, they use the hard debias algorithm, to remove the gender direction from the trained embeddings of most biased words. Combining these two techniques, they benchmark the result of their algorithm by showcasing reduction in bias and limited loss of information in the resultant word embeddings.

Publication
ML Reproducibility Challenge 2020