Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change

Understanding how words change their meanings over time is key to models of language and cultural evolution, but historical data on meaning is scarce, making theories hard to develop and test. Word embeddings show promise as a diachronic tool, but have not been carefully evaluated. We develop a robust methodology for quantifying semantic change by evaluating word embeddings (PPMI, SVD, word2vec) against known historical changes. We then use this methodology to reveal statistical laws of semantic evolution. Using six historical corpora spanning four languages and two centuries, we propose two quantitative laws of semantic change: (i) the law of conformity-the rate of semantic change scales with an inverse…

• NS
  National Science Foundation

  Awards: IIS-1149837, 1149837, 1514268, 1159679
• NS
  Natural Sciences and Engineering Research Council of Canada