Word embeddings with limited memory

Shaoshi Ling, Yangqiu Song, Dan Roth

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper studies the effect of limited precision data representation and computation on word embeddings. We present a systematic evaluation of word embeddings with limited memory and discuss methods that directly train the limited precision representation with limited memory. Our results show that it is possible to use and train an 8-bit fixed-point value for word embedding without loss of performance in word/phrase similarity and dependency parsing tasks.

Original languageEnglish (US)
Title of host publication54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Short Papers
PublisherAssociation for Computational Linguistics (ACL)
Pages387-392
Number of pages6
ISBN (Electronic)9781510827592
StatePublished - Jan 1 2016
Event54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Berlin, Germany
Duration: Aug 7 2016Aug 12 2016

Publication series

Name54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Short Papers

Other

Other54th Annual Meeting of the Association for Computational Linguistics, ACL 2016
CountryGermany
CityBerlin
Period8/7/168/12/16

    Fingerprint

ASJC Scopus subject areas

  • Artificial Intelligence
  • Linguistics and Language
  • Software
  • Language and Linguistics

Cite this

Ling, S., Song, Y., & Roth, D. (2016). Word embeddings with limited memory. In 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Short Papers (pp. 387-392). (54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Short Papers). Association for Computational Linguistics (ACL).