Adapting sequence to sequence models for text normalization in social media

Ismini Lourentzou, Kabir Manghnani, Cheng Xiang Zhai

Research output: Contribution to conferencePaper

Abstract

Social media offer an abundant source of valuable raw data, however informal writing can quickly become a bottleneck for many natural language processing (NLP) tasks. Off-the-shelf tools are usually trained on formal text and cannot explicitly handle noise found in short online posts. Moreover, the variety of frequently occurring linguistic variations presents several challenges, even for humans who might not be able to comprehend the meaning of such posts, especially when they contain slang and abbreviations. Text Normalization aims to transform online user-generated text to a canonical form. Current text normalization systems rely on string or phonetic similarity and classification models that work on a local fashion. We argue that processing contextual information is crucial for this task and introduce a social media text normalization hybrid word-character attention-based encoder-decoder model that can serve as a pre-processing step for NLP applications to adapt to noisy text in social media. Our character-based component is trained on synthetic adversarial examples that are designed to capture errors commonly found in online user-generated text. Experiments show that our model surpasses neural architectures designed for text normalization and achieves comparable performance with state-of-the-art related work.

Original languageEnglish (US)
Pages335-345
Number of pages11
StatePublished - Jan 1 2019
Event13th International Conference on Web and Social Media, ICWSM 2019 - Munich, Germany
Duration: Jun 11 2019Jun 14 2019

Conference

Conference13th International Conference on Web and Social Media, ICWSM 2019
CountryGermany
CityMunich
Period6/11/196/14/19

ASJC Scopus subject areas

  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Adapting sequence to sequence models for text normalization in social media'. Together they form a unique fingerprint.

  • Cite this

    Lourentzou, I., Manghnani, K., & Zhai, C. X. (2019). Adapting sequence to sequence models for text normalization in social media. 335-345. Paper presented at 13th International Conference on Web and Social Media, ICWSM 2019, Munich, Germany.