SIR-ABSC: Incorporating Syntax into RoBERTa-based Sentiment Analysis Models with a Special Aggregator Token

Ikhyun Cho; Yoonhwa Jung; Julia Hockenmaier

SIR-ABSC: Incorporating Syntax into RoBERTa-based Sentiment Analysis Models with a Special Aggregator Token

Ikhyun Cho, Yoonhwa Jung, Julia Hockenmaier

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We present a simple, but effective method to incorporate syntactic dependency information directly into transformer-based language models (e.g. RoBERTa) for tasks such as Aspect-Based Sentiment Classification (ABSC), where the desired output depends on specific input tokens. In contrast to prior approaches to ABSC that capture syntax by combining language models with graph neural networks over dependency trees, our model, Syntax-Integrated RoBERTa for ABSC (SIR-ABSC) incorporates syntax directly into the language model by using a novel aggregator token. SIR-ABSC outperforms these more complex models, yielding new state-of-the-art results on ABSC.

Original language	English (US)
Title of host publication	Findings of the Association for Computational Linguistics
Subtitle of host publication	EMNLP 2023
Publisher	Association for Computational Linguistics (ACL)
Pages	8535-8550
Number of pages	16
ISBN (Electronic)	9798891760615
State	Published - 2023
Event	2023 Findings of the Association for Computational Linguistics: EMNLP 2023 - Singapore, Singapore Duration: Dec 6 2023 → Dec 10 2023

Publication series

Name	Findings of the Association for Computational Linguistics: EMNLP 2023

Conference

Conference	2023 Findings of the Association for Computational Linguistics: EMNLP 2023
Country/Territory	Singapore
City	Singapore
Period	12/6/23 → 12/10/23

ASJC Scopus subject areas

Computational Theory and Mathematics
Computer Science Applications
Information Systems
Language and Linguistics
Linguistics and Language

Library availability

Discover UIUC Full Text

Cite this

SIR-ABSC: Incorporating Syntax into RoBERTa-based Sentiment Analysis Models with a Special Aggregator Token. / Cho, Ikhyun; Jung, Yoonhwa; Hockenmaier, Julia.
Findings of the Association for Computational Linguistics: EMNLP 2023. Association for Computational Linguistics (ACL), 2023. p. 8535-8550 (Findings of the Association for Computational Linguistics: EMNLP 2023).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Cho, I, Jung, Y & Hockenmaier, J 2023, SIR-ABSC: Incorporating Syntax into RoBERTa-based Sentiment Analysis Models with a Special Aggregator Token. in Findings of the Association for Computational Linguistics: EMNLP 2023. Findings of the Association for Computational Linguistics: EMNLP 2023, Association for Computational Linguistics (ACL), pp. 8535-8550, 2023 Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, Singapore, 12/6/23.

Cho, Ikhyun ; Jung, Yoonhwa ; Hockenmaier, Julia. / SIR-ABSC : Incorporating Syntax into RoBERTa-based Sentiment Analysis Models with a Special Aggregator Token. Findings of the Association for Computational Linguistics: EMNLP 2023. Association for Computational Linguistics (ACL), 2023. pp. 8535-8550 (Findings of the Association for Computational Linguistics: EMNLP 2023).

@inproceedings{bb6b00c0ffa14e3fa1280035893a63b1,

title = "SIR-ABSC: Incorporating Syntax into RoBERTa-based Sentiment Analysis Models with a Special Aggregator Token",

abstract = "We present a simple, but effective method to incorporate syntactic dependency information directly into transformer-based language models (e.g. RoBERTa) for tasks such as Aspect-Based Sentiment Classification (ABSC), where the desired output depends on specific input tokens. In contrast to prior approaches to ABSC that capture syntax by combining language models with graph neural networks over dependency trees, our model, Syntax-Integrated RoBERTa for ABSC (SIR-ABSC) incorporates syntax directly into the language model by using a novel aggregator token. SIR-ABSC outperforms these more complex models, yielding new state-of-the-art results on ABSC.",

author = "Ikhyun Cho and Yoonhwa Jung and Julia Hockenmaier",

note = "Publisher Copyright: {\textcopyright} 2023 Association for Computational Linguistics.; 2023 Findings of the Association for Computational Linguistics: EMNLP 2023 ; Conference date: 06-12-2023 Through 10-12-2023",

year = "2023",

language = "English (US)",

series = "Findings of the Association for Computational Linguistics: EMNLP 2023",

publisher = "Association for Computational Linguistics (ACL)",

pages = "8535--8550",

booktitle = "Findings of the Association for Computational Linguistics",

}

TY - GEN

T1 - SIR-ABSC

T2 - 2023 Findings of the Association for Computational Linguistics: EMNLP 2023

AU - Cho, Ikhyun

AU - Jung, Yoonhwa

AU - Hockenmaier, Julia

PY - 2023

Y1 - 2023

N2 - We present a simple, but effective method to incorporate syntactic dependency information directly into transformer-based language models (e.g. RoBERTa) for tasks such as Aspect-Based Sentiment Classification (ABSC), where the desired output depends on specific input tokens. In contrast to prior approaches to ABSC that capture syntax by combining language models with graph neural networks over dependency trees, our model, Syntax-Integrated RoBERTa for ABSC (SIR-ABSC) incorporates syntax directly into the language model by using a novel aggregator token. SIR-ABSC outperforms these more complex models, yielding new state-of-the-art results on ABSC.

AB - We present a simple, but effective method to incorporate syntactic dependency information directly into transformer-based language models (e.g. RoBERTa) for tasks such as Aspect-Based Sentiment Classification (ABSC), where the desired output depends on specific input tokens. In contrast to prior approaches to ABSC that capture syntax by combining language models with graph neural networks over dependency trees, our model, Syntax-Integrated RoBERTa for ABSC (SIR-ABSC) incorporates syntax directly into the language model by using a novel aggregator token. SIR-ABSC outperforms these more complex models, yielding new state-of-the-art results on ABSC.

UR - http://www.scopus.com/inward/record.url?scp=85183312099&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85183312099&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85183312099

T3 - Findings of the Association for Computational Linguistics: EMNLP 2023

SP - 8535

EP - 8550

BT - Findings of the Association for Computational Linguistics

PB - Association for Computational Linguistics (ACL)

Y2 - 6 December 2023 through 10 December 2023

ER -

SIR-ABSC: Incorporating Syntax into RoBERTa-based Sentiment Analysis Models with a Special Aggregator Token

Abstract

Publication series

Conference

ASJC Scopus subject areas

Library availability

Related links

Fingerprint

Cite this