Sentiment analysis on COVID tweets using COVID-Twitter-BERT with auxiliary sentence approach

Publication Date

4-15-2021

Document Type

Conference Proceeding

Publication Title

Proceedings of the 2021 ACMSE Conference - ACMSE 2021: The Annual ACM Southeast Conference

DOI

10.1145/3409334.3452074

First Page

234

Last Page

238

Abstract

Sentiment analysis is a fascinating area as a natural language understanding benchmark to evaluate customers' feedback and needs. Moreover, sentiment analysis can be applied to understand the people's reactions to public events such as the presidential elections and disease pandemics. Recent works in sentiment analysis on COVID-19 present a domain-targeted Bidirectional Encoder Representations from Transformer (BERT) language model, COVID-Twitter BERT (CT-BERT). However, there is little improvement in text classification using a BERT-based language model directly. Therefore, an auxiliary approach using BERT was proposed. This method converts single-sentence classification into pair-sentence classification, which solves the performance issue of BERT in text classification tasks. In this paper, we combine a pre-trained BERT model from COVID-related tweets and the auxiliary-sentence method to achieve better classification performance on COVID tweets sentiment analysis. We show that converting single-sentence classification into pair-sentence classification extends the dataset and obtains higher accuracies and F1 scores. However, we expect a domain-specific language model would perform better than a general language model. In our results, we show that the performance of CT-BERT does not necessarily outperform BERT specifically in understanding sentiments.

Keywords

BERT, COVID, COVID-19, Natural language processing, Sentiment analysis, Text classification, Tweets

Department

Computer Science

Link to Full Text

Share

COinS