Zulkifli, Zulkifli (2025) Evaluasi Akurasi Human Anatator Dan Lexicon Pada Media X Menggunakan Algoritma Bert. Other thesis, Universitas Islam Riau.

Text skripsi_203510500_watermark.pdf - Submitted Version Restricted to Registered users only Download (1MB) | Request a copy

Abstract

This study aims to highlight the difference in accuracy between sentiment labels given by human annotators and the lexicon method on social media data X (formerly Twitter) in the context of the 2024 regional head elections (Pilkada). Data in the form of 7,065 tweets were processed through the stages of crawling, preprocessing, and labeling using two summaries: manual by two human annotators, and automatic based on lexicon. The level of consistency between annotators was tested using the Cohen's Kappa value and showed a value of 0.84, which is included in the category of "Almost Perfect Agreement". Furthermore, the IndoBERT model was drilled with both types of labeled data to measure the performance of sentiment classification. The evaluation results showed that the model drilled with manual labels produced an accuracy of 82.85%, a precision of 0.87, a recall of 0.85, and an F1-score of 0.86. While the model with lexicon labels obtained an accuracy of 81.57%, a precision of 0.85, a recall of 0.84, and an F1-score of 0.84. These findings suggest that manual labeling is more effective in capturing emotional context and complex language nuances, while the lexicon method still provides competitive performance with efficiency advantages. This study provides insights for the development of Indonesian language sentiment analysis systems, especially in choosing the optimal labeling strategy.

Actions (login required)

View Item