Performance evaluation of SVM with synthetic minority over-sampling technique in sentiment classification

Abstract

This study investigates the performance of the Support Vector Machine (SVM) algorithm in sentiment analysis tasks within the context of tourism destination branding, utilizing the Cross-Industry Standard Process for Data Mining (CRISP-DM) framework. Specifically, the research compares SVM performance with and without the Synthetic Minority Over-sampling Technique (SMOTE) across various metrics including accuracy, precision, recall, F-measure, and Area Under the Curve (AUC). The analysis is conducted on a dataset comprising textual data extracted from "Wonderful Indonesia" promotional videos featuring Labuan Bajo. Results indicate that SVM without SMOTE achieves a slightly higher accuracy of 97.79% compared to 96.61% with SMOTE. However, a closer examination reveals that SVM without SMOTE accurately classifies all positive instances, while with SMOTE, one positive instance is misclassified as negative. Precision, recall, and F-measure scores for positive instances are also higher without SMOTE, indicating better performance in classifying positive sentiment