Classifying Thai SMS Messages from Scammers using Machine Learning

Abstract

This research aims to: 1) classify Thai SMS messages related to scams using Machine Learning; 2) investigate the use of PyThaiNLP combined with TF-IDF for text feature extraction and apply the SMOTE technique to address class imbalance before training with the Naïve Bayes algorithm; and 3) evaluate the performance of the model in classifying Thai SMS messages into three categories: normal, scam, and promotional messages, using a dataset of 5,000 samples. The experimental results using the Naïve Bayes algorithm show that the normal class achieved a Precision (accuracy of positive predictions) of 0.94, Recall (ability to detect actual positives) of 0.84, and F1-Score (balance between Precision and Recall) of 0.88. The promotional class obtained a Precision of 0.57, Recall of 0.81, and F1-Score of 0.67, while the scam class achieved a Precision of 0.84, Recall of 0.91, and F1-Score of 0.87. These results indicate that the model performs well, particularly in detecting scam messages. Furthermore, applying the SMOTE technique improved the model performance, achieving the highest Accuracy of 85.00%, with Precision, Recall, and F1-Score at a good level. In conclusion, this approach is effective for Thai SMS scam detection and can be applied in real-world scenarios.