FINAL PROJECT SCORING SYSTEM BASED ON SEMANTIC SIMILARITY OF TITLE AND DESCRIPTION USING INDOBERT

Abstract

Initial evaluation of final project proposals is generally conducted manually, requiring considerable time and potentially resulting in inconsistent assessments. This study aims to develop a semantic similarity-based system using IndoBERT optimized with the SimCSE approach to assist in identifying the similarity and uniqueness levels of final project proposals. The model was trained using three types of positive pairs and evaluated under three scenarios: title, description, and a combination of both, using Precision@5, Recall@5, and F1-Score@5 as evaluation metrics. The results show that model optimization increased the difference in the average cosine similarity values between positive and negative pairs from 0.040 to 0.5412. The description and combined scenarios achieved the highest F1-Score@5 of 0.71 at a threshold of 0.6. The model was subsequently implemented in a web-based application and achieved a user acceptance rate of 84.40%. These results indicate that the SimCSE approach is effective in supporting the initial evaluation of final project proposals in a faster and more objective manner.