Knowledge-Based Approach for Automated Evaluation of Descriptive Answers in Examinations

Abstract

"In exams, essay questions are considered to be the most effective technique of assessing a student's understanding of the learning outcomes. When evaluating essay questions, several factors should be addressed, such as checking for the required keywords, checking for grammatical faults, and considering the overall meaning of the answer. When all of the above considerations are taken into account, manual evaluation can be an extremely time-consuming and challenging task for the evaluators. With the growth of digitalized examinations, numerous techniques for automating the grading of descriptive questions have been developed. However, determining the validity of a response solely based on the meaning rather than considering only the keywords or syntax of the answers remains a difficulty. The suggested system (E-Valuator) can evaluate essay questions based on a predefined answer script. The procedure takes a semantic approach, identifying semantic relationships between the two responses and assigning a similarity score to the provided answer. The similarity score is divided into fuzzy keyword similarity, string similarity, and knowledge graph similarity score. The final score will be calculated by combining these three scores. After evaluating the implemented system, the results obtained a Mean Absolute Percent Error (MAPE) of 0.2% and a Quadric Weighted Kappa (QWK) score of 0.90. After benchmarking the results with the existing systems and evaluating the system with the help of industry experts, it was concluded that the suggested system could improve the descriptive answer grading."