Evaluating the Effectiveness of AI-Driven Adaptive Assessments in Measuring Student Knowledge and Skills

Abstract

The purpose of this study was to analyze the impact of AI-based adaptive assessments on measuring the level of knowledge and skills of students at four public universities in Lahore, Rawalpindi, and Islamabad, Pakistan. The study used a mixed-methods approach and focused on 380 undergraduate students from the fields of computer science, business administration, and engineering. Subjects were assigned to the experimental and control groups, where the experimental group utilized AI-based adaptive assessment platforms and the control group underwent traditional assessments. The data collection lasted for a whole academic semester and involved the use of an assessments administered pre and post experiment and qualitative data collected via semi-structured interviews from twenty-five faculty members and fifty students in focus groups. In the quantitative phase of this study, it was discovered that the experimental group students scored higher, retained knowledge better, and completed tests in a shorter amount of time than those in the control group. In the qualitative phase, further points emphasized were the level of students' engagement, crafting of personalized learning experiences, and introduced challenges in the initial phase of application related to the tech available and the faculty. Statistical evaluations made via t-test, ANOVA, and regression confirmed the use of Adaptive AI assessments for the first time ever in measuring students learning outcomes. The study concluded by stating that Adaptive AI assessments in Pakistan higher education are most likely to flourish, given that adaptive AI assessments are made available to all students, subject to the available tech.