Exploring the effect of ai-facilitated peer-to-peer support on engagement, grades and pass rates: a mixed methods case study

Abstract

This archival longitudinal case study explores the effect of artificial intelligence (AI) in facilitating AI peer-to-peer support and learning, focusing on how these dynamics affect engagement as part of a student belief system, grades and pass rates. The research employs a mixed-methods approach, integrating both quantitative and qualitative analyses. The qualitative component employs thematic reflexive and coded factor analysis to explore AI's peer-to-peer support learning effect on students' beliefs and perceptions. Through t-test, the quantitative aspect evaluates AI's effectiveness by comparing the grade performance of a cohort of students lectured using the AI platform and those lectured using traditional methods. The thematic findings reveal a positive engagement response to the AI platform as it facilitates peer-to-peer learning support. High student response scores indicate a preference for using the AI-facilitated peer-to-peer support platform. T-test outcomes show limited statistically significant change (3-5% improvement) in academic grades following the platform's introduction across several financial management courses. Despite the positive student engagement perceptions of the platform regarding their grades, the peer-to-peer support platform did not lead to significant grade improvements. Implementing the AI platform showed a statistical improvement in the grades of one cohort of students; however, in Financial Management 4, notably, post-graduate students. The finding suggests an impact on grades, which, although not reaching conventional levels of statistical significance, cannot be disregarded. These findings show that engagement with the AI platform suggests a complex relationship between student engagement and academic achievement. Engagement is one component of student activity that improves overall student performance. Students have reported increased active learning experiences through this AI platform, which validates AI peer-to-peer support as an opportunity for institutions to provide additional academic student support. The three elements of traditional peer-to-peer support, exploring, enabling, and infusing, also promoted cognitive learning using this AI platform as peer-to-peer support. There is an increased activity for learning through this AI platform, further validating AI peer-to-peer support rather than encouraging students to remember. These insights add to the discussion on the efficacy of learning and teaching technologies, underscoring the difficulty in linking subjective engagement with objective performance metrics. The findings reveal that AI enhances student engagement and peer-to-peer support, fulfilling the objective of exploring its effect on engagement. However, the minimal improvement in grades suggests that engagement with the AI platform does not directly affect grade performance. The study recognises that AI peer-to-peer support platforms positively influence retention through enhanced engagement, aligning with understanding its impact on grades and pass rates. Recommendations highlight the need for personalised AI feedback and predictive AI to optimise student performance and retention. The study also notes limitations regarding controlling variables and implementing policies for AI-integrated learning environments. The study concludes that AI peer-to-peer support effectively enhances engagement and peer-to-peer interaction but requires further research to understand its full impact on academic performance. This research sets the stage for further exploration to conduct cross-cultural and longitudinal studies to assess AI's impact in varied learning and teaching settings and understand its sustained effect on learning outcomes. Further studies should include long-term data and qualitative evaluations to fully grasp the implications of AI-facilitated peer learning platforms on learning and teaching.