Abstract
This study explores students' perceptions and experiences with the use of generative artificial intelligence (GenAI) technologies for formative assessment of learning in preparation for summative coursework in higher education. As GenAI tools, such as large language models, become increasingly accessible and integrated into academic settings, it is crucial to understand their impact on student learning and engagement. This case study investigates students' perspectives from a business management discipline to provide a comprehensive understanding of the opportunities and challenges presented by GenAI-powered formative assessment of learning.
Formative assessment plays a vital role in supporting student learning by providing timely feedback, promoting self-regulation, and informing instructional strategies (Black & Wiliam, 1998; Sadler 1989). The integration of GenAI into formative assessment has the potential to enhance these processes by offering personalised, immediate, and scalable feedback to students (Dawson et al., 2019). However, the use of GenAI in educational contexts raises concerns about academic integrity, bias, and the potential displacement of human-led feedback (Winne, 2020).
Informed by theories of assessment for learning and technology-enhanced assessment (Bienkowski et al., 2012), this study applied dialogic nature of the Conversational Framework by Laurillard (2002) to design formative learning activities using Gen AI technologies, and the TAM framework by Davis (1989) for qualitative comprehension of students' attitudes, perceived usefulness, and perceived ease of use of GenAI technologies as formative assessment tools.
This case study employed a qualitative approach to capture the diverse perspectives of students across two academic disciplines with participants representing undergraduate and postgraduate programs in the subject areas of Business Management MBA. Data was collected using focus groups to delve deeper into students' experiences, concerns, and expectations regarding the use of GenAI in formative assessment. The focus groups provided a platform for students to share their personal narratives and engage in collective reflection. Students' interactions with GenAI powered tool were observed and documented during class sessions. Additionally, the interaction logs generated by the GenAI systems were observed to understand the nature and quality of the responses provided to students.
Thematic content analysis (Braun & Clarke, 2006) identified patterns, themes, and relationships within the student responses and observation notes. Findings provide valuable insights into students' perceptions and experiences with the use of GenAI for formative assessment of learning in higher education. Initial findings indicate, students were reluctant to use responses generated by Gen AI tools verbatim but preferred to use Gen AI tools as an ‘agent’ to support with critical thinking and assessment of learning needs. The study contributes to the growing body of research on the integration of emerging technologies, such as GenAI, into teaching and learning practices by informing the design and implementation to better align with students' needs, preferences, and concerns. By exploring students' perceptions, this study offers a multidisciplinary perspective on the opportunities and challenges presented by the integration of GenAI into formative assessment practices in higher education. The findings can help inform the development of inclusive and effective technology-enhanced assessment strategies that empower students and promote their academic success.