Understanding CSCL Peer Feedback Contributions: An Automated Content Analysis Approach

Abstract

Peer feedback has been widely used in computer-supported collaborative learning (CSCL) setting to improve students’ engagement with massive courses. Although the peer feedback process increases students’ self-regulatory practice, metacognition, and academic achievement, instructors need to go through large amounts of feedback text data which is much more time-consuming. To address this challenge, the present study proposes an automated content analysis approach to identify relevant categories in peer feedback based on traditional and sequence-based classifiers using TF-IDF and content-independent features. We use a data set from an extensive course (N = 231 students) in the setting of engineering higher education. In particular, a total of 2,444 peer feedback messages were analyzed. The results have shown promising outcomes with both TF-IDF and content-independent features. The Conditional Random Fields (CRF) classification model based on the TF-IDF features achieved the best performance, considering all the metrics computed in the analysis. The results illustrate that the ability to scale up the automatic analysis of peer feedback provides new opportunities for student improved learning and improved teacher support in higher education at scale.