In Georgia Tech's large online MSCS program, students have designed their own course review site, which we see as peer advising: students use it to give feedback to classmates on what to expect from classes. In this session, we cover our research on what they tell each other.
Peer advising is the general philosophy of students giving feedback and advise to other students on how to succeed in a program. Although most peer advising communities are administered deliberately by universities, we observe an instance where peer advising emerged entirely at the direction of students: they constructed their own web site, disseminated it among peers, and passed it off from administrator to administrator.
In our work, we wanted to look at what they tell each other through this community: what kind of advice or feedback do they share with one another? We uncovered six general categories (also available in Duncan & Joyner 2020):
- Advice - Recommendations involving prerequisite knowledge, courses to take before or in conjunction with a course, or the best way to progress through a course were included in this category. Also included were warnings or reassurances about taking a course and information about future offerings of a course. Advice is particularly characterized by being targeted at the reader.
- Review Context - This category included statements about the reviewers themselves, such as their coding or professional experience. Facts that were specific to one semester of a course (and therefore possibly not generalizable across all semesters) were also included, along with non advice, non evaluative statements about how the reviewer or other students progressed through a course.
- Course Description - This category contained statements that provide objective information about a course, such as the number of projects or average final grade in a course. Any factual statements about a specific semester of a course that are likely generalizable across many other semesters are also included.
- Evaluation - Statements in this category were subjective and related to the reviewer’s opinion of a course, often involving likes or dislikes. Statements about a reviewer’s dislikes were only included in this category if they weren’t actionable; otherwise, they were grouped into the Feedback category.
- Feedback - Actionable statements regarding aspects of a course that a reviewer disliked or wanted changed were grouped into this category. These statements were broadly applicable and not specific to one student. Although most of these statements were usually recommendations for course changes, some of them were about aspects of a course that were beneficial and should not be changed.
- Other - Statements that didn’t fall into any of the other 5 categories were grouped into this category. These were often post semester outcomes, musings about education as a whole, or fragments that only make sense in the context of previous sentences.
In this Discover Session, we present what we observed, as well as the relative frequency with which each category is seen in the reviews that students write.
This presentation is based on the papers "Peer Advising at Scale: Content and Context of a Learner-Owned Course Evaluation System" by Alex Duncan and David Joyner at the 2019 ACM conference on Learning @ Scale and "Advising of the Students, by the Students, for the Students: The Case of a Student-Owned Peer Advising Community" by Alex Duncan and David Joyner at the 2020 Hawaii International Conference on Education.