Persistence lift is significantly impacted by student recognition; evidenced by the findings of our study and corresponding to findings of extant studies. Students struggling and on the verge of disenrolling exhibited greater motivation to perform well in a competency-based education model following student recognition of excellence (11-20% persistence lift, n=30,440).
Student enrollment and retention are paramount to the ongoing operation of a college/university. Recognition of achievement is found to be a motivational factor for college students to remain enrolled in their courses and persist in their coursework to graduate, as indicated in past research. This presentation will provide a detailed review on a recently completed study on the impact of student recognition of excellence on student persistence in an online higher ed program. The study includes a retrospective longitudinal study of all excellence award recipients from August 2016 to August 2019, involving the correlation of student’s course progression after receiving the award and its impact on three key student success indicators: On Time Progress (OTP), Student Academic Progress (SAP), and Course Completion Rate (CCR).
The participants will be provided an in-depth look at the student success analytics foundational to this student, particularly regarding the exciting findings related to persistence lift. Persistence lift is significantly impacted by student recognition; evidenced by the findings of our study and corresponding to findings of extant studies. Details will include the findings that students struggling and on the verge of disenrolling exhibited greater motivation to perform well in a competency-based education model following student recognition of excellence (11-20% persistence lift, n=30,440). Participants will be provided the opportunity to explore the relevance of the study methodology and results to the fostering of student success with an interactive discussion on how they may apply study findings to the intentional student recognition at their educational institution.
Presentation Overview
A detailed literature review will be provided, which indicated student recognition affects a student’s self-efficacy, motivation, and level of effort put forth. Students with low academic performance, have higher academic performance after recognition. Students react to receiving recognition with the greater effort expended toward higher academic achievement. However, these findings were not from studies conducted in a competency-based education model.
Research Premise
The study is based on an online university operating in a competency-based education model. The students demonstrate competency (demonstrating what they know and can do) in their courses in order to pass their performance and objective assessments. There is a considerable amount of self-regulated learning on the part of the students in this education model; while the learning is being facilitated and mentored by faculty. Therefore, in this type of learning environment, the students must have a higher level of motivation that fosters persistence than in any other educational model.
Research Questions
Based on the findings of extant research in the area of college student motivation and persistence, the following research questions emerge: (1) What effect is the online competency-based university awards program having on their students? (2) How does student recognition of excellence support and enhance indicators of student success: On-Time Progress (OTP), Satisfactory Academic Progress (SAP), and Course Completion Record (CCR)?
Research Methodology
This research was conducted as a retrospective longitudinal study over a period of three years with a mixed-method investigation in a competency-based education model. The qualitative method was a content analysis using MAXQDA. This is a software program designed for computer-assisted qualitative data, text, and multi-media analysis of content. The quantitative investigation was conducted using a quasi-experimental analysis tool: Civitas Learning Impact™.
The qualitative analysis measured the frequency of trending words and phrases contained within a student's comment written in response to being selected to receive an excellence award. Using MAXQDA, analyzing 11,644 student grateful responses content trends with words and phrases expressing gratitude such as: “thank you,” thanks for the award,” and other similar grateful responses were identified. Another type of trending phrase emerged such as “I was on the verge of quitting,” “was about to give up,” was about to quit school.” In fact, 19% of the phrases written were from students about to give up, quit, and leave school.
The quantitative method measured student’s persistence lift using Civitas Learning Impact™ (hereto referred to as “Civitas”) statistical analysis tool, matching nearest neighbors with one group of students receiving an award and the other group of students not receiving an award. Civitas uses what is known as prediction-based propensity score matching (PPSM), a two-dimensional method that matches students. The similar students are matched to similar persistence probabilities that the two students show, these persistence probabilities are matched to several categories including OTP, SAP, CCR, and demography with behavioral indicators. The action of matching one student to another helps control the amount of bias in any analyzed initiative.
Review of Civitas Analytics
The presentation will provide an in-depth review of the Civitas statistical tool, which uses a quasi-experimental statistical method to determine the Persistence lift for the specified student group. The definition of Persistence lift is defined to be the ability for a student to meet On-Time Progression (8/12 CU’s) and be retained to the next term for a minimum of 45 days. Persistence lift is measured by comparing the Propensity Score and the students’ predicted outcomes to the students who participated in the initiative to those who did not. This methodology was patented by Civitas and is an extension of the Propensity Score Matching methodology that was developed over 30 years ago to calculate propensity. Using this patented methodology provided a way to analyze several initiatives by removing bias and effectively limiting the effects of other confounding factors that might differ participant students to those in the comparison group.
Civitas Data Processing Steps
Participants will be offered in inside look at the key data process steps that are integral to establishing a high level of confidence in this cutting-edge student success analytics tool. In this initiative, the participant group was classified as students who received an Excellence Award, and the comparison group was every student who did not receive the award. There are five high-level process steps to use the Civitas, the first being identifying the eligible students (known as the participant group). This includes nine steps to explore and interpret the results; making sure the research team understands the initiative and context details by asking if they truly answered the research question and making sure that the overall persistence lift is intuitive and not too good to be true. The research team also ensures that the results are statistically significant or that the match rate is greater than 85%.
The presentation will include this level of detail describing the use of our quantitative analysis tool so the audience will not only have a greater understanding of the validity in using Civitas but also gain a level of confidence in the findings. The research team will invite discussion regarding the use of this analysis tool and its applicability in future studies of this nature.
Research Findings
The qualitative analysis showed a significant effect taking place by the student’s comments indicating 19% of the comments were regarding the intention of leaving school and giving up. Another trend identified 20% of the comments expressing the boost of confidence. To the extent, this boost of confidence affected the level of motivation was shown with a measurement of persistence lift.
Using Civitas and after comparing students in the same three year time period terms, the statistical analysis (n=30,340) shows a significant overall lift of 10.9% in persistence (p<0.01), with students in the lower quartile being the ones with the most lift in persistence (20.23%, p<0.01). The content analysis (n= 11,644) shows 19% of the student receiving the excellence award was on the verge of dropping their enrollment, deciding to continue their studies after receiving their award.
Another significant finding in this study was students in their first term who received an award had a greater persistence lift than students who did not receive an award (14.96%, p<0.01). Compared to the persistence lift of students who received an award and were in term 1 - 3 (8.44%, p<0.01) and who were in term 4+ (7.36%, p<0.01), these findings indicate that “early cheering” versus “early warning” had a positive effect on the student’s outcome measures: OTP, SAP, and CCR. Additional findings will be shared with the participants including the measurable impact receiving an excellence award has on a diverse student population at the online university and those attending college for the first time.
Closing Discussion
The presentation includes a closing discussion, noting that the results from this research investigation confirmed the findings of past research as it applied to other education models. The recognition of students’ effort as being deemed excellent has been quantifiably shown to impact their student outcomes and achievement measured by course completion, on-time-performance, and satisfactory academic progress. This is the first study of its nature in a competency-based education model (as known at this time).
Future research in this domain is advised including the use of a student survey measuring motivation, self-efficacy, and self-regulated learning levels to triangulate more of the evidence brought forth in this study investigation. Future studies investigating the effect student recognition has on student success factors will be encouraged as a final takeaway for those attending this presentation.