Assessing Readiness for Online Education - Research Models for Identifying Students At Risk

Problem:
In 2013, over 40 million college students took online classes worldwide; by 2017, that number should triple (Atkins, 2013). Online courses potentially increase college access, but higher attrition rates (for reasons not yet well understood), may also be a stumbling block to degree completion. Yet restricting access to online courses may impede the college progression of "non-traditional" students who need the flexibility that online learning affords. To maximize the benefits of online learning while avoiding the pitfalls, institutions need research-based information about which students are at higher risk online.

This presentation will describe a study that compared online and face-to-face course outcomes taken by students at a large urban university system in the northeastern U.S. We will describe which student characteristics were the most consistent predictors of differential online versus face-to-face course outcomes, and subsequent college persistence. Additionally, we will discuss the implications of these findings and the potential of these kinds of studies to impact policies and programs related to online learning at the college level.

Question:
Identifying at-risk online students can only be done by considering the interaction of student outcomes with the online environment, by identifying factors that predict differential performance between comparable online and face-to-face courses rather than factors (e.g. G.P.A.) which predict academic outcomes more generally. Few studies have tested this interaction, resulting in a dearth of information about factors impacting online outcomes explicitly.
This study specifically sought to answer the question:
When controlling for differences in student characteristics, what differences exist between online and face-to-face course and subsequent outcomes; and which student characteristics are the strongest predictors of such differences?

Methodology:
This study used multi-level logistic regression models and propensity score matching to compare the differential online versus face-to-face performance of students with different characteristics. Models were run comparing the same course taken online versus face-to-face by different students, as well as models that compared online versus face-to-face courses taken by the same student. Factors considered include: whether a student has a child (and age of youngest child); gender; race/ethnicity; age; work hours; income; parental education; developmental course placement; marital/cohabitation status; immigration generational status; native speaker status; college level (two-year, four-year, or graduate); G.P.A; and number of credits/classes taken that semester, as well as scales measuring: motivation to complete the course; course enjoyment/engagement; academic integration (i.e. interaction with faculty/students outside class); self-directed learning skills; time management skills; preference for autonomy; and grit (i.e. perseverance and passion for long-term goals).

Results:
Preliminary results indicate that immigration generational status was a consistently significant predictor of differential online course performance. Foreign-born students were more likely to succeed in the online environment versus the face-to-face environment compared to other students with similar characteristics. However, many of the characteristics often associated with immigrants: autonomy, motivation etc. were not consistently significant predictors across different operalizations of the data set, and the effect sizes were small.

In addition, students with young children were significantly less likely to do well in online versus face-to-face courses compared to other students with similar characteristics. Further models are being run separating community college and results to determine if these patterns might be different for community college students.