Did COVID-19 exacerbate inequities? Course outcomes, on average and when considering students' prior online experience

Session Goals

Attendees will be able to identify inequities in educational outcomes that appear to have been either exacerbated or narrowed by the shift to online learning during spring 2020 because of the COVID-19 pandemic. They will be able to describe the differences in such inequities for a variety of groups (two- vs. four-year college context, gender, and race/ethnicity), first on average, and then after taking into account students’ prior online course-taking.

Study Motivation

During spring 2020, COVID-19 engendered a rapid shift to a fully online mode of instruction, with little-to-no advance preparation. New York City was the first U.S. location hard-hit by the pandemic, and therefore, students at the City University of New York (CUNY) were some of the first to be significantly impacted by the pandemic in the U.S. This created pedagogical challenges for instructors concurrent with many students experiencing significant pandemic-related life challenges and stressors. We explored the extent to which conditions of the spring 2020 pandemic term appeared to impact the resiliency of different college student groups in terms of their course outcomes, drilling down within groups to explore possible reasons and implications of unexpected results. 

Theoretical Framework

This research draws on the concept of resilience, defined as “a phenomenon or process reflecting relatively positive adaptation despite experiences of significant adversity or trauma” (Luthar, 2006, pg. 6). In this context, we conceptualize resilience as relative—the extent to which a student’s course outcomes improved, stayed the same, or worsened during the pandemic term when all courses were forced online.  

Method

The data used included all courses taken by students enrolled in fall 2019 or spring 2020 in the CUNY system, which includes two and four-year colleges. Courses are classified by instructional mode: fully-online vs. traditional(i.e. “not fully-online”). For fall 2019, these designations describe the mode for the entire term; they describe only the start of the Spring 2020 semester before instruction shifted online. Students are classified as: Traditional-mode(students who enrolled in no fully online courses in either term) and Dual-mode (students who enrolled in at least one fully online course and at least one traditional course each term). This study investigated successful course completion, defining this outcome as a course grade of at least C- (typically needed for transfer or credit in a major). 

We report the results of multi-level linear probability models (using Stata’s mixed command) since we cannot otherwise compare coefficients across logistic regression models (Buis, 2010) and we are primarily interested in estimating parameters rather than forecasting outcomes. However, whenever predicted probabilities are presented, we used similar logistic regression models (using melogit) because they better model the probability distribution. The multi-level models included: (level 1) the individual course record; (level 2) the particular student; and (level 3) the college where that student took that course. This allowed us to control for clustering by student and by college, accounting for the tendency of grades in different classes for a given student, and grades for classes at the same college, to be more similar to one another. Separate regressions were run for two- vs. four-year colleges, because student outcomes at these two different types of institutions can be quite different (Attewell, Heil & Reisel, 2018).   

Results and Discussion

In considering the results, it turns out our intuitions were not always a good gauge of the outcomes obtained by different groups of college students. We encourage participants to bring their intuitions to our session and be surprised by what you learn. We will challenge participants to think about reasons why these results occurred and then engage in active discussion of possible alternative explanations. Next, we offer a summary of the results to be shared and discussed (spoiler alert!).

A. Pandemic trends in course outcomes at two- versus four-year colleges

• Four-year students had more resilient course outcomes than students at 2-year colleges during the pandemic, which brings up equity issues. Even when adding controls (such as gender, race/ethnicity, GPA), differences were smaller but did not change the pattern or significance. We had expected all students, regardless of institution type, to do less well in spring than fall because of the abrupt shift to online learning during the pandemic semester, so this result was unexpected.

B. Differential patterns by gender or race/ethnicity and term

• Women had more resilient course outcomes than men during the pandemic, which mirrors pre-pandemic research. Men at 2-year colleges had significantly less resilient course outcomes than women, but this result did not occur at 4-year institutions. This is another surprising result, since it has been documented that women (especially mothers) have been more negatively impacted by the pandemic, including taking the brunt of childcare (Heggeness & Fields, 2020), and 2-year college students are more likely to be parents (Institute for Women’s Policy Research, 2018).
• All racial/ethnic groups did better on average at 4-year colleges during the pandemic compared to pre-pandemic (another surprise). At 2-year colleges, Asian student outcomes did not change, yet Whites, Blacks and Hispanics had less resilient course outcomes during the pandemic, with Hispanic students being particularly negatively impacted. 
• At both institution types, the gap in successful course completion rates between White and Black students was slightly but significantly improved during the pandemic.  Asian/PI students’ course outcomes were significantly lower than that of White students in fall, but on par with those of White students in the spring pandemic term, closing the gap.  On the other hand, the gaps between Asian/PI students and both Black and Hispanic students widened significantly during the pandemic.

C. Voluntary online enrollment, course outcomes, and gender or race/ethnicity

• The gap in course outcomes between men and women increased significantly for traditional-mode students during the pandemic term at both institution types.  However, there was no significant interaction between term and gender for dual-mode students, so being a dual-mode student (i.e., having voluntarily chosen to enroll in a fully online course both fall and spring) appeared to be protective for men during the spring pandemic term.  
• For traditional-mode students, there were also significant differences in academic resilience by race/ethnicity, but these differences tended to be less significant or not at all present for dual-mode students. For dual-mode students at two-year colleges, the resilience of Black students was particularly strong, even reducing gaps between Black and Asian students during the spring pandemic term.  Among traditional-mode students at 4-year colleges, the gap between White and Hispanic students increased significantly.
• Interestingly, patterns indicate that pandemic conditions overall (and perhaps the switch to fully online courses in particular) likely contributed to increased gaps in successful course completion by race/ethnicity for traditional-mode students, but had the opposite effect, contributing to decreased gaps in successful course completion by race/ethnicity for dual-mode students.

Many of our findings are surprising, others are thought-provoking. These results will form the basis of sensemaking discussion among participants during our session, wherein we will explore possible alternate explanations for these results. We believe these results show ways that colleges might invest more to ensure that malleable factors (e.g., skills related to learning online, access to technology) are distributed more equitably. Additionally, if students with prior experience online chose that mode because they needed flexibility, and the pandemic facilitated their getting such flexibility, that suggests that offering more flexibility post-pandemic may help to improve racial/ethnic gaps in outcomes for those students who choose to take courses online; this should be investigated in future research.

Interactivity Plan

This session will include an interactive discussion organized around a series of provocative questions grounded in our research results and intended to inspire future research and intervention direction. After an overview of the research, questions showcasing different types of possible inequities in course outcomes will be posed to the participants. Attendee responses will be compared to actual outcomes found in our study, followed by probing discussions of how our intuition about inequities from prior research and experience does not always translate nicely to pandemic-era student experiences. Take-aways for future research directions will be elicited, aimed at identifying interventions that could be continued or developed to either support promising pandemic trends or counter new problematic developments. This interactive discussion format will help the online research community, as well as faculty, staff and administrators who support online students, understand our results and consider how they may be applied in future institutional research and campus support efforts.

References

Attewell, P., Heil, S., & Reisel, L. (2011). Competing explanations of undergraduate noncompletion. American Educational Research Journal, 48(3), 536-559.

Buis, M. L. (2010). Stata tip 87: Interpretation of interactions in nonlinear models. The Stata Journal, 10(2), 305-308.

Heggeness, M. L., & Fields, J. M. (2020). Working moms bear brunt of home schooling while working during COVID-19. US Census Bureau. Retrieved from: https://www.census.gov/library/stories/2020/08/parents-juggle-work-and-child-care-during-pandemic.html

Institute for Women's Policy Research. (2018). Parents in college by the numbers. Institute for Women's Policy Research. Retrieved from: https://iwpr.org/wp-content/uploads/2020/08/C481_Parents-in-College-By-the-Numbers-Aspen-Ascend-and-IWPR.pdf

Luthar, S.S. (2006). Resilience in development: A synthesis of research across five decades. In Cicchetti, D., & D.J. Cohen, (Eds). Developmental Psychopathology, (2nd Ed.): Risk, Disorder and Adaptation, 739-795. Wiley.