An Adaptive Learning Partnership

Introduction

The higher education innovation landscape involves universities who look for creative instructional technologies and the vendors who market their platforms. Historically, all parties perceive this relationship from their own perspectives. Universities develop strategies for initiatives they consider responsive and transformative, based on their research agendas and the communities they serve. Vendors may work closely with higher education professionals who may advise them on product development. But the vendor perspective of higher education is mediated by their business propositions.

This proposal describes a research relationship formed between two universities and a vendor. We describe our research and positive results achieved only with our cooperative relationship, working together to investigate adaptive learning. Suspending the vendor/university relationship allows for collaborative research partnerships, an essential bond of what should become the new normal in higher education.

Two University Partners: UCF and CTU  

The University of Central Florida (UCF) and Colorado Technical University (CTU) were early adopters of adaptive learning (AL) and began discussing their approaches at scientific meetings, describing their instructional and implementation strategies. Both UCF and CTU adopted Realizeit’s adaptive learning platform as their enterprise solution (realizeitlearning.com). Realizeit has a robust ability to support the sharing of experiences and data, providing insight into successful adaptive learning practices. Both universities were experiencing initial positive results with the technology, but at considerably different scales.

After conversations and collaboration on writing and conference presentations, it became clear that a research partnership between UCF and CTU would be beneficial. A fundamental component for successful cross-institution collaboration was an openness to each other’s work, in spite of considerably different student populations, faculty composition and structure. UCF is a large public university (66,000 students) in the pilot phase of AL adoption while CTU is a private, for-profit institution (21,200 students) that is further along with scaling adaptive learning. The initiative at UCF is research intensive, while implementation centric at CTU.

Enter the Vendor -- The Realizeit Partnership

Vendors develop their products based on a view of how their approach and technology can empower the various instructional roles within an institution. The variety of institutional contexts means that vendors cannot treat a genuinely adaptive learning platform as a simple plug-and-play device. Each implementation must be responsive to the requirements of the courses, instructors, academic standards and goals of individual institutions.

The most successful integration occurs when an open, collaborative relationship is fostered between the vendor and its university partners. While any vendor will have intellectual property that cannot be disclosed, there needs to be transparency in core areas. These include how the platform makes decisions, which data points are available, how the platform provides feedback and guidance to the learners, how it influences the direction of learning, and how it supports the student. When working with Realizeit, all data points generated are owned by the participating institution and are made available to it for measuring the impact of adaptive learning. This information has been the primary source of insights that have influenced the evolution of the platform.

Student Adaptive Behavior

Working with UCF and CTU, Realizeit researchers conducted a study examining students’ behaviors as they engaged with the adaptive learning platform to better understand how they managed their learning in an adaptive environment. In more traditional settings, students progress through the learning cycle in a nearly uniform pace and may not be able to accelerate through new lessons or repeat prior lessons for remediation. The agency of adaptive learning enables a variety of effective behaviors that can inform us on improving student success.

Examining student progression metrics, we were able to identify their behavior patterns indexed by the proportion of concepts completed in courses over time. Common prototypes emerged across different content domains and university settings (a typical CTU course lasts around five weeks compared to fifteen weeks at UCF, except for summer twelve-week semesters). However, depending on how the course was structured and how instructors engaged with their students, not all prototypes appeared in every course. 

The Tortoise and Frog were used to represent the two most common behavior styles of students in higher education. These students make systematic, steady progress throughout the course--a behavior most likely encouraged by many instructors. On the other hand, the Hare and the Kangaroo are prototypes that emerge as a result of adaptive learning self-direction and self-pacing. These students accomplish the majority of their progress in altered time frames, jumping ahead in spurts or in a flash. Since all student prototypes tended to complete, the primary difference was the manner in which students engaged in the course.

Improving Outcomes for Students

One insight that has become evident from the collaborative work of UCF and CTU is that any adopter of adaptive learning must stay on course. Adaptive learning is not an instant solution. Its strength lies in the feedback cycle that it provides to students, instructors, and instructional designers, allowing them to formulate outcomes through an iterative improvement cycle over time. To highlight what is possible, we examined how improvements in both the courses and the adaptive platform have led to an increased attainment level of students taking College Algebra at UCF. For example, the instructor has taken what she has learned after each course, to implement changes in the materials and structure, which produce improvements in student outcomes that are measurable and significant.

The impact of the changes by the instructor becomes much more evident by splitting the students into three cohorts and examining what has happened for each.

• Top 25% - These students have moved from covering at least 86% of concepts to covering 95% or more.
• Middle 50% - These students moved from covering between 49% and 86% in Spring 2015 to covering between 61%and 95% in Fall 2016. Both the top and the bottom boundaries are achieving more.
• Bottom 25% - In Fall 2016 these students covered less than 49% of the course. However, by Fall 2016 some of these lower achieving students are covering up to 60% of the course.

All cohorts have improved, the best have gotten better, and even the lower achieving students have improved. Adaptive learning has shifted the curve in this course, and as we can observe it, has been universally beneficial. The key is staying in the course and committing to using data from each iteration to learn what worked, what didn’t, and where improvements are needed. This same strategy is one that has been used at a much larger scale at CTU.

Student Reaction to Adaptive Learning

We were interested in students’ reactions to adaptive learning and how they might differ across CTU and UCF. UCF developed a survey as part of their pilot evaluation and CTU adopted the same survey allowing for comparisons. While the demographics and campuses vary, it would be beneficial to know if students at both institutions were equally receptive to adaptive learning. At UCF, 300 students were surveyed in General Psychology, an adaptive course delivered in Fall 2014 and Spring 2015 with 244 respondents (81% response rate). CTU sent their survey to all students enrolled in adaptive courses (14,400) and received 1,140 completed responses (10% response rate).

The survey gauged student reactions and experiences with their adaptive classes, capturing details on their interaction with the system, including ease of use, helpfulness of feedback, and guidance and accuracy of platform assessment metrics. In addition, the survey captured overall student attitudes about the nature of using adaptive learning in instruction, including what was most and least positive about this instructional method and how it impacted their interaction with and time spent on the course. Demographic questions allowed examination of differences across student cohorts. Overall, students at both institutions were positive. However, some reactions to adaptive learning illustrated significant differences between students’ reactions at CTU and UCF. We will present highlights in this session and more details can be found in Dziuban, Moskal, Johnson & Evans (2017).

Adaptive Leaning--It’s About Time

Over the past few years, higher education has been able to refocus on adaptive learning because emerging technologies have created more effective capabilities just as they have with artificial intelligence that is emerging from its years of doldrums. Platforms have different strengths and weaknesses but each one adheres to the adaptive paradigm. Our conversation describes a research partnership between two universities with considerably different missions and Realizeit that grew organically through the convenience of a common platform. However, effective partnerships can emerge across several different providers--a development that, in our opinion, would be highly beneficial avoiding the “which is the best platform” narrative, focusing on the notion of adaptive learning as a construct.

We will focus this session on the need for a thoughtful examination of adaptive learning as an instructional process and the value of research findings possible with a university/vendor collaborative relationship. We have presented a small glimpse at our findings here and will illustrate more in the session. In addition, we hope to open a dialog with other universities, encouraging and promoting future collaborative opportunities.