This session describes results from a maturing three-year research initiative among two organizationally different universities and their common adaptive learning platform provider. Current findings indicated that the underlying pattern of learning in adaptive courses remains comparable across disciplines and institutions. These findings have implications for predictive analytics and instructional design.
Background
Adaptive learning continues to grow in popularity as a promising instructional innovation used in the ongoing higher education battle to improve student success and retention. Adaptive learning acts like a GPS for students. As they progress through the course content, it allows for personalized instruction while altering their pathways -- continually assessing their knowledge to help them most efficiently and effectively progress through the course (Moskal, Carter, & Johnson, 2017). This ability to allow students to either advance or remediate is one of the reasons adaptive learning is being investigated for its potential to use mastery to improve instruction (Bienkowski, Feng, & Means, 2012; Dziuban, 2017). The hype is likely to continue in the near future with a number of national reports pointing to adaptive learning as one of the important developments or emerging technologies in education (Becker et al., 2017; Legon & Garrett, 2018; Office of Educational Technology, 2017).
Several national initiatives have provided funding for investigating adaptive courseware’s potential in higher education (Bill and Melinda Gates Foundation, 2014; Association of Public & Land-Grant Universities, 2016; Online Learning Consortium, 2016), and while some preliminary results have been positive, there is much more work to be done. Much of the research indicates the varied campus climates and adaptive courseware implementations can make comparisons and generalizability of findings difficult and the research to date has not been as prolific or as promising as hoped.
Our Cooperative Study
This presentation will describe a cooperative adaptive learning evaluation model between the research unit of the platform provider (Realizeit), the Research Initiative for Teaching Effectiveness at the University of Central Florida and Colorado Technical University. This ongoing collaboration has taken place since 2015 as researchers suspended the focus on the typical vendor/university relationship with the understanding that the three organizations have unique contributions that together strengthen the research possibilities. Our focus has been to investigate the phenomenon of adaptive learning, as opposed to examining individual platforms.
In this session, researchers will share some of the outcomes from the 3-year collaborative relationship including student reactions and student behaviors within these systems. Findings from joint survey research at both UCF and CTU indicate that students across both institutions react similarly to adaptive learning, revealing that they like the feedback and personalization this modality provides. They also feel it makes them more engaged in the learning process and would prefer more adaptive learning in their educational experiences (Dziuban, Howlin, Johnson, and Moskal, 2017; Dziuban, Moskal, Cassisi & Fawcett, 2016; Dziuban, Moskal, Johnson & Evans, 2017).
In addition, an examination of student behavior within adaptive learning identified latent dimensions underlying these courses across multiple disciplines and the two structurally different universities. The objective was to determine if differing disciplines and university contexts impacted adaptive learning patterns, again focusing on examining adaptive learning as a process rather than evaluating an adaptive learning platform.
A subset of key student performance indices generated as students progress through the adaptive content depicted students’ cognitive outcomes and behaviors for three courses from UCF, three courses from CTU, as well as combined samples for each institution. The indices were intercorrelated and subjected to the principal component procedure (Mulaik, 2009) to explain the variance and relationships (correlations) among the indices, and to reduce the dataset to a smaller dimensionality. Operationally, the study examined the question: is the cognitive organization of adaptive learning constant or do the patterns change by institution or course context?
Pattern matrices and similarity confidents indicated that the underlying dimensions of adaptive learning remain stable within disciplines, across disciplines, and across the two universities. With some minor variation across UCF and CTU, the component similarity and invariance was relatively stable and four components were identified:
Knowledge Attainment
Engagement Activities
Growth
Communication
Learning science suggests that there is a clear relationship between these traits -- engagement and communication are prerequisite for growth and achievement although in this study they are statistically independent of each other. This finding supports the notion that this underlying pattern is fundamental to effective teaching and learning using adaptive platforms.
Adaptive learning (AL) creates a fluid educational environment responding to the needs of many student cohorts. Features like initial student knowledge baselines, continuous assessment and feedback, redesigned learning paths, mastery certification and instructional format preference make a more flexible and responsive educational landscape. In the introduction we mentioned AL’s modification of learning time. Adam (2004), in her work with temporal culture, provides insights into what can happen with constant outcomes and variable learning time.
Learning transforms a student’s:
- Time frame: The time boundaries of a course or program of study
- Timing: When will learning will take place
- Tempo: The pace of learning
- Duration: How long will learning take place
- Sequence: In what order will learning take pace
Faculty importance increases in the adaptive environment because they can identify learning objectives for students through course design, and analytics data provided by the system. Therefore, instructors can suggest effective interactions and interventions with students in areas that require support or additional instruction. Faculty members have a real-time view of student progress that is not available in other methods of teaching. For instance, adaptive systems can reliably identify skills or concepts with which the class on average is excelling or having difficulty. In addition, instructors can track individual student progression through course content. This provides faculty the opportunity to adapt their lecture, activities, or homework assignments to personalize instruction.
Conclusion
While we will discuss our research findings, this session will also focus on the importance of establishing collaborative research between universities and vendors. The growth of available learning analytics make this relationships particularly fruitful and also critical to moving educational research forward.
This study is a collaboration among two universities and their common adaptive leaning platform provider. Each organization brings different strengths to the partnership. CTU achieves scale with adaptive implementation. UCF integrates research and data into the decision-making and policy process. Realizeit brings advanced analysis skills and makes transparent analytic data available all its partners. Because of this small network each organization improved its adaptive learning process--the universities with pedagogy and Realizeit with its platform. This happens over time in a nonlinear process that encounters a good deal of productive failure. The technology does not drive the work, but rather the research helps improve the technology. The partners commit to pushing information and flexibility out as far as possible and believe that progress happens in small steps. Simple is more effective. Without the partnership and the sharing there would be no study. None of us could do it alone. Therefore, our major conclusion is that we need more extensive collaborative work. Each university can similarly contextualize adaptive learning and every platform provider can support an active research agenda to form similar, and increasingly productive, collaborative partnerships.
References
Adam, B. (2004). Time. Cambridge, IK: Polity.
Association of Public & Land-Grant Universities. (2016). Personalizing Learning with Adaptive Courseware. Retrieved from http://www.aplu.org/projects-and-initiatives/personalized- learning-consortium/plc-projects/plc-adaptive-courseware/
Becker, S. A., Cummins, M., Davis, A., Freeman, A., Hall, C. G., & Ananthanarayanan, V. (2017). NMC horizon report: 2017 higher education edition (pp. 1-60). The New Media Consortium.
Bienkowski, M., Feng, M., & Means, B. (2012). Enhancing teaching and learning through educational data mining and learning analytics: An issue brief. US Department of Education, Office of Educational Technology, 1, 1-57.
Bill & Melinda Gates Foundation. (2014, November). Early progress: Interim research on personalized learning. Retrieved from http://collegeready.gatesfoundation.org/wp-content/uploads/2015/06/Early-Progress-on-Personalized-Learning-Full-Report.pdf
Dziuban, C. (2017). The Technology of Adaptive Learning. Education Technology Insights.
Dziuban, C. D., Moskal, P. D., Cassisi, J., & Fawcett, A. (2016). Adaptive Learning in Psychology: Wayfinding in the Digital Age. Online Learning, 20(3), 74-96.
Dziuban, C., Howlin, C., Johnson, C., & Moskal, P. (2017, December 18). An Adaptive Learning Partnership. EDUCAUSE Review.
Dziuban, C., Moskal, P., Johnson, C., & Evans, D. (2017). Adaptive learning: A tale of two contexts. Current Issues in Emerging eLearning, 4(1), 3.
Legon, R. & Garrett, R. (2018). The Changing Landscape of Online Education (Chloe) 2: A Deeper Dive. CHLOE2.
Moskal, P., Carter, D., & Johnson, D. (2017). 7 Things You Should Know About Adaptive Learning. ELI.
Mulaik, S.A. (2009). The foundations of factor analysis, second edition. London, United Kingdom: Chapman and Hall.
Office of Educational Technology. (2017, January). Reimagining the Role of Technology in Education: 2017 National Education Technology Plan Update. US Department of Education. Retrieved from: https://tech.ed.gov/files/2017/01/NETP17.pdf
Online Learning Consortium. (2016). Digital Learning Innovation Award. Retrieved from https://onlinelearningconsortium.org/about/olc-awards/dlia/