Since 2012, Colorado Technical University has used Realizeit to deliver adaptive learning. This session will use concrete examples to explore the impact of adaptive learning analytics on students and faculty, as well as the ability of analytics to inform course development and the institutions adaptive learning rollout strategy.
Introduction
Discussions on adaptive learning generally focus on the power of these online platforms to adapt content and generate personalized learning paths for individual students. While this is quite powerful, it is just one slice of the benefits of adaptive learning. Its real power lies in the data it gathers and the potential this has to generate a rich supply of learning insights for the student, faculty, instructional designer and administrator.
This session will use concrete examples to explore two perspectives of the analytics that adaptive learning can provide. The first will examine how analytics can be shared and communicated with students and faculty in a manner that is understandable, digestible and most importantly actionable. This area of analytics is commonly referred to as Learning Analytics. The second perspective is from the viewpoint of the instructional designer and administrator. The remainder of the session will focus on the ability of the analytics to inform both the course development and rollout strategy for adaptive learning, as well as the institution’s overall approach to teaching and learning. This is generally known as Academic Analytics. The examples come from data gathered at the Colorado Technical University who, since 2012, has been using the Realizeit platform, branded as Intellipath, to deliver adaptive learning online.
Adaptive Learning in Realizeit
Realizeit bases its first layer of adaptation around learning networks created by subject matter experts (faculty). Each node on the network represents a concept, skill or competency, tied to course outcomes, to be achieved by the student. Associated with each node are a set of content files, assets (learning objects) and additional resources. At this layer, the platform can adaptively choose between these files to select the most appropriate at that time, based upon student interaction with the adaptive platform and then personalize and adapt the delivery of the files content and questions to match the needs, strengths and weaknesses of the student.
Once an individual begins learning within Realizeit, a baseline set of knowledge is estimated. From here the platform suggests a learning path for the student to follow on the road to mastery. This path is not set in stone and the student is left in ultimate control. However, due the nature of the learning networks, students cannot choose a path for which they are not yet ready.
As students progress along their paths answering questions and interacting with the system they generate evidence which Realizeit uses to continuously update their student models. This includes outcome and progress metrics such as estimates of ability, knowledge acquired, and concepts and objectives that still require mastery, as well as metrics which capture behavior and engagement. This is the data which fuels the learning and academic analytics.
Implementation at Colorado Technical University
Branded as Intellipath, Realizeit has been in use at the Colorado Technical University (CTU) since 2012. In that time, it has captured the learning of over 62,500 unique students. These represent over 285,000 enrollments across more than 180 unique courses. CTU have seen over 150 million questions delivered as part of more than 10 million adaptive learning activities. The length of time that CTU has utilized Intellipath, in addition to the number of students who have used the technology indicates that CTU has successfully implemented adaptive learning in a higher education institution in large volume and scale. In addition to improved student success metrics such as grades, engagement, retention and persistence in a significant number of courses CTU has won several awards for its adaptive learning implementation.
Learning Analytics – Impact on student and instructor
Real-time learning analytics are tremendously powerful as they can impact on learning while it is still in progress. The key is supplying the right feedback. To fulfill this the feedback must fit two criteria. Firstly, it must be easily understandable: given the wealth of metrics and data that is available, it is all too easy to overwhelm with beautifully designed yet overly complicated feedback. Secondly, the feedback must be actionable: if the information cannot be used to positively impact on learning then it is simply distracting.
Realizeit and CTU have worked to create dashboards which integrate the feedback and analytics into the functionality of the platform, they are not a standalone piece separate to the learning. The key to this is the open student model used as the main visualization for faculty and learners. This is based on the learning networks where the nodes on these maps are colored based on students mastery levels. Both the student and faculty can openly see the platform’s view of the students’ knowledge. Strengths and weaknesses immediately become apparent and, due to the prerequisite structure encoded in the networks, the student and faculty can trace back to the root cause of any weaknesses and identify a path to resolving it. Additionally, the dashboards have been used extensively by CTU faculty and administrators. Usability of data to facilitate improvement in course and student performance has been central to the design of the dashboard data.
The impact of the adaptive platform on the students’ learning experience has been very positive. In a recent survey of over 1,400 students, 82.0% agreed that it provided them with the necessary feedback to stay on track and 81.7% of students agreed that it helped them learn the course material better than not having it. These numbers align with similar surveys carried out by other institutions who use the Realizeit platform.
Academic Analytics – Impact on course development and rollout
It is important to understand how CTU achieved their success with adaptive learning. A vital component was the manner of their rollout of online courses in the platform and how it was supported by the analytics at each stage of the process. In the EDUCAUSE Review article Adaptive Learning Platforms: Creating a Path for Success, Dr. Johnson outlines the incremental steps taken by CTU to move from a small pilot of adaptive learning with just 3 courses and 358 students to the large scale implementation they have today.
A key feature of this process is the Review & Adjust element. During this stage, analytic feedback provided by the adaptive platform allowed the CTU instructional designers and administrators to analyze the performance of the content, the courses and the platform itself and to implement critical and targeted adjustments as needed. This, in fact, is an iterative process that continues to become part of the course review and aids in the further development and evolution of the course even once it has been rolled out at scale.
Conclusion
It is when adaptive learning is coupled with learning and academic analytics that it has the power to transform an institution's approach to learning. In a recent review of pass rate, final grade, and retention data, CTU identified an Accounting course as the one with the highest pass rate increase since its launch on the adaptive platform in October 2013. They found that the pass rate went up to an 81% average – a 27% increase. The course retention rate rose about 9% to 95%, while the final grade average increased by 10%, to 79%. It is clear from these numbers that adaptive learning can have an impressive and positive impact on learning. The key to this success is both the feedback that is given to the faculty and learners and the analytics that enable courses to be implemented in a controlled and responsive manner.
The session will center on real-world examples from CTU’s rollout of adaptive learning. The presenters will promote engagement with the topic by inviting attendees to participate in the exploration and analysis of the example analytics through the use of questions, answers, and scenarios. Attendees will be encouraged to relate the examples to their own institutions and share their own experiences allowing each person, including the presenters, to gain multiple perspectives on the topics being discussed.