Managing Variability and Versatility in Adaptive Learning Projects through instructional Designer-Vendor Relationships

Adaptive learning is promising for universities who are seeking to strengthen their long-range academic programming plans in support of student success. Adaptive learning is the ability of a learning experience to deliver a customized personalized experience to each individual learner. This is important because it solves three important problems in higher education--large classes, practice, and feedback--where one individual instructor cannot deliver a radically different experience to every student in the class at the same time. Potentially, adaptive learning technology can. With the right technology, one can orchestrate learning pathways and content that is specific to each student’s individual needs. 

 

While adaptive learning is a promising and novel idea, in reality, it is complex to develop and implement, broad by definition, and highly situated to the contexts it is implemented in. It is complicated because essentially, you are not just building one course, but building multiple learning pathways that address the unique needs of students. This is challenging, but the benefits are worth it because each student's needs are different and adaptive learning can be a path to give everyone exactly what they need to achieve their educational goals. 

 

There is a need to explore best practices and lessons learned from adaptive learning projects implemented at an enterprise level in universities. During this session, participants will: 

1. Examine the variability and versatility of adaptive learning course development projects from three pilot implementations

2. Explore five strategies used to build strong collaboration and partnership among project stakeholders 

3. Reflect with a self-assessment of their current or future adaptive learning initiatives at their university 

 

The case study examined in this session is a two-year adaptive learning initiative  at a five-year public university. Adaptive learning was a university-wide strategy used to meet the critical needs of academic units with large classes, specifically on improving failure rates, personalizing opportunities for mastery learning, practice and feedback, and increasing student engagement. Since Summer 2018, their teaching and learning center, in collaboration with an adaptive learning vendor, has had on-going adaptive development initiatives supporting three courses: Mechanical Engineering, Statistics, and Data Structures. These three courses have now undergone pilot teaching. 

 

Variability 

It is often perceived that the success of adaptive learning initiatives is on the selection and implementation of an adaptive learning platform, however, our experience shows that even in a time of social distancing, it is more critical to draw closer university-vendor relationships. The success of adaptive learning projects is highly dependent on how stakeholders collaborate including administrators, faculty teams, librarians, teaching assistants, instructional designers, and vendor teams.  

 

In this session, we introduce three domains of variability expected from adaptive learning projects, including our findings and recommendations:  

1. Stakeholder involvement - the extent to which stakeholders are involved in the design and development of adaptive learning products

2. Project Management - the extent to which project outcomes, course metrics, timelines, and workflow processes are clearly defined 

3. Design Richness - the extent to which multimedia, content variability, external tool integrations, and learning analytics are integrated into learning

 

Variability is defined as the lack of consistency across course development projects, with the likelihood to vary or change over time. We report our key findings and observations on each domain of variability from three adaptive courses including Mechanical Engineering [MEGR], Statistics [STAT], and Data Structures [CS14] that underwent pilot teaching (Figures 1, 2, 3).

 

Stakeholder Involvement. A key finding in stakeholder involvement is that instructional designers, the vendor team, and faculty teams are the most involved in adaptive projects. They are likely to be the three constant stakeholders involved on an on-going basis in many course development projects. 

Figure 1. Domains of Variability in Adaptive Learning Projects: Stakeholder Involvement

 

Project Management. Our analysis shows that most courses have loosely defined project outcomes, timelines, and process workflow. This is likely due to the novelty or newness of adaptive learning initiatives in the university. 

In effect, it is our experience that instructional designers and the vendor have a dual role, shape-shifting as project managers to move each adaptive learning project forward compared to any other stakeholder. Note the STAT course was a unique case in that it had special funding early in the project to define project outcomes, timelines, and processes. The STATs course has seen positive benefits from engaging in detailed pre-planning activities. 

 

Figure 2. Domains of Variability in Adaptive Learning Projects: Project Management

 

Design Richness. An analysis of course elements shows that most adaptive modules utilize text to present content and practice questions. There is variability in the use of rich media (e.g., video demonstrations, simulations), learning analytics, and integration with external tools (e.g., CodeWorkOut, Zybooks, Piazza, etc.). One common feature to increase student engagement is through the use of practice questions that are variablized to ensure question prompts are not predefined, but unique each time a student revisits the module.  

 

Figure 3. Domains of Variability in Adaptive Learning Projects: Design Richness

 

Versatility 

With degrees of variability in stakeholder involvement, project management, and content, instructional designers and vendor teams are continually having to adapt their strategies to be versatile to meet the needs of the faculty teams in each course. In this session, we further elaborate on five strategies that instructional designers and vendors can apply when working together to achieve versatility in adaptive learning projects: 

1. Artistry: When designing adaptive learning courses, the instructional designer and vendor must work together to maintain a sense of artistry that goes into the arrangement of a variety of types of content, activities, multimedia objects, and interactivity into an effective learning experience. Each adaptive course results in a one-of-a-kind product masterpiece. In practice, there is no template to follow.

2. Discovery: Every adaptive learning project should begin with a joint discovery session among all stakeholders if possible, but most importantly among instructional designers, vendor teams, and faculty teams for the purpose of defining learning gaps, project outcomes, timelines, accountability measures, and process workflows for the design, development, testing, integration, and implementation of adaptive projects.

3. Reusability: A crucial aspect of successful adaptive learning projects is the availability of content in digital form. It is recommended that instructional designers and vendors promote the reusability of existing course materials from the classroom version when building adaptive learning content. Open Educational Resources (OER) are strong sources of content that can be reused or curated further.  

4. Innovation Spirit: Adaptive learning teams should build relationships that have harmony, compatibility, and a genuine spirit to innovate student learning. This is most important among instructional designers, vendor teams, and faculty teams who have the highest level of involvement in the day to day project tasks for adaptive learning. Teams that have misaligned pedagogical approaches, project outcomes, expectations, and working styles are likely to slow down progress and hasten project completion.

5. Resilience. Adaptive learning modules offer the opportunity to be integrated into multiple course modalities including online, blended, and hybrid models. The three courses described in this session continue to be utilized by students through the current remote learning phases, and remain in the pipeline for future developments/improvements. Our experience shows that adaptive learning teams that have strong buy-in from stakeholders, outcomes, timelines, processes, and systems can withstand phenomenological changes (i.e., COVID-19). 

In summary, the session will explore strategies that adaptive learning stakeholders, particularly instructional designers and adaptive learning vendor teams can use to overcome variability and lead with versatility in current and future adaptive learning projects. We support our recommendations with our experience and evidence from three-course pilots.  

Participant engagement: 

Presenters will use a variety of strategies to interact with and engage participants. The participants will gain access to a puzzle toolkit with mosaic tiles to illustrate the key domains of variability and versatility in adaptive learning projects. Participants will be asked to reflect, self-assess, and assemble the puzzle that fits their specific context to gauge their current or future adaptive learning initiative and plan their next steps.