Previous studies reported the low completion and engagement rate at Massive Opening Online Courses (MOOCs). This proposal presents an intervention design by using digital badging system as means to boost and sustain massive learners’ engagement in the self-regulated learning (SRL) environment as MOOCs and conduct analysis to evaluate its effect.
Researchers and educators contend that MOOCs increase educational equity and enable students’ participation and engagement in learning based on their goals, skills, and common interests (McAuley et al., 2010). However, the notoriously low average completion rate ranging from 0.4% to 15% impedes the development and sustainability of MOOCs (Liu et al., 2014; Jordan, 2015) and suggests that MOOC learners lack appropriate incentives to complete courses. Considering the contributing factors, there is a strong call for educators to refine the quality of pedagogical design and boost learners’ motivations to sustain learning in MOOCs. Meanwhile, MOOCs situate in a self-regulated learning (SRL) environment, where learners take control of their own learning from the cognitive, affective, motivational and behavioral points of view (Boekaerts, Pintrich & Zeidner, 2005). Thus, making learners aware of their level of competence, a necessary condition to make autonomous decisions, might propel to sustaining their learning motivations (Cucchiara et al., 2014).
This proposal seeks to propose a research design that adopts the digital badging system, as a formal recognition to boost and sustain massive learners’ engagement in the self-regulated learning (SRL) environment. During the presentation, the discussion of the research design will be the focus for questions and conversations. Due to the early stage of using the digital badging in online education, there is limited study to examine its effect in supporting learning in the recent open education platforms such as MOOCs. Therefore, I propose an intervention study that will explore using digital badges in a MOOC course and its effect on learning outcomes, as measured by completion rates, the level of learners’ engagement and change of motivation. The study will answer the following three questions: R1. Do course learners who use the digital badging system in the treatment group have better completion rate than those in the control group who learn in the regular MOOCs environment? R2. Do learners who use the digital badging system have greater engagement in learning the course? R3. Do learners’ motivation for learning change when using the digital badging system?
To determine the effect of digital badging on learning engagement and motivation, I will design a set of digital badges in three categories that are aligned with course activities and objectives. The badges will purposefully be designed to encourage active participation, quality production and contribution of knowledge, collaboration with others, and additional learning. I will use open badge system offered by Mozilla Foundation (https://openbadges.org/) to design and issue the set of badges that will be attached to the MOOC course system.
To answer research questions 1-3, I will design a randomized controlled trial that serves to investigate the effect of using digital badging to boost learning motivation and engagement in a MOOC, and take a Chinese MOOC as a hypothetical implementation example. The treatment group will be exposed to a game-based learning culture that introduces how the digital badging system work and what badges they will earn during learning processes. Digital badges will be offered automatically when students meet the requirement for the digital badges such as completing a learning task, or continuous learning for weeks. Learners will claim the badges and choose if they would like to add it to their account main docks or not. Learners in the treatment groups will gain completion badges when they complete the course and the course certificate if they reach the requirement. The control group will be offered the original MOOC course, where they will be provided with “business-as-usual” content and learning materials as the treatment group, without providing any information or incorporating the digital badging system. Learners who complete the course obtain a mark by the system as the evidence to show the completion and those who reach the requirement for the certificate still gain a final certificate as the credential.
Three measures will be used to analyze the effect of the intervention: completion, engagement and motivation. Completion rate data will be collected through the course grade book in the learner activity log through the system. The learner engagement will be measured in two ways: the user activities through interpreting behavioral patterns in the learner activity log; and a self-reported engagement survey (Beeland, 2002) that includes 20 items that are categorized as the variable self-reported engagement. I will measure learners’ motivation changes during the learning process in this MOOC course to measure the effect of the badging system on learner’s motivation change. Given the context of MOOC learning and the focus on learning motivation, the adapted Learning Self-Regulation Questionnaire (SRQ-L; Ryan & Cornell, 1989) with 32 likert-type items will be distributed into two groups three times during the course, reducing the effect of the high attrition rate of the MOOC course.
The R1 question will be answered using a two-sample t-test to compare the means of completion rate of treatment and control groups. To answer the R2, Learner engagement is the combined variable of the average mean for user interaction data and self-reported engagement. For the treatment group and control group, a two-sample t-tests will be used to analyze the differences in engagement between week 7 and week 15. In order to test the effect of digital badging in student learning, the use of digital badging system and student demographic information, will be used as the independent variables to predict the outcome variables as user interaction, self-reported-engagement and learner engagement by conducting multivariate regression analysis. To answer question R3, the one-way ANOVA test will be used to examine the differences in learners’ level of motivation by three time points, Week 1, Week 7 and Week 15. The limitation and threats to validity will also be discussed in the presentation.
The goal of the presentation is both to improve my proposed design to explore effective strategies to improve student learning and engagement in the MOOC learning space and support others in designing related research and development projects. It will target educators, course designers, technologists, and faculty in higher education to engage in an active discussion on the improvement of the research design on the intervention study, their best practices and suggestions to boost students’ learning motivation, and the technical operation of the digital badging system. To improve the interactivity, this presentation will use the following strategies to engage the audience:1) Incorporate the PollEverywhere application to set interactive questions for the ice-breaker, design feedback, suggestions and comments. 2) I will divide the audience into groups to discuss some concrete questions that are related to their special area of interest during the presentation. 3) I will also prepare digital access and a printed copy of the handout for the audience.
Academic Sponsor: Samantha Daley sdaley@warner.rochester.edu