This discussion session examines a competency-based version of an online masters program in instructional design that is facilitated through the use of digital badges, resulting in student e-portfolios which house earned badges. Program faculty will discuss with participants the design process and what benefits and challenges they perceive.
This discussion session examines the design and implementation of a competency-based version of an online masters program in instructional design (ID) that is facilitated through the use of digital badges, resulting in student e-portfolios which house earned badges. Faculty of the program will discuss the design process, the challenges to implementing the program, what changes have occurred since the initial design and why, as well as what benefits and challenges they have perceived since the program was fully adopted as a requirement of the online masters.
Since the pilot version of the program being first implemented in 2016 and fully implemented in 2018, multiple revisions have taken place, and the program has been recognized with university, regional, and national level awards for its use of these combined technologies.
Digital Badges
With higher education under increased criticism for its growing costs, challenges in meeting the needs of an increasingly diverse student body, and ineffectiveness in preparing its graduates for the job market, there have been warnings of the need for significant change to how universities approach learning. Today’s employers lament that graduates are entering the workforce without the necessary skills. There is a disconnect between what today’s learners need to learn and what they are taught. There is also no way for employers to accurately evaluate what a graduate is actually competent to perform based on transcripts and letter grades.
Also known as open badges and micro-credentials, digital badges largely emerged due to the Mozilla Foundation’s Open Badges Initiative. Badge earners would provide evidence of their expertise and badge awarders would review the evidence and award the badge if it was merited, essentially functioning much like the boy or girl scout badges so many are familiar with. The difference would be that the badge earner could share these badges digitally, for example on social media sites such as Linked In, while also making the evidence they submitted for assessment public so that others could verify for themselves the worthiness of the badge. Essentially, badges provide an evidential component that separates them from such opaque and unreliable professional competency representations as Linked-In’s “skills”, which can often represent nothing more than participation and popularity within the social network, or to a finer point, grades on a transcript which often provide little information other than what course was taken and how the student compared to her peers. For example, what does a “B” in a course represent? What learning deficiencies might a student have or what is that student really capable of?
Badges function as visual representation of learner competencies and function as a way to demonstration competency outside of the typical accreditation of formal learning institutions. Badges can support learner autonomy and self-regulated learning (Randall, Harrison, & West, 2013). Badges in education are used to incentivize learning (motivate), identify progress in learning and map out the learning path (signposts), and recognize or credential learning and achievement (Gibson, Ostashewski, Flintoff, Grant, & Knight, 2015).
Combining CBE & Portfolios
Competency-based education (CBE) has been around for decades and is largely associated with trade education and/or with higher education programs that reward much of their credit based on life or work experience. However, at its base level, CBE is focused on personalizing the educational process so that learners may advance as soon as they are ready as well as aligning academic programs with specific, targeted outcomes. Similar to mastery learning, CBE assesses learners on whether or not they have met specific thresholds of learning as opposed to comparing them with peers. It often also allows learners to have a degree of freedom in what sort of artifact or performance they will submit in order to try and demonstrate their competence.
E-portfolios have been another approach to assessing student learning in a more evidence-based way. They capture evidence of student competency by storing student projects. These projects allow for learning to be situated in meaningful contexts and problems, and by integrating the requirement for this type of assessment and project-oriented demonstration of student learning, proponent hope to transform education (Batson, 2011). Assessment with an e-Portfolio is student-centered, highlights the interconnectedness of learning activities and how they are relevant, and provides a means for students to demonstrate tangible evidence of their learning to employers (Ring & Ramirez, 2012).
Identifying Competencies
Prior to this program, we had been utilizing the college’s graduate learning outcomes as well as a few specific to our field, but this resulted in very broad outcomes that ultimately did little to guide our curriculum or indicate to our students what they could expect to have learned after completing our program.
In order to identify more specific competencies that better aligned with our field, we conducted a review of similar programs at other universities. We found many did not formally identify outcomes or competencies and those that did largely based them on professional organization’s published competencies. We likewise reviewed the competencies published by our major professional organizations. Finally, we reviewed The International Board of Standards for Training, Performance and Instruction’s published competencies, which are based on decades of research. This ultimately was most beneficial in guiding us, as they break the competencies down into major competencies and sub-competencies, as well as required competencies and advanced competencies.
Our program went through a lengthy and detailed discussion of what competencies we believed our program should require, and also reworded a number of them for clarity. In basing our badges on these competencies, we would be establishing a badge hierarchy, and choosing the number of competencies we would end up was an important consideration. For example, we would have four overarching badges that would be comprised of as many as 19 or as few as 12 sub-badges, based on what we would ultimately choose to require. Each of these sub-badges would represent specific competency with specific tasks/knowledge/skills that students would need to prove they had competence in.
This helped us to realize the potential challenges with evaluating all of the student submissions. We had originally intended to have students earn badges as part of their coursework, but we determined this would too much of a burden on both students and instructors and earning a grade in a course would not guarantee that a badge was earned, meaning that there might be instances where an assignment might need to be graded multiple times. Furthermore, we had to decide how to handle students who could demonstrate competence with artifacts produced outside of their coursework.
We ultimately identified a set of competencies and sub-competencies that we were comfortable with and decided to decouple badges from specific course projects. Instead, students would take a one credit hour badge course each semester in the program where they would be required to make progress towards the earning of badges, whether by aligning coursework artifacts or professional or other external artifacts with badge challenges. We decided to mitigate the evaluation burden on instructors of the badge course by requiring peer evaluation to be completed prior to a student being able to formally submit a product to meet a badge challenge. This reduces the cycle of submission and feedback between instructor and student and allows for a more cursory review by instructors that requirements have been met. It also requires very clear and specific rubrics for each challenge so that peer evaluation will be accurate. By having the badge course in each semester, students would be reflecting on their learning throughout the program, rather than waiting until the end of the program when they work on their final portfolio as they formally did.
A final portfolio course is used to finalize the earning of badges and to create an e-portfolio that provides a more holistic review of the learning experience, incorporating reflection narratives for each supra-badge and sub-badge. These badge courses and the portfolio course also serve as a quality check. Course completion and grades given on course assignments are separate from badge earning. Therefore, if a student receives a grade on an assignment that was perhaps not deserved, it can be refused as meeting badge requirements. A student would then need to remediate or use an alternative artifact. Likewise, if a student had professional artifacts that better represented her skills, she could use that to earn a badge.
A final process was reviewing course outcomes and assignments and determining what courses aligned with which competencies. This was very helpful in identifying existing gaps within the program: things we believed we were teaching our students only to discover that we actually were not. It also helped to identify strengths and weaknesses in our curriculum and areas for improvement as well as helping faculty to have a better understanding of the program overall.
Lessons Learned
The biggest lesson learned was that the identification and crafting of competencies was the most difficult aspect of the prospect. Furthermore, figuring out how to create a process for badge approval that does not overburden either the students or the faculty was a challenging balancing process, that has continuously been refined, particularly in regards to crafting rubrics general enough to be used for a variety of products submitted as evidence but specific enough to be meaningful when used to evaluate. There have been challenges for students, instructors, and faculty in managing the process but also feedback from numerous graduates of the program on the benefits to their learning and employment prospects.