Personalizing Digitized Assessments and Remediation using an Automated Micro-Credentialing Framework for Canvas LMS

1.0 Introduction

The Building-the-Capacity Ecosystem (BC-Eco) developed by the project leverages digitized assessments that automate micro-credentialing of student skills to facilitate personalized tutoring, peer-mentorship, and internships. The Services Provided are listed on the left-hand side of Figure 1.1. The foundational level, referred to as micro-credentialing, utilizes digitized assessment data that is capturable from 29 STEM courses having assessments that were already digitized in the REDACTED UNIVERSITY curriculum from 2014 to 2019. 

Figure 1.1: Building-the-Capacity Ecosystem (BC-Eco). Starting at lower-left corner, students taking STEM courses follow along the gold-colored arrows as Student Mentees to Student Mentors to Interns. Students’ progress upwards through levels of Micro-Credentialing, Peer-Tutoring, and Industry Internships. Learning activities span Culturally Relevant Instruction within gateway courses, Underrepresented Recruitment subsidized by tutoring stipends, and financially-supplemented opportunities for Career Training.

Based on the success of digitized assessments in the College of Engineering and Computer Science (CECS) and in the College of Sciences (COS) at REDACTED UNIVERSITY, their benefits to students, the multi-disciplinary BC-Eco Project has been initiated as shown Figure 1.1. One of the three levels of student-facing mechanisms  is generating digital micro-credentials from digitized assessment results. This is automated by developing micro-credentialing Python programming language computer scripts via a one-time tagging step of the questions within the digitized assessments in STEM courses.

Successfully earning digital micro-credentials constitutes the first step for student participants by becoming a tutoring candidate. For example, a student may earn a digital micro-credential in a topic such as Finite Element Modeling or Human Computer Interface Design. 

2.0 Micro-Credentialing Framework

Micro-credentialing is a subject of active research and offers more portability and flexibility to evaluate student skills beyond grades and test scores. In this NSF project, an open-source framework for awarding and managing badges is developed far beyond what already exists. Micro-credentials provide mechanisms for quantifying student skills, abilities, and knowledge beyond traditional grades and transcripts. In the context of this project, students are awarded through digital badges obtained through data mining of their digitized assessment data. 

Micro-credentialing is an emerging means to authenticate an achievement, accomplishment, or even readiness to advance [1]. It has been suggested that digital micro-credentials may increase equity and improve retention in higher education [2]. Students’ acceptance of the value of micro-credentialing is a key motivating factor for earning micro-credentials which in turn can increase motivation during learning [3]. The BC-Eco micro-credentialing process commences as students complete their assessments, via Python programming language scripts. Operationally, data mining and data-driven discovery approaches provide a bottom-up alternative to using contextual state information and interactive cues as in top-down reasoning approaches [4]. Specifically, the project reaches means to deliver and refine STEM micro-credentials through the annual cycle shown in Figure 1.1.

The inspiration for our Micro-credentialing process was based on the physical badges used by websites such as Stack Overflow, offering students the ability to share and display specific aspects of their achievements compared to the typical transcripts or grade-point averages. Moreover, micro-credentials can be used to represent student learning, to increase opportunities for students to achieve expertise through learning experiences spanning various contexts and to establish learner accomplishment more holistically [1,5]. For instance, since 2011, the Mozilla Foundation has provided open-source digital badges through the Open Badges Initiative (OBI) [1]. Moreover, micro-credentialing opportunities also offer new tangible incremental outcomes for HSI students’ resumes and LinkedIn profiles.

2.1 Tailoring Micro-Credentialing for STEM Assessments

In the first year of this four-year project, substantial progress towards a transportable micro-credentialing framework of the Building Capacity Ecosystem (BC-ECO) project was completed in accordance with the project plan. Activities completed include the following:

• Initiated identification of subject matter areas and expertise in selected content areas while training research assistants in specialties of educational data mining,
• Negotiated allocation of space for computer stations in tutoring room area,
• Refined plan and materials for solicitation of participants in the project,
• Programming language selection and database configuration for micro-credentialing software based on first-round targeted courses, and
• Requirements analysis/prototyping to develop micro-credentialing software framework.

2.2 Foundational Micro-Credentialing Software Currently In-use

For this type of data mining, our NSF project has developed a Python script-based framework which outputs a CSV file listing student performance on a question-by-question basis. The Python script was developed to meet the requirement specification by first traversing all the questions in a quiz to classify each question into a specific category, e.g., Theoretical skill or Applied skill. Next, for each student and for each badging category, the software framework examines student responses to determine if student responses to those questions were correct up to the threshold set by the instructor. Thereby, the software identifies whether to award a badge in the specified category to each student. 

Application Programming Interfaces (APIs) were utilized since they enable user-level applications to communicate seamlessly with each other. It is not practical to download the data manually from Canvas. Nonetheless, a faster and more efficient way is to automatically export data to the micro-credentialing application by using APIs. By sending an API request, it can be possible to automatically obtain the required information as well as alter data on Canvas [6].

The Micro-Credentialing Framework for Canvas requires data extraction. The Python programming language was utilized since this language comprises of libraries that makes it easier to extract data and perform machine learning algorithms. First, the Script extracts the values of specific data fields residing within the Canvas Quiz data export format. Canvas has several APIs that help collect the information without downloading the data manually. Initially, we prototyped using free for teacher canvas website known as Canvas for Teachers. A sandbox course was created to start the implementation of the micro-credential software. 

As shown in Figure 3.1, upon the receipt of API request, a JSON data file is generated by the Canvas LMS using the API call. This API retrieves information about the questions for that specific quiz, including the unique ID for each quiz. This is a vital step since it is required in a subsequent step to drill down to each question text string and response text string. Parsing of the JSON data is conducted since each micro-credential requires only a few specific fields from the entire dataset. The JSON data that is received is being converted to a DataFrame via Panda’s DataFrame library in Python. The DataFrame allows us to store data in a table-like structure that comprise of rows and columns for easy classification. The data that comprised of IDs and names of the students were in a list format. Thus, the Python script acquires the data while performing some field remapping to generate a CSV file each time the script is invoked.

Figure 3.1. Flowchart of micro-credentialing script execution created in this project as a transportable framework for Canvas LMS.

 Later, capabilities will be added so that students can link their Micro-Credentials to their LinkedIn pages, which can advance their internship/hiring connections with potential employers.

3.0 Status and Conclusion

Micro-Credentialling is intertwined with BC-ECO segment such as remediation, where students may consult with tutors to discuss their assessments. Each part of the BC-ECO aims to help garner student learning, where the effects will be seen in the badges received by the students. The current phase of research in the project includes constructing a Python Dictionary data structure that will be used evaluate student skills to issue micro-credentials within each course. The Python programming language scripts developed are available upon request. 

4.0 Acknowledgments 

This research was supported by NSF HRD/EHR-REDACTED.  Authors would like to acknowledge support of REDACTED, REDACTED, and REDACTED contributing to its delivery.

5.0 References

1. B. Fishman, S. Teasley, and S. Cederquist, "Micro-Credentials as Evidence of College Readiness: Report of an NSF Workshop," 2018.
2. D. K. Mah, N. Bellin-Mularski, and D. Ifenthaler, Moving forward with digital badges in education, in Foundation of Digital Badges and Micro-Credentials. Springer, 2016, pp. 511-517.
3. D. Gibson, K. Coleman, and L. Irving, Learning journeys in higher education: Designing digital pathways badges for learning, motivation and assessment, in Foundation of Digital Badges and Micro-Credentials. 2016, Springer. pp. 115-138.
4. V. Hung, A. Gonzalez and R. DeMara, "Towards a Context-Based Dialog Management Layer for Expert Systems," 2009 International Conference on Information, Process, and Knowledge Management, 2009, pp. 60-65, doi: 10.1109/eKNOW.2009.10.
5. National Science Board. (2010). “Preparing the next generation of STEM innovators: Identifying and developing our nation’s human capital,” NSF. [Online]. Available: https://www.nsf.gov/nsb/publications/2010/nsb1033.pdf[Accessed 23-Feb- 2018]
6. Canvas LMS - REST API and extensions documentation. Canvas LMS REST API Documentation. [Online]. Available: https://mitt.uib.no/doc/api/all_resources.html [Accessed 28-Jan-2022].