Applying best practices in the design of formative assessment for blending learning environments

Description and Background

This workshop is based on assessment best practices and informed by prior research of a blended, international academic and professional program, its online assessment methods, and their effects on learning. That previous study had a viable data set of 25,000 students analyzed to determine the effect of formative assessment on summative assessment. The assessments included practice quizzes, chapter tests and summative course tests. The practice quizzes and chapter tests, because of their low stakes, high feedback and ability for retakes from a common pool of items, functioned as formative assessments to the summative chapter tests.

 

Overall, the findings showed that the positive benefits of testing effect were not evident in the data provided and with the analyses possible (and with significant limitations noted above). The highly (and negatively) skewed means for chapter tests hint at mastery of content yet the three curricula show lower summative assessments means. Since the practice quizzes and chapter test items were from the same item pool designed—with appropriate and documented rigor–for the summative, end-of-course assessment, the data show that longer term (until the end of the course) gains in retention of the material did not occur for most students. This did not mean that students did not learn the material, for their means were in the “A” and “B” letter grade ranges.  It meant that the formative strategy did not explain sufficiently why they learned the material or whether they learned the material in an enduring way.

 

The literature is clear on assessment best practices and that study provides a cautionary tale of implementing a less-effective strategy on scale, which, in online environments, is all too common occurrence.  

 

The negative outcome of the testing effect is easily avoidable and this workshop addresses such remedies.  The research literature and that prior study from the presenting team shows that specific assessment strategies in online environments can increased student performance on summative assessments in blended learning environments. Each of these best practices will be explained in detail for the general application in blended learning environments. After establishing an understanding of these practices in a general sense the workshop will show how to apply the practices specifically for a number of the most popular learning management systems. This will include walkthroughs of each system applying the best practices with their unique setups and tools.

Applying the Best Practices in the Classroom

Participants will be able to utilize the information given to them in this workshop to incorporate the assessment best practices into their blended learning courses. They will be shown the steps necessary to enable student performance as well as the reasoning behind the practices so they will be able to fully understand the concepts and why they are applied to blended learning environments.

 

Participants will learn how to harness assessment best practices to blended learning. This workshop will be focused on coordinating formative, performance, and summative assessment strategies in blended learning. It will also address the practices and configurations for multiple learning management systems in order to achieve these goals.

Blended Learning

The primary focus of this workshop is blended learning environments and the best practices for using formative assessments to increase performance on summative assessments. Those interested in blended learning will benefit by gaining insight on the effective and ineffective applications of using practice quizzes and chapter tests as formative assessments in a blended learning environment.

Goals and Objectives

The overall goal is to improve the online assessment practices of blended learning programs.  The following learning objectives guide the workshop:

1. Participants will learn the best practices for using online quizzes and tests as a formative assessment strategy.

2. Participants will gain practice with designing and configuring online assessments within common Learning Management Systems (LMS).

3. Participants will use LMS analytics tools to monitor individual and aggregate outcomes to confirm if assessment best practices are impacting student learning.

Target Audience

Administrators, Design Thinkers, Faculty, and Instructional Support

  Collaboration and Interactivity

The workshop structure will be as follows:

 

Assessment Strategies and Best Practices:

• Brief description of formative and summative assessment

• Discussion of testing effect potentials and pitfalls

• Discussion of unique opportunities and threats to assessment as a result of online testing and scale

• Introduce key variables that affect performance assessment:

  • Lag time between instruction and test

  • Item banks and content validity in pre and post tests

  • Leveraging content reading and assignments to testing opportunities

  • Determine the value of the practice tests

  • Inhibit last-minute test score spiking

 

LMS Configuration and Practices:

Four LMS platforms will be demonstrated: Merlot, Blackboard, Moodle, and Canvas.  Across all platforms (and conceptually any LMS), the following topics will be addressed.  

• Show how to adjust lag time and date

• Provide direct pointers to the reading

• Enforced reading of content before assessments

• Adjust grading percentages

• Show how to adjust test time

• Show how to adjust test frequency

Tangible Deliverables

The participants in this workshop will learn the assessment best practices for blended learning environments and how to specifically apply them using multiple learning management systems. The participants will be receive handouts and access to online materials that tuned for each learning management system and how the assessment best practices are applied within them.