Scalable Formative Assessments in Higher Ed

Historically, there has been a lot of research and application around the concept of formative assessment anchored in the K-12 experience.  Over the recent decades, there has been a greater focus on the evolving higher education industry.  More specifically, how does one go about designing and implementing effective and valuable formative assessments when dealing with variables that include being online, facilitating asynchronous interaction, supporting working adult learners, and how to do this at scale? This was the exact intersection in which we were looking to the literature for insight and guiding inspiration.

As part of building on our direct assessment model, FlexPath, we wanted to integrate a more formalized and scalable approach to formative assessment.  In order to do so, we knew we needed more information, so we formed a working team representing various roles in course development and instruction areas. We broke the concept of formative assessment into a set of related “building-block” terms (such as: practice, authentic context, automated feedback, scaffolding activities, etc), and leveraging each other’s strengths, interests, and expertise, we in essence crowdsourced a literature review and pragmatically distilled recommendations from dozens of academic papers. 

So, what did we find out and how does one make use of it?

Defining Formative Assessment

First, we needed to define exactly what formative assessment was in comparison to summative assessment.  We settled on this simple distinction:

Formative assessments provide instructional opportunities for learners to practice course competencies in a low-risk environment.

Summative assessments assess learners’ achievement of course competencies.

Interestingly, through the process of testing this definition with faculty and course development staff, we quickly realized that the term “assessment” after formative made them immediately equate it with a graded assignment, which by definition is a summative assessment.  In reaction to this common confusion, we began to refer to formative assessments as formative activities, thereby inherently bringing to mind a more accurate picture of what a formative assessment is.  So, for the remainder we will refer to formative assessments as formative activities.

Key Principles of Formative Activities (citations available by request)

By solidifying that first stepping stone, we were able to synthesize a set of key principles for formative activities, divided into two sections:

Why use Formative Activities?

• Metacognitive awareness/ Self-Assessment/Self-regulated learning – Learners can monitor and evaluate the quality of their thinking and behavior when learning and adjust what they are doing. For example, by engaging in a formative activity, learners can identify key misconceptions about a topic area and adjust their thinking based on the challenges posed by the formative activity. Building of metacognitive skills helps build life-long learning skills for adult learners to apply across their career.
• Self-efficacy - “Wins” in formative activities help motivate learners, which can drive persistence and retention. Through these “wins,” and even through “failures” and any subsequent learner self-regulated actions, learners can more accurately assess their current performance and improve their future performance, driving increases in learner investment and motivation.
• Engagement/Motivation – Low risk “failures” can motivate learners to revise their beliefs and strategies to further their learning and successes. These “failures,” tied with substantive feedback, can result in the learners’ perception of their own ability as incremental rather than fixed, and shift their learning orientation towards one of mastery rather than of performance. The learners’ act of self-monitoring or self-assessment through these activities produces a feeling of autonomy, greater meaning of the learning taking place, and results in greater confidence, motivation, and engagement.

What are Key Features of Effective Formative Activities?

• Aligned to relevant learning goals related to future success - Learning goals should be visible to learners, as well as their connection to their summative assessment and real-world contexts. When learners affectively identify with their goals, both near-term learning goals and far-term professional or personal goals, learners find greater meaning in their effort and are more motivated and engaged in achieving their goals.
• Variety, including authentic contexts - Authentic contexts, immersive environments, and games allow learners a low-risk opportunity to practice skills and provide more meaningful applications of critical thinking, problem-solving, and inquiry. Real world contexts are more engaging, which in turn promotes self-regulated learning. Written and open-format questions help learners engage in higher levels of thinking and promote deeper learning. However, simpler applications or knowledge checks may be appropriate to break down more complex content areas or tasks to allow learners to focus their attention on key sub-skills or knowledge content areas.
•  Timely, specific feedback – Feedback should include what good performance is (provide clear targets and criteria), how the learner’s performance compared to good performance, and how to close the gap between current and good performance. Feedback can be direct feedback, but it can also include indirect feedback, such as leading questions, hints, metacognitive suggestions to help learners reflect on their knowledge, and references that help learners self-correct and engage in reflective inquiry. The formative activity may, for example, provide a way to keep notes, reflections, or plans for using the information to adjust what they are currently doing so they can make continuous improvement. Feedback helps learners know where to focus their attention for continuous improvement.
• Appropriate level of rigor – One of the most valued characteristics for learners when presented with instructional problems is having the problem be targeted at an appropriate level of difficulty. A novice learner exhibits significant struggles when they do not possess the knowledge or the problem-solving skills needed to respond to the prompt/question. If that struggle is too great, the learner will not persist. Instead, the principles of Vygotsky play a key role in constructivist learning theory, applying to the designer’s role in targeting the level of difficulty in problems, including the zone of proximal development and scaffolding.
• Belief in the value of formative activities by the institutional culture and learners – Both learners and faculty need to perceive the value of formative activities and believe that they are worth the time for them to be successful. Data from formative activities can help inform future revisions to all learning activities as well as summative assessment in a course – formative activities can help for continuous improvement of the institution, as well as the learner.

Guidelines and Components of Formative Activities

In a parallel work stream, based on  we developed a set of guidelines for the development of formative activities , sort of a “how to” for instructional designers and SMEs working on building FlexPath courses incorporating formative activities.  This guidelines document is proprietary, so we are unable to freely share the details of our process.  But we want to provide you with some of the basics that we found to be highly effective in building a practical knowledge of how to go about conceptualizing formative activities.

Formative activities can be broken down into three components:

Contextual Information

When writing a formative activity, the first step is choosing a strategy for presenting information. When appropriate, try to assess the learner on higher-level Bloom’s Taxonomy skills—like analysis, evaluation, or synthesis of the material. When possible, consider using a specific case study or scenario.  Asking learners to address a specific situation—as opposed to addressing an issue in a more generic way—can help make your assessment more engaging and authentic.

Learner Action

All formative activities require some form of interaction from learners, in order to provide meaningful feedback. This can be a single action, or it could be a sequence of different actions.

Instructional Feedback

In order to guide the learning, we must provide some sort of educational reaction to what learners have done.  This is the reward for the effort the learner has given—make sure this reward matches the level of effort.

In order to implement this approach on the large scale we require, our instructional feedback is written by our instructors during the activity planning and creation phase in more of a generic and objective fashion revealing the expert’s point-of-view on the topic. This feedback is then delivered in an automated way to the learner after they complete the learning interaction.  This results in a learning opportunity without any additional effort by the teaching faculty. We have also since realized that the formative concepts that apply to strengthen FlexPath can easily apply to strengthen our traditional course model, GuidedPath. 

Presenting

We know that the best presentations are ones that break down the items into simple digestible pieces and allow the chance to play with the concepts. So, beyond just showing them the work we’ve done, we’d like to incorporate an activity where they can identify a topic that they might be currently teaching on or interested in and step them through the planning of a formative activity that they could take back with them, giving them practice in this creation process—in essence, a formative activity on creating a formative activity.