Defining and Measuring Personalized Learning in Higher Education

The Harvard Graduate School of Education (HGSE) is engaged in an ongoing initiative to develop How People Learn, a foundational yet personalized online course for all incoming Master’s students. The course leverages innovative design and technology solutions to provide a learning experience that is meaningful and relevant for each individual student. Drawing on research methods and findings from the HPL initial course run as an example, this session will empower creators of learning experiences to build a framework to define personalization for their context and measure the impact of a personalized learning experience on student learning and growth.

HPL is intended as a flexible experience to help students build a foundational understanding of how people develop over the lifespan; how people learn; and how educators successfully design, lead, and create systems to support others’ learning and growth. To succeed, the course will have to serve as a foundation for master's students with widely divergent backgrounds, identities, and professional aspirations. If the course is to faithfully enact the same principles it teaches, it was important to design something that was not “one-size-fits-all. Moreover, it should surface and build on learners’ prior knowledge, values, and goals  ("The Design of Learning Environments," 2000).

So how do we understand and account for the complexity of backgrounds and interests our students bring with them into the course? Many designers of learning experiences are charged with a similar challenge of meeting the needs of a diverse body of learners. While personalization is widely discussed as a strategy to achieve these aims, particularly in the K-12 space, it is less frequently dissected at the level of design, implementation, and outcomes.

There are many possible interpretations of personalized learning; on one end of a spectrum of intervention, personalized learning can mean differentiated instruction using multiple means of representation. On the other end of the spectrum, personalized learning can lead to customized learning pathways that are unique to the learner based on empirical and self-reported data. Drawing from the development process of HPL as an example, this workshop will support participants in developing their own definition of personalized learning aligned with their particular project goals and institutional values.

We will also examine how to use this customized personalization framework to develop a theory of change to guide implementation and evaluation of a personalized learning project. Participants will break the definition of personalized learning into tangible criteria that are relevant for their own individual contexts and learners, and use these criteria to develop a strategy for determining the outcomes of their learning experience.

This session will empower those who design, implement, or assess learning experiences to develop a model to evaluate their work. It is especially, but not exclusively, relevant to audience members who work in higher education.

At the end of this session you will be able to:

1. Reflect on the affordances of personalized learning for your goals and context.

2. Recognize how a personalized learning framework can be used to inform the design of a learning experience.

3. Generate principles of personalization that are relevant to your context.

4. Develop criteria to measure how well a project aligns to your personalization framework.

5. Sketch the outline of a theory of change to guide implementation and evaluation of your personalized learning project.

Session Outline:

Activity (5 min)

As they join the workshop, participants will submit words and phrases in response to the question: “How do you describe personalized learning?” and generate a word cloud via PollEverywhere.

Introduction (10 min):  We will introduce How People Learn and our process of developing a personalization framework that aligns with project goals and institutional values at the Harvard Graduate School of Education. We will share challenges we encountered in developing our own personalization framework for HPL, and learn how our framework has evolved over the course of the project.

Activity (20 min)

Think: As individuals, reflect on a learning experience you are responsible for (re)designing. Probing questions:

• What are the goals for the learners in this experience: the knowledge, skills and attitudes you want learners to obtain from the learning experience?

• Generate some key words and phrases that express the values of the experience and of your institution.

• What are your constraints? (Think about time, technology, production capacity, etc.) 

Pair: Using these thoughts, discuss with a partner how personalization might advance the goals and values of your project within the constraints of your project. Which of the words/ideas surfaced in the word cloud offer promising ways of addressing the goals of the learning experience? Based on these thoughts, what aspects of personalization are relevant to your project? Share via the Padlet.

Share: As a whole group, we will take a look at what we have generated and discuss themes and outliers we find.

Outcome: Reflect on the affordances of personalized learning for your goals and context; outline principles of personalization that are relevant to your context.

Presentation (15 min)

We will take a look at the personalization framework we developed for HPL and share how it aligns with the goals and context of this project. We’ll show how the personalization framework was used to guide design and technology decisions. We will share the rubric we created to evaluate the alignment of an HPL pilot course with our personalization framework, take a look at the results of this evaluation, and discuss how it informed course redesign.

Activity (20 min)

Think: Brainstorm specific criteria by which to measure alignment of your chosen learning experience with the personalization framework you have started.

Pair: In pairs, share the initial personalization rubric you have developed and discuss how it aligns with your project goals and context. Using a tuning protocol, ask probing questions to push your partner’s thinking deeper.

Share: In a whole a group, share examples of how your framework developed based on your conversation with your partner.

Outcome: Recognize how a personalized learning framework can be used to inform the design of a learning experience; develop criteria to measure how well a project aligns to your personalization framework. ​

Presentation (10 min)

We will take a look at how the personalization framework and rubric fit into a larger theory of change for a personalized learning project. We will share how we are using this model to measure the impact of HPL on students’ learning and development, and take a look at results from our initial pilot of the course.

Activity (5 min)

Plan: Take 2-3 minutes to start to populate a theory of change flowchart with initial ideas. Write a next step you will take in order to outline a theory of change for your personalized learning project, and identify one thing you are still struggling with. Submit these answers to the session Qualtrics survey. (Participants who opt-in will receive a nudge of encouragement after the workshop is complete.)

Outcome: Sketch the outline of a theory of change for your personalized learning project.

Activity (5 min)

Wrap up: As a large group, share insights and questions.

 

Materials:

For participants: pens; worksheets, a device to access the Padlet and Qualtrics.

References:

The Annie E. Casey Foundation. 2004. Theory of Change: A Practical Tool For Action, Results and Learning. Baltimore, MD: Organizational Research Services (ORS). Retrieved from https://www.aecf.org/m/resourcedoc/aecf-theoryofchange-2004.pdf

Patrick, Susan, Kennedy, Kathryn, & Powell, Allison. (2013). International Association for K-12 Online Learning, 12 Online Learning, 2013. Retrieved from https://www.inacol.org/wp-content/uploads/2015/02/mean-what-you-say-1.pdf

Jill Abbott, J. (2015). Technology-enabled Personalized Learning: Findings and Recommendations to Accelerate Implementation. British Columbia Teachers' Federation. Retrieved from http://deslibris.ca/ID/10061510

"6 The Design of Learning Environments." National Research Council. 2000. How People Learn: Brain, Mind, Experience, and School: Expanded Edition. Washington, DC: The National Academies Press. doi: 10.17226/9853.