In this interactive workshop, participants will discuss how various educational stakeholders may benefit from using learning analytics methods. We will show and practice what you can do, using real data, drawn from an online learning system, but worry not - there is no need for prior data crunching experience.
Many online learning systems keep detailed log files that document learners' actions within the system. These actions are being continuously and automatically logged in at least three dimensions: Who took the action? What is the action? When was is taken? (That’s the WWW of log files.) The analysis of these data—which is a part of the emerging field of Learning Analytics—allows researchers to explore student behaviors that may otherwise be hidden from eyesight, and therefore may promote our understanding of the overall learning progress.
There are many different ways to analyze a given log-based data set, depending on the questions we have in mind. Some questions that may be answered using this data are: Which are the most active times of the day in this system? What are the best predictors for success in the final task in this learning experience? How could the learners be partitioned into groups in a way that will allow an effective teacher’s feedback? What are the common learners’ mistakes? etc.
This workshop will allow participants at all levels to think about the ways in which data—specifically, data drawn from log files of online learning systems—may be of use for educational stakeholders. Indeed, we will use authentic data from an online platform that teaches basic algorithmic thinking for elementary- and middle-school children. But fear not—there is no need for prior experience with algorithms nor with data analysis.
The workshop will be comprised of the following sections:
Introduction to Learning Analytics (10 minutes) - This section will introduce the audience to the fascinating world of Learning Analytics, focusing on what could be learned from analyses of online learning systems’ log files. This section will provide the participants with the key terms and a framework that will enable us all to communicate in a common language.
Introduction to the online learning system and its log files (30 minutes) - In this section, the audience will gain a deep understanding of the way in which learners’ actions are logged. We will first demonstrate how learners’ actions in the system are logged in real-time.Then, working in groups, participants will be given a username to the learning system, and explore—using their personal devices—how their own actions in the system are reflected in the log files. During this time, participants will get to know the system better.
Experiential Group Exercise (20 minutes) - Participants will be divided into groups (~5 people in each group). Groups will have 20 minutes of guided exploration, using cards that will be prepared for each group. Throughout the exercise, participants will be asked to think about the types of questions they could “ask the data”. They will be guided to connect their questions with as many terms/topics from the framework presented earlier. Participants will be encouraged to think of ways to answer these questions; the questions (and possible ways of answering them) the groups will come up with will serve them in the final (gamified) section of the workshop. No need for prior experience in data crunching is needed, but those of the participants who are capable of doing so, would be encouraged to try to find the answers to their questions. (Files will be in CSV format, which will allow editing them in popular software like Excel or Google Sheets.)
Ideas Discussion (30 minutes) - This section will be gamified, using Bingo as the driving engine. Each group will get a Bingo card with different key terms and topics from the field of Learning Analytics (associated with the framework of Learning Analytics that has been set up at the beginning of the session). Each time, we will randomly announce a term/topic, and groups who could identify it in their suggested questions would mark it on their Bingo card; some of the groups’ representatives will explain to the whole audience how this term/topic is relevant to their question, and we will discuss it. The group that will first have a full row/column/diagonal marked - will be announced as the winner. We hope this game will increase engagement until the end of the session, and will encourage groups to think of as many questions as possible. During this discussion, we will raise issues of validity and privacy. Finally, if time permits, we will present findings from previous studies which are based on these data.
Participants of this workshop will have a clear sense of how data that documents learners’ actions is stored in online learning systems, and how it can be used in practice by education researchers. Participants will become familiar with the key terms of the emerging field of Learning Analytics and will get to know its advantages and limitations. Overall, participants will gain a comprehensive view of using such data for research.