Educational Data-driven Dashboards as Building Bridges between Data and People in Online Learning Environments

People driving a motor vehicle constantly take decisions during the course of their driving, e.g., when to press or leave the gas pedal, when to put gas in the car or when to fill air in the tires, whether they should stop the car immediately, etc. Using the car’s dashboard allows taking these decisions in an effective, timely manner. The car’s dashboard visualizes information that is constantly collected under-the-hood, analyzed and being displayed in a friendly manner. In a sense, teachers are like drivers.

Data-driven decision-making refers to collecting, understanding and analyzing educational data, and is considered as an integral part of teachers' and instructors' professional conduct (we will use the term “teachers” to refer to both teachers and instructors). Using data for educational decision-making is not something new, as teachers have been using grades, students’ work, and behavioral data since the school framework started. Overall, teachers use such data to evaluate their class and individual students, to reflect upon their own teaching, and also to communicate with various stakeholders, such as their students, their students' parents, or the school's educational and management teams.

However, in online learning environments--the most prominent example being MOOCs--many of the learners’ actions and behavior are hidden or hard to track. The former includes, for example, navigating through the course pages, downloading course materials, multiple attempts to solve a problem, affective state during learning, or cheating; the latter is most strikingly exemplified by long, exhaustive threads on discussion boards. Without being aware of such actions and behavior, instructors’ decision-making is based on only a thin layer of easily-observed students’ course activity, mostly submitted assignments that are scored either automatically or manually. For an instructor, to refer to students’ actions and behavior that are hidden from eyesight is a big challenge. Luckily, there is a way to address this challenge. This is where educational dashboards come into the picture.

To put it simply, an educational dashboard is a display which presents educational stakeholders with (usually visual) data-driven information regarding teaching and learning processes in a way that will induce them to reflect upon their behavior patterns and to act accordingly. Educational dashboards could contain various types of information, such as an overview of the course activity, time per tasks, students’ skills, misconceptions, test results, social interaction, and student’s current and historical state in the course. In an era where “big data” is the hottest trend, educational dashboards serve as a bridge between the data and the people, and allow the latter to use the former in an actionable way. In the context of online courses, dashboards would present information that is originated in the learning system’s log files. Many online learning systems log each of the students’ actions in at least three dimensions: Who took the action? What is the action? When was is taken? (That’s the WWW of log files.) Analyzing such data is part of the emerging field of Learning Analytics, and it allows researchers to explore student behaviors that may be otherwise kept unobserved.

Many studies have shown that educational dashboards can be used as a decision-making tool that supports teachers in planning curricula, evaluating the class’ level, and tracking individual students. Furthermore, it was shown that the use of dashboards has led teachers to better tailor their teaching (content and style) to their students' needs, to collaborate more effectively with their colleagues, and to reflect upon their own professional conduct and abilities.

However, it is important to note that the ways information is being presented in educational dashboard is critical to its use. The type of data presented, the way it is visualized, the overall layout, the dashboard interactivity - these are just a few factors that may affect teachers use of the dashboard in an effective way.

In this session, we will design the “perfect dashboard”. After a short introduction about educational dashboards (10 min.), participants will be divided into small groups (~5 in each). Each group will be prompted with a certain learning scenario (e.g., a fully online MOOC, or a LMS-supported blended undergraduate course), each of which will also simply explain the types of data collected through it. Each group will be given with a few questions referring to this scenario; the questions will lead them through the process of designing the “perfect dashboard” for this scenario. Sample questions:

1. What is the information that the teacher will most probably would like to know for improving teaching and learning in this scenario?

2. Can this information be extracted from the existing data? If you answered negatively, which other sources of data would you need?

3. How would this information be best presented to the teacher?

During a 20-min working session, the groups will discuss these questions, and will try to prototype a dashboard, using a template that will be given to them. The groups’ prototypes will be uploaded to a shared Google Slide presentations.

Finally, during a 15-min summary session, we will watch the groups’ products and will discuss their similarities and differences between them.

We believe that participants in this session will leave it with a strong understanding of the need of dashboard in various educational settings. They will understand how dashboards get data, and how they should present it in a way that will allow teachers to take data-driven decisions and to take actions that will eventually promote teaching and learning.