Guess Again: How to Get Better at Estimating Online Course Development ID Capacity and Workload

**This session was originally presented as an education session at OLC Accelerate 2021. The session was highly attended with a lot of follow ups from attendees around details that weren’t able to be covered in the 45 minute presentation. As a result, I am proposing this as a workshop that would allow a more comprehensive look into the process, more opportunity to teach attendees how to handle the data collected with this method, and more opportunity for participants to collaborate together to solve the sticky online course development challenge of how many courses an ID can develop and how  long it takes. 

It is scientifically proven that humans are terrible at estimating the time required to complete a task. As a species, we continually disregard how long it’s taken us to complete tasks in the past and what complications may arise...and we consistently believe that things will be better next time (Kahneman, 2011). 

In a course development process, it is vital to understand the constraints and details that impact how quickly a course can be developed, how open the subject matter expert (SME) is to change, and what sort of factors require additional staffing such as accessibility and media development. Answers to these questions are vital to ensuring appropriate staffing of course development teams, accurate time estimates to completion for stakeholders, and clear communication with faculty SMEs around expectations. 

One common practice when attempting to answer these questions is to refer to the research on how long an hour of eLearning or online, interactive training takes to develop. However, as instructional designers know, building an online course in a Learning Management System is not the same as building an interactive eLearning activity. 

There is little information available about realistic estimates for online course development that take into account common constraints. The result is that every instructional designer (ID) provides their own best guess at capacity and load for projects, relying on their own competencies, working speed, and intuition. While intuition from experienced IDs is mostly correct, intuition only works as an estimation tool when project situations fall within the realm of an ID’s experience. This leads to inaccurate time estimations for course projects with unique elements IDs have not yet encountered.

Some of the questions that are asked about course development projects are: 

• How much ID time will a course project take? 

• How much faculty SME time will be required? 

• Will a course development project be “difficult” or “easy” and what does that mean?

• How much time will accessibility take? 

 Using the answers to these questions, we can begin to answer: 

• How to balance course assignments across a team of IDs 

• How to balance course assignments across cohorts/semesters

• How to account for IDs’ individual working styles 

This workshop will include a walkthrough of a process for answering these questions and findings used within an actual online course development process with a team of 5 instructional designers over 4 cohorts and 70 course projects. 

By the end of the workshop, attendees will be able to:

• Explain the role of an instructional designer’s intuition in an online course development project management process

• Identify biases, blind spots, and surprising findings in regards to questions about capacity and course load

• Use a rubric to score course development projects for “easiness” or “difficulty” in regards to time expectations

• Practice identifying relationships with a sample data set

• Identify strategies to gain buy-in around time tracking

• Identify next steps to implement a capacity tracking project at their own institution

This workshop will include numerous interactive games, small group work, and hands-on practice. 

1. Polling software will be used to help demonstrate how difficult it is to overcome intuitive thinking, even when we know that intuitive thinking leads to inaccurate decision making. 

2. Attendees will be asked to score a sample course using a provided rubric and discuss their findings in small groups. 

3. The audience will be asked to share their own experiences around time management in instructional design course development projects after small group work with question prompts.

4. After using the course scoring rubric, attendees will get the chance to practice data analysis of a sample data set to identify relationships and trends.  

5. In small groups, attendees will identify issues with time-tracking and identify strategies for gaining buy-in from instructional designers for time tracking.

6. Discuss and sign-up for a research collaborative for participants to implement strategy at their own institutions and present at a future OLC event.

Attendees will receive a copy of the presentation, a course scoring rubric, a sample dataset to practice the method, and the step-by-step process to start to answer the questions around estimating course development time at their own institutions, as well as the opportunity to join a larger research collaborative around the topic for continued work and presentations. 

References: 

Kahneman, D. (2011). Thinking, fast and slow. Farrar, Straus and Giroux.