Participants will learn how the University of Missouri – Kansas City evaluated the use of LMS data to determine the effectiveness of the training model for faculty teaching online. Can analytics assist in determining if faculty apply concepts and “practice what we preach” in the online classroom?
In the fall of 2015, the University of Missouri-Kansas City implemented an institutional policy requiring all faculty who teach online courses to complete training to become certified by the fall semester of 2018. There are currently three instructional modes for the faculty certification training including 1) face-to-face week-long boot camp, 2) three week online instructor-led course, or 3) self-paced. All instructional modes of the course have the same learning objectives and assessments. Key topics covered in the certification training include Quality Matters review of online courses, copyright, accessibility, assessments, basic knowledge of the LMS and related tools, active learning, and faculty and student interaction in the course. As a result of the implementation of this policy, 367 faculty have been certified to date.
Faculty certification data is stored in an eLearning Tracker database. This database was recently integrated into the Blackboard Analytics instance on campus for evaluation purposes. The certification course is currently evaluated using an optional survey that is sent to faculty at the conclusion of the course; however, the survey focuses on the quality of the instruction and not necessarily the impact or effectiveness on faculty teaching practices. Using the integration of the eLearning Tracker with Blackboard Analytics, could LMS data be used to determine if certified faculty “practice what we preach” and apply concepts taught during the certification training in the online classroom? Of interest is to investigate if analytics data from the LMS can assist in determining the effectiveness of the certification training in improving teaching behaviors in the online classroom. Measures of teaching behaviors in online courses include: course access minutes, interactions, course accesses, course item accesses, and content accesses.
Key Questions Investigated Include:
- Is there a difference in behavior in online courses between faculty who are certified in online teaching as compared to those faculty not certified?
- Do certified faculty spend more time in online courses as compared to faculty not yet certified?
- Do certified faculty interact more in online courses as compared to faculty not yet certified?
- Is there a difference in teaching behavior in online courses among faculty certified via each of the three instructional modes?
Current analysis is underway to evaluate these key questions. The results of this study will be shared with participants, and an interactive discussion will allow participants to discuss and share how LMS data might be used to assess faculty professional development in the online environment.