Examining the impact of changing teaching models in a blended graduate teacher preparation program

Changes that often occur at an academic institution will undoubtedly impact the courses within an academic program.  For example, an event such as hiring new full-time or adjunct faculty could lead to changes in teaching loads or perhaps alter the teacher-student ratio across an academic program.  Changes in instructor staffing priorities, faculty development opportunities, or learning management systems can also lead to changes in course development in online programs. These changes that are commonplace within an academic institution present a unique challenge to maintaining a degree of consistency within a blended program.  They also, however, present an opportunity to better understand the impact that the change can have on the student experience within such a program. 

This current study is part of an on-going program evaluation, the goal of which is to provide a framework for continuous improvement of the blended program.  As such, it is important to understand the impact of different variations in the program courses on student experience and engagement.  In this evaluation study, the Community of Inquiry model (Garrison, Anderson, & Archer, 2000)  is used as the framework to evaluate the impact of aspects of a different course teaching models on the measures of Social, Cognitive, and Teaching Presence.  

The program on which this evaluation focuses is a blended graduate teacher preparation program at a mid-sized public university in the midwestern United States.  Most of the 80-100 students in this program are enrolled part-time and typically complete 2 courses per term over a 1-2 year period.  The program includes three program tracks including a 1 year teacher licensure program or either a 2 or 2.5  year teacher licensure and Master of Education degree program.  This program is primarily cohort-based, thus students enroll in the same sequence of courses as others who started the program at the same time.   The courses in this program feature 3 on-campus class sessions that are held at the beginning, middle, and near the end of the academic term.  In addition to the class sessions, there is regular online course work during a 8- or 10-week academic term.  

During a two-year evaluation period that will conclude in July, 2017, data have been collected that includes the Community of Inquiry survey (Arbaugh et al, 2008)  as well as data regarding various course characteristics such as teacher/student ratio, frequency of online activity by students and instructors, number and type of course assignments, and course grades.  This data will be used to address the following research questions:

1. In what ways does the teaching model (e.g. team teaching vs single instructor) impact ratings of Social, Cognitive, and Teaching Presence?

2. To what degree do ratings of Social, Cognitive, and Teaching Presence vary according to changes in the instructor teaching assignments?

3. To what extent is there a relationship among Community of Inquiry measures, teacher/student ratio, and frequency of online activity within a course?

Data sources

Data collected were collected using two primary sources: (1) Community of Inquiry survey, (2) course activity records from the learning management system.

The Community of Inquiry survey was administered via Qualtrics Survey Software.  An invitation to complete a program survey was sent to all students enrolled in courses in the academic program at approximately the middle of the academic term.  The survey included a separate 34-item Community of Inquiry survey for each course in which they were enrolled, thus each response could represent ratings of up to 3 courses.  For Fall, Winter, and Spring academic terms, this occurred during the week following the 2nd on-campus class session.  Summer courses in the program include a single on-campus class session at the beginning of the academic term, thus the survey was administered from the 4th-5th week of the summer courses. The survey was available during a period of 7 days and follow-up requests were sent every two days to students who had not completed the survey. During the 2-year time period, response rates (to date) ranged  from 59.5% to 72.5% and yielded 614 responses to the survey, of which 583 resulted in completed surveys.

An additional source of data for analysis was the Learning Management System (LMS).  This data source included records of student activity within the learning management system.  These records of activity in the online courses were gathered using a procedure of downloading individual page-view records for each user within a course.  Using the individual pageview records, a summary of course activity was generated for each student in the course that indicated the frequency of pageviews, pageviews with participation, pageviews in discussion forums, and pageviews with participation in discussion forums.

Data analysis

Data from the Community of Inquiry survey were analysed using SPSS 24.  Data from the Learning Management system were downloaded using a custom-design processed using Python to access data available from the Application Programming Interface (API) provided by the LMS.  Student course activity records were downloaded into a MySQL database.  A frequency summary was then generated for each student in each course using Tableau 10.3 software.  This summary was then imported into SPSS 24 in order to be matched and analyzed with the Community of Inquiry survey data.

Participants

Participation in the study was solicited from the students who were enrolled in program courses.  Total course enrollment varied from a low of 75 students (Winter 2016-17) to a high of 96 students (Summer 2015-16).  All participants were notified of the research student and invited to participate via an email invitation.  The link that was provided by email directed the respondent to the Letter of Informed consent page of the survey in the Qualtrics Survey system.  Participation was voluntary and no incentive was provided.  Data were collected confidentially, including a unique numerical code that facilitated the matching of survey responses with LMS records.  The data collection procedure was reviewed and approved by the appropriate Institutional Review Board.

Results

At the time of this conference proposal submission, data collection remains in-progress and will conclude in July, 2017.  The data that are currently available are limited to the Community of Inquiry survey for courses that have concluded.  Upon conclusion of courses during the Summer, 2017, additional data will be collected and analyzed that includes Community of Inquiry survey results from one additional term.  Because of the technical procedure required to download and analyze the LMC course activity records, those data will not be fully available until the end of the study period in July, 2017. The results presented below are preliminary and will be revised and expanded after data collection is completed.  

Team Teaching vs. Single Instructor

In order to determine the degree to which the teaching model (e.g. team teaching vs single instructor) impact ratings of Social Presence (SP), Cognitive Presence (SP), and Teaching Presence (TP), data were selected from courses that were taught more than once within the study period, but in which there was variation in the teaching model.  There were 3 courses that met this criteria, two of which were taught twice within the study period and one course for which the second term is currently in-progress (concluding in Jul7, 2017).  A preliminary analysis revealed that there was a significant increase in ratings for one course in favor of the team teaching model, whereas there was no difference found in a second course.

Same Course, Different Instructors

A second topic of interest to this study is the degree to which ratings of Social, Cognitive, and Teaching Presence vary according to changes in instructors.   For this analysis, data were selected in which the same course was offered during multiple academic terms, but a different instructor was assigned to lead the course in at least one term.  There were two courses the met this criteria during the study period.  A preliminary analysis revealed that there no significant differences found in one course where the instructor was different in each of the three scheduled offerings.  In a second, course, however, significant differences were found in ratings of Teaching Presence and Cognitive presence that correspond to a change in instructor.

Relationship among Community of Inquiry, teacher/student ratio, and online course activity

Data for this analysis are currently unavailable until the study period is completed in July, 2017.  Although a tests have been conducted to establish the process for downloading and analyzing this data, a preliminary analysis is not possible at this until the study period is completed.

Summary

The presentation that will be prepared for this session will include the full results from this port of a two-year evaluation study.  It is expected that this study will provide the audience with a relevant examination of the impact of certain administrative policies and decisions with regards to teaching assignments on the development of a Community of Inquiry within a blended graduate program.  

References

Arbaugh, J. B., Cleveland-Innes, M., Diaz, S. R., Garrison, D. R., Ice, P., Richardson, J. C., & Swan, K. P. (2008). Developing a community of inquiry instrument: Testing a measure of the community of inquiry framework using a multi-institutional sample. The Internet and Higher Education, 11(3-4), 133-136. doi:10.1016/j.iheduc.2008.06.003

Garrison, D., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education model. The Internet and Higher Education, 2(2-3), 87-105