The purpose of this study was to evaluate the effectiveness, efficiency and impact of a blended tutor training based on the perceptions of the instructor and students. We sought to close the gap in literature and contribute additional evidence to the application of blended teaching and learning.
Introduction
Blended learning is a recent educational approach combining face-to-face and online learning approaches, which makes it possible to enjoy the advantages of both teaching environments and methods (Graham, 2004; Garrison & Kanuka, 2004). It has experienced significant growth during the past years that can be attributed to the following factors: flexibility, reduced costs, and perceived improvement in the teaching and learning experience (Graham, 2004; Napier, Dekhane & Smith, 2011).
The experience of blended learning can be measured with respect to various aspects. Previous studies have been centered on exploring the utility, motivation, and satisfaction from the perspective of either students or instructors, or focusing on introducing different methods of teaching and technologies adopted (Garnham & Kaleta, 2002; López-Pérez et al., 2011). Most case-study research on blended learning to date has come from instructors as researchers analyzing their own students' experiences of blended learning in the context of a particular case in K-12 or higher education (Bliuc, Goodyear & Ellis, 2007). However, very limited research has been conducted in blended learning in the context of graduate-level courses to report students’ learning experiences or outcomes acquired through this format of learning approach (Butz & Stupnisky, 2016; Lim & Morris, 2009). Since graduate students represent a different set of learner characteristics such as more goal-oriented and self-directed, which also requires instructors to adopt relevant pedagogical approaches to accommodate their needs, it is highly important to probe further on their learning and teaching experience of blended learning to inform future course design and development.
In order to close the gap in literature and contribute additional evidence to the application, we sought to evaluate three aspects of a graduate-level training course in blended format concerning instructor’s and students’ perceptions on 1) effectiveness: whether and how the activities afforded by the blended learning environment may foster learning and skill development; 2) efficiency: whether and how the blended learning environment may foster both the learning and instructional process and enhance personal productivity (in terms of time management, distribution of workload, and ease of access in the selected LMS); 3) impact: whether and how the blended learning environment influence the learning/teaching experience positively or negatively.
Research Questions
The purpose of this study was to evaluate the effectiveness, efficiency and impact of a blended tutor training based on the perceptions of the instructor and students. Additionally, we sought to contribute additional evidence from this case study regarding the blended teaching and learning experience with a guidance of the following research questions:
1. What is the students’ and instructor’s perceived effectiveness of the blended tutor training?
2. What is the students’ and instructor’s perceived efficiency of the blended tutor training?
3. What is the students’ and instructor’s perceived impact of the blended tutor training?
Description of the Context
This project was funded by a grant awarded by the Center for the Integration of Research, Teaching and Learning (CIRTL, a NSF funded network) and the Center for Excellence in Teaching and Learning (CELT) at a large Midwestern university for the academic year of 2016-2017. The project involves the design, development, and evaluation of a tutor training in a blended format, which is a prerequisite for students interested in or appointed as writing consultants as part of the Graduate Peer Mentor (GPM) Program in the Center for Communication Excellence (CCE) of the Graduate College.
The primary goal is to equip students with knowledge of how scientific discoveries are effectively communicated to disciplinary communities. Previously, this training was conducted in a three-credit, semester-long course only offered during the Fall semester. However, due to an increasing demand from departments/colleges for training their own writing consultants within a limited time period, the CCE has perceived the need to create a fast-track version of this training, which can have the flexibility of being offered during the Fall and/or Spring semesters.
Two instructional designers (referred to “research team” in this proposal) took a lead on this project and decided to address the design in four stages: 1) May-June 2016, conducting a need analysis on the learners, content, and context; 2) July-Aug 2016, designing and developing all the learning modules; 3) Sept-Dec 2016, running a pilot test in Moodle and collecting formative evaluation results for further improvement; 4) Jan-Apr 2017, implementing the complete training course and collecting summative evaluation results. The research team designed this study upon receiving the award with the aim of evaluating the perceived effectiveness, efficiency and impact of the blended tutor training, informing the future design of other blended courses, and contributing additional evidence regarding the blended teaching and learning experience.
Method
This study used a mixed-methods approach for data collection and analysis. The following data collection procedures have been completed: 1) Pre-survey intended to collect the participants’ demographic information and perceptions of blended learning and expectations about the course through Likert scale questions; 2) Post-survey aimed to gather participants’ perceptions on their overall learning experience, efficiency of learning process, impact of blended learning, and design factors through Likert scale questions; 3) Focus group interview was conducted with a group of voluntary participants in a semi-structured format to explore more in-depth questions pertaining to their learning experience and recommendations for future improvement; 4) Individual interview with the course instructor was conducted to understand the course preparation process and reflective experience before and during teaching the training course.
The data analysis is currently in process including two essential parts: 1) the quantitative data gathered from the pre-survey and post-survey will be analyzed through a descriptive statistical analysis to summarize an overview and comparison of participants' perceptions before and after taking the tutor training; 2) a content analysis of focus group interview and instructor interview by employing the Protocol coding (Saldaña, 2015) for exploring major themes, creating a codebook, constructing a deep analysis, validating the accuracy of the findings by triangulating the data collected in all formats and sources.
Potential Results
Based on a preliminary analysis of the pre-survey and post-survey, our participants show a divided understanding and satisfaction of blended learning experience before and after the course. Most students did not have a clear vision of how to get involved in different types of activities, manage their own time, and ask for assistance at the beginning, but felt comfortable with the blended format at the end of the course. Most students showed a positive expectation on their own performance including active participation in the online discussions, scheduling a regular time for doing course work, and setting up short-term personal goals before the course, but reported relatively negative results on their own performance in the end. As for additional assistance received both from the instructor or peers, students were reporting much lower satisfaction in the online session compared to the face-to-face session.
Overall, participants provided positive feedback on their learning experience in a blended format, felt engaged in online activities, and showed a high level of satisfaction on the course quality, but indicated concerns on the assistance received and the usability of Moodle course page. With more data analysis on the focus group interview and individual instructor interview, we plan to further compare students’ perceptions before and after the course, and triangulate the data from all different formats and sources. If the proposal got accepted, more comprehensive results, discussions, and conclusions will be presented in the conference.
How to Engage the Audience
In order to engage the audience during the entire presentation, we plan to portray our case study as an open-ended question for everyone by introducing our context as a specific scenario for case discussion. Our focus is to use a more participant-oriented approach to get everyone involved in the interactive activities. By using Poll Everywhere, we plan to gather audience's brainstorming ideas on how to interpret the results from our surveys and interviews, and how to improve the instructional design of this blended tutor training in the future.
Blended Session Focus
The primary focus of this proposal is centered on blended learning by introducing the findings of evaluating a blended tutor training re-designed in a blended format. We believe our presentation will actively engage the audience in the conversation by further exploring the highlights emerged from the perceptions of the instructor and students, and rethinking possible solutions to improve the course in order to better take advantage of the benefits of both face-to-face and online learning environments.
References
Bliuc, A. M., Goodyear, P., & Ellis, R. A. (2007). Research focus and methodological choices in studies into students' experiences of blended learning in higher education. The Internet and Higher Education, 10(4), 231-244.
Butz, N. T., & Stupnisky, R. H. (2016). A mixed methods study of graduate students' self-determined motivation in synchronous hybrid learning environments. The Internet and Higher Education, 28, 85-95.
Garnham, C., & Kaleta, R. (2002). Introduction to hybrid courses. Teaching with technology today, 8(6), 1-2.
Garrison, D. R., & Kanuka, H. (2004). Blended learning: uncovering its transformative potential in higher education. The Internet and Higher Education, 7(2), 95–105.
Graham, C. R. (2004). Blended learning systems: definition, current trends, and future directions. In C. J. Bonk, & C. R. Graham (Eds.), The handbook of blended learning: Global perspectives, local designs (pp. 3–21). Zurich: Pfeiffer Publishing.
Lim, D. H., & Morris, M. L. (2009). Learner and instructional factors influencing learning outcomes within a blended learning environment. Educational Technology & Society, 12(4), 282–293.
López-Pérez, M. V., Pérez-López, M. C., & Rodríguez-Ariza, L. (2011). Blended learning in higher education: Students’ perceptions and their relation to outcomes. Computers & Education, 56(3), 818-826.
Napier, N. P., Dekhane, S., & Smith, S. (2011). Transitioning to Blended Learning: Understanding Student and Faculty Perceptions. Journal of Asynchronous Learning Networks, 15(1), 20-32.
Saldaña, J. (2015). The coding manual for qualitative researchers. Sage.