Innovative technologies offer various ways to improve online learning engagement of students. However, how to accurately evaluate the technology effectiveness in the online learning environment remains a significant question online instructors face and deal with nowadays. This study reports a comprehensive evaluation framework to assess the perceived benefits of using innovative technology to improve online learning experience. The findings also exposed several practical difficulties and challenges in using the innovative technology to enhance the consistency, availability, and quality of online instructor support. Methods, insights, and empirical evidence including best practices and lessons learned are also explored and discussed to help online instructors incorporate innovative learning tools efficiently and effectively into their teaching toolkit.
Presentation Outline/Interaction Plan
1. First, introduce the intent of the research briefly to our target audiences:
A wide variety of innovative technology tools have emerged with the aim to enhance online interaction and improve online learning experience (Kirkwood & Price, 2014), while encouraging interaction in online courses in education has long been an interesting research topic (Nandi, Hamilton, & Harland, 2012). Then the critical questions come as well:
- How do online instructors know if a new technology will help increase instructor presence and interactivity, provide a higher degree of presentation clarity, and increase student focus and/or attention to feedback?
- What are the perceived benefits of adopting such an innovative tool?
- How can the instructors be selective and effective in choosing appropriate innovative technologies to improve online learning?
2. Then continue with this specific question to our target audiences:
Suppose you have just noticed an exciting new tool that seems to be great for your online course and you plan to adopt it right away. Here comes the 1st key question:
- How do you know this new tool is going to improve your students learning effectiveness in the online environment?
3. After collecting and summarizing audiences’ initial thoughts and ideas, follow up with the following thought-provoking questions:
- Is there an established process/method of evaluating a new tool before adopting it into online teaching at your institution?
- If yes, do you believe that method offers an objective and accurate way to assess the perceived benefits of using innovative technology to improve online learning experience?
- If not, what rules of thumb have you used in the past for such assessment and were they consistently reliable?
4. Now analyze and categorize the discussed empirical processes, methods, criteria, and rules of thumb for new technology evaluation:
- Was there a pattern? Were there repeated/common themes or fundamental issues?
- Any good experience or lessons-learned applying different evaluation methods in the past? Any missing pieces so far?
5. Map the identified themes and fundamental issues to the proposed comprehensive evaluation framework. Also link each issue to published literature and empirical evidences.
- How to increase instructor presence and interactivity?
- To highly engage in and ensure a strong presence in the online course, instructors need to provide students with in-depth feedback for growth and development (Kelly, 2010).
- How to provide a higher degree of presentation clarity?
- Auditory components bring courses alive by allowing online learners to use their auditory senses to comprehend complex concepts and procedures that are difficult to explain with only text and graphics (Klass, 2003).
- How to increase student focus and/or attention to feedback?
- Listening to the instructor can easily grab student attention and is generally considered easier, faster, more interactive, and less cognitive-effort-consuming than reading and comprehending long written text (Copley, 2007; Oishi, 2007).
6. Real-time and interactive case scenario of using and applying the evaluation framework.
Now back to the very 1st key question we brought up at the beginning: given a perspective new technology tool, how do you know it is going to improve your students online learning effectiveness?
- Ask the target audience to suggest a new tool they would like to adopt or evaluate
- Show step-by-step how to seamlessly operationalize the proposed evaluation framework into a ready-to-use evaluation survey design
- Perform a sample evaluation test and discuss how to interpret the evaluation results
- Identify results that are counterintuitive or provocative and thus provided valuable insights for further investigation
7. Conclude with more discussions and sharing of best practices and lessons learned to help online instructors incorporate innovative learning tools efficiently and effectively into their teaching toolkit
- How to utilize the evaluation framework to provide empirical proof of whether or not to adopt an emerging innovative technology tool in the online learning environment?
- How to use the evaluation results to reveal practical difficulties and challenges in using the innovative technology to enhance the consistency, availability, and quality of online instructor support?
- How to make adjustment to online instructors’ technology adoption strategies in an efficient and effective manner?
Concluding Remarks
This study proposes an integrated evaluation framework to facilitate online instructors new technology adoption strategies. The framework can be operationalized into a ready-to-use evaluation survey to be used to elicit critical student feedback on the tool use in online learning environment. Operational case scenarios were reported as well to illustrate the operationalization and application of the evaluation framework. Methods, insights, and empirical evidence including best-practices and lessons-learned are explored to help online instructors incorporate innovative learning tools efficiently and effectively.