Customizing Learning in an Online Faculty Training Course by Adopting the ibstpi® Instructor Competencies App

The School of Professional Studies (SPS) at Northwestern University hosts a quarterly four-week, asynchronous training for faculty who will be teaching an online course for the first time. Offered through the Canvas learning management system, the training follows the format of an online course offered by the school. This allows faculty to experience a course from a student perspective while learning about technology, school policy and procedures, and best practices for facilitating an online course.

The current version of the faculty training course was first offered prior to fall quarter 2014. The course underwent an extensive redesign prior to fall quarter 2015, and has been updated each quarter since then to integrate new technology and respond to participant feedback. The latest update to the course is the integration of the International Board of Standards for Training, Performance and Instruction (ibstpi®) Instructor Competencies app.

International Board of Standards for Training, Performance and Instruction (ibstpi®)

The International Board of Standards for Training, Performance and Instruction (ibstpi® | http://www.ibstpi.org/) was established in 1977 to promote high standards of professional practice in fields of training, performance, and instruction through the systematic development, validation, and dissemination of competencies. ibstpi® defines a competency as “a knowledge, skill or attitude that enables one to effectively perform the activities of a given occupation or function to the standards expected in employment” (Richey, Fields, & Foxon, 2001, p. 31). The methodology followed by ibstpi® to develop competencies includes literature review, primary research, and input from a broad cross-section of professionals who participate in ibstpi® development and validation studies.

ibstpi® Instructor Competency Set

The standards in the ibstpi® Instructor Competency set (http://ibstpi.org/instructor-competencies/) apply to instructors in face-to-face, online, and blended settings. While it is difficult to identify everything a capable instructor should know and do, the ibstpi® instructor competencies are comprehensive and can be applied in teaching and learning settings throughout the world. The Instructor Standards include 18 competencies clustered in 5 general domains and supported by 98 performance statements. These are the competencies required by instructors, teachers, or training facilitators regardless of the contexts in which they practice.

Customizing Learning in an Online Faculty Training Course by Adopting the ibstpi® Instructor Competencies app

Collaboration with ibstpi® provided an opportunity to integrate a robust, internationally validated set of standards for instructors into the faculty training course. Responses to the self-assessment gave the course facilitator a sense of the background of each participant in the faculty training course. This provided the course facilitator an opportunity for customizing learning (Nistor, 2006). In response to the outcomes of the self-assessment, the facilitator was able to direct participants to specific resources and update the course to include resources that correspond to competencies identified for development. In addition to the foundational information for all new faculty taking the course, the integration of the ibstpi® Instructor Competencies app made the course uniquely relevant to each individual based on their background knowledge and existing competencies.

We asked all eleven students in the course to complete the self-assessment during the first week of the course. In week two, we shared the aggregate results (at the course level) – the overall identified strengths and areas for development. In week three, we shared individual feedback and additional resources related to competencies identified as needing development. During the fourth and final week of the course, we provided an opportunity for students to discuss their results and experiences interacting with the app.

Completion of the self-assessment was a graded portion of the course. We encouraged students to participate in the assessment by providing information in the form of announcements, assignment instructions, and an individual e-mail to each student. All students submitted responses, though there were some incomplete and late submissions.

The ​ibstpi® Instructor Competencies app allowed us to see the individual ratings for each student for each competency, as well as the average competency ratings for the entire class. The individual ratings interface shows the rating each participant assigned themselves on each of the 18 competencies. Each competency can be sorted to show how students scored themselves relative to each other on each competency (see Figure 1). In addition to displaying average results for the entire class, the app also provides a percentage of users who rated themselves in each competency area. This interface allows the instructor to see the percentage of students who rated each competency as an area of strength (green); a competency to be developed immediately (yellow); or a competency needing long term development (red) (see Figure 2).

Figure 1. The admin interface, sorted by ratings in the “communicate effectively” competency.

Figure 2. The admin interface showing percentage of users rating each competency as an area of strength or area for development.  

Although the sample size for this pilot was small (11 users), several results are notable. The overall self-assessment ratings on the 18 ibstpi instructor competencies (http://ibstpi.org/instructor-competencies/) were quite high. In 15 of the 18 competencies, more than two thirds of students identified the competency as an area of strength (Table 1). The course instructor customized the course by focusing on the three competencies that more than a third of participants identified as needing development. Though the students in this course are faculty teaching online for the first time, many have previous teaching experience in other contexts, and four of the eleven students have formal training in education and/or instructional design. The wealth of past experience was reflected in the high self-assessment scores.

Table 1.

Competency Students identifying as area of strength

Establish and maintain professional credibility 90%

Manage the instructional process through the appropriate use of technology 89%

Demonstrate effective questioning skills 89%

Communicate effectively 80%

Comply with established ethical and legal standards 80%

Stimulate and sustain learner motivation and engagement 80%

Prepare for instruction 78%

Demonstrate effective presentation skills 78%

Demonstrate effective facilitation skills 78%

Assess learning and performance 78%

Provide clarification and feedback 67%

Promote retention of knowledge and skills 67%

Promote transfer of knowledge and skills 67%

Use media and technology to enhance learning and performance 67%

Manage an environment that fosters learning and performance 67%

Plan instructional methods and materials 40%

Update and improve one’s professional knowledge and skills 30%

Evaluate instructional effectiveness 22%

As seen in Table 1, the competencies identified as areas of strength by the largest number of students were: establish and maintain professional credibility (90% of students), demonstrate effective questioning skills (89%), and manage the instructional process through the appropriate use of technology (89%). Given the context for the course, this outcome was not surprising, and at the same time it was promising. The school’s faculty are typically practitioners in their fields, so it makes sense that newly hired faculty already possess the skills that contribute to establishing and maintaining professional credibility, which include demonstrating subject matter expertise and relating instruction to organizational contexts and goals. Demonstrating effective questioning skills is an important attribute of an online instructor in our school as our courses are mostly asynchronous and discussion based, so faculty frequently call upon the skills in this area, which include asking clear and relevant questions and using questions to generate and guide discussions. Finally, managing the instructional process through the appropriate use of technology is a key component to facilitating an online course. In this competency, the skills include using technology to seek and share information, and store and reuse instructional resources.

To customize the course specifically to the current cohort, during the second half of the course we focused on the three competencies that less than two thirds of students rated as areas of strength: plan instructional methods and materials (40% identified as an area of strength), update and improve professional knowledge and skills (30%), and evaluate instructional effectiveness (22%) (see Table 1). We added an additional resource to the course to assist students in planning instructional methods and materials: the newly-published Online Course Design section of the school’s Distance Learning website, which assists instructors in planning online courses that follow Quality Matters standards. We focused solely on this resource because it was designed for the exact context of participants in our course.

In regards to “update and improve professional knowledge and skills”:  we recognized this as an opportunity to add significant content to our course. Northwestern University provides a wealth of professional development opportunities for faculty; the students’ response to this criteria indicated that they were not aware. We created a new wiki page in the course with a list of opportunities.

To assist students in developing the evaluating instructional effectiveness competency, we created a new discussion thread in the class: How can you tell if your instruction is effective? We look forward to the student interaction and learning surrounding this topic; this discussion is scheduled for the week after the due date for this proposal.

During the last week of the course we will focus on discussing the self-assessment, the results, and opportunities for future development. We will collect feedback from students in the course about their experience with the ibstpi® Instructor Competencies app.