1. Are you listening? How to listen and use student feedback effectively for continuous improvement.

Introduction

Colleges are typically strong at gathering data from their students. College administrators assess their students’ satisfaction through the use of surveys. Colleges report on quality and performance metrics such as retention and completion each year and publish these rates on their website and/or catalog. Last, colleges collect feedback, typically at the end of each course, about each course and every faculty member. As a result, there is no shortage of data-centered information at a college. However, how do colleges use this data? Once it is collected, is it just reported as required? Does the faculty rating get considered in evaluation metrics? Or is the data used to make meaningful differences? This presentation will explore how one college, American College of Education, uses this data to foster continuous improvement.

American College of Education Background

American College of Education is an institution dedicated to “serve those who serve others”. The college, founded in 2005, is entirely online and offers bachelors, masters, and doctoral degrees in the fields of education and healthcare. American College of Education has a student population of roughly 5,000 students. The student body is comprised of students from 24 – 80 years old, living in 49 of the 50 states and in 48 countries around the world.

The college has experienced tremendous growth over the past four years. Since 2014, new enrollment has grown by over 15 % each year resulting in a 200 % growth in student population. During this period of high growth, the college has continuously achieved strong academic and student satisfaction results. In 2017, the college reported a 77% graduation rate, 85% annual retention rate for master’s students, and an 80% retention rate for doctoral students. Additionally, students reported an extremely high-level of satisfaction: 93% of master’s students reported being satisfied with the college where 84% of doctoral students reported the same level of satisfaction. The students also provided an average rating of 8.3 (out of 10) for satisfaction with their courses and an 8.7 (out of 10) satisfaction rate with faculty members. Although the institution is pleased  with the results; however, wishes to improve in all possible areas in the years to come.

American College of Education Data

As seen in the above section American College of Education is a typical college regarding the amount of data available to administrators. The college collects student satisfaction surveys each year, surveys students at the end of each course regarding the faculty and course, and reports on retention, completion, and other state-certification metrics each year. Where the data collected at American College of Education is not unique, the way the college uses the data is unique. The college uses the feedback garnered from students to continually improve the college. This presentation will explore the ways American College of Education utilizes student feedback to improve the college and the overall student experience.

Turning Course Evaluations into New Year’s Resolutions

Each year the college reviews all course evaluations from the prior year to look for areas of improvement. The course evaluations are reviewed for suggested improvements. Specifically, the college pulls every suggestion that a student made regarding improving the course. The college does not consider the positive comments in this process; instead, the sole focus is on improvement suggestions. To provide some context, in 2017 only 14.4% of the comments focused on improvements. It would be easy for the college to state that 83.6% of the comments did not include any areas for improvement; thus, they college must be doing well.

Instead of accepting the 83.6% positive comments, the college chooses to focus on the other 14.4% to determine a way to continually improve the student experience. Specifically, the college’s academic team and President review the comments, find themes, ensure the suggestions make sense, and then come up with potential solutions for the areas of improvement. Once this evaluation has occurred, the feedback and proposed solutions are provided to the faculty to for feedback and support (the college needs the faculty support to move any change forward).

After the evaluation and discussions take place, the college then officially commits to students to a list of improvements for that year. American College of Education calls this its “New Years Resolutions”. These resolutions are presented to students and is the college’s transparent promise to students to listen to their feedback and to make improvements based on it. The college has used this approach for the past five years and, as a result, fantastic improvements have been made. Some of these improvements include:

1. Restructuring the entire college library and reading access to make the experience easier for students.
2. Opening quizzes earlier to allow for more flexibility for students.
3. Reworking assignments that were confusing to students or that were flawed in the time allowed.
4. Revamping the “Student Commons” area of the LMS to provide more resources to all students.
5. Eliminating lengthy and wordy program handbooks (replacing them with videos and worksheets).
6. Adding in collaboration and technology tools to enhance the student experience.

Student Satisfaction Surveys

In addition to reviewing course evaluations on a regular basis, the college uses results from its student satisfaction surveys to improve the students’ experience. American College of Education, like most colleges, surveys its students to gauge overall satisfaction each year. In this survey, students respond to questions about student support services in addition to the academic experience. The college follows the same pattern to evaluate results and garner additional feedback as it does with course evaluations. Additionally, the college uses these results to pinpoint improvements to services and experience factors. The college introduces these improvements to students and transparently communicates the reason for the change (result of student feedback in the surveys). Some of the largest changes in student support services over the past four years has been the result of these surveys. A few of these improvements are as follows:

1. Creation of “One Stop Shop” for students. All student facing actions and services were moved to the Student Services team providing students with one point of contact for students.
2. Addition of Collegewide Webinars. Students requested more live interaction through webinars at the college: the college now publishes a schedule each year with webinars scheduled each month.
3. Utilization of Instructional Videos. Students expressed dissatisfaction with handbooks and wanted easier ways to find information. The college created a “Toolbox” concept in the LMS with instructional videos (like YouTube videos) to walk them through confusing parts of a program.
4. Assignment of a Student Success Coach: The College moved to a Student Success Coach model, with a specific success coach being assigned at time of enrollment, to respond to students wanting a “dedicated person” for all questions.

Retention and Completion Rates

The college also listens to and hears its students by reviewing trends in retention and completion. As previously stated, the college has high retention and completion rates; however, is continually striving for better. To better understand our weaknesses in these critical areas, we evaluate when students leave us (after a certain course or period), why they leave us, and courses that students are repeating to have to pass. Based on these metrics, the college has done the following:

1. Revamp of the entire doctoral program to provide more support in the dissertation process. In 2017, the college saw that students were not completing the dissertation within the expected timeframe; thus, a new program version was introduced in 2018 with a structured approach, through designated chapters, to the dissertation process.
2. Revision of the Internship Experience in the Educational Leadership program. Prior to 2014, ACE Educational Leadership students were struggling with completing their internship. The college moved the field experience into a 10-week course format which has resulted in 94% completing on the first attempt.

Conclusion

Colleges exist to serve students; thus, it is important to ensure that administrators, staff, and faculty are listening to their students’ feedback. We encourage our students to be receptive to feedback; however, are we receptive to the feedback they are providing to us? Are we modeling the continuous improvement that we ask of our students? If not, what data can we, as college administrators, use to listen and to improve?