Defining and Measuring Quality in Online and Continuing Education: A Cross-Cultural Case in Southwest China

Context

The most critical educational conflict of the new era in China is the conflict between people’s increasing demands for equal access to high quality education and the insufficient unequal supply of quality education resources. Online education, with its powerful potential for educational transformation through new technologies, may provide a significant pathway to redistribute quality educational resources and solve the demand-supply contradict. Such social development demands prelude a nation-wide transformation from scale to quality in the filed of online education development. Since its birth in the early 20th century, online education in China has been tightly connected with continuing education, and is often regarded as a hybrid of correspondence education, adult education, long-distance education, self-organized higher education and all other types of education delivered through on-line and on-ground blended learning mode in higher education institutions. Targeting a large population of students including those who failed the high-stake national higher education examination and would like to continue higher education, those who would like to promote professional capacity during their work, and those who just want to obtain certain degrees to earn a job, online and continuing education (OCE) in China has been providing supplementary learning service to 1/5 portion of the total higher education student enrolment of the whole nation as of 2016 (Yan, 2018).

However, pursuit of scale and quantity while neglecting the quality has led OCE into a pitfall. In recent years, the Ministry of Education of P.R. China has been transforming its visions and strategic plans about OCE from scaling up toward scaling down and quality improvement, as well as from academic compensation oriented towards capacity building oriented education. A series of national policies encouraged higher institutions to diagnose and reflect on their OCE quality issues, develop quality standards and evaluation systems, as well as to explore institutional OCE quality annual report mechanism. This study employed case study approach and illustrated a higher education institution, SU, in the Southwest region of China, where OCE quality is more serious and challenging issue, as an example to showcase how local institutions respond to the national strategic plans and the nation-wide quality oriented transformation. The aim of this case study is to elaborate how SU redefine online education and its unique meanings and measurement in Chinese context when bounded with continuing education through cross cultural perspective.      

Research Question

On the basis of literature synthesis findings and mixed method approach, this paper aims at examining the following questions both in the general sense and in the special situation of SU.

(a) How has the concept of OCE quality been defined in the Chinese context? And what is SU’s integrated view about it?

(b) What are the key factors and components of OCE quality? And how SU build up its own indicator system by involving multiple stakeholders’ views?

(c) How has OCE quality been measured? How does SU build up its quality measurement framework?

Methods

This study employed a mixed method approach to involve both qualitative and quantitative design and data analysis. According to Morse, qualitative or quantitative methods may provide core components or supplementary components respectively in a study, and these components can be conducted simultaneously or sequentially in order to serve the purpose of description and understanding in the study (Morse, 1991; Johnson & et al, 2007). This paper adopted Morse’s definition and chose the sequential mixed approach, with the qualitative method analyzing reflective data from multiple stakeholders’ interviews and literature, and with the quantitative methods (Dephi and AHP) acquiring the core data for building the OCE quality measurement indicator system.

First, the authors synthesized Chinese and international literature regarding conceptualization of OCE quality, measurement approaches, and existing quality standards, rubrics and benchmark including those developed by Chinese governments and associations, Quality Matters, Online Learning Consortium and many other institutions and universities, based on which a theoretical framework was worked out to define the concept of OCE quality and its key factors and compositions. Based on the theoretical framework, together with in-depth reflections on the previous OCE quality evaluation tools of SU university, as well as analysis of its uniqueness, interview protocols were designed to integrate voices from multiple stakeholders such as administrators, faculty, employers and students. An integrated view about OCE quality and a framework of the key components were achieved on this phase. 

Second, on the basis of the OCE quality component framework, a 3-round Delphi method was employed to draw consensus of a group of experts (including designers, administrators, scholars, quality supervisors) and informants in the OCE field to generate an agreed view and shared interpretation of OCE quality measurement framework. AHP method was also adopted to draw weights of the indicators in the framework. The authors further explored what types of data should be collected and how the data could be quantified and measured, or either analyzed by qualitative methods. Implications were finally drawn to other institutions in the similar cultural context.           

Results

Defining quality in any field is a challenge (Parker, 2005). There is hardly one agreed definition about online education quality (Chaney et. al., 2009; Jung & Latchem, 2007). And it is even harder to provide a fixed definition to OCE quality due to its in-borne hybridity and complexity.       Previous meta-synthesis (Esfijani, 2018; Latchem, 2014) about online education quality measurement found that, there is lack of universal agreement on the standards and quality measures for online education, and there is lack of evidence for output- and outcome-oriented approaches for quality factors identification and measurement; and that most approaches focus on students’ views, with faculty and administrators’ views having less attention and very little focus on designers, employers, and clients (Esfijani, 2018). Based on literature synthesis and an integrated view about OCE quality, and taking Logic Model as a conceptual framework, SU focused OCE quality on the most critical outcome dimension of student development quality, and classify all other quality elements as input (such as economic input, policy input, course design) and output (such as online course implementation, students’ engagement and interactions) factors causing short term, mid term and long term impacts/outcomes of student development.   

OCE addresses needs of learners different from those in traditional full time on-ground programs in higher institutions. OCE quality measurement is in essential assessing the extent to which OCE services meet the learning goals of these special groups of students. On its pathway to build up quality measurement framework, SU set up its quality development goals in alignment with the national OCE goals of developing students with better practical skills and stronger professional capacities ready to be applied in service. Accordingly, SU’s framework for assessing OCE quality primarily is comprised of (a) the assessment of students’ learning outcomes in alignment with the national and university learning goals, (b) the assessment of the input elements in alignment with students’ goals achievement status, and (c) the assessment of output activities in alignment with the students’ learning goals. With rudimentary findings from the Dephi study and AHP analysis, an OCE quality measurement framework of 3 dimensions and 19 indicators was established. The overall quality assessment system was centering around the questions of whether the OCE service contributed to the student learning goals, whether they are satisfied with the learning effectiveness, whether employers and faculty are satisfied with students’ practical skills when applied in the job, and whether all the learning inputs and activities are in alignment with students’ development goals.
 

Discussion and Conclusion

The study of OCE quality is a complex project, and to define and measure quality are the key challenges of such a project. In the new era of China, when the priority of online education shifted to quality improvement, the research and practice become hectic in terms of exploring a most effective, appropriate, and scientific definition and evaluation standard agreed upon by multiple stakeholders. More and more Chinese scholars and practitioners in the field of online education seek for existing knowledge and experience proved to be effective by the western countries and international organizations. However, cultural adaptability and locality are new challenges to be faced with. SU university, taking advantage of its unique local characteristics, exemplified such efforts on its pathway of meeting and surviving the national strategic changes as well as the international trends about online education. The study is of great significance both theoretically and practically in that it provides a cross-cultural perspective to understand how online education quality is conceptualized, interpreted, blended and measured with local characteristics, it contributes to existing literature from a different perspective of highlighting outcome quality of student development and through faculty, administrator, employer as well as designers’ perspectives, and it may lend useful practical experience in the pathway of evaluating and improving online education quality in the similar cultural context.