Affective, Culturally Responsive Learner Experience Design for Virtual STEM Peer Mentor Training

A disparity exists in science, technology, engineering, and mathematics (STEM) fields among gender and racial and ethnic populations (National Science Foundation [NSF], 2019); and mentoring is becoming an intervention to promote women's and Black, Indigenous, and People of Color ‘s (BIPOC) STEM engagement, matriculation, and persistence (Carlone & Johnson, 2007; Hill et al., 2010; National Academies of Sciences, Engineering, and Medicine [NASEM], 2019). The research documenting the benefits of mentoring for women and BIPOCs has primarily focused on face-to-face programs (Dawson et al., 2015; NASEM, 2019). However, researchers are beginning to recognize that virtual peer mentoring may be more conducive to these underrepresented populations' needs. Virtual peer mentoring enables women and BIPOCs access to mentors who match their demographic characteristics when otherwise inaccessible due to locations. The virtual environment also provides the flexibility and convenience these populations often need to access such programs (Zambrana et al., 2015), for women and BIPOC students are often unable to participate in traditional face-to-face mentoring programs because the hours and locations in which they are offered do not account for their roles and responsibilities (e.g., busy schedules related to caregiving and work schedules) (NASEM, 2019).

Virtual peer mentoring programs are significantly different from face-to-face ones, particularly in terms of the user interface and learner experience. Learners interact in peer mentoring programs on various smartphones or other Internet-connected devices; therefore, it is commonly considered a best practice to perform a usability or learner experience design (LXD) study before launching a virtual program. This type of study assesses the use, usefulness, and ease of use of the environment to support intended learning outcomes (Gray et al., 2020; Mayer & Moreno, 2003) and seeks to understand how the learner interacts with the interface to facilitate meaningful learning (Tawfik et al., 2021). 

Examining learner experiences is an important step in designing and developing each virtual peer mentoring program element. This includes the training element, as research documents that for any peer mentoring relationship, virtual or traditional, to be effective, training for the mentor and mentee is essential (Butz et al., 2018; Gregg et al., 2017; Subotinik et al., 2019; Rockinson-Szapkiw, Sharpe, & Wendt, 2021). Therefore, the present LXD study examines how content and user experience (UX) supported virtual mentor and mentee training outcomes. This training is intended to be part of a virtual peer mentoring program for women, focusing on BIPOC women, in STEM programs at Minority Serving Institutions (MSIs) and Historically Black Colleges and Universities (HBCUs). This study also sought to engage in the conversation about developing culturally and gender-relevant constructs for LXD studies. 

The results of the study will be discussed. 

Consistent with the literature (Hassenzahl & Tractinsky, 2006), participants created socially and personally constructed models from their previous learning experiences, attributes, and goals. These internal cognitive and emotional models informed their impressions (e.g., structure, navigation, functionality, aesthetics, tone, and modality) and, in turn, their interactions (e.g., value assignment, scaffolding, and dynamic interaction) with the training modules, which ultimately influenced how they believed learning outcomes would be met by the target learners. As discussed throughout the results, these findings validated Tawfik et al.’s, (2021) conceptualization of LXD as the 'confluence' of the constructs of the Learning Environment and the Learning Space. However, the findings also expanded Tawfik et al.’s (2021) conceptualization of these two constructs, adding factors and expanding the definitions of previous proposed factors given cultural perspectives (i.e., codes; see Table 2).

Figure 13

Study findings in visual format

Noteworthy is the expansion of codes to include the affective elements of learning design. For participants who were from diverse populations, they identified the importance of not only the cognitive models but also the emotional models they bring to a learning experience that influences their impressions of the learning environment and interactions with it. Moreover, responses to the physical and abstract learning space were expressed using both cognitive and emotion terms. For this reason, Tawfik et al.’s (2021) original Learning Environment code of “Interface Aligned with Existing Mental Models” was revised to “Learner’s Alignment with Internal Models,” inclusive of both cognitive and emotional models. Internal models are frameworks that an individual use to understand, evaluate, experience, and act. In many casesmental models (Norman, 1988), have been examined in various fields, including design, with most scientists focusing on the cognitive aspects of mental models, often neglecting emotions. Rather, they often describe how learners make sense of their experience and the interrelated nature of variables (e.g., causality). This is not surprising as Craik’s (1943) seminal work on mental models explained this construct as individuals’ cognitive representations of external reality that were used to anticipate events, reason, and form explanations. Johnson-Laird (1983) later extended the seminal work, proposing a mental model is a reasoning (e.g., cognitive) mechanism that exists in an individual’s working memory. Even within the learning and design literature, Nersessian (2002) stated that, “the nature and richness of models one can construct and one’s ability to reason develops with learning domain-specific content and techniques” (p.140); and systems researchers focus on the role of mental models in information feedback loops, focusing on problems which hinder information feedback, and thus, learning (Brehmer, 1992).

While emotions have not been fully neglected in the mental model and learning design literature (e.g., Pekrun, 2000), very little empirical studies have considered both cognitive and emotional models jointly. Moreover, the research has been mixed on the role and interaction of emotions and cognitions in learning and learning design. Debate has ensued if empirical evidence even exists to justify cognitive and emotional models as separate, given their often indistinguishable structure, dependence on each other, and their intertwined influence behaviors (Christensen & Olson, 2002). However, the findings of this research provide evidence that both mental and emotional models of participants were activated as part of the learning experiences design, especially in the Learning Environment and Space and its ipmact on impressions, evaluation, engagement, and interaction. Cognitive models were the knowledge or thought systems (e.g., facts, concepts, principles, and the relations between these constructs;; van Merriënboer, 1997) that provided participant’s immediate expectation about what was going to happen in the learning environment and were used to understand, evaluate, experience, and act. Alternatively, emotional models were the feeling and visceral systems that provided participant’s immediate expectation about what was going to happen in the learning environment and were used to experience and act. The response activated by the emotional models were often specific stimulus to a learning environment element, in this study, and they could be described in terms of dimensions, congruent with previous studies (Morel, 2021; Russell, 2003; Pekrun & Linnenbrink-Garcia, 2012, Pekrun et al., 2017), such as arousal (activating vs. deactivating) or valence. The present study suggests there appears to be an interaction between these systems. Sometimes participants described a thought that activated an emotional response, other times participants described a visceral response that was followed by metacognition about why the visceral response or feeling was experienced. Additionally, in consideration of the emotional component of learning, definitions for several Learning Environment codes are inclusive of the phrase “learner’s impression.” The term impression was chosen to refer to learner’s thought, opinions, and feelings formed both with and without conscious awareness that result in a choice or behavior. The term impression is encompassing of both the cognitive and emotional elements a learner assigns to various physical aspects of the learning setting.

Findings of this study also illuminated the importance of identifying the characteristics salient to the development of learner’s internal models so that inclusive and equitable learning experiences can occur. For this study, participants, who were women, many identifying as BIPOC, noted that their gender, race, ethnicity, and experiences were salient in developing the filters or models they constructed, and in turn, were used to interact with the Learning Environment and Space. For example, they purported their identity as women and BIPOC made them take note that the designers paid attention to the gender and racial characteristics of avatars used in pictures, simulations, and videos. The participants thus found the designers' use of diverse avatars provided them with an inclusive learning experience. These findings imply that if designers want to ensure learners are provided with equitable learning opportunities and experience an inclusive learning environment, they need to understand the salient characteristics that inform learners’ internal models. These finding reiterate the need to assess and develop a deep understanding of learners' diversity and cultural, basic to LXD and other learner-centric design principles (Pillai, Lucas, & Mello, 2014; Shonfield, et al. 2021). Findings remind designers that a broad spectrum of diversity, especially learners who have been minoritized based on demographics (e.g., gender, race, ethnicity) or classifications (e.g., first-generation college students, English language learners), needs to be considered. And, when designing learning experiences with diversity and inclusion at the core, careful consideration is needed of the graphics, resources and activities to ensure all learners feel a sense of belong in the Learning Environment and Space and content is relevant and authentic to diverse learner experiences. Also significant was participants’ warning to designers to avoiding unwanted focus; that is never make learners feel students inappropriately visible (e.g., "an African American perspective" or "a woman's perspective.").

Learner experience design of the virtual STEM peer mentoring training provided us with valuable information, leading us to implement important changes before the peer mentoring training program was launched. We observed both effective and ineffective physical and abstract elements of the learning setting. Whereas LXD was previously focused on the cognitive reactions and perceptions of learners, empirical data from this study suggests that LXD must also take into account the affective learner and her impressions, which are influenced by the very core of who she is and what goals she seeks. Thus, Tawfik et al.’s (2021) Learning Environment construct was expanded from five to eight codes. The new codes moved beyond thoughts of structure, navigation, and functionality to capture the learner’s visceral feelings, about aesthetics, tone, and modality. The Learning Space construct was also refined from four to three codes and organized hierarchically to note how learners’ impressions influenced the value they assign learning features and their perception of scaffolding, both of which ultimately inform dynamic interaction and learning outcomes. These findings are transferable as lessons for any and all individuals desiring to develop a virtual training for a peer mentor program in STEM for BIPOC women. For, there is little research on virtual and blended STEM peer mentoring programs for either the mentor or mentee, especially in STEM at HBCUs and other minority serving institutions. Moreover, this research extended the understanding of learning experience design, which was originally derived from examination of a STEM learning space with cognitively focused outcomes and the interactions of primarily White male users with the space. Using a STEM learning space with cognitively and affectively focused outcomes and examining women and BIPOC learner interaction in the space, original constructs and definition of constructs were broadened and refined.