The Systematic Design Process of Virtual Peer Mentoring Training for Women in Engineering

Overview

Following the design of a virtual STEM peer mentoring training using a systematic deign approach, a learning experience design study was  conducted to examine how virtual STEM peer mentoring training  promoted the STEM self-efficacy of White and Black, Indigenous and People of Color (BIPOC) women undergraduate engineering students at an institution serving a minority population. The authors conducted a remote synchronous usability test and follow up interviews. Following multi-grounded theory (Goldkuhl & Cronholm, 2010; 2018), data were analyzed inductively and deductively. In addition to discussing the design process and study results, recommendations for design and  improving the user experience will be provided by participants as they interact with the virtual peer mentoring training.

 

Relevance

A disparity exists in science, technology, engineering, and mathematics (STEM) fields among gender and racial and ethnic populations (National Science Foundation [NSF], 2019); and mentoring is becoming an intervention to promote both women's and Black, Indigenous, and People of Colors' (BIPOC) STEM engagement, matriculation, and persistence (Carlone & Johnson, 2007; Hill et al., 2010; National Academies of Sciences, Engineering, and Medicine [NASEM], 2019; Rockinson-Szapkiw et al., 2021a; Rockinson-Szapkiw & Wendt, 2021). Lack of representation of White and minority women in STEM degrees and careers has been attributed to myriad reasons; however, research supports that a so-called “confidence gap”, or poor self-efficacy is primary(Hill, et al., 2010). Consequently, growing interest in improving self-efficacy of women to broaden participation have emerged, and engagement in mentoring relationships have been identified as central to the development of self-efficacy and, ultimately, persistence (Carlone & Johnson, 2007;).

The research documenting the benefits of STEM mentoring for women and BIPOCs has primarily focused on face-to-face programs and within the context of research labs (Dawson et al., 2015; NASEM, 2019). However, researchers are beginning to recognize that virtual peer mentoring may be more conducive to these underrepresented populations' needs. Virtual peer mentoring enables women and BIPOCs access to mentors who match their demographic characteristics when otherwise inaccessible due to locations. The virtual environment also provides the flexibility and convenience these populations often need to access such programs (Zambrana et al., 2015; Rockinson-Szpakiw et al., 2021a), for women and BIPOC students are often unable to participate in traditional face-to-face mentoring programs due to their roles and responsibilities. The hours and locations in which programs are offered do not account, for example, for these populations' caregiving responsibilities and work schedules (NASEM, 2019).

Virtual peer mentoring programs are significantly different from face-to-face ones, particularly the user interface and learner experience. Learners interact in peer mentoring programs on various smartphones or other web-connected devices; therefore, it is commonly considered a best practice to perform a usability or learner experience design (LXD) study before launching a virtual program. This type of study assesses the learning environments' use, usefulness, ease of use, and ability to support intended learning outcomes (Gray et al., 2020; Mayer & Moreno, 2003). LXD studies specifically seek to understand how the learner interacts with the interface to facilitate meaningful learning (Tawfik et al., 2021). 

Examining learner experiences is an important step in designing and developing each virtual peer mentoring program element. Elements of these programs may include training and peer-mentoring experiences. Training is essential for an effective peer mentoring relationship (Butz et al., 2018; Gregg et al., 2017; Subotinik et al., 2019; Rockinson-Szapkiw, & Wendt, 2021). Therefore, the present presentation demonstrates the design process and explains the results of an  LXD study that examined how content and user experience (UX) supported virtual peer mentoring training outcomes, namely STEM self-efficacy.  The training is intended to be part of a virtual peer mentoring program for women, both White and BIPOC women, in an engineering programs at Minority Serving Institution (MSI) .This study also seeks to engage in the conversation about developing culturally and gender-relevant constructs for LXD studies, which is in a neophyte stage at this point.

Engagement

After providing an overview of the design process and study results, we will distribute a link to the virtual peer mentoring training. Participants will work in groups to generate strategies for supporting the candidate based on individual characteristics for design and  improving the user experience.

 

References

 

 

 

Andreasen, M. S., Nielsen, H. V., Schrøder, S. O., & Stage, J. (2007). What happened to remote usability testing?: An empirical study of three methods. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 1405–1414). ACM. https://doi.org/10.1145/1240624.1240838

Bandura, A. (1977). Self-efficacy theory: Toward a unifying theory of behavioral change. Psychological Review, 34(2), 191-215. Retrieved from https://www.uky.edu/~eushe2/Bandura/Bandura1977PR.pdf

Bandura, A. (2005). The evolution of social cognitive theory. In K. G. Smith & M. A. Hitt (Eds.), Great minds in management (pp. 9-35) Oxford University Press.

Boren, T. & Ramey, J. (2000). Thinking aloud: Reconciling theory and practice. IEEE transactions on professional communications, 43(3), 261-278.

Carlone, H & Johnson, A. (2007). Understanding the science experiences of successful women of color: Science identity as an analytic lens. Journal of Research in Science Teaching, 44(8), 96-564.

Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35(8), 982–1003. https://doi.org/10.1287/mnsc.35.8.982

Dawson, A. E., Bernstein, B. L., & Bekki, J. M. (2015). Providing the psychosocial benefits of mentoring to women in STEM: CareerWISE as an online solution, New Directions in Higher Education, 2015(171), pp. 53–62. doi:10.1002/he.20142

Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data. MIT Press.

Fan, M., Lin, J., Chung, C., & Truong, K. N. (2019). Concurrent think-aloud verbalizations and

usability problems. ACM Transactions on Computer-Human Interaction, 26(5), 1-35. doi:10.1145/3325281

Gray, C. M., Parsons, P., Toombs, A. L., Rasche, N., & Vorvoreanu, M. (2020). Designing an aesthetic learner experience: UX, instructional design, and design pedagogy. International Journal of Designs for Learning, 11(1), 41–58

Gregg, N., Galyardt, A., Wolfe, G., Moon, N., & Todd, R. (2017). Virtual mentoring and persistence in STEM for students with disabilities. Career Development and Transition for Exceptional Individuals, 40(4), 2015-214. https://doi.org/10.1177/2165143416651717

Goldkuhl, G., & Cronholm, S. (2010). Adding theoretical grounding to grounded theory: Toward multi-grounded theory. International Journal of Qualitative Methods, 9(2), 187–205. doi:10.1177/160940691000900205

Goldkuhl, G., & Cronholm, S. (2018). Reflection/commentary on a past article: “Adding theoretical grounding to grounded theory: Toward multi-grounded theory.” International Journal of Qualitative Methods, 17(1), 160940691879554. doi:10.1177/1609406918795540

Hassenzahl, M. (2004). The interplay of beauty, goodness, and usability in interactive products. Human–Computer-Interaction, 19(4), 319–349. doi:10.1207/s15327051hci1904_2

Hassenzahl, M., & Monk, A. (2010). The inference of perceived usability from beauty. Human–Computer Interaction, 25(3), 235–260. doi:10.1080/07370024.2010.500139

Hassenzahl, M., & Tractinsky, N. (2006). User experience: A research agenda. Behaviour and Information Technology, 25(2), 91–97. doi:10.1080/01449290500330331

Jonassen D. H. (1995). “Operationalizing mental models: strategies for assessing mental models to support meaningful learning and design-supportive learning environments,” in Proceedings of the First International Conference on Computer Support for Collaborative Learning CSCL ’95, (Hillsdale, NJ: L. Erlbaum Associates Inc; ), 182–186.

Lee, S., & Koubek, R. (2010). Understanding user preferences based on usability and aesthetics before and after actual use. Interacting with Computers, 22(6), 530–543. doi.org/10.1016/j.ijhcs.2013.09.006

Lewis, J. R. (2012). Usability testing. In G. Salvendy (Ed)., Handbook of human factors and ergonomics (4th ed., pp. 1267-1312). Wiley.

Lewis, J. R. (2014). Usability: Lessons learned…and yet to be learned. International Journal of Human-Computer Interaction, 30(9), 663–684. doi:10.1080/10447318.2014.930311

Mayer, R. E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38(1), 43–52. https://doi.org/10.1207/S15326985EP3801_6

National Academies of Sciences, Engineering, and Medicine. (2019), The science of effective mentorship in STEMM, available at: https://doi.org/10.17226/25568 (accessed 27 March 2020).

National Science Foundation [NSF]. (2019), Women, minorities, and persons with disabilities in science and engineering, available at: https://ncses.nsf.gov/pubs/nsf19304/ (accessed 27 March 2020).

Nielsen, J., & Loranger, H. (2006). Prioritizing web usability (1st ed.). New Riders.

Rockinson-Szapkiw, A. J., Wendt, J., & Wade-Jaimes, K. (2020). The essentials for building and maintaining trust. In Rockinson-Szapkiw, A. J., Wendt, J., & Wade-Jaimes, K. (Ed.). Navigating the peer mentoring relationship: A handbook for women and other underrepresented populations in STEM. Dubuque, IA: Kendall Hunt.

Rockinson-Szapkiw, A.J., Sharpe, K., & Wendt, J. (2021). Promoting self-efficacy, mentoring competencies, and persistence in stem: Evaluating ethnic and racial minority women's learning experiences in a virtual stem mentor training program. Journal of Science Education and Technology.

Tawfik, A.A., Gatewood, J., Gish-Lieberman, J.J., & Hampton, A.J. (2021) Towards a definition of learning experience design. Technology, Knowledge and Learning. doi:10.1007/s10758-020-09482-2

Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. Management Information Systems Quarterly, 27(3), 425–478. https://doi.org/10.2307/30036540

Zambrana, R. E., Ray, R., Espino, M. M., Castro, C., Douthirt Cohen, B., & Eliason, J. (2015), “Don’t leave us behind”: The importance of mentoring for underrepresented minority faculty, American Educational Research Journal, 52(1), 40-72. doi:10.3102/0002831214563063