Psycho-physiological stress-related changes when using Immersive Virtual Reality in vocational education

The Swiss Vocational Education and Training (VET) system emphasizes the value of work-based training (Gonon, 2002). Being a “Dual System”, it is characterized by an alternance between a work-based apprenticeship in an enterprise (3-4 days per week) and a school-based track in school classes (for the remaining 1-2 days) (Gonon, 2005).

Educational technologies could be an added value to include simulations of professional situations during school classes, allowing students to get closer to real-work scenarios. Virtual simulations could be developed using technologies such as Virtual Reality (VR) and 360° video. The advantages of using these technologies in education include the possibility to experience scenarios that can only be carried out a few times in real practice, even dangerous ones (as emergency rescues), and the reproducibility of the virtual experience as many times as needed. Using 360° video, real-world environments can be recorded and viewed through a head-mounted display (HMD) to provide an immersive VR (IVR) experience (Kavanagh et al., 2016). Realistic stress scenarios can be developed using 360° video in IVR modality to help students to handle emotionally challenging situations. Learning to manage stress in work contexts is essential for novices, who may experience high levels of stress in new working situations.

Stress effects have been highlighted on all body's systems, including cardiovascular, respiratory, endocrine, gastrointestinal, nervous, muscular, and reproductive systems (Chu et al., 2021). Stress as a psycho-physiologic phenomenon, has a multidimensional nature that can be decomposed into three main components: psychological, behavioural and physiological (Giannakakis et al., 2019). Given this multiplicity of factors, a multi-modal approach for accurate stress measurement would be suitable (Carneiro et al., 2012). To detect stress-induced physiological changes, several stress bio-signals can be considered, such as electrocardiogram (ECG), galvanic skin response (GSR) and electroencephalogram (EEG) (Ladakis & Chouvarda, 2021). Bio-chemical indicators can also be used, such as salivary cortisol, the golden standard stress biomarker, that has proven to be a valuable assessment tool in high-stress emergency contexts (Balters et al., 2020).

In this PhD thesis, we would like to focus on the use of virtual simulations to train and prepare students to cope with stress in real-world job situations. The research topic underlying all the planned studies concerns the impact of educational technologies – and of IVR in particular – on learners’ stress management, using psycho-physiological and neuroscientific measurements to detect stress. The progression of studies is thought as follows:

1. A systematic review aimed at investigating psycho-physiological or neuroscientific measurements in studies related to the use of IVR in education.
2. A preliminary field study involving trainee rescuers in emergency conditions. Multiple 360° video recordings of a simulated road rescue are used to develop an interactive 360° educational video. The resulting video is used by learners via HMD. Psycho-physiological measurements have been applied to detect heart rate (HR) and salivary cortisol levels. The aim of this study is to compare psycho-physiological activation during a direct simulation performed in first person with that of a simulation shown in IVR with a 360° video. The underlying hypothesis is that similar levels of stress could be activated in the two conditions. Should this hypothesis be confirmed, we would have an evidence base for using IVR as a training tool in VET for stressful situations.
3. A laboratory study investigating how experiencing a virtual lesson in a 360° video (observation condition) can influence students’ emotions by detecting their stress levels through psycho-physiological measurements (HR and salivary cortisol) and neuroscientific measurements (EEG). The aim of the study is to verify whether there is emotional mirroring between the emotions displayed by a teacher and the emotions experienced by the students. The hypothesis underlying this study is that participants observing a 360° video of a teacher conducting a lesson showing high levels of stress are themselves more stressed than teachers observing an unstressed teacher.

 These three studies adopt a multi-modal approach to evaluate stress (self-reported scales, psycho-physiological and neuroscientific measurements) and are conceived in an ideal progression.

The relevance of this thesis pertains the affordances and impact of 360° video and IVR on teachers' and students' emotions in order to exploit their potential in the world of vocational education, especially for stress management.

References

Balters, S., Geeseman, J. W., Tveten, A. K., Hildre, H. P., Ju, W., & Steinert, M. (2020). Mayday, Mayday, Mayday: Using salivary cortisol to detect distress (and eustress!) in critical incident training. International Journal of Industrial Ergonomics, 78, 102975.

Carneiro, D., Castillo, J. C., Novais, P., Fernández-Caballero, A., & Neves, J. (2012). Multimodal behavioral analysis for non-invasive stress detection. Expert Systems with Applications, 39(18), 13376-13389.

Chu, B., Marwaha, K., Sanvictores, T., & Ayers, D. (2021). Physiology, stress reaction. In StatPearls [Internet]. StatPearls Publishing.

Giannakakis, G., Grigoriadis, D., Giannakaki, K., Simantiraki, O., Roniotis, A., & Tsiknakis, M. (2019). Review on psychological stress detection using biosignals. IEEE Transactions on Affective Computing, 13(1), 440-460.

Gonon, P. (2005). Challenges in the Swiss vocational education and training system. Berufs-und Wirtschaftspädagogik online, 7.

Gonon, P., & St, H. (2002). Arbeit, Beruf und Bildung. Bern: hep.

Kavanagh, S., Luxton-Reilly, A., Wüensche, B., & Plimmer, B. (2016, November). Creating 360 educational video: A case study. In Proceedings of the 28th Australian conference on computer-human interaction (pp. 34-39).

Ladakis, I., & Chouvarda, I. (2021). Overview of biosignal analysis methods for the assessment of stress. Emerging Science Journal, 5(2), 233-244.