Specific Immersive Competence in VR-Based Assessments: Development, Psychometric Evaluation and Associations with Medical Performance (Preprint)

BACKGROUND Virtual reality (VR) is increasingly used in medical education for training and examination purposes, yet learners' performance in VR-based assessments may be influenced by more than clinical competence alone. Immersive competence (IC) has been proposed as a relevant factor in VR-based performance assessment. While general IC captures application-independent VR interaction skills, domain-specific applications may require an additional construct - specific IC - reflecting situational interaction proficiency. OBJECTIVE This pilot study aimed (1) to examine the psychometric properties of a newly developed in situ checklist for assessing specific IC, (2) to investigate the relationship between specific and general IC as well as related human abilities and characteristics, and (3) to explore associations between both IC measures and medical performance in a VR-based OSCE station. METHODS This observational follow-up study was conducted with 21 final-year medical students who had previously completed a curricular OSCE including a VR-based emergency medicine station. General IC was assessed using the VR competence app, and specific IC using a checklist embedded in the original VR simulation. Additional measures included self-reported technological affinity, spatial ability, and OSCE performance scores. Item difficulty, item-total correlations, and internal consistency were calculated for the specific IC checklist. Pearson’s r was used to examine associations among variables. RESULTS The final 13-item specific IC checklist demonstrated acceptable internal consistency (Cronbach’s α = .79) and balanced item difficulty (mean P = 0.56). Specific IC showed a strong correlation with general IC (r = 0.56, p = .008) and with prior 3D application experience (r = 0.57, p = .007), but no relevant association with spatial ability. Both general and specific IC correlated moderately with VR-OSCE performance (r = 0.41, p = .06 and r = 0.37, p = .09, respectively), but neither was related to overall analog OSCE performance. CONCLUSIONS Specific IC appears to be a psychometrically accessible and context-sensitive dimension of VR user readiness. The proposed in situ approach offers a feasible and scalable method to assess this construct within domain-specific VR applications. While correlations with medical performance were limited in this small sample, the findings underscore the importance of considering IC in VR-based high-stakes assessments to support fairness and validity.