Transdiagnostic Mechanisms for Mental Disorders:
A Mixed Reality System for the Assessment of Implicit Social Learning

   
Project summary
Project no.: PN-III-P2-2.1-PED-2019-4599; Contract no.: 347PED/2020
Principal Investigator: Adrian Opre, Babeș-Bolyai University
Co-Principal Investigator: Radu-Daniel Vatavu, Ștefan cel Mare University of Suceava
Funded by UEFISCDI, Romania
Funding scheme: PNIII P2 - Increasing economic competitiveness through research, development and innovation (PED)
Running period: August 2020 - August 2022 (24 months)

Abstract

The goal of this project is to develop a Mixed Reality (MR) experimental model, validated at the TRL-4 level, to facilitate the evaluation of a novel and promising transdiagnostic mechanism - Implicit Social Learning (ISL) - for several mental disorders, such as Depression and Autism Spectrum Disorders. ISL refers to the unconscious acquisition of complex regularities that govern the social environment. Prior research from Cognitive Science has demonstrated that the inferences people make about other people's mental and/or emotional states are largely sustained by regularities that are being extracted unconsciously over time and across social interactions. Due to the centrality of this process for adaptive social functioning, impairments in ISL have been repeatedly inferred as a transdiagnostic etiopathogenetic mechanism involved in several disorders, such as ASD and Depression.


Concrete objectives

We are developing a TRL-4 experimental model for the purpose of assessing Implicit Social Learning. Our concrete objectives are:
  1. Design and development of a Mixed Reality application to support evaluation of ISL
  2. Evaluation of ISL in the general population
  3. Evaluation of ISL in persons with Depression and Autism Spectrum Disorders

Expected results

The main outcome of this project will be an ecological assessment procedure and tool for Implicit Social Learning in the form of a TRL-4 experimental model. Associated expected results are:
  1. Mixed Reality application to support evaluation of ISL
  2. Experimental results
  3. Scientific publications in high-quality conferences and journals and yearly research reports

Team

  1.  Prof. Adrian Opre, Principal Investigator
  2.  Prof. Radu-Daniel Vatavu, Co-Principal Investigator
  3.  Dr. Răzvan Jurchiș (Psychology)
  4.  Andrei Costea (PhD candidate in Psychology)
  5.  Cristian Pamparău (PhD candidate in Computer Science)

Publications

  1. Cristian Pamparău, Andrei Costea, Răzvan Jurchiș, Radu-Daniel Vatavu, Adrian Opre. (2022). Experimental Evaluation of Implicit and Explicit Learning of Abstract Regularities Following Socio-Emotional Interactions in Mixed Reality. Proceedings of DAS'22, the 16th International Conference on Development and Application Systems. IEEE, Washington, D.C., USA, 150-154

    We present results from a controlled experiment with N=47 participants conducted in a mixed reality environment to assess explicit and implicit learning of cognitive structures instantiated by socio-emotional components. To this end, we implemented a custom version of MR4ISL, the Mixed Reality software tool for Implicit Social Learning, with a task involving colors, numbers, and emotions. Our results show evidence of explicit learning with participants’ responses being attributed to conscious response bases, rules, and memory.

    Pamparău et al. (2022)
    PDF | DOI | WOS: FORTHCOMING
  1. Cristian Pamparău, Radu-Daniel Vatavu, Andrei Costea, Răzvan Jurchiș, Adrian Opre. (2021). XR4ISL: Enabling Psychology Experiments in Extended Reality for Studying the Phenomenon of Implicit Social Learning. Proceedings of MUM'21, the 20th International Conference on Mobile and Ubiquitous Multimedia (Leuven, Belgium). ACM, New York, NY, USA, 195-197

    We present XR4ISL, an XR system designed to support psychology experiments examining Implicit Social Learning, a fundamental phenomenon that guides human behavior, cognition, and emotion. We discuss XR4ISL with reference to MR4ISL, a previous system designed for Mixed Reality only, and reflect on differences between Mixed and Virtual Reality for psychology experiments.

    Pamparău et al. (2021)
    PDF | DOI | WOS: 001112164200025 | ARC B (CORE 2021)
  1. Cristian Pamparău, Radu-Daniel Vatavu, Andrei Costea, Răzvan Jurchiș, Adrian Opre. (2021). MR4ISL: A Mixed Reality System for Psychological Experiments Focused on Social Learning and Social Interactions. Companion of the 2021 ACM SIGCHI Symposium on Engineering Interactive Computing Systems (Virtual Event, Eindhoven, Netherlands). ACM, New York, NY, USA, 26-31

    We present MR4ISL (Mixed Reality for Implicit Social Learning), a HoloLens application designed to examine the psychological aspects involved by implicit social learning, a key process responsible for information acquisition at an unconscious level that influences humans' behavioral, cognitive, and emotional functioning. We describe the engineering details of MR4ISL, present our roadmap for identifying technical solutions for believable animations of virtual avatars relevant for implicit social learning, and exemplify use cases of MR4ISL that emerged from discussions with three researchers from psychology. To date, MR4ISL is the only tool that uses mixed reality simulations to increase the external validity of psychological research in the study of implicit social learning.

    Pamparău et al. (2021)
    PDF | DOI
  1. Cristian Pamparău, Radu-Daniel Vatavu. (2020). A Research Agenda Is Needed for Designing for the User Experience of Augmented and Mixed Reality: A Position Paper. Proceedings of MUM '20, the 19th ACM International Conference on Mobile and Ubiquitous Multimedia (Virtual Event, Essen, Germany). ACM, New York, NY, USA, 323-325

    With this position paper, we want to draw the attention of the community toward theoretical work and practical opportunities that have been overlooked so far regarding concepts, principles, and design knowledge for the user experience of Augmented and Mixed Reality content, I/O devices, interactions, applications, and systems. Despite considerable innovations in commercial products and research prototypes enabling Augmented and Mixed Reality worlds, how to design great user experiences in such worlds has been overall neglected at core, while the information and knowledge currently available to practitioners cover usability aspects mostly. Therefore, we advocate for theoretical foundations of the user experience in Augmented and Mixed Reality and propose several directions for more scientific research in this regard.

    Pamparău & Vatavu (2020)
    PDF | DOI | ARC B (CORE 2021)

Media

Project reports (in Romanian)

  1. Scientific report 2020 PDF

Other resources (software, diagrams, and datasets)

  1. XR4ISL, a software application (Unity, HTC Vive) designed to support psychology experiments examining ISL in MR/VR GitLab link
  2. MR4ISL, a software application (Unity, Microsoft HoloLens 2) designed to support psychology experiments examining ISL in MR GitLab link