Reality in a sphere
VR for social attention research
Jonas D. Großekathöfer
24. June 2021
Hi! 👋
I’m Jonas
PhD Student at the University of Würzburg (Germany)
Research on social attention in a wide variety of environments
— —
Outline
Aim of the Talk
- VR has great potential for social attention research
- It might be easier then you think to start with VR in your research
Challenges for social attention research
A lot that we know about …
- Cognitive aspects of gaze following
- Preferential processing of social aspects
… does not generalize beyond the laboratory!
Differences between the laboratory and the real world
- distribution of gaze
- allocation of social attention
- influence of personality traits
No naturalistic stimuli in the laboratory
- Rich, dynamic, multimodal
- Contextual embedded
- Unrestricted head and body movement
- Free attentional flow
- (Social) interaction
Solution: Virtual reality?
- What is virtual reality?
- How can it help?
- How can it be implemented?
What is virtual reality?
a simulated experience that can be similar to or completely different from the real world
How is it related to human experience?
How is it related to human experience?
How can it help?
Naturalistic stimuli or experimental control?
How can it help?
Naturalistic stimuli and experimental control!
Naturalistic stimuli with VR
- Rich, dynamic, multimodal
- Contextual embedded
- Unrestricted head and body movement
- Free attentional flow
- (Social) interaction
And VR enables all of them!
Technical implementation
HMD (Head-mounted Display)
3D scenes
Spherical scenes
 |
 |
Disney’s “The Jungle Book”
Through Mogli’s Eyes
Disadvantages of spherical scenes
- No interactions
- Physical limitations
- No stereoscopy
- Monocular stimuli
Advantages of spherical scenes
- Make VR accessible
- easy and fast stimulus preparation
- very naturalistic stimuli
- including photorealism
VR for social attention research
Reality in a sphere: A direct comparison of social attention in the laboratory and the real world
Motivation
- Suitability of spherical videos for investigating social attention
- Better understanding of the boundaries of typical laboratory settings
Typical laboratory settings
- Restrict head and body movement
- No free decision where to attend
- Introduction of additional tasks in the real world
What did we do?
- Measured eye movements of 44 participants
- in Real environments physically at the location
- in Virtual environments within the laboratory, playing videos of the locations in an HMD
- Manually coded fixations into person, object, background
Locations
Spherical videos
Spherical videos
Eyetracking
Eyetracking
Results
🎯
General findings
Consistency of fixations
Consistency of fixations
Distribution of fixations
Distribution of fixations
Influence of Pedestrians
Influence of Pedestrians
Conclusion
| ✅ | Photorealistic representation of the real world |
| ✅ | Free decision where to attend in the laboratory |
| ✅ | Free head and body movement in the laboratory |
Conclusion
- Reduced social attention in the real environment
- Social attention is more stable across locations in the virtual environment
- Gaze is more concentrated and above a relative horizon in the virtual environment
Limitations
| ❌ | No interaction in the laboratory |
| ❌ | No body representation |
All in all
- Unmatched close and more systematic approximation of the real world within the laboratory
- Valid and reliable measurements of social attention in the laboratory remain challenging
Wrap up
- spherical scenes make VR comparatively easy to access for social attention research
- solves some problems, when embeddedness, free attentional flow, photorealism is important
- does not solve every problem, interaction and body representation remain a hard problem
Future directions
- improve multimodality
- explore possibilities with orchestrated environments
- interactive location changing
- enable social interactions (up to a certain degree)
- think about clinical applications
Thank you for your attention
🙂
References
Literature:
- Großekathöfer, J. D., Seis, C., & Gamer, M. (2021). Reality in a sphere: a direct comparison of social attention in the laboratory and the real world. http://dx.doi.org/10.31234/osf.io/evn2d
- Kleinke, C. L. (1986). Gaze and eye contact: a research review. Psychological Bulletin, 100(1), 78–100. http://dx.doi.org/10.1037//0033-2909.100.1.78
- Emery, N. J. (2000). The eyes have it: the neuroethology, function and evolution of social gaze. Neuroscience and Biobehavioral Reviews, 24(6), 581–604. http://dx.doi.org/10.1016/S0149-7634(00)00025-7
- Blascovich, J., Loomis, J., Beall, A. C., Swinth, K. R., Hoyt, C. L., & Bailenson, J. N. (2002). Immersive virtual environment technology as a methodological tool for social psychology. Psychological Inquiry, 13(2), 103–124. http://dx.doi.org/10.1207/s15327965pli1302_01
- Lee, K. M. (2004). Presence, explicated. Communication Theory, 14(1), 27–50. http://dx.doi.org/10.1111/j.1468-2885.2004.tb00302.x
- Foulsham, T., Walker, E., & Kingstone, A. (2011). The where, what and when of gaze allocation in the lab and the natural environment. Vision Research, 51(17), 1920–1931. http://dx.doi.org/10.1016/j.visres.2011.07.002
- Jarick, M., & Kingstone, A. (2015). The duality of gaze: eyes extract and signal social information during sustained cooperative and competitive dyadic gaze. Frontiers in Psychology, 6, 1423. http://dx.doi.org/10.3389/fpsyg.2015.01423
- Rubo, M., Huestegge, L., & Gamer, M. (2019). Social anxiety modulates visual exploration in real life – but not in the laboratory. British Journal of Psychology, http://dx.doi.org/10.1111/bjop.12396
Online resources:
- Wikipedia contributors. (2021, June 4). Virtual reality. In Wikipedia, The Free Encyclopedia. Retrieved 14:12, June 8, 2021, from https://en.wikipedia.org/w/index.php?title=Virtual_reality&oldid=1026859577
- https://gopro.com/en/us/news/omni-is-here
- http://doc-ok.org/?attachment_id=33
- https://reference.codeproject.com/book/dom/webvr_api/webvr_concepts
- https://www.psychologie.uni-wuerzburg.de/en/bioklin/workgroups/vr-labs-prof-dr-paul-pauli/3d-multisensory-lab-cave/