Introduction and Executive Summary
Katerina Girginova, Kyle Cassidy, Maxwell Foxman, Matthew O'Donnell, Katie Rawson
Welcome to the first issue of the Social Grammars of Virtuality. This is a new type of digital publication providing a high-level, critical summary of social science extended reality research. the Social Grammars of Virtuality will be published annually and is a free resource to the global community of researchers and practitioners interested in the field. At present, there is a version of the publication in English and Spanish.
We weighed each word in the title carefully. Social reflects the need to better understand the changing social identities and relationships between individuals, communities, organizations, and extended reality technologies. Grammar highlights the fundamental, and often structured role of communication in facilitating social formations and creating meaning. Finally, virtuality underscores the context within which the social interactions and meaning-making takes place. The synergy between these key terms reflects our desire to promote critical scholarship, which may better harness how new technologies are used to shape a more just and sustainable world.
Augmented, mixed, and virtual reality technologies – collectively known as extended reality or XR – offer an evolving user experience driven by our desire to extend the capacities of the human body and to approach worlds beyond our reach. While virtuality itself is nothing new, advances and corporate marketing in the past decade have bought it to the forefront of popular and academic attention.
For simplicity’s sake, we can think of XR technologies on a spectrum of technologically-assisted realities with varying degrees of user interaction. On the one end of the spectrum, augmented reality (AR) technologies enhance our physical surroundings by overlaying graphics and information on top of them. Camera filters on applications like Instagram and Live View on Google Maps are popular examples of AR. On the other end, virtual reality (VR) creates an entirely artificial environment where we are fully visually immersed. VR experiences include gaming where the user puts on a headset to see the application. Lastly, mixed reality (MR) lies in the middle of the spectrum and shares characteristics of both AR and XR. MR overlays graphics on top of a user’s physical surroundings and allows for high degrees of user interactivity with the virtual objects, like in medical operation simulations.
XR technologies are becoming increasingly pervasive in our societies. It is estimated that in 2022 there were over 170 million VR users worldwide (almost 70 million were in the US), and there are some 101 million AR users in the US alone[1]. These populations use XR in the contexts of various trainings, health and medicine, and media and entertainment. As the findings of this report also indicate, XR research and development is a national-level priority field of study in many countries. Yet, social sciences-focused research – that is, work that studies society, it’s interrelationships, and XR use – only presents a fraction (~10%) of this total published output.
We argue that a critical understanding of the social dimensions of XR use is key for creating ethical and effective XR experiences. Questions such as who is using XR tools, why, and to what avail are not secondary to issues of technical development and deployment. In fact, they should be driving it. As communication scholars who also straddle a number of other fields of inquiry, we are particularly interested in critically understanding the ways that we communicate about XR tools and, simultaneously, the ways that these tools and practices allow us to communicate, to form relationships, and to create various social structures.
In turn, this report is one part of a larger project that pursues these aims: the Annenberg Virtual Reality ColLABorative, based at the University of Pennsylvania, which is a lab dedicated to the critical and creative study of XR technologies within society. We hope that this report presents a valuable step toward bringing communication and, broadly speaking, social science voices to the fore of XR conversations in society – and to do so in an accessible way. While social science XR research presented a fraction of the total output of XR research in 2022, this still amounted to almost 1,500 peer-reviewed research papers, a significant amount for even the most avid reader. Thus, The Social Grammars of Virtuality becomes the first publication of its kind to provide a systematic review of the literature through:
- A knowledge mapping review of the intellectual, cognitive, geographic, and organizational structure of XR social science research based on all available, peer-reviewed publications in English in 2022.
- A high-level, critical summary of XR research within social sciences from three core academic journals: Frontiers in Virtual Reality and Virtual Reality and Presence: Virtual and Augmented Reality, plus additional sources. (See Methodology section in the Appendix for more details.)
- A discussion of new ideas and approaches to the study of XR.
This report is intended as a tool for researchers and practitioners wishing to better understand the social fabric and communicative dynamics around XR experiences. It contributes to broader, global efforts to develop our collective understanding, vocabulary, and agency to actively intervene within the practical and intellectual developments of XR in society. This report is also intended as a valuable supplement to our cousins in the fields of medical and technical XR research, who have been producing and benefitting from various systematic reviews for years.
Each of the following sections may be read on its own to give a quick, high-level overview of developments in a particular area such as XR theory. The sections may also be read consecutively by the reader wishes to capture the synergy between them. Below, we share five key findings and three recommendations based on our systematic analysis. We look forward to the conversations that will ensue.
Key Findings
- Globally, many more people in 2022 experienced AR technologies on a daily basis than VR technologies. Yet, VR research dominates the academic literature.
- While different geographical regions exhibit some variation in the research undertaken, in general, XR research significantly lags behind technological developments.
- XR, and specifically, VR research is frequently studied from a cognitive psychology perspective and framed around several key terms: immersion, presence, and embodiment. Yet, these terms are often taken as static concepts rather than as ideas to critically interrogate, which limits the types of new knowledge that can be produced.
- XR media consumption often requires new and higher degrees of physicality from its user than previous media forms. For example, virtual tours require their user to move around a space and virtual sport skills training demands the performance of physical movements. Capturing and describing this type of physicality – a new type of audience experience – requires researchers to develop nuanced approaches and vocabularies.
- Major companies like Meta, the global leader in VR hardware, have significantly impacted the popular creation, consumption, and imagination of XR media (often, by strategically limiting development).
Recommendations for XR researchers and practitioners
- Be specific in our definitions and intentional in our use of key terms that define XR experiences and research methodologies. For example, measuring how a user feels present in a VR experience first requires a very clear and explicit understanding of what presence is (or is not), and how it is achieved.
- Adopt more user-centric methods of analysis. Specifically, to include deeper, qualitative analyses of XR engagement that consider diverse user experiences (age, gender, demographics).
- Ethically expand our vocabulary of XR media and experiences. At present, research often subscribes to terms popularized by the corporate world, like colonization, new frontiers, and the metaverse. This type of language frequently perpetuates capitalist regimes of surveillance and invisible labor.
Author Biographies
Katerina Girginova, Editor of the Social Grammars of Virtuality series, is Co-Director of the Annenberg Virtual Reality ColLABorative. She is interested in how global media, audiences, bodies, and events migrate into various virtual realms. Prior to joining the University of Pennsylvania, Katerina worked in a number of media organizations and enjoys combining creative and critical perspectives in media production.
Kyle Cassidy is Co-Director of the Annenberg Virtual Reality ColLABorative and has been writing about technology since the early 1990’s. He has authored a number of books about computer science and has been a contributing editor to Videomaker Magazine for two decades. He’s won four Keystone Journalism Awards and in 2020 he won the University of Pennsylvania’s Model of Excellence award for his work in Virtual Reality.
Maxwell Foxman is an Assistant Professor of media studies and game studies at the University of Oregon's School of Journalism and Communication. His research focuses on the intersection of games and play in non-game contexts, industries, and professions, including immersive media (VR/AR/XR), esports, gamification, and game journalism.
Matthew O’Donnell teaches applied data science at the University of Pennsylvania’s Annenberg School for Communication, with classes focused on data storytelling, automated content analysis, and generative text AI in relation to Communication theory and method. Matt's research uses methods from computational social science, including natural language processing, machine learning and network analysis. He is interested in combining linguistic analyses of media language and persuasive discourse with behavioral, neuroscience, and XR approaches.
Katie Rawson is Director of Library Services and Operations at the University of Pennsylvania’s Annenberg School for Communication. She’s held positions in digital scholarship, learning innovation, and publishing. She has published on data curation, academic collaboration, computational methods, and food culture. Whether studying data models or short-order cooks, her research focuses on ways of knowing.
Zippia. "25+ Amazing Virtual Reality Statistics [2023]: The Future Of VR + AR" Zippia.com. Feb. 27, 2023, https://www.zippia.com/advice/virtual-reality-statistics/ ↑