eXtended Reality: Policy Landscape
The European Commission this week announced a strategy to “steer the technological transition” to a future where virtual worlds “allow an integration between digital and real objects and environments, and enhanced interactions between humans and machines.” Last year we had the privilege of doing a bit of work on policy issues surrounding virtual worlds, and specifically the near-term extended reality (XR) technologies – comprising virtual reality (VR), augmented reality (AR), and related technologies. In the intervening months, collective attention, as it does, has shifted to the next big thing: AI. Like just about everything else, AI will have an impact on the XR policy landscape, but XR itself remains poised to challenge the traditional tech policy landscape in a number of ways.
How is XR different?
This new era of computing interface is different from what came before because of three characteristics: immersion, presence and embodiment. Unlike computing interfaces such as the mobile phone, laptop, and smart watch, Extended Reality (XR) aims to more fully immerse the user in a sensory experience that is different from their actual physical surroundings; it gives the impression that they are fully present in interacting with that sensory experience, that their physical movements affect the surrounding experience; and it provides sensory feedback that fully embodies a user in that experience. Where previous interfaces allowed real-time, multi-user engagement around 2D content and conduct, XR does so in 3D, leading to evolutions in how harms emerge: harassment can take on characteristics of a physical assault; privacy violations can take on characteristics of manipulating your sensory experience; and “social media addiction” may lead to measurable physical side effects and physiological consequences.
As expert Nina Jane Patel observed, XR enables us to “dance with pixels”, not just interpret them. Industry has understood for at least a decade just how powerful the effect of this immersive, embodied presence is. As early as 2013, the technology was advanced enough to produce it — a Wired journalist wrote back then about his experience standing on and then walking off of a plank in VR, which he describes as “stomach churning.” The other popular demo at the time was a beheading demo, just to illustrate the extreme uses to which this technology might be put. Videos of today’s parallel demos leave little room for imagination when it comes to how much more real the experiences seem today.
There remain a number of technical challenges to fully realizing the vision of XR enthusiasts. Matthew Ball’s Metaverse provides an excellent overview of these for those keen to deeply understand the limitations of current technology in this space. But at a high level, one example Ball offers illustrates the issue well: Microsoft Flight Simulator. The video game in 2020 had a square kilometer map equal in size to the entire planet, as well as renderings of two trillion unique trees, 1.5 billion buildings, and nearly every road in the world. It was 2.5 petabytes in size, too big for consumer devices (even computers, let alone today’s VR headsets) to store locally, and required a complex caching solution, and adequate Internet bandwidth, to facilitate game-play. This example illustrates the limitations of today’s bandwidth, consumer devices, and data storage for facilitating at-scale, 3D interactive XR — but doesn’t even address the layers of additional requirements imposed by the real-time multi-user interactivity envisioned by XR enthusiasts.
One might wonder why, if the technology still has so many limitations, policymakers and researchers are already and should continue to pay attention to XR. For one thing, several experts who have gotten their hands on a Vision Pro device relay an experience that is meaningfully different from devices that came before. For another, sales of these devices suggest market adoption even in this most nascent stage: Meta’s Oculus has sold over 20 million devices since 2020.
Isn’t this just gaming?
While it may seem that the commercial use case for XR would primarily be gaming, there are a number of other applications that may have much larger impact over time. We touch on just a few here.
Healthcare: As detailed below, XR is already being evaluated as a potential treatment modality for a variety of conditions, including PTSD and back pain. It is also a technology that will be deployed in healthcare settings and change the dynamic of providing care. The FDA has already authorized an XR device to help surgeons do their jobs better, and Johns Hopkins has been using XR in live spinal surgery for over a year.
Education and Training: XR has already been widely deployed in professional training, with Accenture thought to have done the largest deployment to date. Other educational institutions including Harvard have also piloted XR for training purposes, and startups have emerged aiming to provide immersive educational experiences for students.
Accessibility: A compelling use case for XR is deploying it to assist people with disabilities. For example, today you can purchase glasses that use XR to subtitle reality as it happens, which has everyday uses but tremendous potential upside for the hearing-impaired population.
These are just a few of the applications we find most exciting, that demonstrate the range of use to which XR will be put. McKinsey comprehensively scanned the economy for potential impacts from XR, and found the sectors most likely to be impacted include banking, manufacturing, retail, professional services, and telecom/media. As their report describes, XR is already here and in use by a range of early adopters, and its impact is likely to be felt broadly over time.
What should policymakers and researchers focus on?
In some sense, the range of policy issues raised by XR mirror the range of policy issues in our everyday lives, and can thus feel all-encompassing. In our work there were a few issues that really stood out.
Trust & Safety (T&S)
While addressing harmful content and behavior is nothing new to technology providers, the operational challenges of doing this in XR make it a different type of challenge than, for example, social media has presented.
T&S Tooling: One glaring gap in an XR company’s ability to tackle T&S challenges is the dearth of tooling solutions available to address things like 3D rendered objects, or to analyze real-time interactions and content. There is a need to enhance tooling for T&S operations generally, as we’ve covered in other work, but even the real-time tooling needs of a social media platform like Twitch are incomparable to the requirements XR companies will have. Real-time voice, imagery, and even embodied interactions will need to be moderated, and across any number of simultaneous interactions that are 1:n between users.
Community & User-led Moderation: Services may choose to embrace the community moderation – enabling people to set and enforce rules of appropriate content and conduct in parts of an XR-enabled environment. Doing so would present tooling requirements distinct from those needed for internal company operations.
Privacy
The privacy challenges raised by XR are complex enough that a paper by the European Parliamentary Research Service suggests a revision of the General Data Protection Regulation may be needed. Considering a revision was formally adopted only in 2018, this is a remarkable suggestion.
Updated rules may be needed: A handful of experts have begun analyzing the privacy challenges posed by XR in detail. In particular, Brittan Heller’s work on biometric psychography warrants a close read and details the myriad of ways in which data collection and analysis will be transformed in XR environments, and require new approaches to data protection.
Opportunity for new controls: Engineers have also begun evaluating technical solutions to give users more enhanced control in these environments. For example, Louis Rosenberg describes a tool researchers are building that is designed to shield a range of inferences — from geolocation to the frequency range of one’s voice — from third parties by injecting randomized offsets into the data stream.
Competition & Interoperability
Concerns around competition and monopoly will arise with XR, recalling familiar debates that have arisen around video game systems, app stores, and social media platforms. At each layer of the XR technical stack, there will be opportunities for a company to create a bottleneck. While the challenge of fostering open and interoperable systems is familiar, XR may intensify the issue. Many visions of XR’s future involve it intersecting with every aspect of our daily lives; that is, the promise is the digital world seamlessly weaving into the physical world. If that’s the case, then the stakes are much higher.
Interoperable Standards: Nascent standards efforts have kicked off, including the Metaverse Standards Forum and the Open Metaverse Alliance, both demonstrating the industry’s recognition that interoperability and open standards will play a role in development of the XR ecosystem. The European Commission has put an early stake in the ground in favor of “shaping global standards for open and interoperable virtual worlds” as a key strategic pillar in its virtual worlds work.
Crypto: Many in the crypto community believe that the underlying blockchain technology will be a key component of building an interoperable XR environment. Many believe that NFTs may enable ownership of virtual assets in a way that more closely approximates true ownership than for example licensing has allowed for digital content to date. There are also enthusiasts who believe it will be possible to build a likely-crypto-enabled identity that can be ported from one platform to another.
Health & Wellbeing
Perhaps because it differs the most from traditional policy issues arising from digital media, the health and wellbeing issues and opportunities raised by XR appear to us to be the most rich. XR technologies interact with the human body in a materially different way than the 2D media that came before, and facilitate an embodied interactivity with vast implications for emotional and physical wellbeing.
Social and emotional wellbeing: For instance, some have hailed VR as an “empathy machine.” Professor Bailenson describes experiments where people experienced inhabiting the virtual selves of homeless people or other races and then felt greater empathy for others, and many other anecdotes exist regarding immigrants, victims of natural disasters, and much more. Despite these promising results, hype can outrun data, and, in a recent law review article, Professor Mary Anne Franks raised important questions about how virtual environments might activate biases, traumatize users (by immersing them in another person’s painful experience), or create “compassion fatigue.” Further research could help bring out the true promise (and peril) of this use of XR.
Addiction concerns: In recent years, there has been an increase in the number of experts asserting that social media in particular has a negative impact on mental health, and especially youth mental health. XR will intensify this line of reasoning, especially as XR represents the convergence of social media with online gaming — gaming being another segment that has been criticized for its effects on teen mental health for many years and around which a diagnosable addiction has actually been recognized by medicine.
Medical treatment modality: XR is also poised to be the next darling of medical device technology. The FDA has already invested significant resources into evaluating the safety and efficacy of XR devices. The regulatory agency has granted preliminary authorizations to devices designed to treat specifical medical conditions, including chronic low back pain, schizophrenia and other mental health disorders, as well as XR devices designed to assist physicians in the surgical room and assist in educating future physicians.
Creative Economy & Labor
Creators will be critical to the development of a vibrant XR ecosystem. And as we saw with the emergence of the Internet, there will undoubtedly be disagreements about “ownership” and “earning” rights in various forms of XR content and experience.
Copyright, XR, and Generative AI: While issues around piracy will no doubt arise in XR as they have on the Internet, the most interesting and novel issues arise around AI-generated content. One can imagine generative AI playing a sophisticated role in XR environments, allowing people to start to tell a story or describe a setting and then, automatically, have a rich visual and auditory representation created in real-time. There will be questions about who, if anyone, should own that creation, or whether it would be worthwhile to facilitate a broader commons with open access to information. Already, scholar and advocate Paul Keller has raised important questions about whether copyright is the tool that best serves society’s interests or whether it would mean that the “very environments that we spend increasing amounts of time in will be governed by an inflexible set of legal rules.”
Earnings in XR: Play-to-earn games offer a glimpse of how labor economics might emerge in XR environments. These games enable players to earn cryptocurrency in exchange for playing a game. The most well-known of these is Axie Infinity, which during the pandemic was credited as being a source of income for “workers” across the global South in particular. Players earn cryptocurrency that doubles as a governance token for the game, nominally giving them an ownership stake and vote on the future of how the game develops. But as the game gained in popularity, the barriers to enter got higher, and many players wound up joining “Guilds” to get started, enabling early entrants to engage in rent-seeking and exploit the newer “workers.” Some experts even theorize that an XR-enabled version of a play-to-earn game would have lower wage workers playing the game to fill out “roles” in other players’ canvases — the wealthier players seek entertainment, while other players are primarily playing to earn.
Infrastructure
XR technologies may demand a massive increase in broadband infrastructure and deployment of improved mobile networks. This will reignite infrastructure policy debates along three key themes.
Competition & Network neutrality: Telecom companies have long sought to impose new charges on content and application companies, while public interest groups and Internet platforms have argued these charges would harm speech and innovation. These familiar arguments have already been picked up in Europe in the context of XR.
Spectrum: Expanded access to spectrum for wireless services will be important to facilitate real growth in XR. Just as unlicensed spectrum, which anyone can use but no one owns, enabled competition to flourish in Wi-Fi based devices and services, unlicensed spectrum will be critical to XR. At present, standards for gigabit Wi-Fi are already available, but regulations are holding them back due to fears of the potential impact on licensed providers.
How and Where to Build: XR will raise interesting questions around whether networks and their attendant devices might be provisioned differently in order to create the sort of high-quality, pervasive mobile networks necessary for XR. Perhaps this is an opportunity for municipalities, or user communities facilitated by companies like Helium, to build and control parts of their digital infrastructure.
