Commercial Innovation 2023: A Year in Review

Kyle Cassidy

Introduction

The findings in the various sections of this report indicate a close relationship between industry and academia. So this last section is dedicated to describing key market developments from 2023 that may shape our capacities for future knowledge production. It proceeds with a selected overview of some of the important XR industry developments in the past year, covering virtual reality (VR), augmented reality (AR), Artificial Intelligence (AI), and digital twins.

Evolution of XR Headset Market Share

Since Q4 of 2020, Meta has dominated the XR headset market and by Q4 of 2023 Meta remain in the lead with a 72% dominance. However, in 2023 Sony also (re)entered the market and captured a significant share; between 15-33% depending on the quarter. See Statista (2024) for a graph depicting market share distribution. It will now be interesting to track market shifts in 2024 given Apple’s entry into XR headsets.

Developments in Virtual Reality and Augmented Reality

VR hardware like headsets or head mounted displays (HMDs) have an overall larger market share than AR hardware, such as smart glasses. The primary market driving sales across both technologies remains at the industry and enterprise level (Grandview Research Report, 2023). However, VR hardware saw some decline in sales in 2023, whereas AR hardware sales largely grew (IDC Report, 2023).

In VR, Meta (aka Facebook) still dominated, but with some cracks starting to appear in the armor. Meta released their Quest 3 headset (a $500 follow-up to both the $1,500 Quest Pro and $400 Quest 2) that took features from both and essentially made them obsolete. While the Quest 3 is thinner, it is half an ounce heavier than the Quest 2 on account of a larger battery (though not longer battery life, a key limitation of all standalone HMDs)^[1]. A heavier HMD may lead to comfort obstacles in the long-term use of the device, and may also limit lengthier user engagement, like watching a full-length immersive film^[2]. Still, there were some interesting developments in the Quest 3 like the inclusion, for the first time, of a Time-of-Flight sensor that uses a laser to measure distances and reduces the need to create manual guardian boundaries for VR room scale use (though at the time of writing, using the automatic guardian isn’t easier to set up than the manually created one).

One of the big, though largely unheralded, stories of 2023 has been a series of end-runs around Meta from indie developers. For example, an indie game modder who goes by the name of Praydog released a Meta hack called “FLAT2VR” on New Year’s Eve that allows nearly any flat screen game created with the Unreal Engine 4 or 5 and available on Steam to be played in VR^[3]. Previously Meta’s software store lacked almost any “AAA” or top of the line experiences, but after FLAT2VR’s release, suddenly as many as 11,000 new PC games became available (caveat: while 11,000 games became available, the gaming community has so far (Feb 2024) tested about 1,000 and found them to work well). In addition, Meta’s big news in September of 2023 was that its avatars would finally get legs instead of just being floating torsos. This decision has been long awaited and ridiculed—by comparison, Horizon World’s social VR precursor of VR Chat has had legged avatars since its 2014 release—yet, was delayed not only on account of technical difficulties but also, as suggested by many^[4], as an attempt to prevent virtual harassment.

Taiwanese electronics company High Tech Computer Corporation (HTC), which had been the main competition for the Oculus since the first Vive VR headset came out in 2013, released new headsets in 2023. Their Vive XR Elite is a premium consumer XR device with a nice form factor, a rear-mounted battery and no top head strap to accommodate different hairstyles (an inclusivity problem that all versions of the Meta Quest have had). One big advantage of the XR Elite Pro is its built-in diopters, meaning almost everybody can use it without their glasses. However, the Quest’s lower price, wider field of view, and higher resolution makes the XR Elite Pro a hard mass market sell.

Bigscreen, whose main product until recently has been the multi-user VR theater that allows people to watch films and videos in a realistic virtual theater, also branched out into physical hardware in 2023. Bigscreen Beyond is the world’s smallest VR headset (which only connects to a PC). At the time of writing, Bigscreen Beyond is the only all-day-wearable VR headset and it is custom-fitted to an individual, making it difficult to share or use in a lab space^[5].

Lastly, dedicated niche AR devices (call them “smart glasses”- you’ll get fewer weird looks) took some big strides into the mainstream since their market inception in 2013 with Google Glass. A number of developers realized that the technology for projecting phone apps onto simple glasses is inexpensive and relatively unchallenging. For instance, the Vuzix Blade has a built-in camera, QR Code reader, and single-eye display, which makes it ostensibly useful for things like teleconferencing, museum use, or presentations. Several companies like XRAI, LEION and Vuzix have also debuted speech-to-text AR glasses, which subtitle the live world around you by recording and translating conversations in real time thus, creating the potential for greater inclusivity in use.^[6]

Artificial Intelligence

One of the challenges in designing large-scale VR environments, or “open worlds,” is the resource intensive process of crafting all the elements in them. For example, in creating a hypothetical VR simulation of Revolutionary War era Philadelphia, designers would need to create interior floorplans for every building, contents for every kitchen drawer, and the temperament of every Tory-sympathizing bar patron among a host of other things. In the past, this was done by individual designers at great cost, thus limiting the output of content and the number of organizations and individuals that could afford to build these environments. However, combining AI with VR has the potential to significantly ease this bottleneck.

Building upon last year’s developments, graphic card maker NVIDIA launched generative 3D content creation with text-to-object AI in 2023, allowing VR creators to easily create, add, and manipulate 3D objects into VR content as well as a separate AI application (called move.ai) for animating 3D objects. Mobile application newcomers Luma Labs released a similar product, Geni, for creating 3D objects for AR in 2023, and Unreal Engine released a procedural content generation framework, which uses AI to provide an enhanced toolkit for users. NVIDIA also announced that with 3D face scans they can create mesh framing for easily replicating 3D versions of people^[7]. Still, this process is in its infancy and comes with a range of ethical concerns. A similar model, Google’s Gemini AI, recently received wide criticism and was temporarily suspended for generating ethnically diverse Vikings and Second World War German soldiers (The Guardian, 2024).

Sample image from NVIDIA of generative AI-created 3D animals which now take seconds to create and export. Image Credit: NVIDIA Developer Blog, 2023.

Still, the synergy between AI and VR is useful not only for world-building but for more mundane tasks like upscaling and creating spatial video out of ordinary flat video, squeezing more life out of existing retro footage and potentially expanding its applications.

Digital Twins and Virtual Positioning Systems

2023 saw significant developments in the technology for capturing and recreating real spaces. Niantic’s “Lightship Visual Positioning System” (VPS), launched in 2023, is a Visual Positioning System which provides GPS-style coordinates for augmented reality mapping in the real world^[8]. This is helpful in creating a real world metaverse, allowing for the persistent positioning of virtual objects for multi-user experiences. For instance, dozens of people can look at the same thing at the same time and objects can be shared and manipulated (think virtual art, signage, or even recreations of structures that no longer exist in a space). Lightship is primarily envisioning their VPS as a framework for games, but it would be useful for an array of virtual object that could replace a physical object, particularly in places where putting physical signs is not desirable—for example, virtual lines drawn on the ground to lead you around a historic site or place markers to show habitats of animals.

AR devices, such as smartphones, use cameras to anchor images to local environments, but in order to anchor an AR experience on Earth as opposed to, say, your living room—and for that experience to be persistent and visible to multiple users at the same time—a worldwide positioning system becomes necessary. Lightship claims to have “centimeter precision” (something GPS alone is incapable of). Lightship does this by meticulously mapping certain places they call “wayspots” and anchoring them to one another, which means that practically the system will work more efficiently in some places than others (much as Google Earth has significantly more data around large cities and densely populated places than it does rural areas). As more wayspots get created, the map gets bigger and more accurate.

While Unity stepped back in digital twin hosting, Microsoft Azure pressed ahead with Azure Digital Twins, a cloud-based Internet of Things platform that allows for the creation of digital spaces mimicking real world 3D environments—useful for helping to create a smart building or even city and to analyze data from smart sensors in and around that area to create a visual way to see. For example, you could measure how the opening and closing of blinds on various sides of a building will regulate its temperature. An entire digital twin city might use sensors to monitor traffic flows to help predict and avoid traffic congestion or to slow traffic down at times and places that are particularly dangerous to pedestrians. Yet, developments in digital twin technologies also increase the ability for and concern around surveillance. This work requires massive amounts of data collection and even more massive amounts of storage and processing power, which is a space that Microsoft Azure has carved out.

Lastly, Neural Radiance Field technology (or NeRF) released smartphone software for producing real-time, full volumetric, photorealistic NeRF rendering on the web, which can use images captured not just from latest model iPhone, but even brief drone video flybys. This allows for areas and objects to be very quickly captured from a variety of viewpoints and it uses AI to fill in missing spaces – a boon to anyone studying 3D objects, places or environments.

References

Statista (2024, April 4). “Extended Reality (XR) Headset Vendor Shipment Share Worldwide from 2020 to 2023, by Quarter”. Statista.com: https://www.grandviewresearch.com/industry-analysis/augmented-reality-market
The Guardian, 2024 https://www.theguardian.com/technology/2024/feb/22/google-pauses-ai-generated-images-of-people-after-ethnicity-criticism
IDC Report, 2023: https://www.idc.com/getdoc.jsp?containerId=prUS51574023

1. See: https://arvrtips.com/meta-quest-3-vs-pro-vs-2-vs-1/ ↑

2. The capacity for full-length film was one of the main factors in the success of VHS over the technically superior Beta video format. ↑

3. See: https://arvrtips.com/uevr-praydog-flat2vr-mod/ ↑

4. See: https://www.bbc.com/news/technology-60247542 ↑

5. See: https://www.engadget.com/bigscreen-beyond-review-gunning-for-vr-throne-valve-index-190004793.html↑

6. See: https://www.hearingtracker.com/resources/hear-with-your-eyes-five-ar-live-captioning-glasses↑

7. More about some of NVIDA’s 3D content generation here: https://blogs.nvidia.com/blog/omniverse-accelerates-game-dev/↑

8. More on Lightship here: https://lightship.dev/products/vps/ ↑