Lately, we have been hearing a lot about the metaverse. Although this concept is not new, it has been brought to the limelight because of the huge bets and brand name changes from the potential big players in this space. However, the metaverse that is talked about today is different from what the current state of the internet offers and there are some key technological developments that are drivers of the metaverse and different companies are equipped with different strengths in order to join the competition.

Avatars and game environments have been around for quite some time, especially in immersive video games. The metaverse is much more than avatars or fancy immersive experiences as in video games. It is a platform, rather than a stand-alone application. It is a platform where several applications can be plugged in and get the benefits of the platform. This might be one of the probable reasons a social media messaging company recently rebranded itself. The new brand could mean a platform much bigger than social media, where social media would be just one of the components that could be plugged in. The metaverse combines AR (Augmented Reality) with communication and collaboration tools and in some cases with the help of VR (Virtual Reality) that results in an immersive, photorealistic communication, collaboration and entertainment experience. Many technological developments that happened over the course of the last decade are key to the big bets on the possibilities of metaverse.

The main technological advancements could be grouped under the following categories.

Graphics Hardware (Semiconductors) Computer processors have not just packed more and more transistors but the capabilities of graphics hardware have grown by leaps and bounds the last decade. The semiconductor process technology, new memory technologies, new functions and features driven by deep learning applications, real-time ray tracing, etc. have enabled a new era of computation possibilities at scale that until recently required expensive servers running over long periods of time in order to achieve the same computation. A big part of this is driven by semiconductor process technology (32nm in 2009 to 5nm in 2020) that enabled packing several transistors in a small area (of the die). Similar advances in memory technologies (3D stacked DRAM, advances in DDR as well as HBM and other new memory technologies) helped in increasing the on board storage by a large factor, as required by heavy graphics applications. Similar advances have also resulted in packing enough computation in small form factors like mobile phones, security cameras and other IoT devices. It has resulted in our ability to play games with a reasonably good graphics on our mobile phones. It has also enabled IoT devices to perform data analytics real time as and when they collect sensory data on the field.

Graphics Software (VR, AR, Photogrammetry, etc.)

If hardware was all that was needed, it would have been a commodity. The software layer brings in the innovation that software companies constantly compete to better each other at. For example, irrespective of intel or M1 processor the software layer should ensure that the applications work seamlessly across hardware configurations and operating systems. The more tied the hardware is to the software layer, more performance can be squeezed out of the hardware. Many of the graphics applications need a tremendous amount of compute power. This is especially true for photorealistic graphics. Even given the advances in semiconductors, especially graphics processing units, graphics software innovations require a mix of technology, science and art. While most software companies excel at technology, few excel in combining the three. Graphics components are relatively easy to create, but rendering photo-realistic graphics in real time in different form factors is not straight forward. Advances in AI techniques like Deep Learning Super Sampling (DLSS) and post-processing techniques like Temporal Anti-Aliasing (TAA) techniques have helped tremendously. Techniques like photogrammetry play a vital role in creating digital twins of artifacts, besides 3D cameras and effective and scalable stitching algorithms. AR also involves merging the physical world (camera sensed) augmented with layers of graphics content. Since motion of objects is the undoubted constant in all VR and AR experiences (as anyone who has watched YouTube 360 videos or played with an Oculus headset can attest), simulating physics is the next challenge to overcome. Simulating physics, beyond fire, waves and wind in Computer Graphics is a non-trivial task that requires simulating different materials like stones, marbles, snow, etc., besides their interactions with light rays (reflection, refraction, etc.) and rendering caustics. Although today’s graphics is a major leap from a decade ago, metaverse would demand a much higher quality of photorealistic real-time graphics.

Internet, 5G and streaming games

Metaverse has the potential to change collaboration and media consumption in ways internet changed marketing and commerce. The advances in the same technologies (web2.0, fiber optics, mobile internet, 3G, 4G) that propelled internet adoption would also catalyze technologies like metaverse. Over the last few decades, internet has completely transformed how we consume media content (streaming media, social media, etc. compared to print and television) and how we collaborate (Webex, Zoom, Google Docs, etc. compared to telephone calls and fax messages) increasing both efficiency and effectiveness. The internet infrastructure already being able to stream online games and movies and constantly improving bandwidth and connectivity, companies can safely bet on further improvements that would support applications in the metaverse. New and upcoming technologies like 5G would bring bandwidth intensive applications like gaming within reach for mobile consumers. As adoption increases there would be new opportunities for businesses with new and innovative business models emerging out of it, much in the same way internet created a wide array of marketplace business models (Amazon, Uber, Doordash, etc.). Many doubted Amazon’s and Microsoft’s potential at the beginning of the internet era. The companies that do not see the potential of the metaverse might get left behind when the potentials are realized by the leaders.

Social Networks

The current decade has seen an interesting growth in content creation. The growth of social media enabled everyone to interact and showcase their talent. Specifically, apps like Facebook, Twitter and TikTok made it possible for independent content creators and small businesses to reach and market their products and services effectively in a cost-efficient manner. Gaming is inherently a social activity. The more immersive or innovative games are, the more attention they get (e.g. Pokemon Go). When we think about it, there are many activities like movies, theatre, live sports, education, etc. that are also social activities, or at least the experience is enriched a lot when done as a social activity. Metaverse could offer these activities in the virtual world - for example, right from your living room in Singapore, you could watch US open, sitting in front-row seats at Arthur Ashe stadium, wearing your VR headset sitting next your close friend who is wearing another VR headset and is sitting in his living room in Sydney. Though we might be inclined to think that popular social media players would have an advantage in metaverse, we could also see new innovative players who could change the game. The bottom line is that for many of these social activities, metaverse would offer a much richer experience than what the current internet offers.

There are many other drivers that might directly or indirectly influence the adoption like enterprise communication and collaboration applications like WebEx or Zoom extended to factory floors rather than just computer screens, virtual tourism and adventure trips, etc. It is difficult to predict if metaverse would succeed but it is definitely going to bring lots of innovations in immersive graphics, simulation as well as hardware architectures.