Supercharging NFTs with AR and VR

GIF thumbnail created entirely in Virtual Reality using Tvori… now on Hic et Nunc

Over the past years, we have seen many attempts, some more successful than others, to integrate Virtual and Augmented Reality (sometimes referred to as Immersive technologies) into this new wave of digital art. These experimentations took many shapes and forms, and given the fast pace at which both worlds are evolving, I found it complicated to have a clear conversation with NFT artists, collectors, or creators focused on their creative process, artwork, and the way they were communicating it to the outside world. This article tries to bring some structure into this crazy and exciting world by breaking up the gazillion opportunities out there into digestible chunks that can help the community understand how to take advantage of Immersive technologies and why. Collectors, and anybody in this space, will also benefit from this framework as it will help them attach a subjective value based on a deeper understanding of the artwork’s essence.

If you are already knowledgeable about immersive technologies such as AR & VR as well as NFTs, you can just skip to the next section, on the other hand, if you are not familiar or just want a quick refresher, you are more than welcome to keep reading through this short introduction.

An intro to NFTs and Immersive technologies

The RV continuum as described originally by Milgram and Kishino

Since then, a lot has changed: creating content became much easier, hardware became cheaper, and virtual worlds became accessible directly from a web browser. This led to revisiting that taxonomy to include a more user-centric approach and adding more parameters to categorize this blending of real and virtual. Nonetheless, categorizing a piece of content as augmented reality, mixed reality or virtual reality can still be rather challenging.

Mixing medium and realities can be rather confusing

NFT are an even more exciting phenomenon that took the world of digital art by storm. NFT stands for “non-Fungible-token” and, among its many uses, enables the certification of the authenticity, rarity, and ownership of a piece of digital content. This sparked tremendous interest in the art world as it enabled artists and collectors to start buying and selling digital art on a wide variety of platforms. This topic is so broad that I invite you to do some autonomous research starting from this (rather humorous) article written by the verge encapsulating the power and madness of NFT and crypto art.

Why talk about the role of Immersive technologies within the NFT space?

Looking at the current NFT space, artists are leveraging immersive technologies in a wide variety of ways and it’s sometimes tricky to understand what are the opportunities and where they would add the most value. I consequently decided to group the main use cases drawing a parallel with the typical NFT lifecycle: the creation process, the format of the final product, and the way the product can be experienced or showcased. If we follow this simplification we end up with 3 clear ways in which every digital artwork could take advantage of immersive technologies:

  1. Creators use VR as a tool to produce their artwork
  2. The end product could be an “immersive” piece of content
  3. How the NFT is showcased or experienced

Let’s now dive into each of the three different opportunities explaining what are the main benefits and why.

1. VR as a creation tool

Super Nfty uses Gravity Sketch to model his floating heads

Creating 3D shapes can feel like molding clay, laying a sheet of cloth in zero-gravity, stacking blocks on top of each other, painting on a canvas and SO MUCH MORE. None of this requires months to learn and real-world skills can be ported to the virtual space. This also means that someone with a creative mind who always sticked with 2D creation for the fear of the steep learning curve can start creating in 3D in a matter of hours. On top of that, each and every tool despite having its own look and feel remains a “tool” and every creator can reinterpret its features to align with his/her own style.

@Shanesuttonart reimagines the use of Tiltbrush

The list of tools is rather extensive and varies in accessibility. Some require a powerful VR-ready PC, while some others run on a standalone headset. Another advantage is that the creation process becomes a distinguishing factor from other artists. Mixed reality is a powerful way to grab people’s attention and tools like Liv and Reality Mixer allow to record Mixed reality videos for some compatible apps just using a mobile device and no green screen.

Mixed Reality footage recorded with Reality mixer on an iPad without green screen

If you are not convinced yet, did you know that many of these tools actually support multiplayer? Yes! You can meet with others and create in ways that would be impossible even in the real world. The choice is vast and in time the options are just going to expand. I have created an extensive list on my website that you can check and in upcoming articles and videos I am going to go more into detail on the ones that I think are the most powerful for NFT artists.

2. Immersive experiences as a “content-type”

Instead of being rooted in a static, unmoving screen, immersive and spatial content is present in the physical space that surrounds the user (or at least gives that illusion to some extent).

I realize that this definition is VERY broad and can be VERY easily argued, yet it provides a good starting point to explain how digital artwork could be spatial and/or immersive (you can check out some exchange of thoughts on this LinkedIn post I wrote where many people chipped in with their own ideas).

Now that we have got definitions out of the way let’s start sharing some examples.

360 Images and Videos as NFTs

Another artist experimenting with 360 pictures is @DaveAlberArt who recreated 360 oil paintings but who, as far as I can see from his 360 website, hasn’t minted any of his artwork yet.

In all honesty, I struggled to find NFT of 360 videos, and that left me rather surprised. Here is an example from @MITYAmusic using “in-engine capture” for a music video. The 360 format seems to be still virgin land for the NFT world and I believe has a lot of potential for a couple of reasons:

  1. It is a very flexible format that can be experienced on desktop or mobile (check out the music video I was talking about on youtube to understand what I mean)
  2. Many of the worlds and spaces in the metaverse use skyboxes to literally fill the sky and the horizon. 360 images can be easily used as skyboxes and add a whole new dimension to the space customization that we have seen so far where the focus lies on the objects that populate the virtual space.

Augmented Reality as NFT

I started digging into this topic and realized that it is actually more complicated than expected (why am I surprised?). The closest example I could find is from @sndrv, a real veteran in this field who has actually created an unlockable code that must be used to access the filter created in Snapchat.

This looks like an elegant solution but of course, poses 2 major challenges:

1. The content could be unlocked by every subsequent owner

2. The content is still “centralized (on Snapchat servers)

AR has really the power to augment our world in creative and meaningful ways. Digital content will be on historical landmarks or at your favorite bakery and the upcoming wave of AR glasses is going to free our hands and unleash our imagination. Yet there is a clear disconnect at the moment between the creators, the creation, and the blockchain but it will not take long before more integrations will pop up and trigger the interests of artists and collectors to this brand new augmented world. In that regard, Hololoot seems to be on the right track and promises a much more streamlined creation and distribution of AR-based NFTs.

Virtual Reality as NFT

I then found another project on the YouTube channel of Alaistair Hume, a very talented Unreal Engine developer who created a very evocative space as part of his exploratory journey in the world of NFT ( I strongly suggest you check out the video because it is a very well thought out reflection). The NFT was never sold but you can check out the space he created downloading the file here.

I kept rummaging through until I found something that I thought was VERY special. The first and only VR experience (at least to my knowledge) minted on Hic et Nunc by Michael Hazani.

It can be experienced on the browser or using Metalchromium on PC VR. You can read more about the artwork here, and felt like there is some light at the end of a tunnel.

Throughout this exploration, I certainly learned that marketplaces play a key role in having digital immersive and spatial content legitimized and presented on their platforms. There is still a disconnection and few pioneers are leading the way in what might feel like a crowded space from an insider perspective but, after a closer look, is on the cusp of a revolution.

3. Immersive showcase of NFTs

I am compiling a personal list of platforms that you can check here where I describe some key aspects like supported file format, environment customization, while I journey into this discovery. Some of my favorites are Spatial and Mozilla Hubs.

Another aspect to consider is that the interaction between artists and galleries has the potential to increase the exposure of the artwork via periodic and thematic expositions leading to interesting business partnerships.

If we look a bit beyond the gallery format I just described, the meaning of ownership in the metaverse could take many other interesting twists. Maddie’s already allows monetizing NFT by creating custom physical merchandise and paying royalties to the original artist. A few months ago Maddie’s announced a partnership with Ready Player Me giving the opportunity to bring a list of selected NFT as accessories to the avatars created by the renowned avatar creation ecosystem (now supporting over 180 apps). This means that it is not going to take long before NFT branded merchandise could hit the metaverse and shortly after as AR content visible only through the rumored AR glasses being developed by Apple.

Conclusions

  1. Use VR tools to create any type of content (from spatial to 2D). It is easy, has become more accessible, and you can do a lot of cool stuff with it.
  2. Create Immersive content. Unlike what I expected not many are doing it so you are up for a head start.
  3. Create virtual galleries to show any of your artwork, from GIF and pictures to any type of “immersive content”.

If you are still with me it means you found the content useful and I hope that is going to inspire you to explore these two worlds in brand new and exciting ways. There is a lot of knowledge around and I hope this article brought some structure and gave you an overview of how Immersive technologies could be used to supercharge the NFT world.

Final remarks that you can totally skip

Short clip of how I made the thumbnail GIF in this article in Virtual Reality using Tvori.

Gabriele has a technical past and a creative heart. He is now building a community of designers to democratize XR creation