Supercharging NFTs with AR and VR
Over the past years, we have seen many attempts, some more successful than others, to integrate Virtual and Augmented Reality (sometimes referred to as Immersive technologies) into this new wave of digital art. These experimentations took many shapes and forms, and given the fast pace at which both worlds are evolving, I found it complicated to have a clear conversation with NFT artists, collectors, or creators focused on their creative process, artwork, and the way they were communicating it to the outside world. This article tries to bring some structure into this crazy and exciting world by breaking up the gazillion opportunities out there into digestible chunks that can help the community understand how to take advantage of Immersive technologies and why. Collectors, and anybody in this space, will also benefit from this framework as it will help them attach a subjective value based on a deeper understanding of the artwork’s essence.
If you are already knowledgeable about immersive technologies such as AR & VR as well as NFTs, you can just skip to the next section, on the other hand, if you are not familiar or just want a quick refresher, you are more than welcome to keep reading through this short introduction.
An intro to NFTs and Immersive technologies
Immersive technologies are able to create experiences where the digital content merges or fully replaces reality. Augmented Reality (AR) and Virtual Reality (VR) are at the extreme of the spectrum and the core difference between the two lies in the percentage of digital content that is presented to the user (via for example the mean of a mobile device or a virtual reality headset). Since this percentage is not always a discrete number and can even change throughout an experience in 1994 Milgram and Kishino introduced the concept of the reality continuum that encompasses all possible variations and compositions of “real” and “virtual”.
Since then, a lot has changed: creating content became much easier, hardware became cheaper, and virtual worlds became accessible directly from a web browser. This led to revisiting that taxonomy to include a more user-centric approach and adding more parameters to categorize this blending of real and virtual. Nonetheless, categorizing a piece of content as augmented reality, mixed reality or virtual reality can still be rather challenging.
NFT are an even more exciting phenomenon that took the world of digital art by storm. NFT stands for “non-Fungible-token” and, among its many uses, enables the certification of the authenticity, rarity, and ownership of a piece of digital content. This sparked tremendous interest in the art world as it enabled artists and collectors to start buying and selling digital art on a wide variety of platforms. This topic is so broad that I invite you to do some autonomous research starting from this (rather humorous) article written by the verge encapsulating the power and madness of NFT and crypto art.
Why talk about the role of Immersive technologies within the NFT space?
Giving their digital nature, it is not surprising the world of immersive technologies and NFTs collided leading to astonishing results and a flourishing of experimentations from “technical” and “less technical” creators that found new original ways to blend the two. Immersive technologies offer in fact new creative ways to tell stories making viewers or casual bystanders part of the experience. On top of that, Virtual Reality can be a very powerful tool to fill in the skills gaps of NFT artists or expand their creative powers by providing brand new ways to express themselves… and that is not that hard or expensive anymore. VR headsets like the Oculus Quest offer access to virtual worlds in under 400 USD, VR creative software like Tiltbrush became completely free, and creating a virtual gallery on the web requires just a few clicks (after you have spent a few hours to figure out which free platform to pick 😅). So let’s try to bring some “order” and look more structurally at the many ways these two worlds overlap, connect and synergize.
Looking at the current NFT space, artists are leveraging immersive technologies in a wide variety of ways and it’s sometimes tricky to understand what are the opportunities and where they would add the most value. I consequently decided to group the main use cases drawing a parallel with the typical NFT lifecycle: the creation process, the format of the final product, and the way the product can be experienced or showcased. If we follow this simplification we end up with 3 clear ways in which every digital artwork could take advantage of immersive technologies:
- Creators use VR as a tool to produce their artwork
- The end product could be an “immersive” piece of content
- How the NFT is showcased or experienced
Let’s now dive into each of the three different opportunities explaining what are the main benefits and why.
1. VR as a creation tool
This is undoubtedly the aspect that excites me the most. Virtual Reality lowers the barrier to creation, transporting the process of creating digital art from a screen to space. Moving objects around is not done with a mouse but by stretching your arm and literally grabbing them.
Creating 3D shapes can feel like molding clay, laying a sheet of cloth in zero-gravity, stacking blocks on top of each other, painting on a canvas and SO MUCH MORE. None of this requires months to learn and real-world skills can be ported to the virtual space. This also means that someone with a creative mind who always sticked with 2D creation for the fear of the steep learning curve can start creating in 3D in a matter of hours. On top of that, each and every tool despite having its own look and feel remains a “tool” and every creator can reinterpret its features to align with his/her own style.
The list of tools is rather extensive and varies in accessibility. Some require a powerful VR-ready PC, while some others run on a standalone headset. Another advantage is that the creation process becomes a distinguishing factor from other artists. Mixed reality is a powerful way to grab people’s attention and tools like Liv and Reality Mixer allow to record Mixed reality videos for some compatible apps just using a mobile device and no green screen.
If you are not convinced yet, did you know that many of these tools actually support multiplayer? Yes! You can meet with others and create in ways that would be impossible even in the real world. The choice is vast and in time the options are just going to expand. I have created an extensive list on my website that you can check and in upcoming articles and videos I am going to go more into detail on the ones that I think are the most powerful for NFT artists.
2. Immersive experiences as a “content-type”
Regardless of the creation process been followed, the final artwork can exist in various formats. Currently, the majority of the artwork showcased on many marketplaces is 2D. That means images, videos, GIFs, and even looping videos of animated 3D models are ultimately “flat” products. Immersive and spatial content is a brand new category of content that enables interaction and engagement with the spectator in different ways (please note the stress on the word different).
Instead of being rooted in a static, unmoving screen, immersive and spatial content is present in the physical space that surrounds the user (or at least gives that illusion to some extent).
I realize that this definition is VERY broad and can be VERY easily argued, yet it provides a good starting point to explain how digital artwork could be spatial and/or immersive (you can check out some exchange of thoughts on this LinkedIn post I wrote where many people chipped in with their own ideas).
Now that we have got definitions out of the way let’s start sharing some examples.
360 Images and Videos as NFTs
360 pictures or videos can be a very attractive form of immersive content. The actual file extension is identical to 2D content (e.g. jpg, mp4, etc) but the aspect ratio is 2:1(this is often referred to as “equirectangular” format). There are several ways to create such compositions, from 360 cameras (like the Insta 360 One X 2 that is now rather affordable and lead to very satisfying results) to standard screen tools like Photoshop, to “in-engine” capture. When I mention “in-engine” I refer to the ability to capture a 360 picture or video directly within the tool used to create the artwork like Unity, Unreal or 3D modeling tools like Blender. VR creation tools also offer the ability to capture 360 images or videos like Tvori or Tiltbrush (here is a how-to respectively for Tvori and Tiltbrush on page 8). @Natural_warp is one of the NFT artists pioneering this new format creating kaleidoscopic abstract 360 compositions.
Another artist experimenting with 360 pictures is @DaveAlberArt who recreated 360 oil paintings but who, as far as I can see from his 360 website, hasn’t minted any of his artwork yet.
In all honesty, I struggled to find NFT of 360 videos, and that left me rather surprised. Here is an example from @MITYAmusic using “in-engine capture” for a music video. The 360 format seems to be still virgin land for the NFT world and I believe has a lot of potential for a couple of reasons:
- It is a very flexible format that can be experienced on desktop or mobile (check out the music video I was talking about on youtube to understand what I mean)
- Many of the worlds and spaces in the metaverse use skyboxes to literally fill the sky and the horizon. 360 images can be easily used as skyboxes and add a whole new dimension to the space customization that we have seen so far where the focus lies on the objects that populate the virtual space.
Augmented Reality as NFT
Technically any 3D file could be experienced in AR. Often the composition has some form of animation and the most common file formats used are FBX, GLB, and GLTF (of which only the first two support animation). On the other hand, the fact that you can view a 3D model in AR doesn’t necessarily qualify your NFT as an AR experience. Let’s take the example of @SuperNfty. He creates collectible floating heads, and by every purchase you have access to a variety of file formats: an image, a GIF, and a GLB file. Without much trouble, the GLB file can be imported into platforms like Sketchfab and viewed in AR but in essence, @SuperNfty is not selling an “AR experience”. Along with the usual “game engine suspects” like Unity or Unreal, creating and deploying an AR experience has become easier than ever. Some examples are Adobe Aero, Lens studio, Spark AR and more.
I started digging into this topic and realized that it is actually more complicated than expected (why am I surprised?). The closest example I could find is from @sndrv, a real veteran in this field who has actually created an unlockable code that must be used to access the filter created in Snapchat.
This looks like an elegant solution but of course, poses 2 major challenges:
1. The content could be unlocked by every subsequent owner
2. The content is still “centralized (on Snapchat servers)
AR has really the power to augment our world in creative and meaningful ways. Digital content will be on historical landmarks or at your favorite bakery and the upcoming wave of AR glasses is going to free our hands and unleash our imagination. Yet there is a clear disconnect at the moment between the creators, the creation, and the blockchain but it will not take long before more integrations will pop up and trigger the interests of artists and collectors to this brand new augmented world. In that regard, Hololoot seems to be on the right track and promises a much more streamlined creation and distribution of AR-based NFTs.
Virtual Reality as NFT
Last but not least we have Virtual reality content, and guess what? Same story: not much going on (are YOU even surprised?). I am a firm believer in the immense artistic and storytelling power of Virtual Reality and owning a virtual experience would personally feel unique and precious. I was expecting a multitude of VR experiences running on webGL or some smart ways to access executable files stored in the blockchain but instead, I found just three examples, some of which left me a little puzzled. A few months ago a Swedish developer named Fredrik Ekholm, auctioned an NFT of a VR studio on Opensea but didn’t find any buyer (despite the sharp drop in price after one month the items was listed).
I then found another project on the YouTube channel of Alaistair Hume, a very talented Unreal Engine developer who created a very evocative space as part of his exploratory journey in the world of NFT ( I strongly suggest you check out the video because it is a very well thought out reflection). The NFT was never sold but you can check out the space he created downloading the file here.
I kept rummaging through until I found something that I thought was VERY special. The first and only VR experience (at least to my knowledge) minted on Hic et Nunc by Michael Hazani.
It can be experienced on the browser or using Metalchromium on PC VR. You can read more about the artwork here, and felt like there is some light at the end of a tunnel.
Throughout this exploration, I certainly learned that marketplaces play a key role in having digital immersive and spatial content legitimized and presented on their platforms. There is still a disconnection and few pioneers are leading the way in what might feel like a crowded space from an insider perspective but, after a closer look, is on the cusp of a revolution.
3. Immersive showcase of NFTs
Immersive technologies are also a powerful tool to market and show your NFT from both a collector as well as an artist’s point of view. Unlike the real world, NFT art is confined behind a screen, but thanks to the numerous platforms that appeared over the last years it is possible to create virtual galleries and explore them together to tell the stories behind the artworks, the artists or simply show your collection. This exploration can be often done in a VR headset or from your browser. Images, GIFs and videos are supported by almost all platforms. Other types of content like 3D models, 360 images and videos are not broadly supported and require some tweaking and optimization. If we look beyond the NFTs themselves another important aspect to consider is the space in which the artwork is located. In a virtual world walls, floors, ceilings can take any shape and form and adapt to the artwork to tell an even stronger narrative. Some remarkable examples are the spaces created by Metaxu studio who has been organizing exposition from NFT artists like @E_O_I_N.
I am compiling a personal list of platforms that you can check here where I describe some key aspects like supported file format, environment customization, while I journey into this discovery. Some of my favorites are Spatial and Mozilla Hubs.
Another aspect to consider is that the interaction between artists and galleries has the potential to increase the exposure of the artwork via periodic and thematic expositions leading to interesting business partnerships.
If we look a bit beyond the gallery format I just described, the meaning of ownership in the metaverse could take many other interesting twists. Maddie’s already allows monetizing NFT by creating custom physical merchandise and paying royalties to the original artist. A few months ago Maddie’s announced a partnership with Ready Player Me giving the opportunity to bring a list of selected NFT as accessories to the avatars created by the renowned avatar creation ecosystem (now supporting over 180 apps). This means that it is not going to take long before NFT branded merchandise could hit the metaverse and shortly after as AR content visible only through the rumored AR glasses being developed by Apple.
Conclusions
Let’s summarize all the stuff above in a couple of sentences and examples so that it (hopefully) sticks. How can you Supercharge your NFT with AR & VR?
- Use VR tools to create any type of content (from spatial to 2D). It is easy, has become more accessible, and you can do a lot of cool stuff with it.
- Create Immersive content. Unlike what I expected not many are doing it so you are up for a head start.
- Create virtual galleries to show any of your artwork, from GIF and pictures to any type of “immersive content”.
If you are still with me it means you found the content useful and I hope that is going to inspire you to explore these two worlds in brand new and exciting ways. There is a lot of knowledge around and I hope this article brought some structure and gave you an overview of how Immersive technologies could be used to supercharge the NFT world.
Final remarks that you can totally skip
I thought I would be a bit of a hypocrite if I would preach about these topics and do really nothing about it. As a consequence, I decided to create the thumbnail of this article using a VR tool and turn it into an NFT. You can see a short clip of the creation process below and if you are interested you can get it right now on Hic et Nunc.