Matthew Ball’s “The Metaverse: And How It Will Revolutionize Everything” book (launching today) is the most comprehensive articulation of the Metaverse technology stack, as well as it’s foundational principles catalyzing a paradigm shift from 2D into 3D into spatial and immersive computing. What started as an essay on Fortnite (Feb 5, 2019), then evolved into an essay on the Metaverse (Jan 13, 2020), then evolved into a series of essays with his Metaverse Primer (Jun 28-29, 2021), and now has tripled in size with the publication of Ball’s The Metaverse book (Jul 19, 2022) that dives into specifying each layer of the Metaverse tech stack, where it’s at now, and where he expects it might go in the future.
[See the full transcript of my podcast interview with Ball down below].
I find Ball’s definition of the Metaverse on page 29 of his book to be the most complete and satisfying I’ve seen so far.
[The Metaverse is] “a massively scaled and interoperable network of real-time rendered 3D virtual worlds that can be experienced synchronously and persistently by an effectively unlimited number of users with an individual sense of presence, and with continuity of data, such as identity, history, entitlements, objects, communications, and payments.”
He spends the middle section of the book elaborating in great detail the low-level details of each layer of the Metaverse stack including networking, computing, virtual world engines, interoperability, hardware, payment rails, and the potential opportunities and perils of the blockchain. This middle section was a lot of wonky technical details to sort through, but it’s necessary to understand the bigger picture and larger argument that Ball is making in his book.
Ball sees that the aspirations of real-time, spatial computing and immersive 3D virtual worlds are pushing the technological, cutting edge of each layer of the Metaverse stack. He lays out in the middle part of the book how there are many yet-to-be solved challenges that are preventing the Metaverse from living into it’s fullest potential.
The Metaverse will be a confluence of all of these foundational elements coming together, which will then have the potential to usher in a complete paradigm shift of computing. But we won’t know what that will eventually look like as it’s an iterative and emergent process. He points to paradigmatic examples such as Fortnite, Minecraft, Roblox, EVE Online, and Microsoft’s Flight Simulator to give us some ideas of how these underlying principles have been playing out in a gaming context, but there’s many more unknowns than knowns for how these underlying principles will get translated into other contextual domains.
Ball elaborates in the first section how Metaverse faces many of the similar challenges that tech futurists faced at the advent of the World Wide Web in trying to predict the full scope of their influence on all aspects of how the new technological, cultural, legal, and economic dynamics all fit together. Ball documents many of the wrong predictions of the Internet and WWW that we can look back on and laugh at with 20/20 hindsight, but this makes it clear for how we’re still well within a similar occluded zone with the Metaverse for how it’s really difficult to know how all of these new capabilities will combine together and impact society.
He includes a brief survey at the end of his book of some of the ethical and moral challenges of the Metaverse, and Ball says that the open questions on ethics is a common open question that often comes up. We’re still in the process of trying to understand and deal with the many ethical fallouts of the 2D and mobile web focusing on text, pictures, and video, let alone how the movement from 2D to 3D and asynchronous to synchronous creates some new and even more challenging problems.
There’s still obviously lots of work to be done in the realm of ethics for the Metaverse, but there’s also existing prior work worth mentioning like Madary and Metzinger’s “Real Virtuality: A Code of Ethical Conduct,” XR Access & XR Association’s XR Accessibility GitHub, XRSI’s Ethical Research and Standards, the IEEE Global Initiative on the Ethics of Extended Reality published a series of eight white papers [published in 2022 after Ball’s 2021 writing deadline], and I’ve given a number talks on XR Ethics, XR Privacy, the need for robust responsible innovation frameworks, and the need for Ethics within the Metaverse. [Update: Philosopher Evan Selinger outlines more XR ethical issues in his book review.]
My personal process of peering into the future has been to focus on the direct, embodied, phenomenological and subject experiences of VR and AR applications across many contextual domains for the past 8 years, and to listen to key developers through oral history interviews. My episode #1000 of the Voices of VR podcast elaborates why I’m personally all-in on spatial computing as a foundational paradigm shift as I gathered 120 answers to the ultimate potential of virtual reality since May 2014.
Ball takes a much more objective, distant, and quantitative approach in telling the story of the Metaverse through the economic, cultural, and gaming trends that he’s been tracking. He takes the view that the Metaverse is hardware agnostic, and so it’s not predicated on specific VR or AR technologies. But he also expects that XR devices and the emerging, embodied neural interfaces that are coming along with it will likely be the most compelling ways to access these real-time, interconnected virtual worlds to fully appreciate the new levels of immersion and presence.
I’ve seen a lot of wrong and overly optimistic market predictions about the project size of the VR and AR market over the past eight years, and so I suppose that’s one of the reasons why I’ve paid more attention to the direct experience of XR technologies and listening to what the artists and pioneering XR developers across multiple domains are doing with the tech. There’s not a lot of Ball’s own direct experiences of the Metaverse within his book, or original on-the-record reporting for the companies or experiences he’s featuring. Ball told me that’s because he wanted to show how these broader trends of the Metaverse are happening independent of any one singular company or technology stack. It’s more of a survey and synthesis of publicly-available reports and tech insights of the gaming community in how Fortnite, Minecraft, Roblox, EVE Online, and Microsoft’s Flight Simulator hold some key insights for the future of computing.
The end result is that Ball’s Metaverse book helps to set a broader context for many of the same technological shifts and trends above and beyond what’s happening in XR. Despite my more narrow focus on XR, Ball and I come to many similar conclusions for how the shift towards real-time, interactive virtual worlds is a provocation for the next phase of computing as seen in the major trends of social gaming and technological roadmap for some of the biggest Big Tech players in Silicon Valley. There’s also a clear convergence for what’s happening in the most popular virtual world platforms that are currently primarily accessed through 2D portals, and how XR tech will be some of the most compelling ways to interface with these 3D virtual worlds.
I’d highly recommend checking out Ball’s book to hear more about the specific examples and underlying principles that he lays out for each layer of the Metaverse tech stack. Ball’s writing is helping to tell the broader story of the Metaverse to leaders of the tech industry, but helping bring a more grounded and clear definition of the Metaverse reflecting what’s already happening in gaming right now and where it’s likely to go in the future. It also helps set a broader context for how the future of networking, computing, virtual world engines, interoperability, hardware, payment rails, and potentially the blockchain will play into this Metaverse ecosystem that will very likely include XR devices and new neural HCI input devices that help make the transition from 2D to 3D.
I had a chance to do a half-hour interview with Ball on Tuesday, July 5th, 2022, and you can read the full transcript below or listen to it here.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
FULL INTERVIEW TRANSCRIPT from July 5, 2022.
MATTHEW BALL: My name is Matthew Ball. I’m the author of “The Metaverse: And How It Will Revolutionize Everything”, which comes out on July 19th. But I wear a different set of hats. That is, an investor, an entrepreneur, a producer in TV, film and video games, usually in and around this theme, but sometimes in more traditional media, and so forth.
KENT BYE: I read an interview you did with SIGGRAPH that traces the evolution of this as an idea where you had some essays about Fortnite and then you moved into the Metaverse, and then did a 35,000 word series of essays on the Metaverse. And then now you’ve tripled that into over 100,000 words of the Metaverse. And so maybe you could give a bit more context as to how you started to write some of these series of essays on the Metaverse.
BALL: Sure. So you’re quite right. I talk about in my book the fact that the Metaverse is a nearly 30-year old term. Of course, anyone who’s listening to this podcast know it originated in 1992. Snow Crash. But the ideas it describes can be traced back nearly a century. That starts in the 1930s, with the first known discussion around VR goggles into immersive VR environments, A.I., Holography through the thirties and fifties.
And so it’s not new that we’ve considered the Metaverse. I’ve been familiar with the term since the late nineties, and I’ve known about games that aspired to build it since around that same time. But my personal experience relates to my last job, which was head of strategy at Amazon Studios, which at the time ran nearly everything that we think of Prime Video to be.
And for much of the last decade, that was the frontier. The new frontier in media was direct to consumer streaming services. And I started to get the sense that gaming really was ready for prime time. Of course, it had been growing slowly for 50 years, but it felt like it was on the cusp of cultural domination that really didn’t even seem possible at the start of the last decade.
And then in 2018, I started playing a lot of Fortnite, a lot of Roblox, and I could start to get the sense that it wasn’t just that gaming was moving to the front lines. It’s this fantastical idea of the Metaverse was starting to become a practical opportunity and that some had actually started to build it, that the foundation was in place.
And so I wrote a piece at the end of 2018 called Fortnite is the Future, but Not for the Reasons You Think. And the goal of the piece was to break down many of the narratives about Fortnite at the time that were running, the day that was. “Wow, I can’t believe this game is free. And yet it generates more revenue than any other.” “Wow. Look at this game’s cross-platform nature. Wow. Look at the idea that this is being patched on a weekly basis.”
And I was trying to articulate that none of these things were new, actually. What was new was their creative implementation, the enormity of their popularity. But more important was how it was leading up to the Metaverse and what was starting to become a visible Metaverse strategy at Epic, though, they hadn’t said as much.
A year later, I wrote a dedicated Metaverse piece and this was pre-pandemic but became quite popular as the pandemic began to take off. Then a year later, I wrote The Metaverse Primer, which was an effort to really encapsulate my learnings over the past year and a half about the technical requirements. And then, of course, in 2021, we saw the Metaverse popularized as a theme.
Mark starts talking about it in July and October. He renames the company by the end of the year. Unity and Roblox are the two largest gaming IPO’s ever. Roblox is the largest gaming platform in the world. And so I decided to write this book, which really culminates five years of writing and thinking on the topic.
BYE: Yeah, I just had a chance to finish it and I think it’s a pretty authoritative history assessing all the different confluence and and concrescence of all these technologies coming together. And what I really appreciated about the book was pointing towards different examples from each of these different sections that are paradigmatic examples that can extract the philosophical principles from those examples, extrapolate them out as they continue to diffuse out into larger and larger scales and out throughout the culture.
But before we start to dive more into the book, I wanted to get a bit more context as to your background and your journey into this, because your style is very interesting in your writing because I get the sense that through the process of writing your essays — I know you’ve had conversations with people like Mark Zuckerberg, and I’m not sure if you’ve been in conversations directly with Tim Sweeney, but you’re certainly well-informed, and at the center of a lot of it.
But your writing style is very distant in the sense where I’m not seeing any direct quotes. You’re using quotes from other news articles, so I’d love to hear a little bit more about your process for how you put this stuff together. And if you’re doing consulting with some of these companies behind the scenes? Or if it’s just more of your role as an analyst to be able to immerse yourself to the extent that you do and then try to gather all the publicly-available information to draw out the larger economic story that makes this argument that you’re seeing all these different confluence of technologies come together?
So I’d love to hear a little bit more about your process for how you do this sensemaking process of what’s happening in the realm of technology and where it’s all going.
BALL: So you’re right. In the book, I don’t use any direct quotations, and in fact that was partly a reflection of the constrained writing environment. I wrote the book over three and a half months. At the end of last year. There really wasn’t the opportunity to go deep, to do investigatory pieces, to understand the multi-year history of many of its leading individuals.
And so that was partly a constraint. But the goal here really wasn’t to get deep into any one company. In fact, one of the things that I take personally a lot of happiness from is you’ll note that there’s oppositional or competing endorsements for the book. So I have Epic CEO and Unity CEO. I have the Sony CEO. And then I have the Microsoft Gaming CEO.
And that was because my goal with the book was not to talk about specific instantiation of the Metaverse, not a specific philosophy or ideology, wasn’t to go into the history of Tim Sweeney’s efforts to build the Metaverse nor those of Second Life. It was to provide a survey as to the technologies in the multiple different theses. I got one piece of feedback from the CEO of a large tech company saying, I loved your blockchain section, but I came away not sure whether or not you were pro or con.
And I said, “That’s the best compliment I can imagine.” Because I wasn’t trying to advocate for this. So when it comes to your question as to the writing process, most of the education came from entrepreneurs and founders. I have a venture fund. I’m a partner at MAKERS Fund, I’m an industry advisor at KKR. And so I do early stage investing, kind of mid-stage and then late stage growth equity investing.
And those entrepreneurs are outstanding because they have 20 years of experience. Sometimes they’re disgruntled, sometimes they feel like they wasted ten years of their life trying to solve a problem at a big company that was incapable of doing it. And so there’s this bounty of information, but I think I’ve always been good at doing at distance summaries of the marketplace.
And I see this very much as the manifestation of five years of work on theme, but 15 years of writing as a blogger.
BYE: Yeah, yeah. I think my assessment of the overall book is that you get the overall story completely nailed and correct — in terms of my own assessment. And I really actually did appreciate your critiques of the blockchain because, you know, there’s things that I come to a lot of similar conclusions in terms of like there’s potential for some of it, but there’s also a lot of challenges and risks and other problems that you elaborate in your book.
But I guess one of the other differences that I’d say from how I approach covering the industry is much more direct, phenomenological, immersed into the embodied experience of a lot of these technologies. And I had a harder sense of seeing where you’re oriented when it comes to the direct experience of virtual reality or the direct experience of AR.
Because you talk a lot about Fortnite, and Roblox, and Minecraft and EVE Online as these kind of leading indicators and maybe just one mention of VRChat or Rec Room, which, you know, for me, when I think about the future of the Metaverse and immersive technologies, I think about my own experiences I’ve had in Rec Room and VRChat and the qualities of presence.
And in your definition of the Metaverse, you mentioned presence, but I wouldn’t say there’s a deep elaboration of the concepts of presence within this book, so I’d love to get a little bit more context for you of like your own journey into VR and how you assess the roles of VR and AR technologies into the continued evolution of the Metaverse.
BALL: So I love this question. I really like VR. I get super excited about AR. There are many people in my network who just believe that this technology is so far off that it’s actually absurd that we spend much time focusing on it. I certainly think the technologies are hard, the problems are hard, they’re worth solving. But I am also of two perspectives.
One is that they’re not neigh, which is to say, I don’t think that we’re going to be replacing our smartphones within this decade. I don’t think we’re going to be doing it probably within the first few years of the next decade. The second thing is, I don’t believe that the requirement for the Metaverse there was this great Tweet Thread I’m sure you saw from Neil Stephenson last week or the week before.
And he talks about the idea that when he wrote Snow Crash centering around AR and VR were what he called a good hypothesis, especially if you were a science fiction author at the time. And he said what he couldn’t have imagined was that decades later, you would have billions of people inside 3D render real time environments interacting purely via touch or even WASD, right? Keys on a keyboard for forward, back, right and left.
That would have been an unintuitive estimate. And so I think about VR and AR devices as more intuitive as being essential to further immersion in the Metaverse. They’re doubtlessly destined to become the best, most popular and preferred way to interact with these environments. But I don’t think there are requirements. And in fact, and this is probably one of the reasons why you see them as relatively non-focal is I think that separating the relevance of 3D simulation in the Metaverse from that hardware helps to explain to many people that the Metaverse is not 2050.
The Metaverse doesn’t need you to believe we’re going to replace our smartphone tomorrow. It’s actually about the more underlying technologies, a persistent virtual network that is relatively endpoint agnostic doesn’t mean you’re not going to have different end points with different experiences. My personal experience with VR, I mean at Makers Fund, we’re early investors in VRChat. I use it a lot.
I find it really fun. I’m really lucky that I don’t experience nausea. And I’m also, I think, fortunate in the sense that I just have a more intuitive sense and feel for virtual immersion. And to the extent I might be predisposed to nausea, I think my mental expectations help. But I think it’s an incredible environment for young people.
When you like some old, but like I’m no ranting because I get excited about this topic. But you know, Chris Dixon talks about the fact that when a person realizes that their active brain can’t overcome what they know to be a fantasy in VR, it’s a pivotal moment, right? You put someone at the edge of the cliff in VR, and even though it’s low risk, even though it’s low frame rate, they struggle to jump.
That’s the first time you can tell. This technology may be far from prime time, but it’s not far from substantial immersion that deeply affects us. The second is when you give it to children and they have this natural, instinctive feel, you see this a lot in Rec Room and VRChat where to some extent I still know I’m in VR or when I’m in VR.
But you look at a six year old and they’re just playing around and like that idea that like they’re turning their head to the right and it’s not a physical thing they’re looking at. That boundary just seems totally diffused at this point.
BYE: Yeah. Yeah, I think that makes total sense and also really agree with that because I know Tony Parisi came up with The Seven Principles of the Metaverse, and one of it was that it’s hardware agnostic. And I think another big point that’s made in your book is how Fortnite was catalyzing this cross-platform play that both Fortnite, Minecraft and Roblox are available and all these different platforms. And so Rec Room is another one that I think is taking that same path of being on all these different platforms.
So I think that’s actually a key part to not just tie it to the VR. I’m wondering if you can maybe — I’ve watched the video where you just rattle off your authoritative definition. I’m wondering if you could just share that definition and then how you’ve structured the book, because you’re breaking down each the different chapters going into great detail of your definition. And so I’d love to have you share that and I have some thoughts.
BALL: Sure. So I actually cheat a little bit. I describe the Metaverse primarily around technical keywords and concepts. I’m not actually defining it. The reason why I do this is if you take a look at the definition of the Internet, you’re talking about it either as a network of networks. Again, a little bit of a description, but you talk about it as the Internet protocol suite, not a very helpful definition.
It’s more technical. It’s talking about the protocols. A description of the Internet tends to be more intuitive. So I describe the Metaverse as a massively-scaled and interoperable network of real-time, rendered, 3D virtual worlds, which can be experienced synchronously and persistently by an effectively-unlimited number of users, each afforded an individual sense presence while supporting continuity of data such as communications, identity, history, entitlements, objects, and so forth.
What we’re really doing is describing the technical requirements and experiential observations to have a proper parallel plane of existence. Right? That’s essentially describing the things that exist in the real world today. And so the middle third of my book is a deep dive into what’s required for that to work. We’re talking about networking capabilities, computing requirements, the actual tools to create and render virtual environments, interoperable standards to exchange information coherently, comprehensively and securely.
Talking about the payment systems that are required to actually build a thriving Metaverse, not just a functional one. An examination into blockchain, and why many believe it’s essential. Others believe it’s useless in all applications. And what potential middle grounds might look like? That’s the middle third of the book. It’s building the Metaverse. The first third is focused on “Why now”? What is it?
Explaining that definition in great detail, but also getting into some of the fundamental questions as to why do people believe that there’s a war here? Why is it important? Who wins? And then why — and this was one of the most fun things for me to examine — does it seem that the forerunners of a multitrillion-dollar transformation are gaming companies, which have otherwise been a relatively-trivial part of the economy focused just on consumer leisure?
And then the last third of the book is the more speculative elements. Start to talk about what might the value of the Metaverse be, who might win, which technologies are likely to prevail, what sorts of businesses will be built? And then for consumers concerned about what the future might be, how do they get involved, what should they do?
And then lastly, I should note I finish by trying to remind people what you can and can’t know about the Metaverse today.
BYE: My experience of reading each of those chapters was that there was a lot of really low-level technical details, but as you get through it all, the big picture all comes together. I think as you see the confluence, just like you talk about near the end in terms of the iPhone moment of the iPhone and concrescence of all these technologies coming together. And so I think similarly, you’re telling the story of all of these things that are coming together right now, and why it’s important.
Now, a couple of things I wanted to ask. One is that I know that the challenge of writing a book is that there’s always stuff that happens afterwards. And just a couple of weeks ago, there was the Khronos Group in the Metaverse Standards Forum that had this big coalition.
In the book, you’re talking about how it’s very difficult to have people collaborate. And I guess when I was reading those, I was like, “Yeah, but they have already collaborated on OpenXR. They’re already are a lot of these interoperability efforts. There’s already open standards in terms of object formats, in terms of glTF, and there’s USD.”
So I’m just curious what your reaction is in terms of some of this latest news from the Khronos Group and the Metaverse Standards Forum. And if that supports a lot of this larger thesis that you have and the difficulties of coming up with those standards of interoperability? And if you have any additional thoughts that you didn’t have time to include in your book because it wasn’t even created yet?
BALL: So I think I’m very optimistic in the book about the establishment of interoperable standards. It’s the single biggest area of pushback I receive, which is do you actually believe this can happen? And so I do. And I believe that the primitives or pressures or gravity of expanded networks will deliver. In that regard. The Metaverse Standards Forum is an important first step.
At the same time, I’d say the following the easiest part about establishing standards is always getting a bunch of people in a room together. We’ve seen that numerous times. You’re right, we have OpenXR and WebXR. But very few people support them and more importantly, their antecedents OpenGL and WebGL are very rarely supported. None of the major consoles support them, for example.
Actually, Oculus is the most used console to use these. You’ll also find that there are multiple other standards forums of some way, shape or form in some regard. Khronos exists to do that, but then you have the WesXR and Metaverse Standards Group. You’ve also got the XR Association. There are multiple different groups. Most people sign up for them because they would rather be heard than not heard and they would rather shape the standards than have their competitors shape them.
I use this XKCD comic in the book that basically jokes that a bunch of people get together to say There are 14 standards. We should have just one. That’s how you end up with 15 standards. And so that’s true. But I’m still hopeful, right? You have to have communication to actually end up aligning on something. At the same point, we can see some important omissions from the Metaverse Standards Group.
Most obviously Google and Apple, neither are participating in the forum. But I think more notable is the fact that there aren’t really other application or content-layer members. Epic is in there and Meta’s in there, but Activision isn’t in there. EA’s not in there. Roblox isn’t in there, Ubisoft isn’t in there. And so what ends up happening is lots of technologists can agree on what the best protocol or tech standard will be, but if it’s not then deployed into application-layer services content, it’s just a technical standard, right?
In some regard, it’s like Esperanto. We come up with better artificial languages all the time, but if they’re not actually adopted and deployed, it doesn’t matter. And so, again, I’m optimistic this is right. I actually think that the Metaverse Standards Forum has far more participants than I would have originally guessed, and they’re bona fide a top to bottom.
But no one has been asked yet to make a concession. No one has been asked to deploy a standard that doesn’t optimize for their system or that might advantage one of their competitors. And we don’t yet have an operating network. There’s also another point that many have made at this point, which is there wasn’t too much on ethics, if anything, as to how are those standards going to manage for the softer issues around different platforms rather than just technical interoperations?
But again, it was just announced. The hard work is yet to come.
BYE: Yeah, I was involved with the IEEE Global Initiative on the Ethics of Extend Reality, where we’re been digging into some of those issues. But yeah, it’s huge, huge issue. One other big point I wanted to get in — because I know we have limited time here — is the anti-trust aspects of both Google and Apple seem to be really, really key in terms of, you know, there’s the lawsuit from Epic Games and Tim Sweeney going against Apple.
There’s a lot of information from discovery that was made available that’s in your book that helps paint the picture of some of these different potentially, anti-competitive dynamics of these duopolies in both Google with Android and Apple’s iPhone iOS. In terms of the 30% tax. It seems to me that there likely needs to be some sort of government intervention to break up this.
Otherwise we’re going to have the same type of thing with Meta seems to be wholeheartedly adopting the same thing that they’re critiquing — like Zuckerberg’s critiquing Apple around the 30% tax, but then they’re turning around and doing the exact same thing. And then adding even more in terms of Horizon Worlds in terms of their taxes. So it seems to me that in order to really have this open, interoperable Metaverse that is going to even have the potential to have like an open web manifestation, we need to break apart this 30% tax that’s at the hardware layer.
Otherwise, we’re going to be living into this world where just a handful of companies control our digital future. So I’d love to hear some reflections on that, because that seems to be, for me, at least one of the big takeaways. I’m glad that you are articulating that because I have similar frustrations with how Apple drags it’s feet with implementing WebXR, and not really great implementations with WebGL and it all serves their own ecosystem.
But in order to break that apart, it feels like you need to have at least some level of government intervention.
BALL: So I wholeheartedly agree. I think it’s nice that you and I are speaking today on, what is it, July 5th, where the EU started to announce more of their digital markets reforms. And they’re coming pretty firmly for a lot of the concerns that we have or you and I have, which generally seem to be shared by many in the developer community.
Epic Games as one example. The challenge about the Metaverse is we’re talking about a virtual platform, a persistent network of experiences which exist irrespective of any execution, any hardware, any platform. And yet we have to access them through a platform, a hardware device. There are essentially two of them. And of the others, they’re all contending to be the payment gateway.
Why wouldn’t you? Right? Visa’s one of the best businesses on Earth, and the Apple App Store is an even better one. And it’s not just that it increases payment fees, it’s that, as you’ve also astutely observed, they cripple competitive technologies, they stymie competitive business models. And we know that it’s ultimately transferring money from the pockets of independent creators to the largest and wealthiest companies on Earth.
At the end of the day, I think the challenge here is we’re actually talking about, in some regard, penalizing companies for having been so extraordinarily successful, for having built extraordinarily great products. Right? The Apple iPhone’s integration verticalization is why the mobile era is so accelerated. We wouldn’t be where we are today without that device. And yet we can now start to feel that as we shift to this next platform, it’s impeding us.
Sometimes I think you can very justifiably say in a maliciously, deliberately self-preferential and externally punitive way. But again, talking about the EU, we see evidence of regulatory action. I think you can see — like I’m Canadian, Tim Cook did not used to do interviews with the Toronto Star on privacy. I think that they’re doing that because they understand that they now need to win hearts and minds very differently to maintain their stewardship of the world’s most important computing platform.
But yeah, we need a lot of regulatory action. I’m hopeful there as well. I spent a lot of time at the end of the book saying that we’re disappointed with the last 15 years of digital regulation, but many of us, especially in the millennial generation, assume that that’s the pattern for regulators. And of course, we have political dysfunction that’s new today.
But through all of the 20th century, in the 19th century, new technologies were very vigorously defended by regulators telecommunications, energy, rail, steel. The government was usually pretty early. The Internet Engineering Task Force comes from DoD. The Internet comes from DoD. And the U.S. government had the foresight to relinquish control of those working groups and birds of feathers and standards bodies, understanding the criticality of doing so.
And so I’m hopeful, but the challenges are tough.
BYE: Yeah. Yeah. I guess the last question I have is what do you think the ultimate potential of the Metaverse might be? And what it might be able to enable?
BALL: Well, so this is always a fun question because you have Jensen Huang, the founder and CEO of NVIDIA talking about the fact that he believes that the GDP of the Metaverse will essentially, eventually exceed that of the physical world. The physical world economy today is roughly $70 trillion. We have another $20 trillion that’s coming from digital. And so we’re looking at $50 – $60 trillion over time that might go to the Metaverse.
You can describe it differently as say, billions of individual people reaching almost every consumer, every country, every sector, globally. But the humanist perspective is, look, I believe the digital era has had a lot of bad things, dis and misinformation, data rights, data security, the role of algorithms, toxicity, abuse, harassment, radicalization. But I think that technology is fundamentally agnostic. And I think that overall, I believe that the Internet has been a profoundly good thing for the world, especially when it comes to the democratization of information.
And so I’m hopeful that the Metaverse will allow us to correct many of the problems of the last 15 years that we as developers, users, and consumers can positively affect the trajectory of the Metaverse, who leads it, with which philosophies, and why. But that fundamentally, it will continue that transformation. I talk about in the book that education is an area of extraordinary importance, but which has been barely impacted by technology, especially when it comes to access to educational resources.
And so I’m hopeful that the Metaverse can really improve that, while also bringing more job opportunities to those who, unlike myself, are not born upper middle class in Canada, in a large city, with access to many of the best jobs in the world.
BYE: Awesome. Is there anything else that’s left said that you like to say to the broader immersive community?
BALL: Well, well. Kent, let me ask you a question. What are you least certain about the Metaverse?
BYE: Well, I mean, I really appreciated your elaboration on the blockchain, because I think that the way that I see the blockchain, you know, the Peer-to-Peer Foundation came up with an Accounting for Planetary Survival White Paper, and what they described was the difference between what they see as blockchain based upon libertarian values versus blockchain based upon common-based, shared resources and values.
So every single compelling aspect of the blockchain use [in your book] was around like the RNDR token or shared use of resources to do distributed computing. So I think there’s a real compelling use case for the future of distributed computing. But in terms of the libertarian value exchange that is doing rent seeking behaviors and basically replicating the scarcity model of the existing economy, I’m not convinced that buying and selling virtual land plots is going to be the future.
I think it’s going to be much more experiential-based and more along the lines of what Tim Sweeney and Fortnite have been doing to create really vibrant ecosystems and economies. So I really appreciated that you collaborated some of those different use cases of the blockchain, but I think for me at least, I think there’s a lot of engineering flaws that I don’t know at this point can be overcome. Because there’s like Sybil attacks and with proof of stake moving over, then you could have basically one whale that overtakes and controls everything.
So it’s not actually decentralized. So every functional utility that’s coming out of the decentralized systems have some sort of centralized point that I feel like can be vulnerable to being taken over and replicating the existing power dynamics of the existing economy. So that’s my hot take in terms of where the cryptocurrency is, but it’s actually not a matter of the technology [being] agnostic, it’s more the values that are underneath the technology.
So whether it’s being driven by a libertarian scarcity model that’s trying to do rent-seeking behavior or it’s generating more communal, shared resources to create something that would not be possible as an individual. And I’m more excited about the Decentralized Web efforts from the Internet Archive and more of those technologies, the RNDR token and other things that are going to actually bring utility that are experiential rather than for speculation and creating backdoors for fraud and abuse and money laundering and all the stuff that are the challenges with that.
So when I’m reading your book, that’s the things that I appreciated that you’re articulating those perspectives because I felt like my perspectives are being reflected in your book. But yeah, to me that’s the biggest open question in terms of that.
For me, I’m all in in terms of spatial computing. And I also appreciated the call out to CTRL-Labs and the change from 2D to 3D, meaning moving from a keyboard in a mouse and having the user input that’s using more electromyography and wrist-based inputs, and neural inputs, noninvasive neural implants, and brain computer interfaces.
That for me, the shift from 2D to 3D has to do with the types of embodied and spatial computing that comes from completely new human computer interaction interfaces that seem so much like science fiction of being able to detect the firing of individual motor neuron, meaning that it can detect your intentions without you actually moving. And how that’s going to be translated into how we interface with computing, I think is going to blow people’s minds.
So I appreciated that you included the CTRL-Labs in there as well, because I do think that the ways that we interface with computing is going to be so radically different in the next 15 to 20 years. And just from what we’ve seen already in terms of the neuroscience and these principles, we’re both kind of identifying these new methods of interfacing with the spatial computing, I think are going to be so revolutionary that once they really look into the neuroscience and the technology trends, it’s going to get really weird.
But yeah, the biggest questions are the centralization of power. Having just a small handful of companies that are controlling everything, and privacy and the ethics are the big things — and NeuroRights are the other things. For me, that’s the thing that I’m not knowing how that’s all going to play out. If we’re actually going to come to the other side of this and figure out how to as a collective, overcome some of the power law dynamics of having a small handful of companies that are controlling [a] disproportionate and asymmetrical [amount of] power.
And actually what the business models are actually going to be that go beyond surveillance capitalism, [which] seems to be fueling the existing methods. And if there’s going to be ways for people to take more ownership of their data. But also have business models to go completely away from that more extractive model of surveillance capitalism. So anyway, that’s a little bit more of my thoughts.
BALL: No, I’m — I’m totally aligned with you. I mean, look, the virtual scarcity model, virtual real estate I don’t believe in. I want to reserve the right to change my mind, but I don’t see something there yet. It seems like, you know, we always start every new computing wave by trying to recreate the thing closest to reality. Right? The skeuomorphism, the game center on the iPhone, the yellow line notepad on your iPhone Notes app.
This seems like the worst possible instantiation of trying to reproduce the real world with more tools, virtually. But I think everything the rest that you said is really I’m aligned with, so I think we can leave it there.
The only other thing that I’d say is the thing that excites me most about the advent of new hardware is the accessibility improvements are extraordinary. The Xbox accessibility controller, what we’re starting to see with CTRL-Labs and hopefully BCI can bring so many people into the digital era, not to mention virtual experiences that simply cannot today. And I think that’s a really important good. We talk about the Internet and games bringing people together, but often forget how many people just can’t through physical ability.
BYE: Well, Matthew, thank you so much for writing this book and taking all the time to detail all these things. I think you’ve provided a really nice mapping of where things are at now and where they could be going with a lot of examples to get some insights for the deeper patterns and trends of this Metaverse that you’re helping to specify the structures and forms.
So thanks again for writing it all down in the book — I know it’s not easy to do that — and to join me here on the podcast to help unpack it all.
BALL: Thank you. It was my pleasure.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
I interviewed @ballmatthew about his book "The Metaverse: And How It Will Revolutionize Everything," which launches today.https://t.co/07XsOUeygL
He provides an authoritative Metaverse definition (watch video below) & the most comprehensive elaboration of the Metaverse Stack. pic.twitter.com/TvoZ009oBK
— Kent Bye (Voices of VR) (@kentbye) July 19, 2022
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.452] SPEAKER_00: The Voices of VR Podcast. Hello, my name is Ken Pai, and welcome to the Voices of VR Podcast. A podcast looking at the future of immersive storytelling and spatial computing. If you enjoy the podcast, please consider supporting it at patreon.com slash voicesofvr. So today's interview is with Matthew Ball who has a new book that's coming out on Tuesday, July 19th called The Metaverse and How It Will Revolutionize Everything. So Matthew Ball has been an essayist who's written a number of different influential metaverse essays looking at the structures and forms of spatial computing. It started with Matthew looking at different games like Fortnite and Roblox, and how they were more than just games, that they were a window into the future of computing, being immersed into these spatialized worlds and the new economic dynamics, and thinking about what are all the different primitives that you need in order to have a viable metaverse, which was first explored in Science Fiction explicitly in Snow Crash in 1992, but has prior iterations that Matthew also digs into from more of the historical perspective. But he's looking at different aspects of networking, computing, the virtual world engines, interoperability, hardware, payment rails and blockchains, where things are at right now, and looking at the major players and where they're going in the future. For me, it's the most comprehensive summary of all these different dimensions. diving deep into each of the nuances of each of these realms and digging through a lot of the technical details of that. And at the end of it, you come away seeing there's this larger story building towards this transition from 2D to 3D and into more of a spatial computing realms. But it's not just in the context of XR, of VR and AR, but these larger infrastructures of 5G, cloud computing, distributed computing, All these things emerging together is creating new realms of immersive computing, what's been popularly referred to as the metaverse. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Matthew happened on Tuesday, July 5th, 2022. So with that, let's go ahead and dive right in.
[00:02:09.284] SPEAKER_01: My name is Matthew Ball. I'm the author of The Metaverse and How It Will Revolutionize Everything, which comes out on July 19th. But I wear a different set of hats, that is an investor, an entrepreneur, a producer in TV, film, and video games, usually in and around this theme, but sometimes in more traditional media and so forth.
[00:02:31.410] SPEAKER_00: I read an interview that you did with SIGGRAPH that traces the evolution of this as an idea where you had some essays about Fortnite and then you moved into the Metaverse and then did a 35,000 word series of essays on the Metaverse. And then now you've tripled that into over a hundred thousand words of the Metaverse. And so maybe you could give a bit more context as to how you started to write some of these series of essays on the Metaverse.
[00:02:56.658] SPEAKER_01: Sure, so you're quite right. I talk about in my book the fact that the metaverse is a nearly 30-year-old term. Of course, anyone who's listening to this podcast know it originated in 1992's Snow Crash. But the ideas it describes can be traced back nearly a century. That starts in the 1930s with the first known discussion around VR goggles into immersive VR environments, AI, holography through the 30s and 50s. And so it's not new that we've considered the metaverse. I've been familiar with the term since the late 90s, and I've known about games that aspired to build it since around that same time. But my personal experience relates to my last job, which was head of strategy at Amazon Studios, which at the time ran nearly everything that we think of Prime Video to be. And for much of the last decade, that was the frontier. The new frontier in media was direct-to-consumer streaming services. And I started to get the sense that gaming really was ready for primetime. Of course, it had been growing slowly for 50 years, but it felt like it was on the cusp of cultural domination that really didn't even seem possible at the start of the last decade. And then in 2018, I started playing a lot of Fortnite, a lot of Roblox. And I could start to get this sense that it wasn't just that gaming was moving to the front lines. It's that this fantastical idea of the metaverse was starting to become a practical opportunity and that some had actually started to build it, that the foundation was in place. And so I wrote a piece at the end of 2018 called Fortnite is the future, but not for the reasons you think. And the goal of the piece was to break down many of the narratives about Fortnite at the time that were running the day. That was, wow, I can't believe this game is free and yet it generates more revenue than any other. Wow, look at this game's cross-platform nature. Wow, look at the idea that this is being patched on a weekly basis. And I was trying to articulate that none of these things were new, actually. What was new was their creative implementation, the enormity of their popularity. But more important was how it was leading up to the metaverse and what was starting to become a visible metaverse strategy at Epic, though they hadn't said as much. A year later, I wrote a dedicated metaverse piece, and this was pre-pandemic, that became quite popular as the pandemic began to take off. Then a year later, I wrote the metaverse primer, which was an effort to really encapsulate my learnings over the past year and a half about the technical requirements. And then of course, in 2021, we saw the metaverse popularized as a theme. Mark starts talking about it in July. In October, he renames the company. By the end of the year, Unity and Roblox are the two largest gaming IPOs ever. Roblox is the largest gaming platform in the world. And so I decided to write this book, which really culminates five years of writing and thinking on the topic.
[00:05:58.004] SPEAKER_00: Yeah, I just had a chance to finish it. And I think it's a pretty authoritative history assessing all the different confluence and increscence of all these technologies coming together. And what I really appreciated about the book was pointing towards different examples from each of these different sections that are paradigmatic examples that can extract the philosophical principles from those examples and extrapolate them out as they continue to fuse out into larger and larger scales and out throughout the culture. But before we start to dive more into the book, I wanted to get a bit more context as to your background and your journey into this, because your style is very interesting and you're writing it because I get the sense that through the process of writing your essays, I know you've had conversations with people like Mark Zuckerberg. I'm not sure if you've been in conversations directly with Tim Sweeney, but you're certainly well informed and at the center of a lot of it, but your writing style is very distant in the sense where I'm not seeing any direct quotes. You're using quotes from other news articles. So I'd love to hear a little bit more about your process for how you put this stuff together. And if you're doing consulting with some of these companies behind the scenes, or if it's just more of. your role as an analyst to be able to immerse yourself to the extent that you do, and then try to gather all the publicly available information to draw out the larger economic story that makes this argument that you're seeing all these different confluence of technologies come together. So I'd love to hear a little bit more about your process for how you do this sense-making process of what's happening in the realm of technology and where it's all going.
[00:07:28.531] SPEAKER_01: So you're right, in the book, I don't use any direct quotations. And in fact, that was partly a reflection of the constrained writing environment. I wrote the book over three and a half months at the end of last year. There really wasn't the opportunity to go deep, to do investigatory pieces, to understand the multi-year history of many of its leading individuals. And so that was partly a constraint. But the goal here really wasn't to get deep into any one company. In fact, one of the things that I take personally a lot of happiness from is you'll note that there's oppositional or competing endorsements for the book. So I have Epic CEO and Unity CEO, I have the Sony CEO, and then I have the Microsoft Gaming CEO. And that was because my goal with the book was not to talk about a specific instantiation of the metaverse, not a specific philosophy or ideology. It wasn't to go into the history of Tim Sweeney's efforts to build the metaverse, nor those of Second Life. It was to provide a survey as to the technologies in the multiple different theses. I got one piece of feedback from the CEO of a large tech company saying, I loved your blockchain section, but I came away not sure whether or not you were pro or con. And I said, that's the best compliment I can imagine, because I wasn't trying to advocate for this. So when it comes to your question as to the writing process, most of the education came from entrepreneurs and founders. I have a venture fund. I'm a partner at Makers Fund. I'm an industry advisor at KKR. And so I do early stage investing, kind of mid stage, and then late stage growth equity investing. And those entrepreneurs are outstanding because they have 20 years of experience. Sometimes they're disgruntled. Sometimes they feel like they wasted 10 years of their life trying to solve a problem at a big company that was incapable of doing it. And so there's this bounty of information. But I think I've always been good at doing at distance summaries of the marketplace. And I see this very much as the manifestation of five years of work on the theme, but 15 years of writing as a blogger.
[00:09:39.097] SPEAKER_00: Yeah. Yeah. I think my assessment of the overall book is that you get the overall story completely nailed and correct in terms of my own assessment. And I really actually did appreciate your critiques of the blockchain because you know, there's things that I come to a lot of similar conclusions in terms of like, there's potential for some of it, but there's also a lot of challenges and risks and other problems that you elaborate in your book. But. I guess one of the other differences that I'd say from how I approach covering the industry is a much more direct, phenomenological, immersed into the embodied experience of a lot of these technologies. And I had a harder sense of seeing where you're oriented when it comes to the direct experience of virtual reality or the direct experience of AR, because you talk a lot about Fortnite and Roblox and Minecraft and EVE Online as these kind of leading industries. educators, and maybe just one mention of VRChat or RecRoom, which, you know, for me, when I think about the future of the metaverse and immersive technologies, I think about my own experiences I've had in RecRoom and VRChat and the qualities of presence. And in your definition of the metaverse, you mentioned presence, but I wouldn't say there's a deep elaboration of the concepts of presence within this book. So I'd love to get a little bit more context for you of like your own journey into VR and how you assess the roles of VR and AR technologies into the continued evolution of the metaverse.
[00:11:02.701] SPEAKER_01: So I love this question. I really like VR. I get super excited about AR. There are many people in my network who just believe that this technology is so far off that it's actually absurd that we spend much time focusing on it. I certainly think the technologies are hard. The problems are hard. They're worth solving. But I am also of two perspectives. One is that they're not nigh, which is to say, I don't think that we're going to be replacing our smartphones within this decade. I don't think we're going to be doing it probably within the first few years of the next decade. The second thing is I don't believe that there were a requirement for the metaverse. There was this great tweet thread, I'm sure you saw from Neil Stevenson last week or the week before, and he talks about the idea that when he wrote Snow Crash, centering it around AR and VR were what he called a good hypothesis, especially if you were a science fiction author at the time. And he said what he couldn't have imagined was that decades later, you would have billions of people inside 3D rendered real-time environments interacting purely via touch or even WASD, right, keys on a keyboard for forward, back, right, and left. That would have been an unintuitive estimate. And so I think about VR and AR devices as more intuitive, as being essential to further immersion in the metaverse. They are doubtlessly destined to become the best, most popular, and preferred way to interact with these environments. But I don't think they're a requirement. And in fact, and this is probably one of the reasons why you see them as relatively non-focal, is I think that separating the relevance of 3D simulation and the metaverse from that hardware helps to explain to many people that the metaverse is not 2050. The metaverse doesn't need you to believe we're going to replace our smartphone tomorrow. It's actually about the more underlying technologies, a persistent virtual network that is relatively endpoint agnostic. Doesn't mean you're not going to have different endpoints with different experiences. My personal experience with VR, I mean, so at Makers Fund, we're early investors in VR chat. I use it a lot. I find it really fun. I'm really lucky that I don't experience nausea. And I'm also, I think, fortunate in the sense that I just have a more intuitive sense and feel for virtual immersion. And to the extent I might be predisposed to nausea, I think my like mental expectations help. But I think it's an incredible environment for young people. I sound old, but I'm now ranting because I get excited about this topic. But Chris Dixon talks about the fact that when a person realizes that their active brain can't overcome what they know to be a fantasy in VR, it's a pivotal moment. You put someone at the edge of a cliff in VR, and even though it's low res, even though it's low frame rate, they struggle to jump. That's the first time you can tell this technology may be far from prime time, but it's not far from substantial immersion that deeply affects us. The second is when you give it to children and they have this natural instinctive feel. You see this a lot in rec room and VR chat, where to some extent, I still know I'm in VR when I'm in VR, But you look at a six year old and they're just playing around and like that idea that like they're turning their head to the right and it's not a physical thing they're looking at. That boundary just seems totally diffused at this point.
[00:14:27.511] SPEAKER_00: Yeah. Yeah. I think that makes total sense. And also really agree with that because I know Tony Parisi came out with his seven principles of the metaverse. And one of it was that it's hardware agnostic. And I think another big point that's made in your book is how Fortnite was analyzing this cross-platform play that both Fortnite, Minecraft, and Roblox are available on all these different platforms. And so Rec Room is another one that I think is taking that same path of being on all these different platforms. So I think that's actually a key part to not just tie it to the vehicle. I'm wondering if you could maybe, I've watched the video where you just rattled off your authoritative definition. I'm wondering if you could just share that definition and then how you've structured the book because you're breaking down each of the different chapters, going into great detail of your definition. And so I'd love to have you share that and I have some thoughts.
[00:15:17.154] SPEAKER_01: Sure. So I actually cheat a little bit. I describe the metaverse primarily around technical keywords and concepts. I'm not actually defining it. The reason why I do this is if you take a look at the definition of the internet, you're talking about it either as a network of networks, again, a little bit of a description, but you talk about it as the internet protocol suite. Not a very helpful definition. It's more technical. It's talking about the protocols. A description of the internet tends to be more intuitive. So I describe the metaverse as a massively scaled and interoperable network of real-time rendered 3D virtual worlds, which can be experienced synchronously and persistently by an effectively unlimited number of users, each afforded an individual sense of presence, while supporting continuity data such as communications, identity, history, entitlements, objects, and so forth. What we're really doing is describing the technical requirements and experiential observations to have a proper parallel plane of existence, right? That's essentially describing the things that exist in the real world today. And so the middle third of my book is a deep dive into what's required for that to work. We're talking about networking capabilities, computing requirements, the actual tools to create and render virtual environments, interoperability standards to exchange information coherently, comprehensively, and securely. Talking about the payment systems that are required to actually build a thriving metaverse, not just a functional one. an examination into blockchain and why many believe it's essential, others believe it's useless in all applications, and what potential middle grounds might look like. That's the middle third of the book. It's building the metaverse. The first third is focused on why now? What is it? Explaining that definition in great detail, but also getting into some of the fundamental questions as to Why do people believe that there's a war here? Why is it important who wins? And then why, and this was one of the most fun things for me to examine, does it seem that the forerunners of a multi-trillion dollar transformation are gaming companies, which have otherwise been a relatively trivial part of the economy focused just on consumer leisure? And then the last third of the book is the more speculative element. Start to talk about what might the value of the metaverse be? Who might win? Which technologies are likely to prevail? What sorts of businesses will be built? And then for consumers concerned about what the future might be, how do they get involved? What should they do? And then lastly, I should note, I finished by trying to remind people what you can and can't know about the metaverse today.
[00:18:00.595] SPEAKER_00: Yeah. My experience of reading each of those chapters was that there was a lot of really low-level technical details, but as you get through it all, a big picture all comes together, I think, as you see the confluence, just like you talk about near the end in terms of the iPhone moment of the iPhone, the increscence of all these technologies coming together. And so I think similarly, you're telling the story of all these things that are coming together right now and why it's important now. A couple of things I wanted to ask. One is that I know that the challenge of writing a book is that there's always stuff that happens afterwards. And just a couple of weeks ago, there was the Kronos Group and the Metaverse Standards Forum that had this big coalition. In the book, you're talking about how it's very difficult to have people collaborate. And I guess when I was reading those, I was like, yeah, but they have already collaborated on OpenXR. There already are a lot of these interoperability efforts. There's already open standards in terms of object formats, in terms of GLTF, and there's USD. So I'm just curious what your reaction is in terms of some of this latest news from the Kronos Group and the Metaverse Standards Forum. And if that supports a lot of this larger thesis that you have and the difficulties of coming up with those standards of interoperability. And if you have any additional thoughts that you didn't have time to include in your book, because it wasn't even created yet.
[00:19:10.788] SPEAKER_01: So I think I'm very optimistic in the book about the establishment of interoperable standards. It's the single biggest area of pushback I receive, which is, do you actually believe this can happen? And so I do. And I believe that the primitives or pressures or gravity of expanded networks will deliver. In that regard, the Metaverse Standards Forum is an important first step. At the same time, I'd say the following. The easiest part about establishing standards is always getting a bunch of people in a room together. We've seen that numerous times. You're right, we have OpenXR and WebXR, but very few people support them. And more importantly, their antecedents, OpenGL and WebGL, are very rarely supported. None of the major consoles support them, for example. Actually, Oculus is the most used console to use this. You'll also find that there are multiple other standards forums of some way, shape, or form. In some regard, Kronos exists to do that. But then you have the WEF's XR and metaverse standards group. You've also got the XR association. There are multiple different groups. Most people sign up for them because they would rather be heard than not heard. And they would rather shape the standards than have their competitors shape them. I use this XKCD comic in the book that basically jokes that a bunch of people get together to say there are 14 standards, we should have just one, that's how you end up with 15 standards. And so that's true. But I'm still hopeful, right? You have to have communication to actually end up aligning on something. At the same point, we can see some important omissions from the Metaverse Standards Group. Most obviously, Google and Apple, neither are participating in the forum. But I think more notable is the fact that there aren't really other application or content layer members. Epic is in there and Meta is in there. but Activision isn't in there, EA is not in there, Roblox isn't in there, Ubisoft isn't in there. And so what ends up happening is lots of technologists can agree on what the best protocol or tech or standard will be, but if it's not then deployed into application layer services content, it's just a technical standard, right? In some regard, it's like Esperanto. We come up with better artificial languages all the time, but if they're not actually adopted and deployed, it doesn't matter. And so, again, I'm optimistic this is right. I actually think that the Metaverse Standards Forum has far more participants than I would have originally guessed, and they're bona fide top to bottom. But no one has been asked yet to make a concession. No one has been asked to deploy a standard that doesn't optimize for their system or that might advantage one of their competitors. And we don't yet have an operating network. There's also another point that many have made at this point, which is there wasn't too much on ethics, if anything, as to how are those standards going to manage for the softer issues around different platforms rather than just technical interoperation. But again, it was just announced. The hard work is yet to come.
[00:22:15.924] SPEAKER_00: I was involved with the IEEE global initiative and the ethics that extend reality where we're been digging into some of those issues. But yeah, it's a huge, huge issue. One other big point I wanted to get in, because I know we have limited time here, is the antitrust aspects of both Google and Apple seem to be really, really key in terms of, you know, there's the lawsuit from Epic Games and Tim Sweeney going against against Apple, there's a lot of information from Discovery that was made available that's in your book that helps paint the picture of some of these different potentially anti-competitive dynamics of these duopolies of both Google with Android and Apple's iPhone with iOS. In terms of the 30% tax, it seems to me that there likely needs to be some sort of government intervention to break up this. Otherwise we're going to have the same type of thing with meta seems to be wholeheartedly adopting the same thing that they're critiquing, like Zuckerberg's critiquing Apple around the 30% tax, but then they're turning around and doing the exact same thing. And then adding even more in terms of horizon worlds in terms of their taxes. So it seems to me that in order to really have this open interoperable metaverse, that is going to even have the potential to have like an open web manifestation, we need to break apart this 30% tax that's at the hardware layer. Otherwise, we're going to be living into this world where just a handful of companies control our digital future. So I'd love to hear some reflections on that, because that seems to be, for me at least, one of the big takeaways. I'm glad that you were articulating that because I have similar frustrations with how Apple drags its feet with implementing WebXR and not really great implementations with WebGL, and it all serves their own ecosystem. But in order to break that apart, it feels like you need to have at least some level of government intervention.
[00:23:51.342] SPEAKER_01: So I wholeheartedly agree. I think it's nice that you and I are speaking today on, what is it, July 5th, where the EU started to announce more of their digital markets reforms. And they're coming pretty firmly for a lot of the concerns that we have, or you and I have, which generally seem to be shared by many in the developer community, Epic Games as one example. The challenge about the metaverse is we're talking about a virtual platform, a persistent network of experiences which exist irrespective of any execution, any hardware, any platform. And yet we have to access them through a platform, a hardware device. There are essentially two of them. And of the others, they're all contending to be the payment gateway. Why wouldn't you, right? Like Visa is one of the best businesses on earth. And the Apple App Store is an even better one. And it's not just that it increases payment fees. It's that, as you've also astutely observed, they cripple competitive technologies, they stymie competitive business models. And we know that it's ultimately transferring money from the pockets of independent creators to the largest and wealthiest companies on earth. At the end of the day, I think the challenge here is we're actually talking about, in some regard, penalizing companies for having been so extraordinarily successful, for having built extraordinarily great products, right? The Apple-iPhone's integration, verticalization is why the mobile era is so accelerated. We wouldn't be where we are today without that device. And yet we can now start to feel that as we shift to this next platform, it's impeding us. Sometimes I think you can very justifiably say in a maliciously deliberately self-preferential and externally punitive way. But again, talking about the EU, we see evidence of regulatory action. I think you can see, like I'm Canadian. Tim Cook did not used to do interviews with the Toronto Star on privacy. I think that they're doing that because they understand that they now need to win hearts and minds very differently to maintain their stewardship of the world's most important computing platform. But yeah, we need a lot of regulatory action. I'm hopeful there as well. I spent a lot of time at the end of the book saying that we're disappointed with the last 15 years of digital regulation. But many of us, especially in the millennial generation, assume that that's the pattern for regulators. And of course, we have political dysfunction that's new today. But through all of the 20th century and the 19th century, new technologies were very vigorously defended by regulators. telecommunications, energy, rail, steel, the government was usually pretty early. The Internet Engineering Task Force comes from DoD, the internet comes from DoD. And the US government had the foresight to relinquish control of those working groups and birds of feathers and standards bodies, understanding the criticality of doing so. And so I'm hopeful, but the challenges are tough.
[00:26:46.262] SPEAKER_00: Yeah. Yeah. I guess the last question I have is what do you think the ultimate potential of the metaverse might be and I'd be able to enable.
[00:26:56.825] SPEAKER_01: Well, so this is always a fun question because you have Jensen Huang, the founder and CEO of NVIDIA talking about the fact that he believes that the GDP of the metaverse will essentially eventually exceed that of the physical world. The physical world economy today is roughly 70 trillion. We have another 20 trillion that's coming from digital. And so we're looking at 50, 60 trillion over time that might go to the metaverse. You can describe it differently as say, billions of individual people reaching almost every consumer, every country, every sector globally. But the humanist perspective is, look, I believe the digital era has had a lot of bad things, dis and misinformation, data rights, data security, the role of algorithms, toxicity, abuse, harassment, radicalization. But I think that technology is fundamentally agnostic. And I think that overall, I believe that the internet has been a profoundly good thing for the world, especially when it comes to the democratization of information. And so I'm hopeful that the metaverse will allow us to correct many of the problems of the last 15 years, that we as developers, users and consumers can positively affect the trajectory of the metaverse, who leads it, with which philosophies and why. but that fundamentally will continue that transformation. I talk about in the book that education is an area of extraordinary importance, but which has been barely impacted by technology, especially when it comes to access to educational resources. And so I'm hopeful that the metaverse can really improve that while also bringing more job opportunities to those who, unlike myself, are not born upper middle class in Canada, in a large city with access to many of the best jobs in the world.
[00:28:34.369] SPEAKER_00: Awesome. Is there anything else that's left unsaid that you'd like to say to the broader immersive community?
[00:28:39.716] SPEAKER_01: Well, Kent, let me ask you a question. What are you least certain about with the metaverse?
[00:28:45.517] SPEAKER_00: Well, I mean, I really appreciated your elaboration on the blockchain, because I think that the way that I see the blockchain, you know, the Peer to Peer Foundation came up with an Accounting for Planetary Survival white paper. And what they described was the difference between what they see as blockchain based upon libertarian values versus blockchain based upon commons based shared resources values. So every single compelling aspect of a blockchain use was around like the render token or shared use of resources to do distributed computing. So I think there's a real compelling use case for the future of distributed computing, but in terms of the libertarian value exchange that is doing rent sinking behaviors and basically replicating the scarcity model of the existing economy, I'm not convinced that buying and selling virtual land plots is going to be the future. I think it's going to be much more experiential based and more along the lines of what Sweeney and Fortnite have been doing to create really vibrant ecosystems and economies. So I really appreciated that you've elaborated some of those different use cases of the blockchain. But I think for me, at least, I think there's a lot of engineering flaws that I don't know at this point can be overcome because there's like civil attacks and with proof of stake moving over, then you could have basically one whale that overtakes and controls everything. So it's not actually decentralized. So every functional utility that's coming out of the decentralized systems have some sort of centralized point that I feel like can be vulnerable to being taken over and replicating the existing power dynamics of the existing economy. So that's my hot take in terms of where the cryptocurrency is that it it's actually not a matter of the technology agnostic. It's more the values that are underneath the technology. So whether it's being driven by a libertarian scarcity model that's trying to do rent-seeking behavior, or it's generating more communal shared resources to create something that would not be possible in an individual. And I'm more excited about the decentralized web effort from the internet. archive and more of those technologies, the render token and other things that are trying to actually bring utility that are experiential rather than for speculation and creating backdoors for fraud and abuse and money laundering and all the stuff that are the challenges with that. So and when I'm reading your book, that's the things that I appreciated that you're articulating those perspectives, because I felt like my perspectives are being reflected in your book. But yeah, to me, that's the biggest open question in terms of that. For me, I'm all in, in terms of spatial computing. And I also appreciated the call out to like control labs and the change from 2D to 3D. Moving from a keyboard and a mouse and having the user input that's using more electromyography and risk-based inputs and neural inputs, non-invasive neural inputs, brain-computer interfaces, that for me, the shift from 2D to 3D has to do with the types of embodied and spatial computing that comes from completely new human-computer interaction interfaces that seem so much like science fiction of being able to detect the firing of an individual motor neuron, meaning that it can detect your intentions without you actually moving. And how that's going to be translated into how we interface with computing, I think is going to blow people's minds. So I appreciated that you included the control labs in there as well, because I do think that the ways that we interface with computing is going to be so radically different. in the next 15 to 20 years. And just from what we've seen already in terms of the neuroscience and these principles, we're both kind of identifying these new methods of interfacing with the spatial computing, I think are going to be so revolutionary that once they really look into the neuroscience and the technology trends, it's going to get really weird. But yeah, the biggest questions are the centralization of the power, having just a small handful of companies that are controlling everything, and privacy and ethics are the big things, and neurorights are the other things. For me, that's the thing that I'm not knowing how that's all going to play out. If we're actually going to come to the other side of this and figure out how to, as a collective, overcome some of the power law dynamics of having a small handful of companies that are controlling disproportionate and asymmetrical power. And actually what the business models are actually going to be that go beyond surveillance capitalism seems to be fueling the existing methods. And if there's going to be ways for people to take more ownership of their data, but also have business models that go completely away from that more extractive model of surveillance capitalism. So anyway, that's a little bit more of my thoughts.
[00:32:58.328] SPEAKER_01: No, I'm totally in line with you. I mean, look, the virtual scarcity model, virtual real estate, I don't believe in. I want to reserve the right to change my mind, but I don't see something there yet. It seems like You know how we always start every new computing wave by trying to recreate the thing closest to reality, right? The skeuomorphism, the game center on the iPhone, the yellow line notepad on your iPhone notes app. This seems like the worst possible instantiation of trying to reproduce the real world with more tools virtually. But I think everything, the rest that you said is really I'm aligned with. So I think we can leave it there. The only other thing that I'd say is, The thing that excites me most about the advent of new hardware is the accessibility improvements are extraordinary. The Xbox accessibility controller, what we're starting to see with Control Labs and hopefully BCI can bring so many people into the digital era, not to mention virtual experiences that simply cannot today. And I think that's a really important good. We talk about the internet and games bringing people together, but often forget how many people just can't through physical ability.
[00:34:09.602] SPEAKER_00: Well, Matthew, thank you so much for writing this book and taking all the time to detail all these things. I think you've provided a really nice mapping of where things are at now and where they could be going with a lot of examples to get some insights for the deeper patterns and trends of this metaverse that you're helping to specify the structures and forms. So thanks again for writing it all down in the book. I know it's not easy to do that and to join me here on the podcast to help unpack it all.
[00:34:35.896] SPEAKER_01: Thank you. It was my pleasure.
[00:34:37.696] SPEAKER_00: That was Matthew Ball. He's a strategist and venture capitalist, as well as an essayist who's been writing a number of different articles on the metaverse and has a new book that's coming out on July 19th called The Metaverse and how it will revolutionize everything. I've had a number of different takeaways about this interview. First of all, for me, the overall story that's being told is a really authoritative summary of how to start to define what the Metaverse is and where it's going to go. From the full tech stack perspective, I think it's a really great model to look at. There's been a lot of hype around the Metaverse, a lot of crypto-based projects that have co-opted as the term. I was actually really excited to see the pushback even that Matthew was saying, trying to get Be a little bit more cautious how some of the blockchain and Web3 and crypto-based attempts or co-opting of the phrase in the metaverse, that his definition is trying to make firm boundaries, that it could include some of that, but it's not requiring that. It's also not requiring any specific hardware from VR or AR. So there's also hardware agnostic in that way, especially when you start to look at the cross-platform experience of something like Fortnite, Roblox, or Minecraft, that eventually over time, and Roblox is already starting to integrate different aspects of VR, And there's been VR versions of Minecraft. And as we move forward, there's going to be more and more dimensions of XR technologies that are going to be part of the portal of the metaverse. My experience of reading the book is that it's diving into a lot of the low level technical aspects and details to sift through. But I think at the end of it, I have a much greater appreciation an understanding of nuances of like the latency and all the other trends. And I think of Simon Wardley's ways that he breaks out technological diffusion and evolution. And so there's different ways technology is diffusing out into the culture, but it's also evolving in different ways that come up into an academic prototype that's kind of proving out that concepts even possible. And then it moves into more of a custom bespoke enterprise instantiation, where it's something that's maybe something handcrafted or something that's a very unique experience. Those principles then get embedded into more of a consumer product that's operating more at consumer scale, and it eventually gets into mass ubiquity. A lot of books like this are trying to say, OK, what is the pattern here between this is a concept or idea, let's look at where the first idea of this was, maybe the first deployment of the idea in a context and in a community, and then how can we take those ideas and then Extrapolate that out and see how that gets spread out to all these other various different contexts I think that's the idea of the metaverse is to look at different things like fortnight and minecraft and roblox and Eve online and you know the muds and all the other long history of all these different virtual worlds a second life but adding the more real-time dimensions of it and dimensions of cross-platform across all these different media and then adding in their virtual and augmented reality and There's only one stylistic thing. It's very distant and looking at things from an objective and quantified perspective, looking at a lot of numbers and trying to make these arguments for what's happening in the industry, and just trying to focus on what those numbers are and use that to paint a larger picture for these markets as they're growing. It's something that, as a venture capitalist as well as a strategist, is part of his job his previous head of strategy at Amazon Studios, of being aware of all these different dynamics and understand where things are at right now, and to project out how things are going to be shifting out into the future and what scale they're going to be at. And so there wasn't a lot about his own personal embodied and phenomenological experiences. That for me, a really connection to that, but he told me that he made the explicit decision to try to make it that he's not focusing on any One specific aspect of these companies, but that he's trying to paint this larger picture for all these different players that by the end of it It's not Biasing towards any one of those individual players or experiences or his own direct embodied experiences for what the metaverse is for me a huge part of this future of spatial computing is the direct embodied phenomenological experience of all the different qualities of presence and being immersed into these different worlds and what the experience of that is like and where the structures and forms of that are going. And so there's a little bit of that in terms of looking at the meta-businesses and the different contexts and, you know, who the players are and who we expect to be influential in shaping where this is all going. It's very difficult to know how all these things combine together. I mean, we had a whole pandemic and the pandemic has shifted things in a way that is been a catalyst towards a lot of these different virtual worlds, especially things like Roblox and Fortnite and Minecraft and Discord, have become these de facto social spaces for people to congregate on online. And that's at the mass scale. And then within the VR context, you have VRChat and RecRoom and Metas, Horizon Worlds, that's still emerging and growing. You have these spaces, these virtual worlds, and moving into these virtual world platforms, reflecting that the market is not at the point where it's at a huge, massive scale, and that by 2030, citing John Riccatella from Unity, they expect around 250 million headsets. Relative to smartphones, when there's billions of smartphones out there, 250 million is a lot smaller by 2030. But who knows what that number actually is? Part of my resistance is that I've saw so many numbers over the years, and most of them were just completely wrong. So I just stopped looking at them, just because, for me, it wasn't what was so interesting about it. For me, the interesting thing about XR is how compelling some of the experiences are. But in terms of trying to look at the larger decisions that these companies make, they have to make businesses out of it, and they have to look at the scale of where things are at and how much of their entire business they can start to rely upon. I think by 2030, it could be a lot more than 250 million. I don't know. It's hard to know what will continue to happen within the culture and the technology driving things. I expect a lot of new XR devices that are coming out in the realm of 2024, 2025. That is starting to have the next generation of augmented reality. But there's going to be four different VR headsets that are coming out over the next couple of years, what's been reported from the information on Meta's roadmap. hard to tell whether or not something that's been exponentially growing and doubling if there's gonna be a device that's just so compelling that it actually exceeds that mark of 250 million by 2030. So, highly recommend checking out the book. At the end of it, I'd be really happy to have a much better understanding of where things are at and where they're going and to get some of the thoughts. And I hope to, at some point, maybe have more of an extended conversation with Matthew to get into more of the nuances of some of my reactions to the book or where I think things are at and what I'm looking at in terms of seeing where the future of the medium is going across the variety of different contexts, and also a lot of the ethics as well. I mean, there's some stuff that he's starting to talk about, but that's what I see as some of the biggest open questions in terms of privacy, accessibility, harassment, diversity, inclusion, but also the threats to privacy and neural rights, the right to mental privacy, the right to identity, the right to agency, some of these neuroethics dimensions of XR technologies that are going to continue to be a big part of the conversation as well. That's been a big part of my own work, as well as organizations like the XR Association, XRSI, and the IEEE Global Initiative on the Ethics of Extended Reality that I've been involved with for the past couple of years. So, yeah, overall, just a really authoritative and compelling summary, helping to point out some of the deeper philosophical principles of the metaverse and where it's at and where it's going in the future. So that's all I have for today, and I just wanted to thank you for listening to the Voices of VR podcast, and if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listener-supported podcast, so I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
