Science Fiction becomes our (Mixed) Reality

I was looking through a couple of talks on mixed and augmented reality, and I came across one from the 2015 SVVR conference by David Holz (CTO, Magic Leap). You can watch the entire talk here. But the Sci-Fi nerd in me wants to repost just my favorite parts below.

“Right now, when people say ‘digital,’ they mean it doesn’t look real. It doesn’t act real. It’s not a real thing. It’s this whole other separate thing that is not part of my normal experience. Over time, as all of these things come together, that really changes fundamentally. The digital medium becomes a physical material of the everyday world. Kids might say, some things are made of atoms and waves, and some things are made of bits and bytes. How we perceive the world changes at a philosophical level.”

“Our kids are going to get really weird. When I say I see something, I’m very biased to the things in the real world. Now all of a sudden, sights don’t have to be sights, sound doesn’t have to be sounds. All of these things can be remapped and interchanged. Reality in the future will mean something different than it does to us now, and the human experience is going to vastly expand. Kids won’t just play with soccer balls, they’ll play with atoms, galaxies, and quantum particles.”

So why did I post this almost Descartian view of reality?

Augmented Reality has been around in Sci-Fi literature since 1893 with the first mention of La Stilla in Jules Verne’s The Carpathian Castle. The Italian prima donna is thought to be alive, but is later shown to be a projected still image accompanying a high-quality phonograph recording.

1893 - First appearance of Augmented Reality in Literature

1893 – La Stilla in The Carpathian Castle (the first appearance of Augmented Reality in Literature)

But how can we discuss science fiction depicting the futuristic technologies, the result of new scientific discoveries, or different social systems and the consequences of this change without discussing Asimov.

My favorite reference to Augmented Reality in Asimovian literature has to be during a Seldon Crisis in the original Foundation Trilogy (1942). A Seldon Crisis, named after Hari Seldon, is a fictional socio-historical series of events culminating in a seemingly catastrophic social and political situation that, to be surmounted, would eventually leave only one possible, inevitable, course of action. At the climax of each Crisis, Seldon would appear to the citizens of the city in a pre-recorded hologram, reassuring the citizens of Terminus that they remained on track with the Seldon Plan.

Fast forward a couple of decades, and we start to see more and more traces of Augmented Reality turning into Mixed Reality with Stargate Atlantis (2004), Doctor Who (2007), Black Mirror (2011) and more.

AR Holographic projection of Hari Seldon in the Terminus Time Vault in Asimov's Foundation Series.

AR Holographic projection of Hari Seldon in the Terminus Time Vault in Asimov’s Foundation Series.

With mixed reality we have the potential for Sci-Fi to come into our daily life with the real world suddenly augmented with virtual objects and characters, walking around and interacting with the us as much as we can interact with them. Initially it was only the live world and live experiences that would involve all our senses (visual, audio, haptic, olfactory and more). With the advent of mixed reality, and the ability of current procedural content generation algorithms (insert shameless plug to my own research) to use their breadth of experience in VR to tailoring the virtual elements in mixed reality, this balance changes. In the future, VR may pale in comparison to Mixed Reality.

Isaac Asimov summarised the importance of science fiction in 1978, and his statement is (not so) surprisingly very relevant to our Mixed Reality discussion. He stated:

“It is change, continuing change, inevitable change, that is the dominant factor in society today. No sensible decision can be made any longer without taking into account not only the world as it is, but the world as it will be… Science fiction writers foresee the inevitable, and although problems and catastrophes may be inevitable, solutions are not. Individual science fiction stories may seem as trivial as ever to the blinder critics and philosophers of today – but the core of science fiction, its essence…has become crucial to our salvation if we are to be saved at all.