Averting Nuclear Disaster with AR, with Packet39’s Shachar “Vice” Weis

XR for Business

English - October 25, 2019 09:49 - 41 minutes - 37.9 MB - ★★★★★ - 12 ratings
Technology Business Homepage Download Apple Podcasts Google Podcasts Overcast Castro Pocket Casts RSS feed

Previous Episode: The YouTube of 3D Models, with Sketchfab CEO Alban Denoyel

Next Episode: XR via Sexy Sunglasses, with Vuzix' Paul Travers

Nuclear energy is no joke, and to
train to work in the field can be risky and costly… unless you’re
training in a virtual environment. That’s the kind of technology
Shachar “Vice” Weis, co-founder of VRAL, has been
developing for the last several years. Alan and Vice discuss the pros
and…well, are there really any cons to non-radioactive training
simulations?

Alan: Welcome to the XR for
Business Podcast with your host, Alan Smithson. Today’s guest is a
great friend, Shachar “Vice” Weis. He’s a software
developer with 25 years experience. He’s worked in many fields and
disciplines, from ancient mainframes to tiny system-on-chip units.
Vice has extensive experience with 3D frameworks, game development,
robotics, UX design, and automation. He has broad the R&D
experience from managing an R&D in a startup environment, to
developing enterprise solutions in HP Labs and leading an R&D
team in the Israeli Navy Computer Center. Vice’s worked in many
areas, including datamining, web development, virtual reality, 2D and
3D graphics, image and video processing. And he brings acute
analytical skills, system system-wide vision, and experience with
clients and knowhow in R&D work methodologies. You can learn more
about his company, Packet39, it’s packet39.com.

Vice, welcome to the show, my friend.

Vice: Hey, good morning. Thanks
for having me.

Alan: It’s my pleasure. I’m
really excited. Your presentation is at Virtual Reality Toronto
meet-up was mind-blowing. I got there and I sat down, and all of a
sudden this guy on stage is talking about nuclear reactors and using
Hololenses for training and virtual reality training and simulators.
And I was sitting there with my mouth open the whole time, taking
photos and trying to capture all of the goodness. And I’m really
honored to have you on the show. How did you get into nuclear? Like,
what happened there?

Vice: Well, as most things in
life, it was mostly chance. I met a guy at VRTO — the Toronto VR
conference — three years ago, and he was working for a company that
provides services for nuclear power, specifically Oakajee here in
Canada. And we got to talking and we understood that there was a lot
of a lot of need and virtual reality could solve some really
interesting problems. And we took it from there.

Alan: VRTO, it’s a small
conference, but man, the level of quality of the attendees and the
speakers at that conference every year is just phenomenal. And it
feels like the show keeps getting smaller but more important in its
stature. So it’s cool to hear that you–

Vice: It’s getting smaller and
more condensed and I’ve given a talk at VRTO every year in the last
three years and every time it was–

Alan: Yeah, it’s amazing. This
is the first year I missed it. I was traveling, but I’m really
excited to see what comes next year because I know it got smaller,
but it just got– the people that attended it are really deep into
this stuff. So tell us about this nuclear reactor training, kind of
what was the first step with that? How do you start training people
in VR for nuclear facilities?

Vice, welcome to the show, my friend.

Vice: Hey, good morning. Thanks
for having me.

Vice: It’s getting smaller and
more condensed and I’ve given a talk at VRTO every year in the last
three years and every time it was–

Vice: Well, there’s a lot of
stuff you can do in VR and a lot of stuff that you shouldn’t. And the
trick is finding the correct path. We started with a proof of concept
project, that was the airlock. And this was new to me as well. I
didn’t have any experience in nuclear power specifically, back then
when we started. And it turns out that the entire core, the entire
facility where the core is housed is airtight. And to get in and out,
you have to go through an airlock, which is very similar to a
submarine airlock. It is very small. It has a very big metal door.
And it’s pretty terrifying, especially if you haven’t done it before.
And they had the problem where people would go through the airlock
for the very first time. People were new or training and only
experienced this in the classroom before, and they had panic attacks
where they had issues or they had claustrophobia. And it was just a
very unpleasant experience for everybody. We decided that let’s try
to do this in VR and see what happens. We can try to recreate this
airlock, this feeling of closeness or this– basically trying to get
people scared, to figure out who can and who shouldn’t be going
through these devices.

Alan: And one thing, there’s a
video on your site of the airlock and it’s kind of this monstrous
door. You kind of do the thing and it’s got real-time sound, when you
as you kind of screw it open, it’s like [creaking sound] and then you
open this [groaning sound] you know, this big door sound. And it’s
funny because that sound of that door opening, it’s really– it feels
like you’re grabbing a really big door. It feels heavy. Even though
it’s digital, it weighs nothing, but it feels heavy because it’s such
a an ominous sound. And then when you close it behind you, it’s that
locking, that [thudding sound] you know, that– Oh, man.

Vice: We learned a lot from that
one door. We learned a lot about designing for VR, the challenges.
And I actually talk about the door in many of my presentations,
because you can learn a lot from that. There’s a lot of things that
come up when you’re designing for VR or building for VR, that you
don’t really experience in any way in any other medium. And for
example, in a regular desktop application, you have a door. There’s
also an animation attached to it with the door opening and then you
run the animation in reverse. The door closes and there’s one audio
file that you play, that is synced exactly for that 2D animation, and
then you’re done. With VR, everything becomes 10 times more
complicated, because the user can grab the door and he can pull it
open and stop halfway, and then what happens, right? You need to stop
the audio. He can pull it quickly versus pulling it slowly. He can
slam it shut, he can close it gently. All of a sudden, you’re not
just playing an audio file in an animation. All the sudden, you have
to synthesize audio in real time and you have to work with physics.
And it’s no longer simple. But all of that is required because these
are things that people expect the VR experience to do. And if it
doesn’t, it feels weird then just breaks the immersion. And immersion
is your Holy Grail. That’s what makes games effective and that also
is what makes training applications effective. It’s the same goal.

Alan: What other senses are you
hijacking? Did you put the smell of the nuclear reactor in there,
too?

Vice: [chuckles] I don’t think
we should do that. That kind of negates the whole safety thing. But
audio is very important. As you saw yourself, all the hissing sounds,
and the creaks of the door, and the slamming, and audio is super
important. Those are super hard to do right. Haptic feedback is
important, and you don’t have much to work with. The controllers
vibrate, but you can’t control how much it vibrates and the
frequency. And you could do a lot with that. And if you want to get
really fancy, you can add stuff like fans or vibrating floors, but
that’s really mostly for entertainment.

Alan: Yeah, it seems for now,
but I think we will probably get there where for things like if
there’s a simulator where you want to simulate the motion of
something, even though it’s not really moving, I think you could add
haptic plate or something to simulate the motion.

Vice: Yeah. Some things work and
some things don’t. It’s very complicated, especially when you start
talking about motion sickness or simulation sickness.

Alan: Indeed. So have they ruled
this giant airlock out, have they run people through it yet?

Vice: They did a bunch of tests.
It was never published as a real training application, unlike the
Geiger counter one, which they are using.

Alan: But let’s talk about the
door for a second. The door one, how many people did they run through
there before they realize, was anybody scared in it? Because it seems
like I’ve seen this for mining as well, where they’ll they’ll put
people in VR and take them down into a mineshaft before they hire
them, because are you going to spend 50, 60 thousand dollars training
somebody, then they go to a mineshaft and go, “I can’t do this.
This is beyond me.” So being able to give them that exposure
therapy of the environment, I think is very beneficial to companies.
The cost of building that one demo is far paid for by not hiring
people that are going to be terrified in there.

Vice: Absolutely, yes.
Unfortunately, there’s also always a lot of politics involved and the
decisions are not always the ones that make the most sense,
especially in big organizations such as nuclear power. And everything
takes time. So we’re still talking about this. I mean, it’s been two
and a half years, two years. That’s nothing in nuclear power time.
These guys think in projects that span decades, like literally, I’ve
seen projects that are 30 years.

Alan: Wow. That’s incredible.

Vice: I don’t think any other
industry has that. It’s unbelievable.

Alan: That’s amazing. So let’s
talk about the Geiger counter thing, because that is being used. What
is that about? Because that was a really cool experience as well.

Vice: Yeah, that was the most
complex one we’ve done. And the premise is that in some situations, a
worker comes out of the power plant and he needs to scan his
equipment for contamination. And to do that, he will use a handheld
radiation monitors, similar to what you see in the movies, the ones
that click when you sense radiation, it goes “click, click,
click” with background [radiation] and then
“clickclickclickclick”, very fast clicks if there’s more
radiation than background. And they need to train on how to use this
device, and to do that they go to a certain facility, they go down to
the basement, they sign off on a lot of paperwork. They take out of
the safe a small piece of contamination, a small piece of radioactive
material. There’s a trainer, the trainer takes the radioactive
material, he puts it on the table. And usually it comes in this giant
yellow box with a lot of warning signs on it. And he tells the
trainee. “OK, now you have to sweep this table, and you have to
pretend that you don’t see where this contamination is” because
again, it’s sitting inside a big yellow box that is possible to miss.
“So pretend that you don’t see it. And show me how you sweep the
table, how you search for this contamination and find it.” And
this is how it has been done for decades. In VR, it’s actually a
much–

Alan: It is really tedious.
Like, you showed me how it works, and it was like, oh my god. And the
fact is they have to pull out a piece of radioactive material,
exposing people to real radiation to test you. [chuckles] Like it’s
kind of counterintuitive if you really think about it.

Vice: Yes. Because really, there
wasn’t an easy way to do it until now, right?

Alan: Yeah.

Vice: In VR, this works perfect.
So this is a golden project, what I call, because it lends itself
perfectly for VR. There’s a table, which you can either calibrate to
your real table or not. There’s a virtual piece of radiation that the
instructor can put on the table. And the trainee in VR doesn’t see
where that contamination is. So there’s no cheating involved. The
training is actually better in VR than in the real world, in this
case. And the system will monitor how fast you’re moving the
detector. You have to move at a specific speed. Not too slow, not too
fast. There is a pattern that you have to follow, a sweeping pattern,
and the software will check that you’re actually getting this pattern
correctly. And these are things that are hard for human operators to
eyeball and measure and estimate. And for the software it’s very
accurate. So the software will tell you exactly where you missed a
spot, if you went too fast, if you went too slow, and you can pass or
fail this test. And it works perfectly in VR, in VR you get that
sense of how to move– how to correctly move the detector, get the
muscle memory of operating this device, with the knobs and the
handle. It was a perfect project.

Alan: And it decreases the
riskiness, the safety quotient there by not having to pull out a
piece of radioactive material. [laughs]

Vice: Yeah. It decreases it to
zero and decreases the cost significantly. And you can do this
training anywhere and not in that specific basement where the
material was held in a safe.

Alan: You’ve explored a lot
within the nuclear realm. You’ve got this airlock door where people
go in and test. Then you’ve also got the Geiger counter, which is
pretty awesome. But then there’s another one that you guys have
worked on, that really stands out as something that could save lives
immediately. And the Geiger counters, great, airlock door, but the
cone of radiation. Talk to us about that thing.

Vice: That was an augmented
reality project. The nuclear reactor face, the reactors in Canada are
a design, it’s called a CANDU design, where– and none of this is
confidential, by the way, you don’t have to come and shoot me, OPG,
please. Don’t burst into my house at middle of the night.

Alan: It’s all public knowledge.
It’s a public utility. So you can go on and you can Google the
building plans for the CANDU reactor, if you choose.

Vice: I totally Googled it, just
to make sure that it’s public knowledge. The reactor is vertical in
the sense that it’s basically this wall with tubes in it and each
tube has radioactive material inside. And when the tubes are uncapped
or open, there’s radiation beam that comes out, and of course the
radiation is– the radiation beam is invisible. And stepping into the
radiation beam is not a good idea.

Alan: You don’t want to step in
front of a radiation beam? Come on. What will happen?

Vice: Generally, it is not
advised to walk into the radiation beam. They have a mockup for this
facility, which is a one-to-one copy of the real thing. Of course,
without any radiation in it. And they bring in teams to train about
welding, to train all sorts of work they have to do under the reactor
face. They train in this mockup facility. They wanted to show
radiation coming out of the beam. And they used basically tape that
they put on the floor, to show where the beam more or less is. They
wanted a better way to do this. And we suggested using augmented
reality in this case, because the facility is already there. The
rooms are very big and they have to do stuff in the room. They have
to weld. They have to do physical activity. So it really lends itself
beautifully for augmented reality. And this project we opted for the
Hololenses, which was the only one available when we started working
on this, almost two years ago. And we discovered a lot of things in
the process.

I’ll talk about the end result and then
I’ll talk about the process itself. In the end result, the user puts
on the Hololens and he can see the radiation beam coming out of a
specific fuel channel tube behind the wall. If he walks into the
radiation beam, he gets a simulated virtual dose. And if he stays too
long, eventually he’s fake dose meter — which is something you wear
in your pocket or on your lapel — will start to vibrate and tell him
that he needs to get out immediately. There’s an administration
station, where the operator or the person in charge of this training
session can see where everybody is in space. He can see everybody’s
fake virtual dose and he can control which tube is open. He can do
all sorts of activities or he can set warnings. And he just oversees
the experience or the training session. That is it, right? There are
five Hololenses, all network together walking inside this facility
and all seeing the same radiation beam.

Alan: It’s pretty amazing the
fact that you went from taking tape lines on the ground — like,
literally duct tape on the ground, saying “don’t walk here or
you’ll die” — to being able to visualize that in a three
dimensional space that is as accurate as possible.

Vice: And the cone, the
radiation cone coming out is not simple, which we discovered. It’s
not just a simple cone, it’s a complex, three dimensional object
which has different intensities depending on where exactly you stand.
And we worked with their physicists to get to model this correctly as
much as possible.

Alan: You know, normally we talk
about ROI, and we talk about what are the key performance indicators.
I think just the fact that you’re able to keep people alive, that’s a
pretty good ROI.

Vice: Yeah, that’s a nice thing
to have in any application, that keeping people alive should be on
everybody’s list of success criteria.

Alan: [laughs] Profits aside, I
think keeping people alive is a pretty good idea. So what’s next
then? I mean, you’ve got Geiger counters, you’ve got airlock doors,
you’ve got *The Cone of Nuclear Radiation*. What’s
next? What is the next thing that they’re working on?

Vice: For them, we’re working on
a bunch of things, but nothing I can talk about at this point.

Alan: OK. So what other projects
are you working on with different companies? Because, I mean, this
must have led to some other opportunities as well.

Vice: Yeah, there’s a bunch of
stuff that we’re working on. One of the more exciting ones that they
would be happy to talk about is in the medical domain. And this is
another company that I’m partly involved in, apparently co-founded
with a physician in Toronto. He’s a radiologist in SickKids Hospital.
For those who don’t know, there’s a hospital in Toronto called
SickKids, which, let’s be honest, is the worst name in the world for
a hospital. But it is a very good hospital. It’s one of the top ten
paediatric hospitals in the world. And we built a virtual reality MRI
simulator for kids. And the premise is this: I don’t know if you’ve
done an MRI, but it is a very scary experience. It is very loud and
you have to stay still in this tube. It’s very scary for children.

Alan: You know, I’ve never had.
But, yeah, it looks terrifying. You hear this [ominous pulsing sound]
you know, these clicking sounds in the big ominous–

Vice: And there’s more than
clicking. There’s like hissing and then there’s high pitched noises
and grinding sound. It’s a very, very loud. They usually give you
earplugs. But even with earplugs, it’s very loud.

Alan: Wow.

Vice: And kids– and it is a
process that takes 10, 15, 20 minutes. And kids, some kids can’t
handle it. And they need to be sedated. And sedation with kids has
risks. And it has also very high costs. And we developed the virtual
reality MRI simulator, which we call Emma
Rye, because that’s the fairy that appears inside of the VR,
she’s Emma. Get it? MRI. Yes, it’s very clever. And–

Alan: I like it. Emma Rye, I
love it.

Vice: Yeah, it’s very clever. We
were very proud of ourselves. She comes up inside of the VR
experience and she walks the child through this. What to expect and
how to deal with it. This is an experience that a child does either
before– either immediately before the actual scan, or a few days
before the actual scan. It’s something you do before the procedure
itself. Because you cannot take a virtual reality headset into the
MRI, because of the magnetic fields involved, right?

Alan: Would be very bad.

Vice: And we’ve run a bunch of
kids through this. And so far we had really, really good results.
We’re starting clinical trials soon, which is something that’s
mandatory for any kind of this application, to get real scientific
data. Anecdotally, I can tell you that every child sedation costs
about $50,000, because you need anesthesiologist there. You need to
usually keep the child overnight. The costs really just explode
whenever there’s sedation involved. The waiting time for an MRI
without sedation is about a week, and the waiting time for an MRI
with sedation is about six months.

Alan: So if you can use this
tool to help kids get through this without sedation, you’re saving
them six months and $50,000. Is that correct?

Vice: Just one child, yes.

Alan: Wow. That’s incredible. Oh
my god. This is– why isn’t this in every hospital?

Vice: Exactly. So we’re working
on bringing this to every hospital.

Alan: I want this for needles,
too. My daughter is terrified of needles.

Vice: Yes. I also have a son
that’s terrified of needles. And we’re working on other distraction
therapy, physical distraction therapy as well. And it works. It’s
particularly– it works for everybody, but for children, it seems to
work particularly well, just build the experience carefully and you
have to do clinical trials, which also unfortunately takes a long
time. But we’re working on it.

Alan: With something like VR, I
wonder if it really requires clinical trials, because if it works, it
would be no different than putting a TV, for example, entertainment.
You know, I wonder if you could position it as an entertainment
device rather than a medical device.

Vice: Yeah, it is actually. But
even even for that, you’re still in clinical trials, because you want
to see real numbers. You want to see scientific evidence, right? And
clinical trials is the only way to do it, especially when–

Alan: It’s true.

Vice: –it involves kids. You
want to be a very rigorous way of measuring success. And clinical
trial is the way to go.

Alan: You’re working on this–
It’s VRAL, right?

Vice: That’s name of the other
company, VRAL.ca. We have actually a separate website for this MRI
simulator, which is emmarye.com.

Alan: I love it, I love it. Emma
Rye. Very Canadian too, we drink rye instead of other types of
whiskey.

Vice: The VR experience itself
has two parts. One is more of a game where you do first and the
fairy, which is called Emma Rye shows up and she talks to you, the
biofeedback loop where the child is encouraged to stay still and get
acclimated to all of these noises and all of this environment inside
of the MRI. And it’s kind of a process that builds up to a real full
blown simulation of the MRI scan itself. The whole thing you do in VR
first. And doing it in a controlled environment and doing it in a
game-like manner really helps children. And we’ve seen several cases
already just in a small, very small pilot that we’ve done in a few
locations in Toronto, in Stanford, in San Francisco and soon in
Israel. We see really good results.

Alan: I mean, what an amazing
use case for virtual reality, being able to take kids through an MRI
in a way that reduces their stress, probably gives much better MRI
results because they’re staying still. They’re experienced with it.
They’re not– it’s not this big scary tube that they’re going to go
into. They have an understanding of what it is, what’s coming,
they’ve heard it before. How did you guys do the spatial audio, or
how did you do the sounds?

Vice: We did a bunch of things.
We recorded audio. You cannot stick a microphone inside of MRI, of
course, because the magnets will tear it apart.

Alan: Oh, that’s right.

Vice: But you can record inside
of the room. We’ve done that and then cleaned it up and also
synthesized a bunch of audio just by using that as a reference. And
we also found one MRI machine that was powered down. Again, this is
not something that most people know, but these machines work 24/7 for
years. It is very rare to find a machine with a magnet powered down.
Because it costs a lot of money to power it up. It uses liquid
nitrogen, and you have to flush the entire system.

Alan: Oh, OK.

Vice: It takes a few days. So
they keep them running 24/7. So we found one that was down for
maintenance and we kind of recorded stuff in there, 360 video, which
we used later for reference, and all sorts of measurements we did.
And we use that to model everything.

Alan: How did you find that?

Vice: Again, my partner’s a
radiologist, so he knew people.

Alan: Ah, he’s like “Hey, I
know this machine’s not working right now.”

Vice: “I have a guy who has
a machine. It’s cool.”

Alan: “I got a guy.”
It’s like, yeah, I got a guy for a car parts and you got a guy for a
spare MRI machines just kicking around.

Vice: Yeah.

Alan: Amazing. So you’re working
on nuclear reactors. You’re working on MRI machines. There’s
something very radioactive about the work that you guys are doing. So
are you yourself really excited about this whole field of radiation
or what’s the deal?

Vice: I don’t know. I mean,
there’s just something that happened. The first time I was in the
reactor, I was wearing a what’s called the rubber suit. It’s just
this big yellow thing that they wear as protective gear. And I took a
selfie with the reactor core in the background, and this giant rubber
suit all over my face. And I sent it to my wife and she said, “What
are you doing? Get out of there right now!”

Alan: [laughs] No kidding.

Vice: Of course it was a mockup.
I wasn’t in a real–

Alan: We have to use that. You
have to send me that photo. We’ll use it as the photo for this
episode.

Vice: Sure, I’ll sent it to you,
I have it somewhere.

Alan: Amazing. So what are the
things like– you guys are really– let’s face it, you guys are an
R&D lab making cutting edge stuff. And what is something that you
see in the future of virtual/augmented/mixed reality, AI, that you’re
like, “Wow, I see this coming. I want to start working on the
R&D for it now.”

Vice: That’s an excellent
question because we are working on a few things that I’m specifically
excited about. We are — like everybody else — are kind of waiting
for better augmented reality headsets to come out. What we have now,
the Hololens and even the Hololens 2, and the Magic Leap. They’re a
good start, but they’re very limited in terms of field of view, and
battery life, and size, which is very big. So everybody’s waiting for
the next generation of augmented reality headsets, which will really
open up the market. The market for AR is ten times bigger than the
market for VR. But we’re not there yet. The technology is just not
ready. So that’s one thing we’re kind of keeping on the back burner.
And always, always keeping an eye out and doing experiments. Another
thing I’m really excited about is volumetric video.

Alan: Yeah. Let’s talk about
volumetric video. Speaking of which, the people that put on VRTO —
that you talked about at the very beginning — Keram
[Malicki-Sanchez] and his team, they also host the FIVARS, the
Festival of International Virtual and Augmented Reality Storytelling.

Vice: Yep.

Alan: And I actually went
yesterday and I recorded about a two minute video in volumetric and
they’re using 180 instead of 360. They don’t need to have behind me
and stuff. But they were just recording volumetric. But the way they
did it was, it was Joanne [Popińska] and her partner, Tom [Hall].
And the great thing about it was when you’re in VR looking at this
capture, most people are not going to walk around the back of
somebody talking to them. Let’s be honest. Like when you’re– when
somebody’s talking to you, you’re not going to walk around the back
of them. So what they did was they just focused on the front facing
capture. So they’re using video cameras to capture it in stereoscopic
and then using a depth sensor to give it that sense of depth. And it
just– it felt like the person was right there. And so they recorded
me with that. I’m really hoping to see this, but it felt like I was
right there in VR with them. And it was like a one-to-one, they were
talking to me, it was really cool. So volumetric video is something
that I think we’re going to see a lot more of as well. Have you
started with volumetric capture rigs, have you messed around with
that type of thing or…?

Vice: Yes, a little bit. We’ve
worked with DepthKit with a bunch of Kinect cameras. I just finished
working on a project that wasn’t ours, but I was part of for the
Dallas Cowboys. This is a project that’s making a lot of headlines
right now. And there’s also why, as you know, I’ve been in Dallas for
the past three weeks.

Alan: Yeah, I saw it today. You
can get a selfie with the Dallas Cowboys players in AR.

Vice: Exactly. We did a fluffy
with AR selfie with a volumetric video of the Dallas Cowboys players,
and people absolutely loved it. We just did it in the first game of
the season. And they used a capture rig in San Francisco– no, sorry,
in LA to capture the players.

Alan: Were these– they used
Metastage?

Vice: I think so, yes. They used
Metastage and another company as well.

Alan: Cool. Very cool. Yeah. We
had Christina Heller on the show from Metastage on an earlier
episode, so
you can listen to that. We also had Raj
Puran from Intel, and Intel’s got their Intel Studios volumetric
capture stage, which is a 10,000 square foot volumetric capture
theater, I guess, stage. It’s crazy.

Vice: Right.

Alan: So what do you think is
the future of volumetric capture then?

Vice: So very much capture, in
my honest opinion, is going to disrupt the entertainment market in
the same way that HD did 10 years ago, where it took a long time, but
eventually HD completely took over the market. I expect we’ll see the
same process starting with volumetric capture. It’s going to take 10
years. It took HD 10 years, it’s gonna take volumetric capture also
10 years. But eventually we’ll have that it will become standard, and
it will take a while.

Alan: It’s interesting that this
is such a hot topic. I’m writing an article on the different
volumetric capture rigs around the world right now, and in doing our
quick analysis we came up with 55 companies that are doing volumetric
capture, and we’re in 2019 right now. So I think that’s only
scratching the surface and Metastage is certainly out there in front.
But you have companies like 8i, Jaunt is working on it now. So
there’s a number of companies working in the volumetric space. But to
be honest, it’s not that difficult to capture volumetrically. It’s
just difficult to do it well and compress the data.

Vice: Exactly. Everything is
difficult to do it well. And it takes a lot of money right now,
because you need a massive amount of cameras and you need a massive
amount of computing power to process it and stitch everything
together. We are very interested in it from the training perspective
because volumetric video in training would be awesome, right?
Something that I’m really interested in.

Alan: Yeah, absolutely. There’s
so many opportunities there with training. I mean, you can– I almost
feel like there’s gonna be two camps. There’s gonna be volumetric,
real people doing acting and being caught into VR and AR or whatever.
But there’s also gonna be CGI actors and AI driven actors. And I
don’t know– in the long run, I think the CGI ones are gonna win out,
personally, because you can spend the money upfront, record, make an
avatar, and then that avatar can be doing anything. You can have its
head spin around, you can have it talk in multiple languages. There’s
so many things you can do with it. Whereas volumetric video, you
capture it once. And if you need to change something, you’ve got to
go and recapture it.

Vice: Well, actually, you know,
you can do a lot of stuff with volumetric as well. You can rig up
people and then change their movements. You can do a lot of things.
It’s just– we’re just scratching the surface, as you said. So all of
these tools are still being built, but we’ll see a lot of stuff
happening and we’ll see a lot of hybrid systems, where it’s a mix of
CGI, 3D models, volumetric, real live 2D video. It’s going to be a
whole mess of content.

Alan: And it has to. Here’s the
thing. We’ve got many, many devices. So we’ve got a AR headsets,
mixed reality headsets or Hololens and Magic Leap, we’ve got the ones
that are capturing the world around you, as well as projecting onto
the world. Then you’ve got things like Nreal, or some of the glasses
that are just heads-up displays, they’re not really capturing the
world around you. And then you’ve got something very simple, the
glasses that just give you a little heads-up display, almost like the
Google–

Vice: Google Glass.

Alan: [That’s]
what’s it called? Google Glass? Those types of things. And
they’re bringing real enterprise value right now. And as we push the
limits of this technology between you, the stuff you guys are doing,
the stuff we’re doing, we’re really pushing the limits of the
technology. How far can we push it? Do we need haptics, spatial
sound, all of these things? And then you look at a company like
Strivr, and they’re just using 360 cameras to capture people in their
workplace and creating a very simple process pipeline for taking
knowledge from one person and giving it to many. I think maybe
sometimes we overthink these things.

Vice: You know, there’s a place
for all levels of simulation and 360 is one of them. It’s good for
some things and not good for others. Same with VR. They all have
their pros and cons, and the same with the volumetric video. We just
have to find the right place to do this stuff in the right times.

Alan: Absolutely. And I think
that’s one of the reasons why we took a different approach than most
companies. Most companies out there are building a 360 video editing
platform, or they’re building an AR editing platform, or they’re
building a VR collaboration. We took a different approach. We said,
“look, different types of training, different types of learning
are going to require different types of technologies.” As you
know. I mean, you just take Packet39 and look in there. You’ve got
VR, AR, spatial audio. You’ve got everything in five different things
and they all overlap somewhat.

Vice: Yes.

Alan: And the technologies
overlap and the thing is, who knows what’s going to be invented in
next year, or this year, or tomorrow? And so being able to keep our
platform open so that new technologies can be developed not just by
us, but by anybody and plug it in so that we really become the
central hub for learning is what our goal is. And I think that’s
really important because we were trying to keep up with everything.
We were building volumetric capture rigs and we were building CGI
avatars and we’re trying to drive them through artificial
intelligence using IBM’s Watson. And in doing all those things we’re
like, “Holy crap, there’s no way we can possibly keep up on all
this.” So why not focus on bringing together the best talent in
the world, the best platforms, plugins, all of that together in a
community where everybody is contributing to the long term success of
training and education? Sorry, my little rant there, but this
conversation with you was really kind of hammering it home for me.
It’s like all the great research that you’re doing is a drop in the
bucket, when you consider all the research everybody is doing right.
And while the stuff you’re working on and what you’re doing is very
important and it’s very hard to do, if we can harness everybody’s
work together, I think we can truly democratize education and
learning globally, within the next 20 years.

Vice: Yeah. Open it up for
everyone.

Alan: Exactly. And if you look
out 10 years, you talked about the next generation of glasses and
stuff. So look out 10 years. We’ll have five generations of glasses
ahead. It’ll run on 6G, not even 5G. Another hundred times faster,
which is ten thousand times faster than what we’re doing now. And the
glasses will be cheap. The compute power will be in the cloud, not in
our face. So we’ll be able to make the glasses really cheap. And the
content development will be inexpensive too, because all the work
that you guys are doing and all the companies and individuals and
universities and labs around the world is really laying the
foundation to make content creation inexpensive and easy to do. And
once we have that, we can have global scale of education,
democratization.

Vice: Amen.

Alan: All right, my friend.
Well, we’re gonna work on it together. Before we go, we’re at the 40
minute mark. So I want to make sure that I use this time wisely,
because I really love learning from you. What is the most important
thing that businesses or customers can do to leverage the power of XR
technologies right now?

Vice: I would say keep an open
mind, because we deal with this stuff all day. But the clients, the
customers, most of them are still not even experienced VR, right? And
they don’t– it’s hard to sell VR to somebody was never tried it.
It’s like selling a color television on a black and white television.
You have to see it for yourself, you have to try for yourself. So
keep an open mind. Let us kind of show you what can be done and ask
the right questions. Is this a good idea to do in VR? That’s a good
question. And we try to answer that truthfully, right? We don’t push
VR just automatically. The question should be, how does this help my
training? Does this make my training safer or cheaper or easier?
Where can I try this? Show me a demo. Show me how this works. Show me
how it feels like. And most important question would be. Tell us
about your experience in VR, what do you think we should do in this
training?

Alan: That’s a great question.

Vice: You get this a lot, where
you deal with clients that they come through with some kind of idea.
And because they don’t have experience in designing this stuff, it’s
not necessarily a good idea. You can see where they’re going. You can
see where they’re trying to get to. But the way they’re trying to get
there is not necessarily a good way to do it. So just keep an open
mind, because it’s such a new medium that very few people actually
have the right tools and the right experience to design for this. And
your stuff that you’ve done– I’ve seen a team that worked for ages
in video or in desktop applications, and a lot of this stuff is
simply not applicable to VR. It’s just bad idea.

Alan: Yeah, I agree. And the
medium is the message sometimes. And being able to use a medium where
you can have something, create an entire environment that doesn’t
exist, train people on it and run people through it is a magical
experience, but sometimes it’s not necessary. Sometimes just a video
will suffice. And being able to choose the medium to the message and
match that is great advice, Vice. Great advice from Vice!

Vice: Yes. And “Vice”,
by the way, is how my last name is pronounced. So my name is Shachar
Weis.

Alan: All right. Got it. I’m
just going to call you Vice from now on.

Vice: It’s easy. Everybody does.

Alan: [laughs] Amazing, man. So
my last question, what problem in the world do you want to see solved
using XR technologies?

Vice: Ooh, of course. Sedation
with children before MRI. That’s my main goal right now.

Alan: Just to take all the money
off the table, take all of the crazy things we do in this industry,
from nuclear training to everything. Being able to make an individual
child more comfortable during a procedure that is terrifying is a
really, really great way to use this technology. It’s worth it. So
thank you for all the work you put in, man. And thank you for being
on the show.

Vice: It was my pleasure. Thanks
for having me.

Sound effects sourced here,
here and
here.