Combining the Best of AR and VR with Varjo's XR-1, featuring Niko Eiden

XR for Business

English - September 18, 2019 09:28 - ★★★★★ - 12 ratings
Technology Business Homepage Download Apple Podcasts Google Podcasts Overcast Castro Pocket Casts RSS feed

Previous Episode: Helping Firefighters Douse Blazes Around the World, with RiVR's Alex Harvey

Next Episode: It's Okay to be Small, with Virtual Reality Marketing's Terry Proto

The human eye
is a wonderful and complex thing, and it's a technological feat just
to even come close to its natural revolution. Well, the folks at
Varjo have created something that is pretty darn good at it. Alan may
have trouble remembering how to say their company's name, but he can
attest to the clarity of the XR-1's display. Varjo CEO Niko Eiden
comes by to give us a look behind the curtain of its creation.

Alan: Welcome to the XR for
Business Podcast with your host, Alan Smithson. Today we have Niko
Eiden, CEO of Varjo. He's formerly held top product leadership
positions at Microsoft and Nokia. At Nokia, Niko led a product
program team in 2006-2007, and together with researchers from Nokia
Research Center, his team developed the basis for the optical
technology that later became the Microsoft Hololens. Niko has a
Master's in science in aeronautical engineering. Varjo is redefining
reality by jump starting a new era in computing. Their hardware and
software lets people seamlessly mix realities together, moving from
the real world to extended reality into pure virtual reality, all
with human eye resolution. Their new headset, XR-1, is a mixed
reality developer device for engineers, researchers, and designers
who are pioneering a new reality. With photorealistic visual
fidelity, ultra-low latency, and integrated eye tracking, the XR-1
seamlessly merges virtual content with the real world for the first
time ever. If you want to learn more about Varjo, you can visit
varjo.com. And I want to welcome to the show, Niko. Thanks so much
for joining me.

Niko: Thanks, Alan. Nice to be
here.

Alan: It's my absolute pleasure.
We got a chance to meet at AWE this year and I got to try the XR-1,
which was -- oh my God -- what an incredible experience. You put on
the headset, the pass-through cameras were as if I didn't have a
headset on at all, I could just see the whole world. And then all of
a sudden a car appeared in front of me; in the space I was in, there
was a car. Then I got in the car, and the space around me
disappeared, and I was in the car. And the one thing that really
stuck out with me that blew my mind was. I was looking -- it was a
Volvo -- and I remember looking at the steering wheel. And the little
Volvo symbol in chrome was so crystal clear. It looked like a real
car. Let's talk about your technology and how you guys ended up at
this place.

Niko: Sure. We really had the
initial vision-- the founding team, we had a long background in
different augmented reality and VR devices. And we had always been
talking about that video see-through type of devices could actually
combine the best of both worlds of AR and VR devices. And every time
you started talking about them, somebody kind of commented basically
that don't bother: the latency, the lag between what you're seeing
through the cameras, and what's happening in reality is gonna be too
long. Or the resolution is not going to be good enough, it's not
going to look great. But we were in the summer of 2016, we were
looking at a demo and the demo was shown with the Hololens first gen
device. And I got really thinking that this would be such a cool
demo, but it's really missing a big part of the experience, because
it wasn't able to show the image in a photorealistic fashion. It was
a very high fidelity graphic scene that we were looking at with
Hololens and we were just thinking that this would be so much better
with a VR device. And combining the reality with a video see-through
device could actually really work. We were a bit curious. We had some
extra time at that point together with Urho [Konttori] -- one of the
other founders -- and we built our first prototype in 24 hours, and
the experience was really magical. So it was a very simple
experience. But the main thing of that first experience was that we
were able to dim an existing room and make it

Niko: Thanks, Alan. Nice to be
here.

Alan: It's interesting, I
actually tried-- there was a company based in Toronto -- and I can't
remember the name of them -- but they had a VR device with
pass-through cameras, and then they had this demo where a giant
crashed through your ceiling and it picked you up. And so it was this
blend of real world and then mixed reality, and then you were in full
virtual reality. And that was my first foray into it. But to your
point, the resolution just wasn't there and there was such a lag in
the latency that it kind of made me feel queasy, and I felt none of
that with your headset. It was crystal clear, and not only crystal
clear with the latency, I waved my hands. There's actually a photo of
me waving my hand in front of my face, because I just couldn't
believe that I was seeing my hand. Your secret sauce or your magic
that's behind it is, you've kind of created almost like a false
foveated rendering, meaning if you look directly in front of your
eyes, your fovea are 100 percent focused. And then as you move out
from there, it gets less and less focused. So if you're staring at
your computer and you hold your hand out to the periphery, your hand
is kind of blurry, you can't really see it in detail. You guys have
leveraged that by putting a higher resolution screen within a screen.
Can you walk us through that?

Niko: Yeah. So that was actually
the first pivot that we had to make. As I mentioned, the big vision
was to do mixed reality with video see-through device. But in the
beginning, we didn't plan to set up a company that would produce VR
headsets. So we actually were thinking that it would make sense to do
a good accessory and equip the existing VR headsets with a very good
accessory that would solve the latency problem. We had a few ideas
how we could crack that problem all the way from the very beginning.
But when we were testing it, fairly quickly we realized that we can't
leverage any of the existing VR headsets on the market still today,
just because the resolution was not on par what we could achieve with
the video see-through system. So fairly early on we decided that
okay, even though we want to do the mixed reality with the video
see-through, we have to park it for a while. We we have to work a bit
in a serial fashion on this problem and we have to tackle the
resolution problem on the VR headsets first. The idea that we came up
with, as you mentioned, was exactly that. So we are utilizing four
displays in a headset, so two displays per eye. And we are using more
of a traditional setup that we call "context display" that
provides you the full field of view and you get the periphery and
that doesn't have to be in such a high resolution because the way how
your eye works, it's impossible to see and in high detail for the
periphery part. And in the center we have a second display that we're
overlaying with an optical mirror, and that way we are able to mix it
almost borderlessly in front of the complex display we call "focused
display." And this display is able to match the human eye
resolution by something that we call "60 pixels per one degree
field of view" in that high resolution area. Very early on we
had a target that we want to be able to read a newspaper in VR. That
headset is able to perfectly allow you to read a newspaper in VR.

Alan: Now, for those who haven't
spent a lot of time in VR, that is something that is a real challenge
for VR, is just reading text is a problem. And there's companies like
Monotype, who has created all the type fonts that you see, -- Times
New Roman and all these things -- but they actually started creating
type fonts that would be visible in AR and VR. Then you guys took a
different approach and just said, well, the problem isn't the text,
the problem is the resolution.

Niko: Yeah. And then there were
quite a lot of restrictions, and still are. I mean, one is the
compute power available on desktop PC. So if everything would be high
resolution VR, we would talking about not 4K display; we'd need,
what, 16K, 20K displays that would be required to pull it off in a
reasonable field of view. So that was not the track that we could
take. I mean, the displays didn't exist, the processing power didn't
exist. But with this way, by combining these four displays that
existed on the market, -- basically off-the-shelf components -- and
combining them in a smart way allowed us actually to do something
that we can produce today, and that will work with the computers of
today and provide you this region of high resolution. It would be
bombastic if everything would be high resolution and you couldn't
distinguish anything anymore from resolution perspective, but
unfortunately that we aren't get there.

Alan: From a user's perspective,
when I put it on, I didn't notice that the periphery wasn't focused.
As I looked at the steering wheel, I looked around the car, I looked
at the leather stitching and actually my brain just totally ignored
it. It wasn't until the second part of the demo -- I was looking at a
whole area where there was a moose crossing, and it was really neat
-- but it wasn't until then that I kind of realized that there was a
little bit of a border around what I was looking at. The way you guys
have done it, with the super high resolution in the middle and it
kind of almost feathers or fades out to the other resolution. It's
almost imperceivable if you didn't know what to look for. I think
it's really magical. You guys have a partnership with Volvo. How did
that come about?

Niko: The Volvo partnership, we
started off developing the VR-1, which is our headset that brings
human eye resolution to the market and brings an eye tracker to the
market, that also differentiates us from pretty much all of the other
companies out there today. But the Volvo case was a bit special, and
Volvo had a vision of stuff that they wanted to do in the mixed
reality space, and our dream of doing mixed reality with a video
see-through actually matched. And we were able to offer them first
very early prototypes, so that they could start testing. And it
resonated really well. So Volvo had a fantastic engineering team. So
they had very good capabilities of actually doing work from an
engineering perspective. And for us, it was a fantastic case to
pressure test whether the vision, whether the product that we were
actually signing made sense from an enterprise perspective, and in
the case of Volvo it definitely did. So it was these kind of early
discussions. It started around the VR-1, doing VR in high resolution.
And we mentioned that we have this mixed reality prototype that we're
still working on, and we were able to show. It clicked from there on
that, and we collaborated pretty tightly from there on. We were able
to finalize and understand what an enterprise company would need and
Volvo could get their hands on very early on something that simply
wasn't available anywhere else.

Alan: What are they using it
for?

Niko: Volvo is planning to
accelerate their design process for cars. The team that we've been
working on is mainly working on the interior of future cars. And if
they have a new idea, with our device they can actually test it
before they have to build a single physical prototype. So what they
can do, is they can replace the interior of an existing car and
virtually replace it with a future car interior. And they can start
testing the interior of the instruments, switch locations. And with
the eye tracking, they can actually do proper testing so they can
drive on a real road. And with our headset, we are able to segment
the window so that the windows from inside the car show the reality.
And that view is to our camera setup and then the rest of the
interior is then the future car interior, which is completely
synthetic and created graphics. And with the eye tracking, they can
light up, for example, a warning light and they can check in a real
driving condition how quickly a driver could, for example, see that
light, just has a crude example of the stuff that they could do. Or
head-up display system, mixed graphics and show-- your virtual moose
you mentioned, they can show it while driving. There doesn't have to
be a real moose, but they can simulate the moose and they can start
testing driver reactions.

Alan: How are you capturing
the... I guess the window views? Is that through a 360 camera setup
or something like that?

Niko: No, we have the
see-through cameras, obviously, and then we have a full model of the
car. If you want to replace the interior of the car, you have to have
a full model of the car, and then you just have to have good
calibration so that the tracking of the headset and the head pose is
synchronized and calibrated.

Alan: So, wait a second. Hold
on. People are driving a real car wearing the headset?

Niko: Yes.

Alan: Oh, wow. So they're
getting in a car, putting on this headset -- I'm assuming there's a
computer in the back seat or somewhere -- and they're driving around
a closed track.

Niko: Yep. Officially, it is a
closed track. Absolutely.

Alan: Wow. Oh, my God. That's
next level product testing.

Niko: And you don't have to
build a single thing in order to start testing. So this early on
testing, fail fast type of stuff, completely new world from a design
perspective.

Alan: So what are some of the--
it's almost silly to talk about ROI because they're probably seeing a
dramatic decrease in the design times. Do you have any data around
how this is benefiting them?

Niko: Yeah, we have a discussion
on this with Volvo. They didn't want to go into detail about the ROI
on that one, but it's as you mentioned, it's extremely obvious. Being
able to create a faster design cycle and especially doing
auto-validation of something that's still virtual, just on a computer
is pretty bombastic. The interior design is just one example. Car
design in general, I guess you've seen pictures of these full-size
wax models as well, where car designers go around a full-size car,
built out of wax and modify the lines.

Alan: It seems so antiquated.

Niko: Yeah.

Alan: [laughs] Now that we have
VR, it seems so antiquated to do because one, you have to have the
physical model. Two, you have to have everybody physically there to
look at it. And three, you can't make any changes on the fly.

Niko: Yes.

Alan: So dramatic, dramatic
decreases in design times, which as we're entering into this area
where AI, and VR, and XR, all these technologies are culminating in
creating this exponential growth pattern. Every single efficiency
that we can afford these companies is going to be snapped up and used
really, really well. So this is a great way to do it.

Niko: It's not just vehicle
design. I mean, architecture, everything. The fact that you're able
to see photo-realistically a room, and see how the light plays inside
the room is big. Previously you had to imagine that stuff and wait
until the building is built. Now you can experience light inside a
building before it's been built.

Alan: There's so much happening
around ray tracing as well, having light shine in from different
angles, reflect off of materials. There's been so much research done
on that, Unreal Engine and Unity are really pushing towards that.
"How do we get photorealistic renders without blowing up your
computer?" What are some other businesses that are using this,
and how are they using your headset?

Niko: Well, apart from design,
training is another really big focus for us. Easy to imagine is
really high profile training of airplane pilots, for example, if you
are able to reduce the simulator time required to train a pilot, for
example, by allowing them to retrain in a virtual cockpit beforehand.
Again, the business case is absolutely a no-brainer for even aiming
for a high-end device like ours. But it's not just the pilot
training. It scales down to everything, to all professions where
there's something unusual might happen during your day of work, but
it's hard to prepare and train. So emergency rooms, control rooms,
police, firefighters, those type of professions, I think they will
benefit immensely from proper mixed reality and virtual reality
training programs.

Alan: You know, people say,
what's the killer use case for VR? Well, I think training is the
killer use case and design is a close second to that. Can you talk
about some other specific examples of companies that are using your
device, and how they're using it?

Niko: We are still learning how
companies plan to use this device, but it varies quite a bit. And for
us, the key learning has been also we don't know how a business or
enterprise wants to use a mixed reality or VR device. Running a
business and then, for example, designing something or training
something that's really the core of that company. We need to work in
close collaboration to understand their specific use case. And for
those companies now thinking whether they should deploy or think
about using mixed reality or VR: for some specific cases, yes, there
will be very clear places to read how you could use it for your own
business, but in most cases you need to understand what you can do,
and then you need to start dreaming. And the good news is there are
thousands of companies out there willing to help, to program and
create the software tools required to do stuff, especially in
enterprise and business segment. But it needs to come from the inside
out. And those companies who are kind of really advanced, like Volvo,
like Audi, for example, a lot of the car companies that're really
advanced, they usually have a big team of specialists, who are able
to support than those people who have the need and were able to dream
the use case. And let's say in an automotive company, they can use it
for the accelerated design process that we discussed. They can use it
to accelerate the design process of their car factories and they can
use it to help configure a future car model that they selling to an
end customer in a showroom. So there are plenty of different
completely different use cases with completely different needs, even
within just one large corporation. And to me, that's super exciting.
But it does require that those companies are pretty active themselves
as well.

Alan: There's really no part of
any business-- and I published a post on LinkedIn a couple weeks ago
and I said: "Is there businesses that won't be impacted by this
technology?" Can you think of any?

Niko: No. I have a strong
feeling that this will change fundamentally how we will be working in
general, and I think everybody will be impacted one way or the other.
This is kind of a pretty bold statement I'm making. But if you think
about the tools with what we are working in general today, it's a
phone, it's a PC, maybe in some cases it's a tablet. That's it. It's
very restricted, it's very 2D. Once you start to have a device that
can match the resolution of reality -- from the VR or mixed reality
perspective -- you can move all of the existing tools and make them
part of the virtual experience, the mixed reality experience. And
that's why I have a feeling that this transition, once these products
start to be out there, is going to happen faster than we anticipate.
And when we hit the first professions where using virtual reality or
mixed reality will be part of their daily routine everyday and it's a
few hours, I think that's when we start to see a really big
transition happening also, from professional space all over.

Alan: I was at the PTC LiveWorx
conference in Boston just recently. And some of the use cases that
they're bringing online now are not even really mixed reality so much
as the RealWear headset where you can just see a little screen out in
front of you and give you a heads up display. And what they're able
to do is capture the standard operating procedures that experts know
intuitively because they've been doing the job for 20 years. As those
experts start to retire, they need to capture that knowledge somehow,
and then transfer that knowledge to younger workforces. And I think
when people start wearing this a couple hours a day on a regular
basis to help them with their work, and I think in some very small
instances that's happening already and the results across the board--
Shelly Peterson is coming on the show and she's talking about how
Lockheed Martin is using augmented or mixed reality glasses for
assembly. The original trials that they did had 85 percent reduction
in task completion times. 85 percent reduction! So when you have
these crazy reductions in speed-to-product or speed-to-development,
if you can take a process that takes a year of designing a car and
shrink that to six months, that is a massive savings to a company.
Where I'm going with this is that, it offsets the cost. If, for
example, it costs me $10,000 or $20,000 for a VR headset that shaves
millions off of my design process, who cares what it cost at $20,000?
It doesn't matter. You guys have the premium headset in the market.
Talk to us about the costs with assembling not just the headset, but
the whole package. What would something like that cost to fully
outfit one designer with a setup, the ideal setup?

Niko: It's around the 10K mark
and that includes already the state of the art workstation. It's very
similar than a few years ago, just to workstation alone from a
pricing perspective. We haven't seen the price be really the issue of
adoption. It's more about the day to day practicalities and how do
you work. But I think one great example has been on the architecture
and civil engineering side. For them, the benefits are huge. I feel
that they are fairly slow in picking up new technologies in general.
But if you think about what happened during the 90s, I was an
engineer at that point. Then I started my university studies by
drawing with a pencil to a big white paper. And when I ended,
everybody had transition to a CAD system. And that happened all
across the world, not just in the university, but all the companies
transitioned as well. So civil and architect engineers -- from my
perspective -- they are fairly slow to pick up, but once they pick
up, it's a huge snowball effect that everybody does it at once. And
that's something that I'm expecting and hoping that would happen as
well, from a VR/mixed reality perspective fairly soon.

Alan: Your timing is probably
very, very perfect because we're also seeing this as well. You get
these proof points out and it's only recently in the last kind of six
months where big companies are releasing their original pilot
information. We're just starting to come out of pilots with some
companies. They're going, "Hey, Lockheed Martin, we decreased
our time to production by 85 percent." Macy's is using VR for
sales in their stores and they-- averaged across 110 stores, they've
increased their sales by 45 percent and decreased their returns from
7 to 2 percent. So once you kind of start to see these use cases, if
you're going to remain competitive and your competitors are seeing
these kind of transformational shifts, it's only a matter of time
before you have to do it. It's not nice to have, it's a must have. So
what is your timeline around where most companies start to roll this
out? Is it the 3, 5, 10 year rollout or what are we looking at?

Niko: We are seeing every
Fortune 500 company looking at this stuff. Some are doing trials and
pilots still fairly early on. Some are super advanced. It's hard to
say a timeline, but what I can say is that the transition has
started, and this is going to happen. There's been a lot of
discussion, whether VR or mixture reality. Is it real? Will it take
place? I mean, the consumer side didn't work out as everybody was
expecting a few years ago. But from an enterprise perspective, from
the perspective of using this stuff to accelerate and make your work
more efficient, I think that has already started and there's no way
to stop it anymore.

Alan: So what is the most
important thing that businesses can do to get started?

Niko: Take it seriously. The
minimum is, as I said, it's very hard for externals. It needs to come
from the inside because you need to have the understanding of how
stuff works in the specific business in order to make it more
efficient. And obviously, consultants can come in and do the
interviews and then they provide you an answer. But it really has to
come from inside. So put the few creative people, give them a mandate
to study and really understand what mixed reality in the next few
years can do. And I think it will be very easy to come up with three
to five things that yes, could actually make our life a lot easier
from a business perspective. And then just find a partner with whom
to build a pilot. It's an upfront investment that companies will need
to do within their own context, but it's going to be worthwhile.

Alan: I agree. You know,
recently we started XR Ignite, which is a community hub based on the
idea that startup studios, developers, they all have a role to play.
And a lot of VC investment funds only look at platforms and products,
and they're looking for giant billion dollar unicorns. What we're
seeing is that there's going to be a lot of acquisitions around the
studios and developers, because as big companies start to realize
that they spin up a small team, they go, "Hey, this is working
for us, we need to start doing this at scale. We can either hire
people, which is hard because not a lot of talent out there, or we
can acquire a small team that's already working together, that
already has what we're looking for." And Deloitte just acquired
a startup studio, rather, and a couple of other companies have
acquired studios. What do you think the investment landscape is going
to look like? Because VCs don't typically invest in studios and
content. They're more invested in the platforms, where we took a very
holistic approach and said, "well, as a community hub, we can
all help each other get more clients, but also facilitate these
smaller acquisitions, because there's going to be a massive need for
content soon." How do you think we're going to address that?

Niko: I think in general, the
appetite from a VC perspective and the interest in VR and mixed
reality, I think it's turning, and it's turning towards the positive
side. I mean, the past few years have been fairly difficult for many
startups working in this arena. It hasn't been easy with the
software, whether platforms, whether content to get funding,
especially if it had a title VR in front of it. It's a bit easier if
you were somehow in the augmented reality domain in the past years.
But I have a good feeling, and based on the discussions that I've had
recently, I think the-- there starts to be a lot of appetite, and
this appetite is actually being fueled by these real world use cases:
the companies who haven't played a role in the VR/AR before actually
showcasing. And I think the Volvo case is a fantastic example of one
of those cases. You would definitely not put them into a VR/AR
bracket, but hey, they are showing something that's completely
advanced and then blows everybody out of the water at the moment. And
it's coming from a company who you would position into a fairly
traditional automotive sector and not into the latest stuff of
computer graphics. So I think those showcases are fantastic, and it's
gonna be easier in the next few years to get funding because there is
a clear need. There is-- the business case is there, and demand is
strong. And I think the training and simulator market is a fantastic
example. I think there are thousands of companies -- if you go to a
trade show focusing on training and simulation -- there are thousands
of companies showcasing already a product. But they're showcasing it
with hardware that actually doesn't match the needs of the product or
is somehow inferior, but it doesn't matter. Despite the fact that the
hardware hasn't been up to the standards that they would need
actually, they have already created the product and somehow they
found the investment to create that product. And timing wise, it's
from our perspective, it's fantastic. We can give them now a product
that fulfills many of their needs and they can go and promote it
again with a complete new spin. I think it's going to happen and I
think we will be seeing more and more investment in the space as
well.

Alan: I think we're already
starting to see it. I think the trough of disillusionment when people
went, "Hey, we're gonna-- VR going to be huge! Everybody is
going to have it, and every consumer is going to wear AR glasses!"
Yes, this will all happen, but it will take 10 years. And I think
that the consumer market is really fickle. It has to be perfect,
cheap and lots of content, whereas the enterprise just needs to make
an ROI. And we've already proven that across every industry -- almost
everything from oil and gas, to automotive, to training, to retail --
every aspect of business is being impacted by this.

But, let's take this in a different
direction for a second. What problem in the world do you want to see
solved using XR Technologies?

Niko: Change the way how we
work. As I mentioned earlier, for me, that's one of the big visions.
In the beginning, we wanted to change and create the future displays.
The transition from a 2D display -- like a monitor or a cinema screen
-- into a more immersive display. That's still something that I see
as a transition perspective. But before that, for me, the big thing
is to change the way how we work.

Alan: One last thought I was
thinking, people buy all different VR headsets for different reasons,
for consumer reasons now. If you buy a Varjo headset and you're
designing on it, can you still play Beat Saber on it?

Niko: Not yet.

Alan: Oh, you got to fix that.

[laughs]

Niko: Yeah, we'll fix it. I
mean, that's the problem with that is, that all of the games, they've
been optimized for the consumer grade headsets, which means very
limited resolution and two displays. So in our case, you would have
to drive four displays and you would have to have enough resolution,
so it would make sense.

Alan: Do you guys have an SDK
then, that helps people -- developers, for example -- create content
specifically for this?

Niko: Absolutely, yes. So we
have an SDK that plugs in into Unity, Unreal, all the game
development engines are fairly straightforward. We've also been
working quite extensively with the vertical engines like Lockheed
Martin, for example, with a CAD system so that you can you can use
our headset directly with those systems with which are not
necessarily on a gaming platform or available and then those consumer
type of stores. And then we are focusing quite a bit on the OpenXR
standard. And at CIGREF we're actually showing our XR device with a
pure OpenXR demo as well. So I think OpenXR is going to unify quite a
bit all the different standards and it's going to make everybody's
life a lot easier.

Alan: I really, really hope so.
Having the ability to pull this up on web, being able to do what you
need to do and then switch between programs seamlessly, I think is
gonna be where this takes off. If we can get it right with
enterprise, it's going to lay the groundwork and foundations for the
consumers to just enjoy the best resolution, the best everything at
the lowest cost possible. So, Niko, is there anything else you want
to share with the listeners?

Niko: It's been a pretty
fantastic grind. And I mean, you experienced when we came out with XR
at AWE. For us, that was a huge milestone. It was our initial vision
to do a mixed reality device video see-through. We had to do a couple
of other things before that. So we had to develop our human eye
resolution. We had to do bombastic eye tracking technology. We had to
do the tech, we had to productize it. But the fact that we are able
to bring it to the market and create the product, that's something
I'm super proud of. And it's always a big milestone, actually,
instead of just dreaming and envisioning technology, but being able
to make the technology available on a commercial market, that's a
fantastic achievement. And this is gonna be a fantastic year for us,
and I hope everybody else as well.