Letting Workers Qualify Themselves in AR, with AR Expert’s Dr. Björn Schwerdtfeger

Getting future workers excited for the jobs they might have tomorrow can be challenging, especially when many young workers tend to enjoy challenging themselves with new tasks. Dr. Björn Schwerdtfeger says that AR training can allow those workers to qualify themselves for all sorts of tasks, and have fun doing it, to boot.

[Transcript coming shortly]

Alan: Hey, everybody, and thanks
for joining in on the XR for Business Podcast with your host, Alan
Smithson. I’m really excited today. I have Dr. Björn Schwerdtfeger
from Germany. He has more than 15 years experience in augmented
reality. Together with the German industry, he’s evaluated almost
every idea for AR in applications in the industry. He’s been a
co-inventor of Pick-By-Vision at TU Munich. And during that time —
when computers for AR glasses were still carried in large backpacks
— Björn holds a PhD in industrial augmented reality as a serial
entrepreneur. And among other things, his company, AR Experts, is
advising about a third of Germany’s most important production
companies, and is shaping their augmented reality roadmaps. You can
learn more about them at ar-experts.de. And they have another product
that they’re gonna be talking about today. It’s ar-giri.com. Björn,
welcome to the show, my friend.

Björn: Yeah. Welcome, Alan.
Nice to meet you. Nice to meet you online. I’m looking forward to
this podcast.

Alan: It’s so exciting. The work
that you’ve been doing over the last few years — like a decade and a
half — is really starting to come to fruition now. I mean, all of
the hard work that you and your team have done to evangelize a
technology that — let’s be honest — 15 years ago, the technology
really wasn’t ready for the market. Tell us, how did you get into
this, into AR?

Björn: It was actually quite
funny, while still studying at the university, computer science, and
then somewhere else, augmented reality which popped up. And someone
had a demonstrator, where someone took some glasses and glued a
webcam — we had external webcams earlier — just hot-glued to some
glasses and using some [unclear] stuff and highlighting it. I think
it was just a cube. A virtual cube… And it was so fascinating that
you can bring this computer interface into the real world. Quite a
long time ago. But it was really nice.

Alan: Björn, did you say there
was a webcam hot-glued to a pair of glasses?

Björn: Exactly. That’s how we
did augmented reality 15, 20 years ago.

Alan: Amazing. You are one of
the OG, the originals of this industry. You’ve been building and
advising brands and companies around their strategy for production.
What is the one thing in augmented reality right now that you’ve seen
the most ROI?

Björn: It’s probably… we’ve
seen a lot of companies trying to do everything. Basically every
single one of us have tried to out, in the last three decades, and
failed with it. And we’re figuring out what is actually the core of
augmented reality. And the core of augmented reality is not– it’s
not a measurement tool, it’s not a tool for everything. It looks like
a display, and it is a good display. But where its core is, where
it’s so good in, is in communication. It displays communication and
augmented reality is big. It’s so much more close to your reality,
that perception is getting much better. So what you tried to
communicate with exosheets, nice PowerPoints; it’s getting so closer
to the user with augmented reality. And they figured out that the
communication got so much better using augmented reality — using
*good* implemented augemnted real

Getting future workers excited for the jobs they might have tomorrow can be challenging, especially when many young workers tend to enjoy challenging themselves with new tasks. Dr. Björn Schwerdtfeger says that AR training can allow those workers to qualify themselves for all sorts of tasks, and have fun doing it, to boot.

[Transcript coming shortly]

Alan: Hey, everybody, and thanks
for joining in on the XR for Business Podcast with your host, Alan
Smithson. I’m really excited today. I have Dr. Björn Schwerdtfeger
from Germany. He has more than 15 years experience in augmented
reality. Together with the German industry, he’s evaluated almost
every idea for AR in applications in the industry. He’s been a
co-inventor of Pick-By-Vision at TU Munich. And during that time —
when computers for AR glasses were still carried in large backpacks
— Björn holds a PhD in industrial augmented reality as a serial
entrepreneur. And among other things, his company, AR Experts, is
advising about a third of Germany’s most important production
companies, and is shaping their augmented reality roadmaps. You can
learn more about them at ar-experts.de. And they have another product
that they’re gonna be talking about today. It’s ar-giri.com. Björn,
welcome to the show, my friend.

Björn: Yeah. Welcome, Alan.
Nice to meet you. Nice to meet you online. I’m looking forward to
this podcast.

Alan: It’s so exciting. The work
that you’ve been doing over the last few years — like a decade and a
half — is really starting to come to fruition now. I mean, all of
the hard work that you and your team have done to evangelize a
technology that — let’s be honest — 15 years ago, the technology
really wasn’t ready for the market. Tell us, how did you get into
this, into AR?

Björn: It was actually quite
funny, while still studying at the university, computer science, and
then somewhere else, augmented reality which popped up. And someone
had a demonstrator, where someone took some glasses and glued a
webcam — we had external webcams earlier — just hot-glued to some
glasses and using some [unclear] stuff and highlighting it. I think
it was just a cube. A virtual cube… And it was so fascinating that
you can bring this computer interface into the real world. Quite a
long time ago. But it was really nice.

Alan: Björn, did you say there
was a webcam hot-glued to a pair of glasses?

Björn: Exactly. That’s how we
did augmented reality 15, 20 years ago.

Alan: Amazing. You are one of
the OG, the originals of this industry. You’ve been building and
advising brands and companies around their strategy for production.
What is the one thing in augmented reality right now that you’ve seen
the most ROI?

Björn: It’s probably… we’ve
seen a lot of companies trying to do everything. Basically every
single one of us have tried to out, in the last three decades, and
failed with it. And we’re figuring out what is actually the core of
augmented reality. And the core of augmented reality is not– it’s
not a measurement tool, it’s not a tool for everything. It looks like
a display, and it is a good display. But where its core is, where
it’s so good in, is in communication. It displays communication and
augmented reality is big. It’s so much more close to your reality,
that perception is getting much better. So what you tried to
communicate with exosheets, nice PowerPoints; it’s getting so closer
to the user with augmented reality. And they figured out that the
communication got so much better using augmented reality — using
*good* implemented augemnted reality, it’s quite important — you can
do a lot of mistakes there. But this is helping so much. And that’s
why you’re seeing currently augmented reality mainly in marketing,
because marketing is a form of communication. You see it in museums,
because museums are also a form of communication. And then work
training, because training is also a lot of communication.
Communicating the knowledge someone has to another user is key. It’s
the same in the museum. A few people have some knowledge of what has
happened the past, of the stories. This is a tool to improve the way
you’re telling stories, basically. That make sense to you?

Alan: Makes total sense. So,
when you say communications, are you meaning, I put on a pair of
glasses and an avatar pops up and starts talking to me, or maybe a
video screen? Run a certain example of what that would look like in
an industrial space.

Björn: It’s– augmented
reality, it doesn’t do a lot, but what it does, it really improves
the way we communicate things. An avatar can pop up, but it’s also
communicating technically-complex things, like someone has
constructed a car, or a lot of people have constructed a car, and a
lot of other people need to produce that car. There’s a lot of
technical challenge you need to talk about. You can make those things
visible. AR is a good tool to make things visible. Also, for training
processes, AR is really good thing, a tool to make things visible.
And also museums, you’ve got some boring things there, but then you
could put the story over it as a new layer, and the story is being
told through augmented reality. And that’s why it’s so nice.

Alan: I recently read a book
called “The Age of Smart Information” by Mike Pell, and he
talks about how information now is just information. I mean, we have
access to the world’s information — I can Google pretty much any
details — but it just gives me the information. It’s not in context
to my world. It’s not in context to me. I can ask Google questions
now; Voice, Amazon, Alexa, and Google Voice. But really, what
they’re– what you’re discussing here, what you’re talking about is
being able to look at a machine and the machine providing you with
the story on how to fix it, or how to deal with it. Is that what I’m
gathering here?

Björn: Well, for part of it,
and also bringing this information, registering to the real world.
That’s what you’re saying. Yeah, you’ve got the information in
Google, but Google has information somewhere in servers, you can pick
it up using your smartphone. But actually, you want to look at a
machine or an exhibition, and you want to get information which is
there exactly for that machine. And that’s where augmented reality
can help, and where it makes so much more fun; finding the
information so much faster and with making it fun, basically.

Alan: I recently read an article
that came out. It’s a study by IBM saying that over 120 million
people are going to need reskilling and upskilling as AI and robotics
replace human workers. 120 million people. In my opinion, AR and VR
are the fastest ways to do this. But you mentioned something that
really struck me. You mentioned that it’s a lot more fun. And I
think, if you look at education as a system, it’s competing with
Hollywood movies and AAA video games. If we don’t start making
education faster and more fun, then I think we’re gonna start falling
behind, as we need to reskill people as fast as humanly possible.
What are your thoughts on this?

Björn: We’re talking about a
lot of topics now. So, one part is that 120 million people need to be
reskilled. Seems that, I guess it’s one billion people working in
production. One billion people producing things every day. I see that
are a lot of more people need to be reskilled every day, because also
the jobs are getting more complex. So when you’re working with
clients, you always try to do an internship first with the customer.
So you’re trying to do the job, which we try them. They get the
support. We then analyze the situations and realize, hey, this guys
doing a job since 10 years, 20 years, but now life is getting more
complex, because they need to handle the more complex machines. The
tasks are getting more complex, and they need to do so much more
work, so much more different work. The world’s moving faster and the
environments that you’re working in is getting so much more complex.
The answer is always “Okay how can you fix it in the future?”
The always say that you need more people. It’s always an answer,
because those people are not thinking about making it easier to work
on the more complex tasks. At least in Germany, we face a problem.
You cannot plant more people for doing your job.

Björn: This is universal, my
friend. They just did a study recently — I’ll have to put it in the
show notes — but they did a study of 3,000 youth in the US and
China. And they asked them what jobs they wanted. They gave them
seven jobs to choose from. And in China, the number one job was
astronaut. In America, the number one job was YouTube influencer.
Think about that for a second: in Western society, our kids would
rather try to be a YouTube influencer traveling around the world on
Snapchat and Instagram, than actually contributing to a business. And
the mindset of that, I think is going to start haunting us in the
future, where AI is being taught to grade five children, and grade
school in China.

Björn: And also, China used to
be the workbench of the world 20 years ago; everybody was proud and
rising up to get a good job. And now, as you’re saying, everybody
wants to be an astronaut. What we figured out is, there’s some old
people who like their job, but they don’t find new people for the
actual boring jobs. What we figured out is, we did a lot of–
obviously a lot of interviews with the workers, and they’re saying,
when you’re doing this with glasses and I can self-qualify myself for
that workplace? That’s really interesting, because on the one hand,
you don’t put production workers on some different things because
then they’re liable to do a lot of mistakes. But on the other hand,
they want to do so many different jobs. But it won’t be so expensive
to qualify them for so many different jobs. But for the business on
the one hand, they need to do it. And on the other hand, they want to
do it. So when you give them the possibility to qualify themselves
for so many more, different jobs, that’s making it interesting.

Alan: If you look at people’s
LinkedIn — especially younger people — they’re changing jobs every
three years. And that from a process standpoint — from a factory, if
you’re running a factory, you need somebody to work on a machine, and
every three years you’re having to change it out — and there are
people that are working there now, but the average age of people
working in industry is escalating above 50. And people are starting
to retire en masse. So you have this huge group of people retiring
from jobs that they’ve been in for 20, 30, 40 years. People coming
into those jobs, they’re looking at them going, “Well, I only
want to do something for three years, most. And I want to try
something new. I always want to be challenged.” And I think you
mentioned something earlier — and I thought it was bang-on — is
that we can make training fun. And by doing that, you’re not only
being able to train people on new jobs fast, so they can feel like
they’re always growing. But I think also you said that these jobs are
becoming more and more complex, which I think is actually a benefit
to everybody, because if it’s more complex, it’s actually more
challenging for the learner or for the person working. And I think
that’s really what gets people excited, is a real challenge in life.
What are your thoughts on that?

Björn: As you’re saying, it’s a
better fit for everybody. You can look at some jobs and say, okay, I
need more people for this. Perhaps that’s not a good fit for
everybody, because more people means more stress, and more cost. And
everybody’s just doing one job, and people want to do different
things. Or you look at the other side — you give people the
possibility that they are getting the power to do more complex jobs,
and different jobs — qualify themselves. Be more proud of what they
can do, being more interested in that they can do different jobs.
From the business side, that’s more flexible workers, they make less
mistakes. They, say, remember 70 percent more. So it’s a better fit
for everybody. If you look at this from a broader perspective, the
benefit is — as with most things — is survival of the fittest.
That’s the English saying for it, right? “Survival of the
fittest,” Right?

Alan: But I think now we have
the ability to– imagine, 20 years ago, if you wanted the facts about
something, you had to look them up. There was no– well, I guess the
Internet’s been around, but it was really difficult to do it. Now, I
don’t even have to pull my phone out; I can just ask Google and every
answer is there. So, it’s not really about what you know anymore. It
used to be what you know, and what you knew gave you an advantage.
But it’s no longer about that. It’s about how you apply that
knowledge. And I think this is where augmented reality becomes a
tool, like nothing we’ve ever had. You did a PhD in industrial
augmented reality. What did that entail? Cause I know that that was a
big part of your life. What does a PhD in industrial augmented
reality mean, or what does it look like?

Björn: You need to look quite
holistically at augmented reality. You cannot simply say, “oh,
this improves a job,” but you also need to find a way how it
improves the job. Industry is using things only if they work. So,
does it work from the technical side — does it really do the job? If
they work from the business side, does it save you money? And if it
works for the people? If you don’t find a solution for all three
areas — technology, people and business — it’s not working.

Alan: That is pretty much– if
nobody takes anything else away from this podcast, that is it. You
got to have the technology that works. It has to serve the people.
And it has to make good business sense.

Björn: Exactly. And then there
are also a lot of other obstacles. What was funny in the last years
was — particularly in the last year — quite often sitting in
management rounds, and the managers always were saying, “oh, we
cannot give it to the workers. They are not accepting it and they
don’t like it. We’ve got old workers. They cannot work with it. They
were rejecting a lot of technology.” But if you work close
together with people, with the workers, and figure out what they
really need — what really helps those workers standing there, also
the old guys — it’s really age-independent things. “It’s cool.
When can we get it?”

Alan: It’s so true. You
mentioned that– I had somebody else in the podcast, and they were
saying, “Look, you really need to engage. You need three people
in a conversation about rolling this out. You need the very high
levels — you need maybe the CEO or somebody in the C level — to
champion to say, yes, we’re gonna do this, you have my full support,
go for it. Then you need an internal champion from the management
level, who’s going to keep the communication between the C levels to
keep the funding going for the project. And then you need somebody
who’s actually going to be using it. Somebody who’s on the factory
floor really doing this. And by having the buy-in of the three
different levels, that’s how you get real change.” Is that what
you’re seeing?

Björn: Yeah. Also, I think the
most important — you need you need he OK from CEOs, from the
innovation manager, but what you really need is commitment from the
users who will really tell you what they need. It needs to be a
solution for the workers. If it doesn’t work with workers, they are
simply destroying your glasses and they’re not going to use them
anymore.

Alan: Yeah, I can see that. So
if you look at augmented reality in the enterprise, are you seeing
more of this moving to glasses, or is it tablet-based for now? What’s
being used most, and what is kind of delivering the most effective
ROI?

Björn: You see some stuff
happening on iPads. And it’s always quite nice. It’s scalable.
Everybody says it’s a more robust technology. But if you’re doing a
deep dive into technology — into the perception of information —
it’s very different with glasses. You have a window to the world;
you’re basically standing in your house. You’re looking through the
window into the nice world. It’s like there’s some distance. What’s
cool about augmented reality is that there’s this real-world overlay
and it’s the distance between the point where you need information.
It gets information presented. It’s going to zero. And this effect,
you only have those glasses. So that’s why we’re seeing more and more
people working on glasses. Also doing a lot of rollouts with glasses,
for sure. Now you see a lot of rollouts for smart glasses, which is
already — we don’t call it augmented reality in Germany. I know
Americans do — but it’s good that it’s happening. So you see a lot
of rollouts with smart glasses. We hope to see also more rollouts
with the augmented reality glasses, and a lot of VR glasses getting
rolled out — AR glass are not currently not getting rolled out
because no one’s currently shipping AR glasses. There’s Magic Leap–

Alan: Still a challenge.

Björn: Yeah, but there are a
lot of things are solved. The Hololens line. This device is like 4
years old. So yeah.

Alan: Yeah.

Björn: Everybody’s got the
HoloLens 2, but they’re also waiting for some competitors [to start]
shipping their glasses. Because from the general point of view, the
technical challenge which is AR, there are some recent companies that
have solved those problems. But it’s a matter of time. It’s a matter
of months now, that Microsoft is shipping a new HoloLens; some other
companies are shipping glasses. NReal is going to be shipping some
glasses. Magic Leap is putting out their next generation of glasses
and also shipping to the European market for sure — we have the
glasses here, but we cannot currently roll out those glasses, because
they don’t even have certifications for the European markets. So
these are the next-level things which need to happen now; the glasses
getting all the certification for the market.

Alan: It’s interesting, Björn,
you mentioned– the first thing you mentioned was technology, then
people, then has to make business sense. I think we’re still
struggling with the technology side. We have glasses. We’ve proven
the business cases; they work. And it’s just a matter of “Oh,
you’re shipping them at scale now.” I think over the next six
months, we’re going to see an absolute tsunami of technology being
introduced to the marketplace. You mentioned three different types of
glasses. You’ve got overlay glasses — or ones that kind of show you
a heads-up display — like RealWear, for example, who just raised $80
million. And those are not really augmented reality glasses, they’re
more heads-up display, similar to Google Glass. And then you’ve got
VR headsets. And then you’ve got AR glasses. And the latter — the AR
glasses — are the ones that are able to look at a machine and
project images on top of it in context to that, in the real world.
And VR has its place, heads-up displays have their place. But really,
the magic is in the these AR glasses. Is that what you’re saying?

Björn: Exactly. And at the end,
has the biggest benefit also, because the way you’re perceiving
information, it’s made better if you get it presented in a
stereoscopic way. That’s what glasses are doing. They benefit this
way. But if you want to achieve this 70 percent of memory effect —
that works really good, that people really are remembering so much
more information because this is information in place — you need to
do it with glasses.

Alan: I don’t know if you can
give any specifics around companies that are deploying this. Are
there any companies that you know that have deployed AR glasses at
scale? Like, real AR glasses? Or are we still in the pilot purgatory
of this?

Björn: Not really in scale yet.
So, scale is like thousands of units. There are some companies using
several thousands of devices, but really count as a problem. No
devices are available, and the last Hololens was shipped one year
ago. Now it’s like, all the glasses are breaking, because they so
old. So you cannot roll out 1,000 glasses, because you cannot ensure
that they will hold for such a long time until you get a replacement.
That’s currently not happening. But I think it’s a matter of time.
And also, the companies, the glass makers are fulfilling all the IT
requirements that you can integrate into those glasses, and large
infrastructures that can have property-wide management
certifications, you’ve got health certifications, and all that stuff.
But we think, from the technology point of view, it’s actually
solved… well, not every glassmaker has solved every piece of
puzzle.

Alan: Yeah, I think that’s
interesting. You look at all the different glass manufacturers and
they all have different parts solved.

Björn: Technology’s not a
problem. There are many possible companies because we are scouting
for all the AR/VR glasses. So in the last three years we’ve been
reviewing, I think, 228 glasses. 228.

Alan: [chuckles] Holy crap.

Björn: That’s really quite a
lot. For most glasses, half of them are shipped. For most of the
other glasses, we’ve figured out, what is missing there? They all
have different focus; what they want to be. You’ve also seen — which
is quite sad, actually — we’ve seen the Meta glass, for example,
quite early. And it was looked over by physicists and optical experts
who said, “there’s fundamental problems here. Why the hell are
they doing it?” And one year later or two years later, they went
bankrupt. It’s the same device. But it’s so complex, augmented
reality — also, virtual reality — and the companies also need to
learn this. But we’ve got the feeling there are a few companies now
who’ve learnt a lot, and will be able to ship a lot of glasses by the
beginning of next year or so, I guess.

Alan: All right. So let’s talk
about specifics. What are, in your opinion, the top five AR glasses?
Out of– you reviewed 228 pairs of them, which is incredible. I would
love to get that information to share with the listeners, if
possible. But yeah. What are your top five, then?

Björn: I mean, the bigger ones
you probably know as well. Yes, sure. Microsoft tries to be the
leader of market, but they’re not shipping. It doesn’t show up in the
top five company because they don’t ship. Magic Leap for sure. I
mean, everybody’s saying Magic Leap is not so good, and they oversold
what they ship. But a few things about Magic Leap are still better
than Hololens 2.

Alan: Like what?

Björn: Wearer comfort is still
better with Magic Leap than HoloLens 2. Because in the end, it’s way
more lightweight.

Alan: Yeah, because they took
the compute power off of the headset. Have you tried the Hololens 2
on?

Björn: Yeah, for sure. It’s
really good. From the control, it’s way better than the HoloLens. The
interaction is really good — they placed the HoloLens 1 on so many
heads, and they always needed to explain a lot of things.

Alan: Yeah, yeah. That little
pinch to click, nobody gets that. Anybody over 30 doesn’t get that
right. We’re kind of in this weird place, where it’s like we know
where the ROI is, companies want to deploy it, and we can’t [laughs]
For whatever reason.

Björn: And then there are
problems with a second row of companies won’t ship until the
beginning of next year. I kind of was like, you know, all those
companies. NReal doing quite a good job.

Alan: Yeah. NReal really
impressed me as well. NReal is a spin off from Magic Leap. They took
all of the best parts of the Hololens and Magic Leap idea. They’re
like, “hey, really, what we need is just a lightweight display
with a bit of tracking, and that’s it. We don’t need 8 cameras, and
we don’t need a computer on the head, and all this.” And so they
run it through USB-C — which now actually this week you can plug it
in to your computer, I think they just announced — and you can also
plug it into your phone, so your phone becomes the compute device,
which makes the glasses much, much cheaper. I think they’re shipping
at $599. Or that’s what they’re taking pre-orders for. I can tell you
right now, we’ve done a lot of VR and AR development, and we used to
haul around a big huge computer to do demos for people. We’d bring
around, set up the computer, wire it up, put up the sensors, all of
this. And I got the Oculus Go, thinking, “Oh, this is gonna be
great for VR.” And it was just underwhelming. It was only 3
degrees of freedom. You could look around. But it overheated. The
battery didn’t last very long. It was glitchy. And so when the Oculus
Quest came out, we kind of said, “We’ll just wait, it’s not
going to be as good.” But I recently tried it about a month ago,
and I was very, very impressed with the Oculus Quest. As a VR headset
that is standalone, the tracking is amazing. And now they just
introduced a plug-in, where you can actually, via Wi-Fi, stream your
computer to the Oculus Quest, so you can have computer-based
rendering and graphics pushed to the Quest, which now makes the Quest
an incredible tool for VR. I mean, it’s not AR, but man, the
technologies behind these things really are getting much, much more
impressive over the next little bit. So we’ve talked about Hololens,
Magic Leap, NReal. What are some other ones that are kind of popping
up in the top of your head that maybe are shipping in the near
future?

Björn: We’ve gotten a lot over
the years, so I’ve got a huge heap of glasses next to me.

Alan: Got a big pile?

Björn: Yeah.

Alan: You’re going to start a
museum, Björn.

Björn: We called it– it’s a
kind of museum. Yesterday, by the way, I was at the Deutches Museum
— Germany’s technology museum — and they had the Oculus DK1. It was
looking so good, because it’s looking so old already — it’s only 6
years old. But it was already looking so old. This technology has
improved so much.

Alan: Yeah. The DK1 was the
first VR headset I ever put on my head, and I remember looking at it.
Somebody put it on, they had these big headphones. I thought, “Man,
I don’t know what this thing is, but it looks ridiculous. [laughs] So
big!”

Björn: [chuckles] Yeah, it’s
like scuba diving.

Alan: Yeah, it was ridiculous.
And I got to try, was it the Pimax? The one that’s 8K and it looks
like a big V. Oh my goodness. It felt like I strapped a fishbowl on
my head.

Björn: Yeah, but now it’s
getting smaller, actually.

Alan: Yeah.

Björn: It’s small. And then
also some other glasses are getting smaller. So technology’s
advancing quite a lot.

Alan: You know what’s
interesting about mixed reality? The AR glasses are going from a
glasses standpoint, where you can see the real world and then you’re
kind of putting holograms on top. But another company — Varjo from
Finland — they’re taking a different approach. They built a really,
really good VR headset and then they use pass-through cameras to
create the mixed reality, or AR effect. And I tried it and I thought
I was pretty skeptical, because usually when you have pass-through
cameras, it’s nauseating. It makes you very sick — or makes me very
sick. And so I put it on. And we had the pass-through cameras. I
looked at my hands. And first thing you do is look at your hands, to
see if it’s real time. And it was really, really accurate. It didn’t
make me sick. It looked proper. And then being able to bridge that
gap between, OK, you’re in VR — or sorry, in AR — you can see the
whole world and then all the sudden they’re adding virtual layers to
it. And I went from being in the real world with a car in front of
me, to being in a completely virtual space. It was incredible. It’s
maybe not practical for certain applications, but for applications
where you need to see the real world and you can wear a big, bulky
headset connected to a computer, I think it’s great.

Björn: It has some benefits.
Varjo’s doing a great job, also has visors with a rich display, which
helps a lot. Varjo is doing a great job, particularly with their
retina center display. It’s amazing, what they are doing. It already
helps a lot because you can see much more clearer situations in the
industry to evaluate things. That’s quite good. What I didn’t get was
a story with the see-through glasses. I saw it. I tried it. They’re
doing a good job but still see-through. We had also the glasses of a
Canadian startup. I forgot to name that it’s the same. They were
acquired by Apple.

Alan: Oh, Vrvana!

Björn: Vrvana, yeah.

Alan: Yeah, out of Montreal.

Björn: Yeah. Yeah. We’re having
basically the second device that they ever shipped.

Alan: Oh, very cool. You know
why Apple bought them?

Björn: My guess was, as they
were quite good in the latency and [unclear]. What was it?

Alan: There was those two
things. But the one thing that they did — that nobody else could
figure out — was occlusion from a single camera source. And if you
look at the new ARKit system, the new ARKit allows you to do
occlusion. And for people that don’t know what that means, if I’m
looking in AR glasses and I look at a hologram that’s 3 feet away
from me and somebody walks in front of that, it should know that
they’re walking in front of it and to block it out as they walk
through. If they walk behind it, it should walk behind it. But in
most AR, it doesn’t recognize the depth. So as somebody walks through
it, the object just becomes really big because it’s now projected on
top of them instead of into the real world. And these guys solved
that. And if you look at the new ARKit release that just came out,
that’s actually one of the things that’s embedded in it now. So I
think that’s one of the reasons they did that, because occlusion from
a single camera is very, very difficult.

Björn: I saw the demo of the
ARKit was showing this occlusion. I was really impressed. I was
really impressed that this was working because I know about all the
technical challenges behind it. And I was questioning whether it
would only work with a stereoscopic iPhone, or a structured light
sensor. But it’s not. It was only done by a single camera. This was
looking so good, they a person walking around, you could see this
person was walking in front of or behind the table. And this is
amazing for augmented reality, because we test the technology to make
the augmented reality look really good, to be more immersed to it.

Alan: Yeah, there’s nothing
worse than somebody walking through your hologram when you’re trying
to do something. [laughs] It’s very distracting.

Björn: Going back to the Vrvana
device and media see-through. That’s the scientific term we’re having
for it, media see-through augmented reality, in comparison to optical
see-through. Media see-through part has a lot of other obstacles. So
you’re having the frame rate issues, the latency issue. So, saying
“is it real-time” when you move your hands, which is very
important.

Alan: That’s what I think Varjo
got really well. They nailed it. I mean, it was imperceivable, the
latency.

Björn: That’s quite important.
Then you having the amount of between black and white.

Alan: The contrast, yeah.

Björn: Contrast. That’s
important. I mean, if you’re in a dark room, that’s fine.

Alan: How many people work in a
dark room, though? Not very many. I think HoloLens 1 was pretty
bright, and had some good colors and had a good contrast. But I think
we need to do better, because most people work in very bright
environments.

Björn: Yeah, but it’s even more
complex when you’re looking at the biggest issue, because the cameras
have low contrast and it’s only just to a certain area of
environment. So what you can see is the monitor of the device, it’s
only part of the world. If you’re standing in front of the window,
you can’t help that it’s getting quite bright, or it’s getting dark
and you can’t see what is outside the window. That’s the challenge.

Alan: I think it’s gonna be a
challenge. For now, we need to be in windowless rooms for that.

[laughs]

Björn, I want to switch gears, because we have a little bit
of time left and I want to switch gears and talk about AR Giri.

Björn: That’s our approach for
training. For worker training, for training processes. So we figure
it out in 2016 — a long time ago — we were asked to make some
training apps. We did a lot of innovations at the assembly line,
along with interviews with the workers, creating a lot of prototypes.
And then yeah, we went live with some system and performance tests
and we get random people coming in and we taught them, using
Hololens, how to assemble a car engine and everybody managed it. In
general, everybody managed it. And afterwards a guy came out of the
one day experiments and say, “Hey, I never, ever assembled a car
engine — I just assembled a car!” I said, “wow.”
We’re doing it since so many years and now it’s working. There’s some
room for improvement. Yes. But after so many years, now, it’s
working. We also could figure out together with our partners or
customers that the a learning benefit is so high — it is so really
high — that people can quantify themselves. Motivation is higher.
They remember more. But also, the old management guys who aren’t used
to work in augmented reality said, “OK, that’s nice, but that’s
not a business case because you can never, ever manage to scale this
because of the content crash.”

Alan: Yeah. It’s a problem.

Björn: I want to get all this
content inside. We also need to get 3D data. Once we get that figured
out, you don’t get so many 3D data. Arrows are much more important
than 3D data.

Alan: It’s so true. Somebody
else came on the show and I can’t remember, but they were saying that
at the beginning — it might even be you, I think, when we recorded
this previously — you were saying that we used to take a machine and
recreate it in three dimensions and overlay it on top of the real
machine. And then what people realize is that it’s kind of a pain in
the butt, because you couldn’t see the real machine because the
digital one was on top. So just take all that away and just put the
arrows of what you need. Very simple. “Do this.” An arrow,
a finger pointing at it. Seriously, as a technology industry, AR/VR
technology, we really overthink things. And sometimes the most simple
things are what end up being the most impactful.

Björn: Exactly. And we need to
figure this out. So complex to figure this out.

Alan: Yep.

Björn: At the end, we’re only
working with arrows and videos. This really helps a lot. And someone
needs to generate the content.

Alan: Yep.

Björn: And if you’re having,
for example, the trainer who knows how the process works, and then
you’re having the guy who knows how the AR works, those two guys play
ping-pong.

Alan: Yep.

Björn: “Oh I want to have
it like that.” OK. You do it like that. Then the programmer
gives it back to trainer to translate. If it’s not what I tried to
say, they do it again; they play ping-pong all the time. So in the
end, what we’re doing is that we the trainer — the guy who knows
what needs to be trained — he can generate everything by himself.
But where we need to generate it? Not somewhere in the office.
Production’s happening in the production line. It’s the assembly
line. So the trainer puts on the glasses, goes into the editing mode,
drags and drops what needs to be done. Records basically what he’s
doing. It’s pictures, live-recording, using the Hololens or using
some other glasses. Let me just say, “Okay, now start the
training,” and the training is generated automatically.

Alan: Isn’t that amazing? That’s
incredible. So if people are interested in learning more about that,
it’s ar-giri.com. Being able to enable trainers to create this
content is going to be the key. And I think one more piece of the
puzzle is being able to allow companies that create training —
because some training, they’re going to recreate environments and 3D
models and all this — but being able to have some way for them to
share that as well, because the content creation right now is very
expensive, it’s time consuming. But I think what we’re going to see
is companies that spend a lot of money on developing content, they’re
going to want to be able to monetize that as well, because there’s
got to be a way to share that content across different entities. And
that’ll decrease the time to training right cross the board.

Björn: Exactly. And it’s also
important to share the content. Someone could be working with a
civilian training company on sharing some 3D model content. Sometimes
it’s quite good to test 3D models. Also, you need to think about
different training stages of car. For example, the car doesn’t exist;
the assembly line doesn’t exist yet. You need to train others in
virtual reality, because you don’t have a car. And then you are
moving over and the latest car exists, the training is also getting
more concrete. What needs to be done? That’s more augmented reality.
If you can reuse some 3D models — all those items which already
exists — makes the whole process much more scalable. There’s another
thing we realized is content creation from what the workers are
saying. “It’s nice if you guys generate us a training, but
problem is, production is not static. It’s not like you make a
product for years and you do it always the same way.” It’s like,
if you’re having errors and you’re having errors every week, then
they change something, it improves the process. This new process
needs, again, to be trained towards the other people. Basically, a
training can change every week because production is changing. It’s
also not that you’re only assembling one car at one place. You ship
different cars; you produce different cars. It’s the same assembly
line. And there’s a lot of variation in the trainers all the time.
And so for each work place, you need to improve or modify the
training all the time, basically.

Alan: Well, Björn, we’re coming
to the end of the conversation. But before I let you go, and it has
been a fascinating conversation about augmented reality, the
different glasses, how companies can roll it out. To package this
whole conversation of that, those are the key points. What is one
thing, one challenge in this world, one problem in this world that
you want to see solved using XR technologies?

Björn: We’re kind of doing the
training for the boring jobs. Which is our business, which makes a
lot of fun for us. But there are bigger problems in this world, like
in education. I feel like my history education was so bad, really so
bad. I learned a lot of things later. But augmented reality is such a
great tool which is ready to tell stories, stories of our past. Also
to tell stories about technology and about complexities about the
basics of technology. For example, yesterday in the Deutches Museum
they reproduced the labs, the office of Galileo.

Alan: Oh, cool.

Björn: Galileo was the first
guy doing structured experiments and filling out a lot of the basic
principles. It was looking so nice. And there were a lot of
experiments and a few of them explained, but it would have been so
nice to have Galileo be inside there. anyway. It would have been
stirring and would have so much impact to entertain people so much.
It’s the possibility.

Alan: And it’s not hard to do.
I’m writing an article on volumetric capture right now. And there’s
55 volumetric capture companies in the world. So far. That we know
of.

Björn: And that’s good.
Learning and education, you having… in Germany, the best-paid
people that the German government pays are teachers. Sorry to say
this, but most of them — I’m feeling — are not doing the most
wonderful job. There are some good teachers, but most of them, no.

Alan: Well, it’s hard to have
the best teachers in a system where a teacher is in one school
teaching a group of students. If that teacher now can teach students
around the world in a one-to-many VR presentation — or AR — you can
now start to bring really contextualized, personalized learning to
the world. And I think the world’s education system — systems,
because there are multiple systems around the world — but they’re
just not going to be adequate for a world where the jobs are changing
every few years now.

Björn: Exactly. That’s a
different story of what skills that you need in five and 10 years,
and 20. But if you’re starting to digitalize, it’s not about
replacing old school teachers, but it’s about them giving… teachers
need to do different jobs later. And and it’s also about, for
example, if you need to teach a certain topic, and there are also
different views in the world on this topic. But if, for example, you
could take the three best people in the world — you ask a Chinese
opinion, a European opinion, and you take the best teacher from the
US — who has a certain opinion. You take the best teacher from
Europe who has a different opinion. And the teacher from China who is
also different. But you would have the chance to record those people
once and then giving every student the possibility to experience all
those different opinions presented in the best way to give them the
best teachers. And then you can get it for the next topic. Also, the
three best teachers.

Alan: Björn, I really thank you
for joining us and thank everybody for listening. This has been the
XR for Business Podcast. You can learn more about Björn and his team
at the ar-experts.de. And you can learn more about their training
platform, ar-giri.com. Björn, thank you so much.