State of the AV/VR technology in 2019

As the year comes to the end, it is time to look back at how the AR/VR technology progressed in 2019. I will be focused on the enterprise market here since at UNIGINE we are dealing with such customers more frequently.

Augmented Reality

First, AR. This year we evidenced a lot of loud failures there. MagicLeap is a huge disappointment (largely because it was overhyped way too much), Hololens 2 by Microsoft is only slightly better than the first version (after 3 years of development). Hordes of other AR companies collapsed or silently buried into oblivion. Mobile AR is still in the early stages of development with very simple apps, mostly games.

Why is it so? Here is the list of current AR showstoppers:

  • Very small field of view, 30-40 degrees. To make things worse, expect a very close far clipping plane, leaving a user with a tiny “hologram” right next to his nose. Virtual objects simply disappear when leaving this small area.
  • Bad position tracking. Delays are very noticeable, hundreds of milliseconds or even seconds. This is not acceptable even for slow head movement, not to mention any dynamic actions. Motion sickness is guaranteed. If you try to use AR guidance for making actual work, the synthetic 3D objects are very shaky and it’s impossible to do accurate work.
  • Low visibility outdoors. The contrast is really disappointing for now, making outdoor works barely possible.
  • Short battery life. You have only maybe 2-4 hours, depending on the app.
  • Indoor positioning problems. A lot of AR applications require exact geo-position – which is still hard to get indoors or at an industrial site. GPS doesn’t work there, GSM can be also unavailable (due to security reasons or simply due to the remoteness of the location), Wi-Fi can be banned as well. One of the viable solutions for indoor navigation is ad-hoc beacons, but they require substantial investments in each site.
  • Lack of computing power. We already have powerful mobile CPUs and GPUs, but the blockers are heat and battery drain. As a result, AR headsets are equipped with low-end chips, which are only capable to render primitive graphics.
  • Safety problems. You can’t combine AR headset with a protective helmet, the device is also obscuring your vision. Both factors are no-go for a lot of industrial sites.
  • Very high price for consumers. Even these primitive headsets cost several thousand USD, leaving them only in the enterprise sector.

Still not convinced? OK, Magic Leap sales: 6000 devices (first year), Hololens 50000 (in 3 years). Good luck with building apps for this tiny market.

I’m a big believer in the AR concept myself, but the harsh truth is that the hardware is not mature enough yet, I think it will take 3-4 generations until it will be usable by the majority of users. A lot of people are bullshitted with aggressive AR marketing, but I would recommend trying the experience first hand before betting on it.

All of the above is the reason why we decided not to support AR in our UNIGINE engine for now. Anyway, once the hardware will be there, the porting will be very easy for us.

Virtual Reality

Unlike AR, VR is pretty much alive. The VR hype is over, though – but now we have a somewhat healthy technology evolution and gradual growth of the install base.

Luckily, these simple setups like GearVR or zillions cardboard-copycats, where you use your phone for almost everything are effectively dead now. Still, some people are having a hard time understanding the difference between a phone with a couple of lenses and a fully-fledged VR headset with proper optics, sensors, and desktop computing power. Come on, 360 video is not real VR!

The hardware landscape is constantly changing: HTC seems to be slowing down, Oculus has nice inside-out tracking in Rift S, Varjo is doing great on the enterprise market, Valve Index is the hottest consumer-grade gadget now.

Some things are still holding VR back:

  • Limited GPU performance. Too many pixels to render at too high framerate. 8K@90 Hz x2 (stereo) is a tough challenge even for high-end RTX 2080 Ti, which costs alone more than $1000 now. That’s why the majority of VR apps looks like 15-year-old flashbacks.
  • Wires. Being tethered by a short cable running to your head is far from being a comfortable experience. Wireless solutions do exist here, but they have drawbacks like additional lags or image quality degradation.
  • Small install base. We are talking about less than 10M fragmented devices here, it is still not enough to emerge a growing market. Luckily, things are getting better.
  • Limited content. It is coming from the previous item. Conservative game publishers tend to say these days “bring me any game unless it is VR”.

To sum up, we have VR ready for enterprise use and there is some limited consumer market as well (mainly due to high prices and lack of content).

Haptics

Okay, we are pretty much fine with tricking our eyes now, but what about hands and the rest of the body?

This year I have tried several state-of-the-art haptic devices and all of them were disappointing. Electrical impulses, sonic feedback, mechanical motors restricting movement – none of them are working properly yet.

Current problems:

  • Force feedback. No, simply vibrating is not enough here. A user must have the motion restricted completely in order not to fall through virtual objects. As of now, it is only achievable with the full-body exoskeleton (expensive, bulky, scary). Myostimulation (small electrical impulses to your skin) doesn’t work properly as well – not precise, hard to calibrate for different skin resistance, hard to encode due to complex nerve network construction.
  • Sizes. Unlike headsets, where you are mostly fine with adjusting only one parameter (eye distance), people have very different body sizes. None of the VR gloves I’ve seen had broad enough set of sizes to cover them all, from kids to brutal lumberjacks.
  • Precision of tracking. Human hands are very accurate and versatile tools. We feel even slight shape difference with our fingers – which is not possible to imitate now with the current precision of position tracking.

Unfortunately, 2019 hasn’t brought any major breakthrough here. Still looking for at least good gloves to come.

5G and Streaming

A lot of hype is around 5G potentially enabling VR and AR experiences by cloud streaming. I’m very skeptical here because AR/VR is very sensitive to latency. Even with the edge computing concept (which is pretty cool by itself), there will still be noticeable lag and all sorts of problems with edge deployment. You still can’t cheat the speed of light, you know. Google Stadia is one of the very evident failures here.

Conclusion

Pardon me for sounding negative – sometimes it is hard to know too much about the inner side of the technology. Actually, I strongly believe in potential of both VR (almost now) and AR (a few years later). We have VR support in UNIGINE engine since 2013, and our professional services team already delivered a number of enterprise VR projects, not to mention our Superposition benchmark with VR mode, so we plan to keep investing in VR now.

Looking forward to 2020 for more exciting news!

My first GDC in 2010

Found my old talk at NVIDIA Game Technology Theater, “Practical Use of Tessellation in Unigine Heaven Benchmark”, and it brought up memories of my first Game Developers Conference in SF.

It was the third day of GDC, and everybody who was a booth staff at least once, know how it feels. What’s worse, I caught cold and lost my voice, so it was not so easy to talk that morning, but some good guy from NVIDIA booth brought me hot tea and vitamins, so I survived. The funny thing was that we invited a lot of AMD folks to the talk, and they were all sitting together in the last row, looking a little bit afraid of being beaten by competitors =)

-QI8ThAeolI

The UNIGINE booth at GDC 2010 was fun. We had two rigs with updated Heaven Benchmark 2.0: one from AMD, another one from NVIDIA (with brand new GeForce GTX 480 delivered to us by our NVIDIA DevRel manager just 30 minutes before the tradeshow opening). Both PCs were working side-by-side, but we tended to stay away from the direct performance comparison in order not to upset some partner. GTX 480 was just announced, and it was the first NVIDIA card with hardware tesselation units, so we got a lot of attention from the hardware journalists.

We quickly ran out of brochures, so tried to keep them under the counter and dispense only to the VIP visitors.

3Kjrv7nKM2c-2.jpg

There were only three people (me, our CTO and a sales manager) back then, while there was always a line of people with questions.

Jnz4HNqugIg

We also had nice chat with Aras Pranckevičius from Unity, who stopped by at our booth.

Lx7CWJlQv9Y

We also talked with a lot of nice people from the industry, who I knew from online but never met in person before, like Ruslan Abdikeev and Simon Kozlov.

It was a good show!

UambfBfYHaQ