How the Microsoft HoloLens 2 works

It’s been over half a year since Microsoft announced the further development of the market leader for AR glasses at the MWC in Barcelona and with this great video:

If you are also one of the people wondering how the new HoloLens works in detail, we have the answers for you below:

Microsoft has basically improved the HoloLens in 3 areas: immersion, comfort and time to value.

While the first HoloLens was already a milestone, Alex Kipman and his team have gone the extra mile for the successor. No less than “everything” should be better in the second edition compared to its predecessor. What can already be said without any background knowledge: The level of complexity has definitely been increased, as the comparison of the two models before the design clearly shows:

One of the main problems with the HoloLens 1 was the perceived lack of immersion. The relatively small field of view was a frequent point of criticism. And even though a successful immersion experience requires more than a large field of vision, this point has been significantly improved. The HoloLens 2 offers a more than doubled field of view. This is without having to compromise on the holographic density of 47 pixels per degree of view. Sounds abstract at first, but it is. Put simply, the improvement means an upgrade from a 720 pixel video to 2K – per eye.

In addition to the enlarged field of vision, a new technology provides an improved visual experience. The HoloLens 2 has a new display system that uses lasers instead of LEDs as before. Three RGB lasers per eye ensure optimum visualization.

The light from the three lasers is projected directly onto the back of the eye via several mirrors. This process corresponds to “normal”, natural vision. As our brain is already familiar with this process, the experience is perceived as significantly less stressful for our brain, which is particularly beneficial for concentrated work.

Another factor in favor of improved immersion is the significantly improved interaction with objects. It should now be possible to interact directly with holograms and manipulate them. This even goes so far that the user does not have to learn new gestures as usual, but the HoloLens recognizes “natural” gestures. This is primarily made possible by significantly improved hand recognition.

In the following photo you can see a shot of the depth camera. This has two room sensors, one of which is only for hand detection:

It is interesting to see how Microsoft deals with errors when measuring the room: depending on the surface (mirrors, leather sofas, etc.), sensors struggle to display a room correctly, which leads to holes in the displayed objects and thus display problems. An AI algorithm specially developed for this “spatial mapping” problem recognizes errors and corrects them. This leads to significantly “better” spaces and improved interaction with the displayed content.

Another new feature compared to its predecessor is that the HoloLens 2 tracks the user’s eyes instead of the movement of the head. The advantage of this approach is that it is much closer to biological reality and therefore less strenuous for the brain.

AR is becoming increasingly interesting for industrial companies in particular. No wonder Microsoft has put a lot of effort into voice control. A total of 5 microphones filter out ambient noise and ensure that spoken commands can still be executed even at 90db background noise. A feature that can be worth its weight in gold, especially in industrial and production facilities.

It is also important for use in a productive environment that users can wear the HoloLens for a long time without experiencing pain or problems. According to Kipman, wearing comfort has been tripled. This is mainly due to better balance of the device. This means that the device, which is only 2 grams lighter, is perceived as significantly less heavy and therefore more comfortable.

Another issue that Microsoft tried to address is the “time to value”. In other words, how much time passes between the purchase of the glasses and their actual productive use.

Without a competent development team, HoloLens 1 was not yet worth much, and suitable software had to be specifically created first. The company is now working with various large companies (mainly from the industrial sector) to offer a range of functionalities at launch. However, it remains to be seen how good these will actually be after the official release.

One of our projects that we have already implemented with HoloLens is the VBZ Holotram.