Microsoft Hololens

Microsoft HoloLens - virtual reality glasses. They are a hoop put on the head with tinted lenses with a wavy prismatic structure located in front of the eyes, which refract and send images from microdisplay located on the sides to the user's eyes.

Microsoft HoloLens - mixed reality glasses . Use the 32-bit Windows Holographic operating system (Windows 10 version ). The device is autonomous and does not require connection to a PC , smartphone or game console.

In addition to the central and graphics processors, the glasses have processor a holographic processing unit designed Microsoft specifically for HoloLens.

2024: HoloLens virtual reality helmet discontinued because no one wants it

In early October 2024, Microsoft announced that it was retiring its HoloLens virtual reality helmet. The company intends to continue supporting users who own HoloLens 2 headsets for several more years, but the development of HoloLens 3 is completely frozen.

Microsoft said the HoloLens 2 headset will receive updates to address "critical security and regression issues" software through 2027. Security updates for the original 2016 HoloLens will be discontinued as early as December 10, 2024. By 2028, support for all devices will be discontinued completely. All that remains of the HoloLens project is an integrated visual complement system, or IVAS, glasses augmented reality for the army. USA

𝓲

Gizmodo

HoloLens were originally introduced as a business device for working in factories, rather than a headset for ordinary users, but never found their place in the world of competition with Meta augmented reality glasses. The U.S. Army is reportedly still enthusiastically awaiting the IVAS project, although a 2022 report found that soldiers who used IVAS 1.0 were less effective in combat than those without it. However, Microsoft has signed a 10-year contract worth about $2.2 billion with the armed forces to develop this headset. 

According to the National Defense, IVAS 1.2 field tests were more successful due to the reduction of the form factor and the improvement of low light sensors. The US Army plans to conduct new tests of the headset in 2025. Military officials must decide whether to launch IVAS into broader production. The Army requested $255 million from Congress, which would allow them to purchase more than 3,000 IVAS headsets in 2025.^[1]

2022: Colour-blind VR glasses released

On April 13, 2022, the company Microsoft released Windows the Holographic platform version 22H1, which offers HoloLens 2 users, as well as -administrators and IT developers, a large set of new features, including color blindness mode. More. here

2021: HoloLens 2 Industrial Edition sales start

In early February 2021, Microsoft begins selling an industrial augmented reality headset called the HoloLens 2 Industrial Edition. Microsoft said it tried to take into account feedback from customers who used the previous version - Lockheed Martin, Imperial College Healthcare NHS Trust and Mercedes-Benz USA.

Lockheed Martin is known to use HoloLens 2 to build the Orion spacecraft, significantly reducing costs, as actions that previously required an eight-hour shift can now be performed in just 45 minutes. Imperial College Healthcare NHS Trust uses HoloLens 2 to limit health workers' contact with COVID-19 patients, achieving an 83% risk reduction, and Mercedes-Benz uses a headset to improve the efficiency of service technicians.

Microsoft begins selling AR headsets for industrial enterprises, it has already been acquired by Mercedez-Benz and Lockheed Martin

However, many companies would like to use HoloLens 2 in highly regulated areas, where devices must meet particle emission standards, for example, in the pharmaceutical and semiconductor industries. And in the energy sector, companies need a device that can be safely handled in potentially dangerous environments.

The HoloLens 2 Industrial Edition headset complies with ISO 14644-1 class 5-8, which allows it to be used in clean rooms. From the point of view of intrinsic security, the headset complies with the UL class I standard, section 2. In addition, the HoloLens 2 Industrial Edition has a two-year warranty and is included in a quick replacement program, under which a faulty device can be replaced with a new one as soon as possible.

The HoloLens 2 Industrial Edition is already available for pre-order at $4,950. The first deliveries to customers will begin in the spring of 2021.^[2]

2020: Windows Autopilot Service Support

On May 8, 2020, Microsoft added support for augmented reality glasses HoloLens 2 to its Windows Autopilot solution. Read more here.

2019

Launch of HoloLens 2. Price - $3500

On November 7, 2019, Microsoft announced the start of sales of the HoloLens 2 mixed reality headset. Its cost turned out to be the same as the American corporation named during the announcement of the product - $3,500 (more than 223 thousand rubles at the exchange rate on November 7).

Residents of the USA, France, Germany, Ireland, New Zealand, Australia and Britain will be the first to purchase HoloLens 2. The timing of the product in other countries, including Russia, Microsoft is not named.

Microsoft announces the start of sales of the HoloLens 2 mixed reality headset

The manufacturer addresses HoloLens 2 to the corporate sector, mainly companies engaged in production or repair work. For an additional monthly fee, a business can purchase Microsoft Remote Assist software designed for real-time troubleshooting.

Journalists from The Verge, who managed to test HoloLens 2, noted some differences between the device and its predecessor. First, the developers have increased the viewing angle from 34 to 52 degrees. Secondly, gesture control capabilities were significantly expanded: if in the first version they were limited to clicks on virtual objects, then in HoloLens 2 you can scale and move objects in view. In addition, in the new headset, you can invoke a pop-up menu by pressing a holographic button on your wrist.

Microsoft Communications Director Greg Sullivan says the company retains support for the former's HoloLens, but some developers may start building applications that require gesture control implemented only in HoloLens 2.

Among those who left bids for HoloLens 2 were both new customers and those who decided to buy the device to replace the first-generation HoloLens, Sullivan said.^[3]

Emulator output for HoloLens 2

In mid-April 2019, Microsoft released an emulator for HoloLens 2 that will test applications on a computer without using the mixed reality device itself. HoloLens Emulator is a very useful product, given the high cost of the gadget - $3500. Read more here.

HoloLens 2 announcement

On February 24, 2019, the mixed reality presentation of the second generation Microsoft HoloLens helmet took place. The device is addressed exclusively to corporate customers and will not cost cheap.

Compared to the first version, the new headset received twice the field of vision while maintaining a holographic density of 47 pixels per degree of view. The company explains that the improvement is equivalent to switching from a 720p to 2K TV for each eye. 

HoloLens 2 allows you to track hand movements without any additional controllers. During a live demonstration, a Microsoft employee launched various applications in 3D space, including a virtual piano. 

Microsoft introduced the HoloLens 2 mixed reality helmet. Price

HoloLens 2 contains a new display system that allows devices to demonstrate improved performance with low power consumption. Using the new time-of-flight depth sensor combined with built-in artificial intelligence and semantic understanding, the wearable device allows interaction with holograms similar to how it happens in the real world with physical objects.

The device is equipped with eye tracking sensors, which makes interaction with holograms even more natural. face recognition Windows The Hello system is also supported. The developers claim that the new helmet is "lighter and more comfortable" compared to the previous model.

Microsoft notes that HoloLens 2 is focused on use in companies that can create their own programs and app stores for the gadget. In addition, the helmet will support the installation of third-party web browsers, and Mozilla has already begun creating such a product.

After the announcement of HoloLens 2, Microsoft began to accept pre-orders for the purchase of a mixed reality helmet. The price is $3500.^[4]

2018

How Microsoft's AR headset helps surgeons on surgeries

By the end of November 2018, surgeons began using a mixed reality headset to intelligently plan reconstruction operations on the lower extremities - Microsoft's new HoloLens AR headset helps understand which tissues and vessels can be used in recovery operations using autotransplantation.

Patients are prescribed standard contrast CT scans that provide visualization of passable vessels and allow isolation of the most viable tissues. A 3D reconstruction then simulates the limb and all of its vessels and transfers the data to HoloLens. Surgeons using the headset see the projection of blood vessels, bones and muscles superimposed on the patient's leg, as well as the marks of radiologists, so they can accurately predict the volume of the upcoming operation.

Microsoft's AR headset helps surgeons with surgeries

Traditionally, Doppler ultrasound was used to assess the location of blood vessels before surgery, but this technique is inferior to HoloLens in the reliability of the data obtained. In addition, the use of HoloLens allows you to reduce the time of surgery, and therefore the time of anesthesia, which is important not only for the work of the department, but also for the patients themselves - it is known that reducing the time of surgery and anesthesia improves the outcome and reduces the risk of complications.

Proving that the HoloLens system can be effectively used in some branches, Microsoft developers have begun to consider other ways to apply an AR headset - in particular, it is already used in operations to remove colorectal cancer.

However, there are a number of problems the researchers hope to address before the HoloLens system begins to be used more widely. Developers are trying to reduce contrast image processing time, which currently takes about an hour, achieve higher image resolution and improve compatibility between HoloLens and hospital software.^[5]

Application during spacecraft assembly

On October 23, 2018, it became known that Lockheed Martin engineers, who are commissioned by NASA to assemble the Orion spacecraft, use Microsoft Hololens glasses in their work. The headset helps visualize instructions and the final build result, eliminating the need to study thousands of pages of paper guides. Orion is being developed to deliver astronauts and cargo to the ISS, and later for manned flights to the Moon and Mars.

Thanks to Hololens, engineers can see holograms modeled in a special program from Scope AR, which are superimposed on the elements they are working on. Each part of the model is provided with a comment on how to install it, and information on how to twist the bolts is displayed next to the corresponding hole. Also, for each engineer, only those parts that correspond to his specialization are highlighted in a certain color.

Earlier, Microsoft and NASA launched a joint project Sidekick, in which ISS astronauts used Hololens headsets for repair, experiments and communication with engineers on Earth^[6].

Dynamics 365 Layout and Dynamics 365 Remote Assistant Exit for HoloLens

In September 2018, Microsoft introduced Dynamics 365 new applications that use artificial intelligence mixed reality. With their help, organizations and enterprises can fully use the capabilities of the helmet mixed reality HoloLens for business tasks. More. here

Use for blind navigation

In May 2018, the California Institute of Technology demonstrated an application that allows people with vision loss to navigate space using virtual reality Microsoft a HoloLens helmet.

This device is equipped with several cameras, including a depth camera, as well as speakers located around the perimeter. This and the ability of the headset to create a three-dimensional map of the surrounding area was used by a group of engineers led by Markus Meister of the California Institute of Technology.

They developed a program that allows blind people to move indoors and "see" objects (steps, furniture , etc.) around using audio navigation.

The application can lead a person along a given route, informing him of approaching turns and all kinds of obstacles. The glasses capture the video image, extract important environmental data, and transmit this information through auditory augmented reality. So, a person in a Microsoft helmet hears prompts such as "Up the stairs," "Railings on both sides," "Go to the right," etc.

The VR navigator was tested on seven volunteers who were asked to overcome the rugged route over two floors. According to the developers, all participants in the experiment went the first time.

Marcus Meister hopes that after the study, it will be possible to create devices that can be used by visually impaired people in public places, including in shopping centers and hotels.

According to the World Health Organization, by May 2018, about 253 million people around the world are visually impaired or completely deprived of vision. The development of the California Institute of Technology has great potential, but a lot of work needs to be done. For example, the app cannot build routes that were not laid by someone in advance, and does not respond to people who may be in the same room.^[7]

Application in complex reconstructive surgery

In February 2018, Microsoft reported that doctors from Imperial College London used the HoloLens mixed reality device in reconstructive plastic surgery. As part of the experiment, a 3D computed tomography (CT) image was loaded into a headset and shown to a doctor in the form of a hologram during surgery. This technology allows you to establish the exact location of the patient's blood vessels, bones and muscles, which greatly simplifies the execution of operations for fractures, and also reduces potential risks, Microsoft said.

Surgeons used Microsoft HoloLens in complex reconstructive surgery

As you know, a feature of surgery for fracture and tissue transplantation is the need to have an accurate picture of anatomical features for a successful and maximum operative connection of the vessels and the subsequent restoration of the blood flow of the operated area. According to doctors, the presence before the eyes of a visual scheme of the location of vessels, bones and muscles can be a key factor in the success of the operation, usually requiring careful preliminary preparation.

When conducting such interventions, surgeons traditionally use a hand-held ultrasound scanner to detect blood vessels under the skin along the movement of blood flow. However, this process is very laborious and does not provide accurate data on the location of vessels under the skin.

Demonstration of the experiment, February 2017

Application in training physicians to take complex births

As it became known in January 2018, CAE created the LucinaAR simulator, designed to help doctors work out actions for different options for birth complications. The basis of the simulator is the detailed mannequins of the mother and baby. The kit also includes HoloLens mixed reality glasses, with which you can see the movement of the fetus and assess the effectiveness of the measures taken.

In particular, the simulator is designed to train doctors to work with brachial dystocia - a rare type of complication, it can occur up to once per thousand cases. If the obstetrician is familiar with techniques that help release the infant's shoulder, this greatly increases the chances of a successful birth. A detailed simulation in this case is indispensable.

LucinaAR is a real breakthrough in educational simulation, thanks to which students will be able to see the patient's anatomy with their own eyes, "said CAE President Dr. Robert Amio. - The simulator will be combined with our mixed reality platform, because we are confident in the immediate and powerful effect of preparing for childbirth with brachial dystocia.

CAE, a Canadian company that specializes in the production of avionics and medical training equipment, demonstrated its first product in 2017 using HoloLens, a VimedixAR project aimed at training medical students to perform ultrasound examinations.

2017

Use in tumor removal surgery

Being in different parts of the world, three outstanding surgeons from India and Britain, using Microsoft Hololens augmented reality glasses, were able to virtually be present in one operating room and jointly perform surgery to remove the tumor. This was announced on October 23, 2017 in a blog post by BMI Healthcare, the UK's largest group of private clinics.^[8]

Microsoft AR glasses helped surgeons from different countries to perform surgery to remove the tumor

Surgery for colorectal cancer took place at the base of The Royal London Hospital in London. UK surgeon Shafi Ahmed was assisted by colleagues - Professor Shailesh Shrikhande, a cancer surgeon at Tata Memorial Hospital in Mumbai, India's largest cancer clinic, and Hitesh Patel, a consultant colorectal surgeon at BMI's private clinic The London Independent Hospital. Another virtual consultant was Ian Nott from the USA, co-founder and CTO of Aetho, which developed the Thrive AR application for Microsoft Hololens glasses.

Specialists worked in AR glasses and could not only hear, but see each other in the form of graphic avatar figures, as if they were really in the same operating room. Surgeons had access to tomographic images of the patient's internal organs, which were displayed as three-dimensional holograms, and doctors could make virtual notes on them while discussing the course of the operation.

Thrive's AR app for Hololens glasses was developed by Aetho

"We've really combined technology with medicine. By bringing together specialists from different parts of the world and time zones, we have demonstrated that we can make operations safer, provide better patient outcomes and democratize surgical practice, "commented Professor Shafi Ahmed[9]

Using HoloLens Glasses in Cancer Surgery, October 24, 2017

It is worth noting that this is not Shafi Ahmed's first experience in using innovations in the operating room. In 2014, with the help of VR Glass glasses, he Google organized an online broadcast of the operation, which was seen by several thousand medical students from 115 countries.

Microsoft HoloLens 2.0

According to Microsoft representatives, the second generation of HoloLens glasses, in addition to the "holographic" processor module (Holographic Processing Unit/HPU 2.0), will also be equipped with an artificial intelligence coprocessor (AI coprocessor), thanks to which it will be possible to support deep neural network technologies (Deep Neural Networks, DNN).

Executive Vice President of Artificial Intelligence and Research Group Microsoft (Artificial Intelligence and Research Group) Gary Shum in July released the first details about the second generation of HoloLens glasses, the development of which is ongoing.^[10]

Photo: s9.tarhely.com

Among the many additional features and capabilities of HoloLens 2.0, the most interesting is the AI coprocessor, which will allow the device to acquire support for deep neural network technologies. This means that HoloLens will be able to process large amounts of data on the fly without resorting to external cloud resources.

As Gary Shum specified, the AI coprocessor will be completely reprogrammable and will be able to work in real time, which, in turn, will allow the second generation HoloLens to autonomously and without delay analyze what the user sees and hears. In addition, the chip will be used by developers of the HoloLens 2.0 software platform to create mixed reality applications with AI enabled.

As with the first generation HoloLens, a feature of the HoloLens 2.0 glasses hardware platform will also be a "holographic" processor module (Holographic Processing Unit/HPU 2.0), responsible for processing data from all sensors of the system in real time, including Microsoft's own developed real-time depth sensor, cameras for tracking the user's head position, sensors of the inertia measurement unit (IMU) and infrared camera.

Hardware

HoloLens 2.0 is based on a processor with a performance of at least Intel Mobile Core i5, the built-in graphics module is not weaker than Intel HD Graphics 620 (GT2) with support for the Direct3D 12 API. The device is equipped with at least 8 GB of dual-channel memory, HDMI 1.4 interface with support for 2880x1440 mode with a clock frequency of 60 Hz or HDMI 2.0 with a resolution of 2880x1440 and a clock frequency of 90 Hz, a hard drive with a capacity of 100 GB, a USB 3.0 or USB 3.1 port, as well as Bluetooth 4.0.

Use as a guide to the museum of the Japanese temple

Microsoft HoloLens, (2016)

On August 18, Microsoft announced that HoloLens mixed reality glasses are now being used as a guide to the museum of the Japanese Kennin-ji temple. The ancient exhibits will come to life with visual effects, sounds and pop-up information. Thus, the management of the Kennin-ji temple expects to attract new visitors, as well as strengthen their impression of the exhibition.

One of Kyoto's oldest Zen Buddhist temples Kennin-ji will use Microsoft HoloLens as a museum guide

One of the first artifacts involved in the project is a 400-year-old painted screen with images of the Japanese wind and thunder gods, Fujin and Raijin. In mixed reality, it will be complemented by realistic sounds and 3D elements developed by the Hakuhodo group in partnership with Microsoft. Wearing HoloLens glasses, the visitor will see clouds under the deities, from which lightning strikes the ground. The electronic "performance," accompanied by a voice certificate about the exhibit, will last from three to five minutes.

We want people to take a fresh look at our national treasure. This is the first exhibition at which HoloLens glasses will be applied in this way, - said Kazuhiro Suda, creative director of the advertising agency Hakuhodo Inc. - I believe that each museum should try to use mixed reality, including in order to introduce ordinary visitors to the technology.

Use of glasses in complex spinal surgery

In February 2017, it became known about the use of Microsoft HoloLens virtual reality glasses during complex and risky spinal surgery. It was conducted at the Institute of Orthopedics and Traumatology of Jaraguá do Sul (Instituto de Ortopedia e Traumatologia de Jaraguá do Sul) in the state of Santa Catarine (Brazil).

Surgeons Henrique Lampert and Bruno Gobbato used HoloLens in arthrodesis, an operation to create artificial ankylosis of the joint, in which complete immobility of the joint is achieved. The Microsoft gadget helped doctors accurately install screws to fix the spine in places where movement hurt to avoid nerve damage or implant destruction.

Using HoloLens, surgeons visualized a three-dimensional model of the patient's spine with optimal screw arrangement, created on the basis of computed tomography (CT). Scientists note that such imaging is very difficult, since it requires the use of expensive CT equipment during surgery, which not all hospitals have. HoloLens is a portable device and costs $3,000, which is much less than specialized medical solutions.

Enrique Lampert and Bruno Gobbato expressed hope that the new versions of HoloLens will be able to display a voluminous picture with a higher resolution and frame rate. At the same time, Lampert and Gobbato said, they liked the "limited" field of view of the device, as it allows you to simultaneously follow other screens, recordings and items not related to the headset.

Doctors intend to develop a training course on the use of holographic technology implemented in HoloLens glasses in surgical operations.^[11]

Builder Version

Microsoft, together with the American manufacturer POTRMB and the University of Cambridge, has developed HoloLens augmented reality glasses designed for use in the construction industry, with an announcement expected in 2017.

The device is based on Microsoft HoloLens mixed reality glasses and looks like a VR helmet. The glasses will run on Windows 10.

The glasses have built-in sensors that read the technical parameters of the buildings and identify both errors made in the design and construction, as well as purchased damage to the structure. In addition, using the gadget, you can create your own 3D model of a building - a building or an infrastructure object, for example, a bridge - and save it for subsequent computer processing.

In 2016, Hololens glasses went on sale, allowing you to work with the interior and layouts of buildings.

2016: Computer made in the form of glasses

Microsoft HoloLens is a headset worn over the head, a hoop with tinted lenses and a wavy prismatic structure located in front of the eyes. The lenses are refracted and directed into the user's eyes from microdisplay, from the spirit of the sides.

Glasses are equipped with adjustment means, individual adjustment to the physiological features of the user's head: interpupular distance and others. The speakers are located at the bottom edge of the device, they provide virtual reality sounds and external noise.

Microsoft HoloLens, (2016)

Product development began in 2012. Microsoft The company unveiled the prototype in January 2015. In March 2016, sales of the developer version began in and. USA In To Canada October 2016, pre-order of the device became available in,,,, Great Britain Ireland France Germany Australia and New Zealand.

Microsoft HoloLens, (2016)

The HoloLens design uses:

• 64-bit 4-core Intel Atom x5-Z8100 processor with a frequency of 1.04 GHz.
• GPU
• a holographic processor developed by Microsoft specifically for HoloLens.

The holographic processor in the 12 × 12 mm package uses 28 digital signal processors manufactured by Tensilica to process and integrate data coming from sensors, spatial scanning of the room, gesture recognition, voice and speech. SoC (system on chip) and holographic processor share 8 MB of embedded SRAM memory and 1 GB of LPDDR3 each.

Built-in data storage of 64 GB, of which ~ 10 GB is the operating system. The amount of RAM is 2 GB.

According to the developer, Microsoft HoloLens is a computer made in the form of glasses. The picture displayed on their glass is superimposed on the world around them and embedded in it through data received from the device's cameras. Interactive applications are controlled by gestures or voice.

Notes