RSS
Логотип
Баннер в шапке 1
Баннер в шапке 2

Microsoft Hololens

Product
Developers: Microsoft
Last Release Date: February 2019
Branches: Pharmaceuticals, medicine, healthcare

Content

Microsoft HoloLens - virtual reality glasses. They are a head-mounted hoop with tinted lenses with a wave-like prismatic structure located in front of the eyes, which refract and send images from microdisplays located on the sides into the user's eyes.

Microsoft HoloLens - mixed reality glasses . Use the 32-bit operating system Windows Holographic (version of Windows 10). The device is autonomous and does not require connection to a PC , smartphone or game console.

In addition to central and graphics processors, glasses have a holographic processor processing unit, designed Microsoft specifically for HoloLens.

2022: VR glasses for daltons released

On April 13, 2022, Microsoft released the Windows Holographic platform version 22H1, which offers users HoloLens 2, as well as IT administrators and developers a large set of new features, including color blindness. More details here.

2021: Start Selling HoloLens 2 Industrial Edition

In early February 2021, Microsoft begins selling an augmented reality headset for industrial enterprises called HoloLens 2 Industrial Edition. Microsoft said it tried to take into account feedback from customers who used the previous version - Lockheed Martin, Imperial College Healthcare NHS Trust and Mercedes-Benz USA.

Lockheed Martin is known to use HoloLens 2 to create the Orion spacecraft, significantly reducing costs, since actions that previously required an eight-hour shift can now be performed in just 45 minutes. Imperial College Healthcare NHS Trust uses HoloLens 2 to limit healthcare professionals' contact with COVID-19 patients, achieving an 83% reduction in risk, and Mercedes-Benz applies a headset to improve the performance of maintenance technicians.

Microsoft begins selling AR headsets for industrial enterprises, it has already been acquired by Mercedez-Benz and Lockheed Martin

However, many companies would like to use HoloLens 2 in strictly regulated areas where devices must comply with particle emission standards, for example, in the pharmaceutical and semiconductor industries. And in the energy sector, companies need a device that can be safely operated in potentially dangerous environments.

The HoloLens 2 Industrial Edition headset complies with ISO 14644-1 class 5-8, which allows it to be used in clean rooms. In terms of intrinsic safety, the headset complies with the UL Class I standard, Section 2. In addition, HoloLens 2 Industrial Edition has a two-year warranty and is included in the quick replacement program, in which the faulty device can be replaced with a new one as soon as possible.

HoloLens 2 Industrial Edition is already available for pre-order at $4,950. The first deliveries to customers will begin in the spring of 2021.[1]

2020: Support for Windows Autopilot service

On May 8, 2020 Microsoft added to the decision of Windows Autopilot support of points added realnostiHoloLens 2. More details here.

2019

Start of sales HoloLens 2. Price - $3,500

On November 7, 2019, Microsoft announced the start of sales of the mixed reality headset HoloLens 2. Its cost turned out to be the same one that the American corporation called during the announcement of the product - $3,500 (more than 223 thousand rubles at the exchange rate on November 7).

Residents of the USA, France, Germany, Ireland, New Zealand, Australia and Britain will be the first to purchase HoloLens 2. The timing of the product in other countries, including Russia, Microsoft is not called.

Microsoft announced the launch of mixed reality headset sales HoloLens 2

The manufacturer addresses HoloLens 2 to the corporate sector, mainly to companies engaged in production or repair work. For an additional monthly fee, the business can purchase Microsoft Remote Assist software for real-time troubleshooting.

Journalists of The Verge, who managed to test HoloLens 2, noted some differences between the device and its predecessor. Firstly, the developers increased the viewing angle from 34 to 52 degrees. Secondly, the gesture management capabilities have been significantly expanded: if in the first version they were limited to clicking on virtual objects, then in HoloLens 2 you can scale and move objects in the field of view. In addition, in the new headset, you can call up the pop-up menu by pressing the holographic button on the wrist.

Microsoft Communications Director Greg Sullivan says the company retains support for HoloLens first, but some developers may start creating applications that require gesture control, implemented only in HoloLens 2.

According to Sullivan, among those who left applications for the purchase of HoloLens 2 were both new customers and those who decided to buy the device in return for first-generation HoloLens.[2]

Emulator output for HoloLens 2

In mid-April 2019, Microsoft released an emulator for HoloLens 2, which will allow you to test applications on a computer without using the mixed reality device itself. HoloLens Emulator is a very useful product, given the high cost of the gadget - $3,500. More details here.

Announcement HoloLens 2

February 24, 2019 the presentation of the mixed reality helmet of the second generation Microsoft HoloLens took place. The device is addressed exclusively to corporate customers and will cost a lot.

Compared to the first version, the new headset received a doubled field of vision while maintaining a holographic density of 47 pixels per degree of view. The company explains that the improvement is equivalent to switching from a TV with a resolution of 720p to 2K for each eye. 

HoloLens 2 allows you to track hand movements without any additional controllers. During the live demonstration, one of the Microsoft employees launched various applications in 3D space, including a virtual piano. 

Microsoft introduced the mixed reality helmet HoloLens 2. Price

HoloLens 2 includes a new display system that allows devices to demonstrate increased performance at low power consumption.  Using a new time-of-flight depth sensor in combination with built-in artificial intelligence and semantic understanding, the wearable device allows you to interact with holograms in the same way as it happens in the real world with physical objects.

The device is equipped with eye tracking sensors, which makes interaction with holograms even more natural. facial recognition Windows Hello is also supported. The developers claim that the new helmet is "lighter and more convenient" compared to the previous model.

Microsoft notes that HoloLens 2 is focused on use in companies that can create their own programs and application stores for the gadget. In addition, the helmet will support the installation of third-party web browsers, and Mozilla has already begun to create such a product.

After the announcement of HoloLens 2, Microsoft began to accept pre-orders for the purchase of a mixed reality helmet. The price is $3,500.[3]

2018

How Microsoft's AR headset helps surgeons with surgeries

By the end of November 2018, surgeons began using a mixed reality headset to reasonably plan reconstruction operations on the lower limbs - Microsoft's new AR headset HoloLens helps understand which tissues and vessels can be used in recovery operations using autotransplantation.

Patients are prescribed standard CT tests with contrast, which provides visualization of passable vessels and allows to isolate the most viable tissues. Then, 3D reconstruction simulates the limb and all its vessels, and transmits the data to the HoloLens. Using the headset, surgeons see a projection of blood vessels, bones and muscles applied to the patient's leg, as well as radiologists' tags, so they can accurately predict the volume of the upcoming operation.

Microsoft AR headset helps surgeons with surgeries

Traditionally, doppler-ultrasound has been used to evaluate the location of blood vessels before surgery, but this technique is inferior to HoloLens in the reliability of the data obtained. In addition, the use of HoloLens allows you to reduce the time of surgery, and therefore the time of anesthesia, which is important not only for the work of the department, but also for the patients themselves - it is known that reducing the time of surgery and anesthesia improves the outcome and reduces the risk of complications.

Having proved that the HoloLens system can be effectively used in the work of some departments, Microsoft developers began to consider other ways of applying the AR headset - in particular, it is already used in operations to remove colorectal cancer.

Nevertheless, there are a number of problems that researchers hope to solve before the HoloLens system begins to be used more widely. Developers are trying to reduce contrast image processing time, which currently takes about an hour, achieve higher image resolution and improve compatibility between HoloLens and hospital software.[4]

Use in spacecraft assembly

On October 23, 2018, it became known that Lockheed Martin engineers, who were commissioned by NASA to assemble the Orion spacecraft, use Microsoft Hololens glasses. The headset helps visualize the instructions and the final build result, eliminating the need to study thousands of pages of paper guides. Orion is being developed for the delivery of astronauts and cargo to the ISS, and later for manned flights to the moon and Mars.

Thanks to Hololens, engineers can see holograms modeled in a special program from Scope AR, which are superimposed on the elements on which they work. Each part of the model has a comment on how to install it, and information on how to twist the bolts is displayed next to the corresponding hole. Also, for each engineer, only those parts that correspond to his specialization are highlighted in a specific color.

Earlier, Microsoft and NASA launched the Sidekick joint project, in which ISS astronauts used Hololens headsets for repairs, experiments and communication with engineers on Earth[5].

Exit Dynamics 365 Layout and Dynamics 365 Remote Assistant for HoloLens

In September 2018, Microsoft introduced Dynamics 365 new applications for using artificial intelligence mixed reality. With them, organizations and enterprises can take full advantage of helmet capabilities mixed reality HoloLens for business tasks. More. here

Application for navigation of the blind

In May 2018, the California Institute of Technology demonstrated an application that allows people with lost vision to navigate space using virtual reality Microsoft a HoloLens helmet.

This device is equipped with several cameras, including a depth camera, as well as speakers located around the perimeter. This and the ability of the headset to create a volumetric map of the surrounding area was used by a group of engineers led by Markus Meister of the  California Institute of Technology.

They developed a program that allows blind people to move in the premises and "see" objects (steps, furniture, etc .) around using audio navigation.

The application can lead a person along a given route, informing him of approaching turns and all kinds of obstacles. Glasses capture the video image, extract important environmental data and transmit this information through auditory augmented reality. So, a person in a Microsoft helmet hears hints such as "Up the stairs," "Railing on both sides," "Go to the right," etc.

The VR navigator was tested on seven volunteers who were invited to overcome the crossed route on two floors. According to the developers, all participants in the experiment went the first time.

Marcus Meister hopes that after the study, it will be possible to create devices that can be used by people with impaired vision in public places, including shopping centers and hotels.

According to the World Health Organization, by May 2018, about 253 million people around the world are visually impaired or completely deprived of vision. The development of the California Institute of Technology has great potential, but much more work needs to be done. For example, an application cannot build routes that have not been laid by someone in advance, and does not respond to people who may be in the same room.[6]

Application in complex reconstructive surgery

In February 2018, Microsoft announced that doctors from Imperial College London used a mixed reality device HoloLens in reconstructive plastic surgery. As part of the experiment, a 3D photograph of computed tomography (CT) was loaded into the headset and shown to the doctor in the form of a hologram during the operation. This technology allows you to establish the exact location of the blood vessels, bones and muscles of the patient, which greatly simplifies fracture operations, and also reduces potential risks, Microsoft said.

Surgeons used Microsoft HoloLens in complex reconstructive surgery

As is known, the peculiarity of operations in fracture and tissue transplantation is the need for an accurate picture of anatomical features for successful and maximum operational connection of vessels and subsequent restoration of blood flow of the operated area. Having a visual vascular, bone and muscle layout in front of your eyes could be a key factor in the success of the operation, usually requiring careful preparation, doctors said.

In carrying out such interventions, surgeons traditionally use a hand-held ultrasound scanner to detect blood vessels under the skin along the movement of blood flow. However, this process is very time consuming and does not provide accurate data on the location of vessels under the skin.

Demonstration of the experiment, February 2017

Use in teaching doctors to take complex births

As it became known in January 2018, CAE created a LucinaAR simulator designed to help doctors work out actions in various versions of birth complications. The simulator is based on detailed mannequins of the mother and baby. The kit also includes mixed reality glasses HoloLens, with the help of which you can see the movement of the fetus and evaluate the effectiveness of the measures taken.

In particular, the simulator is designed to teach doctors how to work with shoulder dystocia - a rare type of complication, it can occur up to once per thousand cases. If the obstetrician is familiar with techniques that help release the baby's shoulder, this significantly increases the chances of successful birth. Detailed simulation in this case is indispensable.

File:Aquote1.png
LucinaAR is a real breakthrough in educational simulation, thanks to which students will be able to see the anatomy of the patient with their own eyes, "said CAE President Dr. Robert Amio. - The simulator will be combined with our mixed reality platform, because we are confident in the immediate and powerful effect of preparation for childbirth with shoulder dystocia.
File:Aquote2.png

Canadian company CAE, which specializes in the production of equipment for training in avionics and medicine, in 2017 demonstrated its first product using HoloLens, a VimedixAR project aimed at teaching medical students to conduct ultrasound examinations.

2017

Use in tumor removal surgery

Being in different parts of the world, three prominent surgeons from India and Britain with the help of augmented reality glasses Microsoft Hololens were able to virtually be present in the same operating room and jointly perform surgery to remove the tumor. This was announced on October 23, 2017 in his blog by BMI Healthcare, the largest group of private clinics in the UK.[7]

Microsoft AR glasses helped surgeons from different countries perform tumor removal surgery

Colorectal cancer surgery took place at The Royal London Hospital in London. UK surgeon Shafi Ahmed was assisted by colleagues - Professor Shailesh Shrikhande, an oncosurgeon at Tata Memorial Hospital in Mumbai, India's largest cancer clinic, and Hitesh Patel, a consultant to the Private Hospital Hospital of the Onbrectal Hospital. Another virtual consultant was Ian Nott from the USA, co-founder and technical director of Aetho, which developed the AR application Thrive for Microsoft Hololens glasses.

Specialists worked in AR glasses and could not only hear, but see each other in the form of graphic avatar figures, as if they were really in the same operating room. Tomographic images of the patient's internal organs were available to surgeons, which were displayed as three-dimensional holograms, and doctors could make virtual markups on them discussing the course of the operation.

AR application Thrive for Hololens glasses developed by Aetho
File:Aquote1.png
"We have truly combined technology with medicine. Bringing together specialists from different parts of the world and time zones, we demonstrated that we can make operations safer, ensure better patient outcomes and democratize surgical practice, "commented Professor Shafi Ahmed[8]
File:Aquote2.png

Use of HoloLens glasses in oncological surgery, October 24, 2017

It is worth noting that this is not the first experience of Shafi Ahmed in the use of innovations in the operating room. In 2014, with the help of VR glasses, Google Glass organized an online broadcast of the operation, which was seen by several thousand medical students from 115 countries.

Microsoft HoloLens 2.0

According to Microsoft, the second generation of HoloLens glasses, in addition to the "holographic" processor module (Holographic Processing Unit/HPU 2.0), will also be equipped with an artificial intelligence coprocessor (AI coprocessor), thanks to which it will be possible to support Deep Neural Networks (DNN) technologies.

The executive vice president of Microsoft's Artificial Intelligence and Research Group, Gary Shum, released the first details of the second generation of HoloLens glasses in July, the development of which is ongoing.[9]

Photo: s9.tarhely.com

Among the many additional functions and capabilities of HoloLens 2.0, the most interesting is the coprocessor of artificial intelligence (AI coprocessor), which will allow the device to acquire support for deep neural network technologies. This means that HoloLens will be able to process large amounts of data on the fly without resorting to external cloud resources.

As Gary Shum specified, the AI coprocessor will be fully reprogrammed and will be able to work in real time, which, in turn, will allow the second generation to HoloLens autonomously and without delay analyze what the user sees and hears. In addition, the chip will be used by developers of the HoloLens 2.0 software platform to create mixed reality applications with an activated AI function.

As well as in a case with HoloLens of the first generation, the "holographic" processor module (Holographic Processing Unit/HPU 2.0) which is responsible for data processing from all sensors of a system in real time including the sensor of depth of real time of own development of Microsoft, cameras for tracking position of the head of the user, sensors of the block of measurement of inertia (IMU) and the infrared camera will also be feature of the hardware platform of HoloLens 2.0 glasses.

Hardware

HoloLens 2.0 is based on a processor with a performance of at least Intel Mobile Core i5, the built-in graphics module is no weaker than Intel HD Graphics 620 (GT2) with support for the Direct3D 12 API. The device is equipped with at least 8 GB of dual-channel memory, HDMI 1.4 interface with support for mode 2880x1440 with a clock frequency of 60 Hz or HDMI 2.0 with a resolution of 2880x1440 and a clock frequency of 90 Hz, a hard drive with a capacity of 100 GB, a USB 3.0 or USB 3.1 port, as well as Bluetooth 4.0.

Use as a guide to the museum of the Japanese temple

Microsoft HoloLens, (2016)


On August 18, Microsoft announced that mixed reality glasses HoloLens are now used as a museum guide for the Japanese Kennin-ji Temple. Ancient exhibits will be revived with visual effects, sounds and pop-up information. Thus, the leadership of the Kennin-ji temple expects to attract new visitors, as well as strengthen their impression of the exhibition.

One of Kyoto's oldest Zen Buddhist temples, Kennin-ji, will use Microsoft HoloLens as a museum guide

One of the first artifacts involved in the project is a 400-year-old painted screen with images of Japanese gods of wind and thunder, Fujin and Raijin. In mixed reality, it will be complemented by realistic sounds and 3D elements developed by the Hakuhodo group in partnership with Microsoft. Wearing glasses HoloLens, the visitor will see clouds under the deities, from which lightning strikes the earth. The electronic "performance," accompanied by a voice certificate about the exhibit, will last from three to five minutes.

File:Aquote1.png
We want people to take a fresh look at our national heritage. This is the first exhibition in which HoloLens glasses will be applied in this way, "stressed Kazuhiro Suda, creative director of the advertising agency Hakuhodo Inc. - I believe that each museum should try to use mixed reality, including in order to connect ordinary visitors to the technology.
File:Aquote2.png

Using glasses in complex spinal surgery

In February 2017, it became known about the use of Microsoft HoloLens virtual reality glasses during a complex and risky spinal surgery. She was held at the Instituto de Ortopedia e Traumatologia de Jaraguá do Sul in Santa Catarine, Brazil.

Surgeons Henrique Lampert and Bruno Gobbato used HoloLens in arthrodesis - the operation of creating artificial ankylosis of the joint, in which complete immobility of the joint is achieved. The Microsoft gadget helped doctors accurately install screws to fix the spine in places where movement caused pain in order to avoid nerve damage or destruction of the implant.

With the help of HoloLens, surgeons visualized a three-dimensional model of the patient's spine with optimal screw arrangement, created on the basis of computed tomography (CT). Scientists note that such imaging is very complex, since it requires the use of expensive CT equipment during the operation, which not all hospitals possess. HoloLens is a portable device and costs $3 thousand, which is much less compared to specialized medical solutions.

Enrique Lampert and Bruno Gobbato expressed the hope that the new versions of the HoloLens will be able to output a volumetric picture with higher resolution and frame rate. At the same time, according to Lampert and Gobbato, they liked the "limited" field of view of the device, since it allows you to simultaneously monitor other screens, recordings and elements not related to the headset.

Doctors intend to develop a training course on the use of HoloLens holographic technology implemented in glasses in surgical operations.[10]

Builders Version

Microsoft, together with the American manufacturing company POTRMB and the University of Cambridge, has developed augmented reality glasses HoloLens for use in the construction industry, the announcement is expected in 2017.

The device is based on Microsoft HoloLens mixed reality glasses and looks like a VR helmet. Glasses will work on Windows 10.

Glasses have built-in sensors that read the technical parameters of buildings and detect both errors made during design and construction, as well as purchased structural damage. In addition, using the gadget, you can create your own 3D model of construction - a building or an infrastructure object, for example, a bridge - and save it for subsequent computer processing.

In 2016, Hololens glasses went on sale, allowing you to work with the interior and models of buildings.

2016: Computer made in the form of glasses

Microsoft HoloLens is a headset worn on the head, a hoop with tinted lenses and a wavy prismatic structure located in front of the eyes. Lenses are refracted and directed into user's eyes of images from microdisplays, from the spirit sides.

The glasses are equipped with means of adjustment, individual adjustment to the physiological features of the user's head: inter-transparent distance and others. Speakers are located at the lower edge of the device, they provide the sound of virtual reality and external noise.

Microsoft HoloLens, (2016)
Microsoft HoloLens, (2016)

The design HoloLens use:

  • 64-bit 4-core processor Intel Atom x5-Z8100 with a frequency of 1.04 GHz.
  • GPU
  • a holographic processor developed by Microsoft specifically for HoloLens.

The 12 × 12 mm holographic processor uses 28 Tensilica digital signal processors to process and integrate sensor data, spatial room scanning, gesture recognition, voice and speech. SoC (system on chip) and holographic processor divide 8 MB of embedded SRAM and 1 GB of LPDDR3.

64 GB of built-in data storage, of which ~ 10 GB is the operating system. The RAM volume is 2 GB.

According to the development company, Microsoft HoloLens is a computer made in the form of glasses. The picture displayed on their glass is superimposed on the world around them and embedded in it by data received from the cameras of the device. Control in interactive applications is performed by gestures or voice.

Notes