Research on Space Monitoring and Positioning Technology of ARVR Equipment

All ARVR products must have accurate space monitoring and positioning capabilities to achieve a full range of head and hand interactions and interactions in order to provide users with a more realistic virtual experience.

First, the current status of ARVR products

There are quite a few ARVR products on the market, and the price difference varies from tens of dollars in Cardboard to tens of thousands of Hololens. Let's talk about VR products, which can be divided into the following categories as a whole:

1. Cardboard products

Research on Space Monitoring and Positioning Technology of ARVR Equipment

This kind of product is the simplest VR product, which is derived from Google's carton VR box. Simply put, the two convex lenses are used to project the contents of the mobile phone screen to the glasses, and the user's head is judged by the built-in IMU of the mobile phone. The motion pose to adjust the direction of motion of the VR display content. This kind of product began to ship a large number of white cards in China in 2015, but the overall effect of VR depends on the configuration of the mobile phone and the quality of the optical lens. It can only be said to be an entry-level VR experience product.

The first generation of VR products of some domestic video content and game companies are basically in this form. The most representative one is the Storm Box, which is equipped with a BT control handle for menu selection. The emergence of such products has indeed played a significant role in the promotion of the entire VR industry, but its drawbacks are also very obvious, so the integrated machine such as the storm machine, big friends, and small birds will be listed in 2016.

Cardboard-style VR devices have ushered in a particularly big opportunity in 2016, that is, Google’s Daydream VR, Google’s user experience in VR Cardboard, and hopes to dominate the entire VR ecosystem in 2016. The Daydream mobile phone Pixel, Pixel XL and Daydream Viewer are equipped with Cardboard, and a remote control handle with built-in IMU is also available. According to Google's requirements for the user experience of VR devices, it is necessary to reach a mobile phone with a certain hardware configuration. In conjunction with the Daydream Viewer running related VR applications, users will not be dizzy. Since 2016, with the launch of Huawei P9, Lenovo MotoZ and other products, more and more mobile phones claiming to support Daydream are expected. In 2017, such products will completely break the deadlock of the original Cardboard user experience.

2. Head motion control integrated machine

Research on Space Monitoring and Positioning Technology of ARVR Equipment

As one name suggests, the all-in-one machine is a VR device that can be used without having to plug in a mobile phone or connect to a PC. At present, there are representative big friends, small birds, etc. Most of the products are listed in 2016, and the characteristics of these products are basically It is a 1080p or 2K resolution display (some also use 2Kx2 dual screen) with a Fresnel aspherical optical lens specially designed for VR products, the overall picture quality and distortion have improved a lot. However, at present, such products have a common feature, that is, only simple head movement control, can only sit or stand in place to shake the head to achieve VR control, can not do the first-hand interaction. In order to make up for this defect, there are some external accessories with space position detection function (built-in IMU handle, somatosensory backpack, space positioning light ball, etc.) to enhance the overall VR user experience.

3. All-in-one with space monitoring and positioning

Research on Space Monitoring and Positioning Technology of ARVR Equipment

There are two ways to monitor and locate space, Inside-out and Outside-in. Inside-out uses the built-in sensor of the all-in-one to sense the spatial information outside the device and make related positioning. Outside-in, on the other hand, uses external sensors to sense the location of the VR device, providing location information to the VR application. For the all-in-one machine, the Inside-out solution has been selected without exception, thus avoiding the need to additionally arrange external sensors in advance, and the user can pick up the VR device anytime and anywhere. It is expected that some of the big products that will be launched in 2017, such as the second generation of the big friend, the storm machine, etc., will have the spatial positioning capability, which will greatly enhance the user experience of VR, especially VR games.

4. PC VR

Research on Space Monitoring and Positioning Technology of ARVR Equipment

The most representative of these products are HTC Vive, Sony PSVR and HTC Oculus. After the launch of the Three Musketeers in 2016, the special HTC Vive has become the standard of the VR Experience Pavilion. Many other industry customizations are also based on Vive. Sony PSVR sold about 1 million units in less than half a year, which has greatly improved the overall level of user experience in the entire VR industry. The characteristics of these three products are the use of external spatial positioning technology. For example, Vive uses an external infrared laser transmitter, the position of the head display and the handle sensor to determine its position and motion track, PSVR and Oculus. The external sensor is used to sense the visible light or infrared light emitted by the head display and the handle to judge the position of the head display and the handle; so that the head display and the handle can accurately know their respective relative positions and movement trajectories in space at any time. This will make the application of VR more realistic.

Next, I will talk about AR products. At present, the AR products are very different. Some AR glasses products only project virtual information and images directly. They are not actually related to the actual environment where the users are located. The projected virtual information follows the head movement to achieve 6DOF motion tracking, but the virtual object cannot be merged with the actual environment where the user is located. For example, the famous ODG, the latest products only support 6DOF motion tracking, no spatial positioning function.

Research on Space Monitoring and Positioning Technology of ARVR Equipment

Shanghai Zhishi's smart glasses are similar to the functions of ODG, but their positioning is the live broadcast of the first perspective, so it is only necessary to display the relevant information and video of the shooting in the glasses, without spatial positioning. However, it is understood that they are also developing a lens with a built-in TOF depth camera, which is capable of 6DOF+ spatial positioning.

Research on Space Monitoring and Positioning Technology of ARVR Equipment

Among the AR products currently available, the space monitoring and positioning capabilities are the best. The virtual objects match the environment with the best two products, one is Lenovo Phab2 Pro based on Google Tango's spatial positioning technology, and the other is Microsoft's Hololens. Although these two products are very different in form, they are essentially a complete combination of 6DOF+SLAM, which realizes spatial positioning and motion tracking. Their AR effects can basically achieve virtual objects and actual environments. Fusion, no matter how the user moves and from which angle, the feeling is the same as the real object. However, from the final use effect, Hololons' 6DOF+SLAM is still more accurate and stable than Tango, which is closely related to the whole hardware design platform, the choice of sensor type and quantity, and the core algorithm.

Research on Space Monitoring and Positioning Technology of ARVR Equipment

In addition, there are still many companies in China that are also developing related AR products. Among them, Micro Eye's SMAKKEST is not only similar to Hololens, but also similar in function to Hololens. The effect of 6DOF+SLAM is said to be recognized by Hololens.

Research on Space Monitoring and Positioning Technology of ARVR Equipment

Second, the spatial positioning technology category

In essence, space monitoring and positioning technology can be divided into two major categories, one is Inside-out space monitoring and positioning, and the other is Outside-in space monitoring and positioning.

1. Inside-out space monitoring and positioning technology

Inside-out space monitoring and positioning technology, the technology is essentially similar to the process of human glasses sensing the environment, and the external space is perceived from inside the device, so it is called the Inside-out positioning method. Inside-out space monitoring and positioning technology is also divided into two categories, one is based on SLAM technology for space monitoring and positioning, which is also a basic function required by the current hot robot industry, SLAM is the English abbreviation for simultaneous localizaTIon and mapping. It means that the device itself starts to move from an unknown location in an unknown environment, and locates itself according to the location estimation and map during the movement process, and builds an incremental map based on its own positioning to realize the self-detection and positioning navigation of the device itself. . Most AR products currently use Inside-out space positioning technology.

The representative products of SLAM-based space monitoring and positioning technology are Lenovo Phab2 Pro Tango mobile phone and Hololens, which use deep monitoring + Fisheye and IMU fusion for motion monitoring. The difference is that the technology used for depth monitoring is different (Tango uses TOF, while Hololens uses structured light) and Fisheye motion and feature point monitoring are different. Hololens' spatial positioning effect is more accurate and stable than Tango, and in the low light, glass light transmission, etc., Hololens effect Winning the current Tango, but the Tango technology is not static, the next generation of Tango technology also supports multiple Fisheye camera.

Lenovo Phab2 Pro Tango phone, using a TOF depth camera, a 155 degree FOV Fisheye motion monitoring camera and an RGB camera, Fisheye motion monitoring camera with IMU data fusion, feature point matching, to provide a complete motion track for the device The monitoring, together with the cloud image information of the TOF depth camera, can map the position of the device itself in space and the trajectory of the movement in real time, and at the same time, through the area learning function of Tango, when returning to the space that has already passed, once the Fisheye camera By monitoring the already stored feature points and matching them, it is possible to quickly perform spatial positioning, which is very important for AR devices, ensuring that users can quickly use them anytime, anywhere. The working principle of Tango is not explained here. You can refer to Lin Shigong's other article for a detailed understanding.

Research on Space Monitoring and Positioning Technology of ARVR Equipment

Microsoft's Hololens design is very unique and very rich and self-willed. It has opened a new HPU (Holographic Processing Unit) based on Intel's CPU and GPU technology. The device uses CPU+GPU+HPU configuration. Like Tango, it does not need External PCs can be used to achieve complete spatial monitoring and positioning. On the sensor side, depth monitoring uses two structured light cameras with four structured light grating emitters for depth monitoring. In addition, there are two Fisheye cameras on the left and right sides of Hololens, which can detect more feature points in the space environment. Therefore, the spatial positioning accuracy and stability of Hololens is much higher than that of Tango.

Research on Space Monitoring and Positioning Technology of ARVR Equipment

The other type is based on the marker (point) spatial monitoring and positioning technology. Simply put, some patterns like the two-dimensional code, special color graphics or light spots are placed in the actual space as the marker points, and the device monitors these markers. The position of the point to determine its position and movement trend. HTC Vive is the representative of this type, the following figure is the working principle of HTC Vive spatial positioning.

Research on Space Monitoring and Positioning Technology of ARVR Equipment

Vive emits laser light through two fixed laser emitters. Each base station has an infrared LED array that distributes two rotating infrared laser emitters that are perpendicular to each other. One is X-axis scanning and the other is Y-axis scanning. The lasers have a fixed 180 degree phase difference. When one of the lasers emits an X-axis scan, the other laser emits a Y-axis scan.

Research on Space Monitoring and Positioning Technology of ARVR Equipment

HTC Vive's head-mounted housing is densely covered with 32 light sensors, which are oriented in different directions. These sensors are used to receive infrared laser light from a fixed laser transmitter. The computer connected to Vive will control all devices to operate synchronously. The light sensor on the head display respectively monitors the time when the laser reaches the sensor in the X-axis and Y-axis directions and the phase relationship of the two different laser emitters, so that the phase difference of each light sensor can be calculated, thereby Accurately position the head and motion trajectory.

There are 24 light sensor on the Vive handle. The working principle of the handle is the same as that of the head. It will not be repeated here.

Research on Space Monitoring and Positioning Technology of ARVR Equipment

In addition, it must be mentioned that the domestic Ximmerse, they also introduced the Outside-in space positioning package design for VR devices. With this spatial positioning package, VR devices without spatial positioning function can be provided with similar accuracy to HTC Vive. Space monitoring and positioning capabilities. The positioning package includes a dual camera module and two control handles with a light ball. When used, the dual camera module is externally attached to the head display, and the light ball of the control handle is captured by the dual camera to determine the position of the handle in the space while the handle is Through BT, the information of 6DOF is transmitted to the module, and the module combines the 6DOF and the position information of the photosphere to achieve omnidirectional positioning, thereby enhancing the interaction of the head hand. The advantage of this solution is that the positioning accuracy can be protected from external ambient light under any ambient lighting conditions, and the positioning accuracy is high. However, the solution cannot monitor and locate the overall space environment of the user, and can only monitor the spatial position information of the ball handle, but the advantage is that the VR device without spatial positioning can be quickly made at least like HTC Vive. Spatial positioning capability.

Research on Space Monitoring and Positioning Technology of ARVR Equipment

2. Outside-in space monitoring and positioning technology

Outside-in's space monitoring and positioning technology is currently a mature space positioning technology for VR devices. Sony PSVR and Oculus use similar solutions, but Oculus is also equipped with an external camera plus infrared active mark points. Higher accuracy and faster response time. The PS VR directly uses the PS 3 MOVE system. The principle is similar to that of the Kinect. The external binocular depth camera is used for motion recognition and tracking of the head display and the handle. There is a light indication on the head, and the top of the handle also has a light ball of different colors. The external dual camera checks the movement of the light ball on the head display and the handle at any time, and the PS StaTIon receives the IMU information of the head display and the handle transmitted back through the BT. Through the calculation, the complete head display and the movement track and positioning of the handle in the space can be obtained.

Research on Space Monitoring and Positioning Technology of ARVR Equipment

From the principle and the actual use effect, the accuracy of space monitoring and positioning is not as high as HTC Vive. Although Sony's game effect and PSVR wearing itself are better than Vive, the spatial positioning accuracy is not high, which affects VR. The overall experience of the game. Therefore, we also saw Sony's latest space positioning patent from Sony's official website. This patent should be prepared for the next generation of PVSR. From the patent description, it should be a method similar to HTC's fixed laser transmitter. At present, PSVR is the opposite. Instead of using an external camera to shoot (Outside-In positioning), the external signal is used to locate the direction of the VR helmet in space. This can provide HTC Vive-like accuracy (possibly higher), while capturing the user's various poses and motion trajectories at 360 degrees without dead ends.

Research on Space Monitoring and Positioning Technology of ARVR Equipment

Oculus's space monitoring and positioning technology is similar to that of Sony PSVR. The difference is that PSVR uses an external dual camera to monitor visible light. Oculus uses active infrared light. The infrared display is placed on both the head display and the handle. The external infrared camera is used to capture and capture the information of the infrared light spot on the head display and the handle, thereby obtaining the movement track and spatial position information of the head display and the handle.

Research on Space Monitoring and Positioning Technology of ARVR Equipment

Research on Space Monitoring and Positioning Technology of ARVR Equipment

Regardless of whether it is Sony PSRV or Oculus's Outside-in solution, there is a problem that the external camera is fixed, and when the user is facing the camera, the camera cannot detect the light spot on the head display and the handle. There is a possibility of leaking supervision, which affects the user experience, which does not exist on HTC Vive.

Another Outside-in space monitoring and positioning technology based on Marker points is also Ximmerse's light ball + dual camera solution, which is the opposite of their previous Inside-out solution, where the dual camera monitor is placed. The fixed position in the room, then the light ball is arranged on the VR head display and the handle, the head display and the handle are in motion, the dual camera monitor records the three-dimensional position of the head display and the handle in the space and their movement data, through BT or WiFi is passed to the VR head to perform related operations and processing, and feedback is implemented on the screen to realize free movement in the virtual environment and interact with things in the virtual world.

Research on Space Monitoring and Positioning Technology of ARVR Equipment

Research on Space Monitoring and Positioning Technology of ARVR Equipment

The following is a summary and comparison of several different scenarios:

Research on Space Monitoring and Positioning Technology of ARVR Equipment

Flat Cable

Antenk offers flat cables from its own production – the location of the in-house Flat cable production is the antenk factory in shenzhen,Made in China. The FLAT CABLES are produced with latest state of the art production facilities and fulfill the Automotive standards.

Flat cable is a standard organization and is suitable for mobile electrical equipment with AC rated voltage of 450v/70v and below. Flat structure is especially suitable for frequent bending occasions, without kinking and folding neatly, such as driving. Yb, YBF and YBZ products can meet the needs of various occasions. It is suitable for the electrical connection between mobile electrical equipment in the harsh environment of power generation, metallurgy, chemical industry, port and so on.

Flat Cable

ShenZhen Antenk Electronics Co,Ltd , https://www.antenkcon.com