Binocular stereo vision principle

Binocular stereo vision is a subject with broad application prospects. With the development of optics, electronics and computer technology, it will continue to progress and gradually become practical. It will not only become a key technology in the fields of industrial inspection, biomedicine, virtual reality It may also be applied in the fields of space telemetry and military reconnaissance. Currently abroad, binocular vision technology has been widely used in production and life.

Binocular stereo vision is an important branch of computer vision. From the application of binocular vision, it is mainly divided into four parts:

-Binocular vision navigation and positioning;

Two three-dimensional reconstruction;

Three binocular stereo measurement;

Four-dimensional three-dimensional stereo tracking.

This article only describes its basic principle and implementation process from the aspect of "navigation and positioning based on binocular vision".

The principle of binocular vision:

The basic principle of binocular vision is to simulate the human eye and use spatial geometric models to derive corresponding algorithms to solve practical problems. To be more straightforward, the most basic purpose of binocular vision is to extract the "points", "lines", and "surfaces" we are interested in from the complex objective world, and then describe them with numbers in order to accurately understand and control them. This principle contains three parts:

The following uses a robot navigation application as an example for illustration. In this application, what needs to be done is to tell the robot that there is an obstacle in front of it and also to tell it the position information of the obstacle in the robot's navigation coordinate system. Then the robot can make a judgment and avoid it. When implementing this function, the following three parts are included.

First, extract points of interest (feature detection). It is to segment the object we are interested in. For navigation and positioning, it is to find the obstacle in front of the robot. The features of this application in feature point extraction are: fewer feature points to be extracted and high extraction speed requirements. The specific speed needs to be measured according to the overall accuracy of the robot. Generally, at least five signals are given to the robot in one second, that is, at least 5 frames of images are collected every second and processed separately.

In this process, we must first extract the information of the obstacles. It is easy to think that we can install a camera on the robot to take pictures of the road ahead. However, if a camera is used, we can only get the plane information of the obstacle, that is, the X and Y coordinate information. The robot does not know how far it should go before it can perform the obstacle avoidance action. So we need to add another camera to shoot this obstacle from another angle. In this way, you can get 2 pictures describing the obstacle from different angles. Then add some appropriate pre-processing algorithms (image processing), such as binarization, edge extraction, feature point denoising (need to choose the appropriate algorithm according to the specific scene), etc. to extract and segment the obstacles in the two pictures come out. This completes the extraction of feature points and lays the foundation for the next step of "precise digital description."

Second, accurate digital description (stereo matching, attitude measurement). This part refers to the need to describe the feature points of the obstacle after segmentation with valid values. Of course, in the binocular vision system, it is described by three-dimensional coordinates. If the monocular camera calibration and binocular camera calibration are performed at the same time as the binocular camera is installed, the parameter matrix of the dual camera and lens is confirmed, and the translation vector and rotation matrix are obtained. Then according to the principle of binocular stereo matching, the three-dimensional coordinate value of the obstacle can be obtained. Enter the three-dimensional coordinate value into the robot control system, and the robot can intelligently avoid obstacles.

 Binocular vision.png

In this part, the "stereo matching" algorithm is used. In fact, the algorithm is combined with dual target determination. This algorithm calculates the basic matrix based on the coordinate points of the feature points in the left and right images, and the coordinate points of the same name are left and right. correspond. The "translation vector" and "rotation matrix" used in this process are the parameters given in the next dual target determination.

Third, dual goals. Binary targeting is to use the correspondence between the known world coordinate system (calibration plate) and the image coordinate system (the result of image processing on the calibration plate) to calculate the parameter information of the binocular camera under the current positional relationship. After the calibration is completed, the three-dimensional information can be obtained when the binocular system is used to observe the unknown world coordinate system. In fact, before the dual-target calibration, a single camera calibration needs to be performed on each camera to determine its distortion coefficient and camera internal parameter matrix. The purpose of this is so that the images acquired by the left and right cameras can be corrected to a standard image before processing.

In general, as long as it involves a binocular project, this step is universal and essential. For different applications, it is only necessary to do different processing when obtaining the specific world coordinate system.