1. A double-camera combined calibration method is characterized in that: comprises the following steps of (a) carrying out,

s1, controlling the left camera and the right camera to shoot checkerboard images on the calibration board simultaneously, and establishing the relationship between the checkerboard corner points to the pixel coordinate systems of the checkerboard images shot by the left camera and the right camera and the position relationship between the left camera and the right camera according to the checkerboard images shot by the left camera and the right camera; wherein, the left camera is a standard camera with known internal reference;

s2, based on the z direction of the calibration board coordinate system, the relation between the corner points of the checkerboard to the pixel coordinate system of the checkerboard image shot by the left camera is transformed in a homogeneous way, and the outer parameters of the left camera are solved according to the known inner parameters of the left camera and the relation between the corner points of the checkerboard after the homogeneous transformation to the pixel coordinate system of the checkerboard image shot by the left camera;

s3, performing joint optimization on the internal parameters of the right camera according to the relationship between the corner points of the checkerboard and the pixel coordinate system of the checkerboard image shot by the right camera, the external parameters of the left camera and the position relationship between the left camera and the right camera to obtain the optimal solution of the internal parameters of the right camera;

and S4, calculating the external parameter of the right camera according to the external parameter of the left camera based on the position relation between the left camera and the right camera.

2. The dual-camera joint calibration method as set forth in claim 1, wherein: in S1, the relationship between the corner points of the checkerboard to the pixel coordinate system of the checkerboard image captured by the left camera is,

the relationship of the checkerboard corner points to the pixel coordinate system of the checkerboard image captured by the right camera is,

wherein: p_wIs the coordinate vector of the checkerboard corner point in the coordinate system of the calibration plate, and P_w＝[x_w,y_w,z_w]^T，x_wFor the component, y, of the checkerboard corner in the x-direction in the coordinate system of the calibration plate_wFor the component of the checkerboard corner in the y-direction in the coordinate system of the calibration plate, z_wThe components of the checkerboard angular points in the z direction in a coordinate system of a calibration board are taken as the checkerboard angular points;

is a coordinate vector of the checkerboard corner point in the pixel coordinate system of the checkerboard image shot by the left camera, and the component of the checkerboard corner points in the z-direction in the left camera coordinate system,for the components of the checkerboard corner points in the u direction within the pixel coordinate system of the checkerboard image taken by the left camera,the components of the checkerboard angular points in the v direction in the pixel coordinate system of the checkerboard image shot by the left camera are taken;

M_sIis an internal reference correlation matrix of the left camera, anu_0s、v_0s、α_sAnd beta_sKnown internal references for the left camera;

M_sEis an external parameter matrix of the left camera, anR_sAnd t_sAll external references of the left camera, in particular, R_sIs a 3 × 3 rotation matrix, t_sA translation vector of 3 × 1;

is a coordinate vector of a checkerboard corner point in a pixel coordinate system of a checkerboard image shot by a right camera, and the component of the checkerboard corner points in the z-direction in the right camera coordinate system,for the components in the u direction within the pixel coordinate system of the checkerboard image taken by the right camera for the checkerboard corner points,the angular points of the checkerboards are components in the v direction in a pixel coordinate system of the checkerboard image shot by the right camera;

M_bIis an internal reference correlation matrix of the right camera, anu_0b、v_0b、α_bAnd beta_bAre all internal references of the right camera;

M_bEis an external reference matrix of the right camera, anR_bAnd t_bAll external references of the right camera, in particular, R_bIs 3 × 3 rotationMatrix, t_bIs a translation vector of 3 x 1.

3. The dual-camera joint calibration method as set forth in claim 2, wherein: the position relationship between the left camera and the right camera is,

wherein, T_bIs a position matrix of the right camera, anT_sIs a position matrix of the left camera, anT is a position relation matrix between the left camera and the right camera, andr and t are external parameters between the left camera and the right camera.

4. The dual-camera joint calibration method as set forth in claim 3, wherein: specifically, the step S2 is,

s21, making the component z of the checkerboard corner point in the z direction in the coordinate system of the calibration board_wAnd the relation from the corner points of the checkerboard to the pixel coordinate system of the checkerboard image shot by the left camera is transformed in a homogeneous way to obtain a homogeneous transformation relation, wherein the homogeneous transformation relation is specifically,

wherein H is an intrinsic matrix, andh₁、h₂and h₃Three column vectors, r, of the eigen-matrix H₁And r₂Is an external reference R of the left camera_sλ is a parameter to be estimated;

s22, calculating the external parameter R of the left camera according to the eigen matrix H and the internal parameter of the left camera_sAnd t_sAnd R is_s＝[r₁ r₂r₃]，

Wherein A is₁Is an internal reference matrix of the left camera, an Is an estimate of a parameter λ to be estimated, and r₃＝r₁×r₂。

5. the dual-camera joint calibration method as set forth in claim 4, wherein: specifically, the step S3 is,

s31, substituting the position relation between the left camera and the right camera into the relation between the corner points of the checkerboard and the pixel coordinate system of the checkerboard image shot by the right camera, and based on the external reference R of the left camera_sAnd t_sThe relation from the checkerboard corner point to the pixel coordinate system of the checkerboard image shot by the right camera is transformed to obtain the transformation relation from the checkerboard corner point to the pixel coordinate system of the checkerboard image shot by the right camera, the transformation relation from the checkerboard corner point to the pixel coordinate system of the checkerboard image shot by the right camera is,

wherein the content of the first and second substances,x′_wis the transformation component in x-direction, y 'of the checkerboard corner points from the calibration plate coordinate system to the left camera coordinate system'_wIs the transformation component in the y-direction, z ', of the checkerboard corner points from the calibration-plate coordinate system to the left-camera coordinate system'_wThe angular point of the checkerboard is a transformation component in the z direction when the coordinate system of the calibration board is transformed to the coordinate system of the left camera;

s32, according to the transformation relation between the corner points of the checkerboard and the pixel coordinate system of the checkerboard image shot by the right camera, a transformation equation for transforming the corner points of the checkerboard from the left camera coordinate system to the right camera coordinate system is deduced, and the transformation equation is,

wherein, P is a checkerboard corner point, x ″)_wIs the transformation component, y ″, in the x direction when the checkerboard corner points are transformed from the left camera coordinate system to the right camera coordinate system_wIs the transformation component, z ″, in the y direction when the corner point of the checkerboard is transformed from the left camera coordinate system to the right camera coordinate system_wThe transformation component in the z direction when the checkerboard angular point is transformed from the left camera coordinate system to the right camera coordinate system;

s33, normalizing the transformation equation to obtain a track with the radius of r circle, wherein the track with the radius of r circle is,

r²＝x²+y²；

wherein x ═ x ″, in the formula_w/z″_w,y＝y″_w/z″_wX is the component of the trajectory of the r-circle in the x-direction of the right camera coordinate system, and y is the component of the trajectory of the r-circle in the y-direction of the right camera coordinate systemA component of (a);

s34, performing Brown distortion on the track with the radius of r circle in a Cartesian space to obtain a Brown distortion model,

x_c＝x(1+k₁r²+k₂r⁴+k₃r⁶)+2p₁xy+p₂(r²+2x²)，

y_c＝y(1+k₁r²+k₂r⁴+k₃r⁶)+2p₂xy+p₁(r²+2y²)；

wherein k is₁、k₂、k₃As radial distortion coefficient, p₁And p₂For tangential distortion coefficient, let distortion coefficient d ═ k₁ k₂ k₃ p₁p₂)^T；x_cComponent of the Brown distortion model in the x-direction of Cartesian space, y_cIs the component of the Brown distortion model in the y-direction of cartesian space;

s35, based on pinhole imaging principle, projecting the Brown distortion model to the pixel plane of the checkerboard image shot by the right camera to obtain the projection coordinates of the checkerboard angular points in the pixel plane of the checkerboard image shot by the right camera, and the projection coordinate expression of the checkerboard angular points in the pixel plane of the checkerboard image shot by the right camera is as follows,

wherein the content of the first and second substances,for the projection component of the checkerboard corner point in the u direction in the pixel coordinate system of the checkerboard image captured by the right camera,projection components of the checkerboard angular points in the v direction in a pixel coordinate system of a checkerboard image shot by a right camera are obtained;

s36, according to the internal reference and distortion coefficient d of the right camera and the external reference R and t between the left camera and the right camera, carrying out joint optimization on the pixel coordinates of the checkerboard image shot by the right camera and the projection coordinates in the pixel plane of the checkerboard angular point, enabling the error between the pixel coordinates of the checkerboard image shot by the right camera and the projection coordinates in the pixel plane to be minimum, and taking the internal reference of the right camera with the checkerboard angular point with the minimum error between the pixel coordinates and the projection coordinates in the pixel plane of the checkerboard image shot by the right camera as the optimal solution of the internal reference of the right camera.

6. The dual-camera joint calibration method as set forth in claim 5, wherein: in S36, the formula for jointly optimizing the pixel coordinates of the checkerboard corner points of the checkerboard image captured by the right camera and the projection coordinates in the pixel plane is,

wherein a ═ α_b β_b u_0b v_0b)^T。

7. A dual-camera combined calibration system is characterized in that: the system comprises a bracket, a left camera, a right camera, a calibration board with checkerboards and a combined calibration subsystem;

the left camera is a standard camera with known internal reference, the left camera and the right camera are respectively fixed on the left side and the right side of the bracket, and the calibration plate is fixedly placed on any point in the visual field range of the left camera and the right camera;

the combined calibration subsystem comprises a relation establishing module, a left camera external reference calibration module, a right camera internal reference calibration module and a right camera external reference calibration module;

the relation establishing module is used for controlling the left camera and the right camera to shoot the checkerboard images on the calibration board simultaneously, and establishing the relation between the checkerboard angular points to the pixel coordinate systems of the checkerboard images shot by the left camera and the right camera and the position relation between the left camera and the right camera according to the checkerboard images shot by the left camera and the right camera;

the left camera external parameter calibration module is used for performing homogeneous transformation on the relation between the checkerboard angular points and the pixel coordinate system of the checkerboard image shot by the left camera based on the z direction of the calibration board coordinate system, and solving the external parameters of the left camera according to the known internal parameters of the left camera and the relation between the checkerboard angular points subjected to the homogeneous transformation and the pixel coordinate system of the checkerboard image shot by the left camera;

the right camera internal reference calibration module is used for carrying out joint optimization on the internal reference of the right camera according to the relationship from the checkerboard angular points to the pixel coordinate system of the checkerboard image shot by the right camera, the external reference of the left camera and the position relationship between the left camera and the right camera to obtain the optimal solution of the internal reference of the right camera;

and the right camera external parameter calibration module is used for calculating the external parameters of the right camera according to the external parameters of the left camera based on the position relation between the left camera and the right camera.

8. A dual-camera combined calibration device is characterized in that: comprising a processor, a memory and a computer program stored in said memory, said computer program, when executed by said processor, implementing a dual camera joint calibration method as defined in any one of the preceding claims 1 to 6.

Background

In image measurement processes and machine vision applications, in order to determine the correlation between the three-dimensional geometric position of a certain point on the surface of an object in space and the corresponding point in the image, a geometric model of camera imaging must be established, and the parameters of the geometric model are the parameters of the camera. Under most conditions, the parameters must be obtained through experiments and calculation, and the process of solving the parameters is called camera calibration. In image measurement or machine vision application, calibration of camera parameters is a very critical link, and the accuracy of a calibration result and the stability of an algorithm directly influence the accuracy of a result generated by the operation of a camera. Therefore, the camera calibration is a precondition for subsequent work, and the improvement of the calibration precision is a key point of scientific research.

However, most of the existing camera calibration methods are complex, and the calibration precision is not high enough, so that the existing application requirements cannot be met.

Disclosure of Invention

The invention aims to provide a double-camera combined calibration method, a system and a device, which have simple calibration process and high calibration precision.

The technical scheme for solving the technical problems is as follows: a dual-camera combined calibration method comprises the following steps,

s1, controlling the left camera and the right camera to shoot checkerboard images on the calibration board simultaneously, and establishing the relationship between the checkerboard corner points to the pixel coordinate systems of the checkerboard images shot by the left camera and the right camera and the position relationship between the left camera and the right camera according to the checkerboard images shot by the left camera and the right camera; wherein, the left camera is a standard camera with known internal reference;

s2, based on the z direction of the calibration board coordinate system, the relation between the corner points of the checkerboard to the pixel coordinate system of the checkerboard image shot by the left camera is transformed in a homogeneous way, and the outer parameters of the left camera are solved according to the known inner parameters of the left camera and the relation between the corner points of the checkerboard after the homogeneous transformation to the pixel coordinate system of the checkerboard image shot by the left camera;

s3, performing joint optimization on the internal parameters of the right camera according to the relationship between the corner points of the checkerboard and the pixel coordinate system of the checkerboard image shot by the right camera, the external parameters of the left camera and the position relationship between the left camera and the right camera to obtain the optimal solution of the internal parameters of the right camera;

and S4, calculating the external parameter of the right camera according to the internal parameter of the left camera and the external parameter of the right camera based on the position relation between the left camera and the right camera.

Based on the double-camera combined calibration method, the invention also provides a double-camera combined calibration system.

A double-camera combined calibration system comprises a bracket, a left camera, a right camera, a calibration plate with checkerboards and a combined calibration subsystem;

the left camera is a standard camera with known internal reference, the left camera and the right camera are respectively fixed on the left side and the right side of the bracket, and the calibration plate is fixedly placed on any point in the visual field range of the left camera and the right camera;

the combined calibration subsystem comprises a relation establishing module, a left camera external reference calibration module, a right camera internal reference calibration module and a right camera external reference calibration module;

the relation establishing module is used for controlling the left camera and the right camera to shoot the checkerboard images on the calibration board simultaneously, and establishing the relation between the checkerboard angular points to the pixel coordinate systems of the checkerboard images shot by the left camera and the right camera and the position relation between the left camera and the right camera according to the checkerboard images shot by the left camera and the right camera;

the left camera external parameter calibration module is used for performing homogeneous transformation on the relation between the checkerboard angular points and the pixel coordinate system of the checkerboard image shot by the left camera based on the z direction of the calibration board coordinate system, and solving the external parameters of the left camera according to the known internal parameters of the left camera and the relation between the checkerboard angular points subjected to the homogeneous transformation and the pixel coordinate system of the checkerboard image shot by the left camera;

the right camera internal reference calibration module is used for carrying out joint optimization on the internal reference of the right camera according to the relationship from the checkerboard angular points to the pixel coordinate system of the checkerboard image shot by the right camera, the external reference of the left camera and the position relationship between the left camera and the right camera to obtain the optimal solution of the internal reference of the right camera;

and the right camera external parameter calibration module is used for calculating the external parameters of the right camera according to the external parameters of the left camera based on the position relation between the left camera and the right camera.

Based on the double-camera combined calibration method, the invention also provides a double-camera combined calibration device.

A dual-camera joint calibration apparatus comprising a processor, a memory and a computer program stored in the memory, wherein the computer program, when executed by the processor, implements the dual-camera joint calibration method as described above.

The invention has the beneficial effects that: the double-camera combined calibration method, the double-camera combined calibration system and the double-camera combined calibration device adopt the left camera with known internal reference to calibrate the right camera to be calibrated, the calibration method is simple, the internal reference of the right camera is jointly optimized in the calibration process, the optimal solution of the internal reference of the right camera is obtained, and the calibration precision is improved.

Drawings

FIG. 1 is a flow chart of a dual camera joint calibration method according to the present invention;

FIG. 2 is a schematic diagram of a calibration structure in a dual camera combined calibration system according to the present invention;

fig. 3 is a block diagram of a joint calibration subsystem in a dual-camera joint calibration system according to the present invention.

In the drawings, the components represented by the respective reference numerals are listed below:

1. left camera, 2, right camera, 3, calibration board.

Detailed Description

The principles and features of this invention are described below in conjunction with the following drawings, which are set forth by way of illustration only and are not intended to limit the scope of the invention.

As shown in fig. 1, a dual camera joint calibration method includes the following steps,

s1, controlling the left camera and the right camera to shoot checkerboard images on the calibration board simultaneously, and establishing the relationship between the checkerboard corner points to the pixel coordinate systems of the checkerboard images shot by the left camera and the right camera and the position relationship between the left camera and the right camera according to the checkerboard images shot by the left camera and the right camera; wherein, the left camera is a standard camera with known internal reference.

The relationship of the checkerboard corner points to the pixel coordinate system of the checkerboard image captured by the left camera is,

the relationship of the checkerboard corner points to the pixel coordinate system of the checkerboard image captured by the right camera is,

the position relationship between the left camera and the right camera is,

wherein:

P_wfor checkerboard corner points in the coordinate system of the calibration plateA coordinate vector of, and P_w＝[x_w,y_w,z_w]^T，x_wFor the component, y, of the checkerboard corner in the x-direction in the coordinate system of the calibration plate_wFor the component of the checkerboard corner in the y-direction in the coordinate system of the calibration plate, z_wThe components of the checkerboard angular points in the z direction in a coordinate system of a calibration board are taken as the checkerboard angular points;

is a coordinate vector of the checkerboard corner point in the pixel coordinate system of the checkerboard image shot by the left camera, andthe component of the checkerboard corner points in the z-direction in the left camera coordinate system,for the components of the checkerboard corner points in the u direction within the pixel coordinate system of the checkerboard image taken by the left camera,the components of the checkerboard angular points in the v direction in the pixel coordinate system of the checkerboard image shot by the left camera are taken;

M_sIis an internal reference correlation matrix of the left camera, anu_0s、v_0s、α_sAnd beta_sKnown internal references for the left camera;

M_sEis an external parameter matrix of the left camera, anR_sAnd t_sAll external references of the left camera, in particular, R_sIs a 3 × 3 rotation matrix, t_sA translation vector of 3 × 1;

is a coordinate vector of a checkerboard corner point in a pixel coordinate system of a checkerboard image shot by a right camera, and the component of the checkerboard corner points in the z-direction in the right camera coordinate system,for the components in the u direction within the pixel coordinate system of the checkerboard image taken by the right camera for the checkerboard corner points,the angular points of the checkerboards are components in the v direction in a pixel coordinate system of the checkerboard image shot by the right camera;

M_bIis an internal reference correlation matrix of the right camera, anu_0b、v_0b、α_bAnd beta_bAre all internal references of the right camera;

M_bEis an external reference matrix of the right camera, anR_bAnd t_bAll external references of the right camera, in particular, R_bIs a 3 × 3 rotation matrix, t_bIs a translation vector of 3 x 1.

T_bIs a position matrix of the right camera, anT_sIs a position matrix of the left camera, anT is the position between the left and right camerasA relationship matrix, andr and t are external parameters between the left camera and the right camera.

And S2, based on the z direction of the calibration board coordinate system, performing homogeneous transformation on the relationship between the corner points of the checkerboard to the pixel coordinate system of the checkerboard image shot by the left camera, and solving the external parameters of the left camera according to the known internal parameters of the left camera and the relationship between the corner points of the checkerboard after the homogeneous transformation to the pixel coordinate system of the checkerboard image shot by the left camera.

Specifically, the step S2 is,

s21, making the component z of the checkerboard corner point in the z direction in the coordinate system of the calibration board_wAnd the relation from the corner points of the checkerboard to the pixel coordinate system of the checkerboard image shot by the left camera is transformed into a homogeneous form (namely, the formula (1) is transformed into a homogeneous form) to obtain a homogeneous transformation relation, wherein the homogeneous transformation relation is specifically,

wherein H is an intrinsic matrix, and

h₁、h₂and h₃For three column vectors of the eigen matrix H, a 3 × 3 eigen matrix H can be obtained from equation (4), and then H ═ H₁ h₂ h₃]3 column vectors H of the eigenmatrix H can be determined₁、h₂、h₃；

r₁And r₂Is an external reference R of the left camera_sλ is a parameter to be estimated;

s22, calculating the external parameter R of the left camera according to the eigen matrix H and the internal parameter of the left camera_sAnd t_s；

Due to internal reference of left cameraKnown, so the internal reference matrix of the left cameraIt is known that, in the formula (5), r₁And r₂Is an external reference R of the left camera_sThe column vectors of the first two columns, i.e. R_s＝[r₁ r₂ r₃]From equation (5), the following can be obtained:

r₃＝r₁×r₂， (9)

R_s＝[r₁ r₂ r₃]； (11)

wherein the content of the first and second substances,is an estimated value of the parameter lambda to be estimated.

And S3, performing joint optimization on the internal parameters of the right camera according to the relationship between the corner points of the checkerboard and the pixel coordinate system of the checkerboard image shot by the right camera, the external parameters of the left camera and the position relationship between the left camera and the right camera to obtain the optimal solution of the internal parameters of the right camera.

Specifically, the step S3 is,

s31, substituting the position relation between the left camera and the right camera into a checkerboard from the corner point of the checkerboard to a checkerboard shot by the right cameraIn the relation of the pixel coordinate system of the image and based on the external reference R of the left camera_sAnd t_sTransforming the relation from the checkerboard corner points to the pixel coordinate system of the checkerboard image shot by the right camera to obtain the transformation relation from the checkerboard corner points to the pixel coordinate system of the checkerboard image shot by the right camera;

namely, formula (3) is substituted into formula (2), and the transformation relation from the checkerboard corner point to the pixel coordinate system of the checkerboard image shot by the right camera can be obtained based on formula (10) and formula (11);

the transformation relation from the corner points of the checkerboard to the pixel coordinate system of the checkerboard image shot by the right camera is,

wherein the content of the first and second substances,x′_wis the transformation component in x-direction, y 'of the checkerboard corner points from the calibration plate coordinate system to the left camera coordinate system'_wIs the transformation component in the y-direction, z ', of the checkerboard corner points from the calibration-plate coordinate system to the left-camera coordinate system'_wThe angular point of the checkerboard is a transformation component in the z direction when the coordinate system of the calibration board is transformed to the coordinate system of the left camera;

s32, according to the transformation relation between the corner points of the checkerboard and the pixel coordinate system of the checkerboard image shot by the right camera, a transformation equation for transforming the corner points of the checkerboard from the left camera coordinate system to the right camera coordinate system is deduced, and the transformation equation is,

wherein, P is a checkerboard corner point, x ″)_wIs a chessboard with angular points from leftTransformation component in x-direction, y ″, when the camera coordinate system is transformed into the right camera coordinate system_wIs the transformation component, z ″, in the y direction when the corner point of the checkerboard is transformed from the left camera coordinate system to the right camera coordinate system_wThe transformation component in the z direction when the checkerboard angular point is transformed from the left camera coordinate system to the right camera coordinate system;

s33, normalizing the transformation equation to obtain a track with the radius of r circle, wherein the track with the radius of r circle is,

r²＝x²+y²； (15)

wherein:

x＝x″_w/z″_w,y＝y″_w/z″_w， (16)

x is the component of the trajectory of the r-circle with radius in the x direction of the right camera coordinate system, and y is the component of the trajectory of the r-circle with radius in the y direction of the right camera coordinate system;

s34, performing Brown distortion on the track with the radius of r circle in a Cartesian space to obtain a Brown distortion model,

x_c＝x(1+k₁r²+k₂r⁴+k₃r⁶)+2p₁xy+p₂(r²+2x²)， (17)

y_c＝y(1+k₁r²+k₂r⁴+k₃r⁶)+2p₂xy+p₁(r²+2y²)； (18)

wherein k is₁、k₂、k₃As radial distortion coefficient, p₁And p₂For tangential distortion coefficient, let distortion coefficient d ═ k₁ k₂ k₃p₁ p₂)^T；x_cComponent of the Brown distortion model in the x-direction of Cartesian space, y_cIs the component of the Brown distortion model in the y-direction of cartesian space;

s35, based on pinhole imaging principle, projecting the Brown distortion model to the pixel plane of the checkerboard image shot by the right camera to obtain the projection coordinates of the checkerboard angular points in the pixel plane of the checkerboard image shot by the right camera, and the projection coordinate expression of the checkerboard angular points in the pixel plane of the checkerboard image shot by the right camera is as follows,

wherein the content of the first and second substances,for the projection component of the checkerboard corner point in the u direction in the pixel coordinate system of the checkerboard image captured by the right camera,projection components of the checkerboard angular points in the v direction in a pixel coordinate system of a checkerboard image shot by a right camera are obtained;

s36, carrying out joint optimization on pixel coordinates of a checkerboard corner point in a checkerboard image shot by a right camera and projection coordinates in a pixel plane according to internal parameters and distortion coefficients d of the right camera and external parameters R and t between the left camera and the right camera, enabling the error between the pixel coordinates of the checkerboard corner point in the checkerboard image shot by the right camera and the projection coordinates in the pixel plane to be minimum, and taking the internal parameters of the right camera with the minimum error between the pixel coordinates and the projection coordinates in the pixel plane of the checkerboard image shot by the right camera as the optimal solution of the internal parameters of the right camera;

the formula for jointly optimizing the pixel coordinates of the checkerboard corner points of the checkerboard image shot by the right camera and the projection coordinates in the pixel plane is as follows,

wherein, the reference matrix of the right cameraLet a ═ α_b β_b u_0b v_0b)^T。

And S4, calculating the external parameter of the right camera according to the external parameter of the left camera based on the position relation between the left camera and the right camera.

Since the external parameter of the left camera is known, the external parameter of the right camera can be directly obtained according to equation (3).

It should be noted that, in the present invention, the right camera may be a standard camera with known internal reference, and the left camera is a camera to be calibrated; the left and right in the present invention do not represent a limitation of the position.

Based on the double-camera combined calibration method, the invention also provides a double-camera combined calibration system.

As shown in fig. 2, a dual-camera combined calibration system includes a bracket, a left camera 1, a right camera 2, a calibration board 3 with checkerboards, and a combined calibration subsystem;

the left camera 1 is a standard camera with known internal reference, the left camera 1 and the right camera 2 are respectively fixed on the left side and the right side of the bracket, and the calibration plate 3 is fixedly placed on any point in the visual field range of the left camera 1 and the right camera 2;

as shown in fig. 3, the joint calibration subsystem includes a relationship establishing module, a left camera external reference calibration module, a right camera internal reference calibration module, and a right camera external reference calibration module;

the relation establishing module is used for controlling the left camera and the right camera to shoot the checkerboard images on the calibration board simultaneously, and establishing the relation between the checkerboard angular points to the pixel coordinate systems of the checkerboard images shot by the left camera and the right camera and the position relation between the left camera and the right camera according to the checkerboard images shot by the left camera and the right camera;

the left camera external parameter calibration module is used for performing homogeneous transformation on the relation between the checkerboard angular points and the pixel coordinate system of the checkerboard image shot by the left camera based on the z direction of the calibration board coordinate system, and solving the external parameters of the left camera according to the known internal parameters of the left camera and the relation between the checkerboard angular points subjected to the homogeneous transformation and the pixel coordinate system of the checkerboard image shot by the left camera;

the right camera internal reference calibration module is used for carrying out joint optimization on the internal reference of the right camera according to the relationship from the checkerboard angular points to the pixel coordinate system of the checkerboard image shot by the right camera, the external reference of the left camera and the position relationship between the left camera and the right camera to obtain the optimal solution of the internal reference of the right camera;

and the right camera external parameter calibration module is used for calculating the external parameters of the right camera according to the external parameters of the left camera based on the position relation between the left camera and the right camera.

In addition, the specific functions of the relationship establishing module, the left camera external reference calibration module, the right camera internal reference calibration module and the right camera external reference calibration module refer to the technical characteristics of the corresponding steps in the dual-camera combined calibration method.

Based on the double-camera combined calibration method, the invention also provides a double-camera combined calibration device.

A dual-camera joint calibration apparatus comprising a processor, a memory and a computer program stored in the memory, wherein the computer program, when executed by the processor, implements the dual-camera joint calibration method as described above.

The double-camera combined calibration method, the double-camera combined calibration system and the double-camera combined calibration device adopt the left camera with known internal reference to calibrate the right camera to be calibrated, the calibration method is simple, the internal reference of the right camera is jointly optimized in the calibration process, the optimal solution of the internal reference of the right camera is obtained, and the calibration precision is improved.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.