1. A camera calibration stand, comprising:

the straight bar comprises three straight bars with known lengths, wherein each of the three straight bars is provided with three round balls, two of the three round balls are arranged at two ends of the straight bar, the other round ball is arranged in the middle of the straight bar at a certain position, the three round balls divide the straight bar into two sections with known lengths, and the diameters of the three round balls are known;

the planes of the three straight bars are different planes.

2. The camera calibration stand of claim 1, wherein the three linear bars are the same or different lengths.

3. The camera calibration stand of claim 1, wherein the three spheres on each one have the same or different diameters.

4. The camera calibration stand of claim 1, wherein all of the balls on the three linear bars are the same or different in diameter.

5. A calibration method of a single image camera based on a calibration frame is characterized by comprising the following steps:

acquiring a calibration image, wherein the image is a photo obtained by photographing the camera calibration stand of any one of claims 1 to 4;

acquiring the coordinates of the center points of the sub-pixels of the spheres on the three straight bars;

constructing a first equation from world coordinates to two-dimensional pixel coordinates of the calibration image according to the collinear coordinates of the central points of the spheres of each linear bar and the distance from the object point to the camera plane;

solving three first equations respectively corresponding to the three linear rods as an equation set to obtain a first external parameter of the camera;

constructing a second equation from the world coordinate corresponding to each sphere to the two-dimensional pixel coordinate of the calibration image;

solving by taking a second equation corresponding to the nine spheres as an equation set to obtain internal parameters of the camera;

obtaining the second external parameter according to the corresponding relation between the internal parameter and the external parameter;

and according to the external parameter matrix, the scale factor and the deflection parameter of the image coordinate system are resolved by the unique orthogonal solution of the internal parameter matrix.

6. The method of claim 5, further comprising:

calibrating the camera using the internal parameters, the first external parameters, and the second external parameters.

7. Method according to claim 5 or 6, characterized in that said first external parameters comprise: a third component of the rotation matrix R and the translation matrix t; and/or, the second external parameter comprises: a first component and a second component of the translation matrix t; and/or, the intrinsic parameters include: the scale factor of the image coordinate system, the image principal point coordinates and the deflection parameter.

8. A single image camera calibration system based on a calibration stand is characterized by comprising: calibration stand and camera according to any one of claims 1 to 4, wherein the camera is used for taking a picture of the calibration stand.

9. A single image camera calibration system based on a calibration stand is characterized by comprising: software for photographing the calibration stand of any one of claims 1 to 4, and a camera for performing the method of any one of claims 5 to 7.

10. The system of claim 9, wherein the system is mounted on a vehicle or other device requiring calibration.

Background

In computer vision, three-dimensional reconstruction with two-dimensional images requires camera calibration to extract three-dimensional structural information from the two-dimensional images. The camera calibration aims at determining the internal and external parameters and the distortion coefficient of the camera, thereby laying a foundation for computer vision. The camera calibration method not only needs to solve the internal and external parameters and distortion coefficients of the camera very accurately, but also meets the actual requirements of various application scenes. The camera calibration is not simply applied to three-dimensional reconstruction, and has many applications in various scenes, such as robot navigation, industrial control, medical diagnosis and the like, and application scenes in the 5G era will be more vigorous.

The existing camera calibration method determines most internal and external parameters, needs iterative computation, multiple images, needs a nonlinear method to compute an orthogonal matrix or reduce the number of the internal parameters, and the like, so that the computation is not fast and accurate enough, and the design of a calibration plate is too complex.

Disclosure of Invention

The embodiment of the application provides a camera calibration frame, a calibration frame-based single-image camera calibration method and a calibration frame-based single-image camera calibration system, at least solves the problem that the existing calibration technology needs a nonlinear method to calculate an orthogonal matrix when determining internal and external parameters, and clarifies the cause and effect association of the internal and external parameters through orthogonal decomposition.

According to an aspect of the present application, there is provided a camera calibration stand including: the straight bar comprises three straight bars with known lengths, wherein each of the three straight bars is provided with three round balls, two of the three round balls are arranged at two ends of the straight bar, the other round ball is arranged in the middle of the straight bar at a certain position, the three round balls divide the straight bar into two sections with known lengths, and the diameters of the three round balls are known; the planes of the three straight bars are different planes.

Further, the three linear bars may be the same or different lengths.

Further, the diameters of the three spheres on each of the three spheres are the same or different.

Further, all the round balls on the three straight bars have the same or different diameters.

According to another aspect of the present application, there is provided a calibration stand-based single image camera calibration method, including: acquiring a calibration image, wherein the image is a photo obtained by photographing the camera calibration frame; acquiring the coordinates of the center points of the sub-pixels of the spheres on the three straight bars; constructing a first equation from world coordinates to two-dimensional pixel coordinates of the calibration image according to the collinear coordinates of the central points of the spheres of each linear bar and the distance from the object point to the camera plane; solving three first equations respectively corresponding to the three linear rods as an equation set to obtain a first external parameter of the camera; constructing a second equation from the world coordinate corresponding to each sphere to the two-dimensional pixel coordinate of the calibration image; solving by taking a second equation corresponding to the nine spheres as an equation set to obtain internal parameters of the camera; obtaining the second external parameter according to the corresponding relation between the internal parameter and the external parameter; and according to the external parameter matrix, the scale factor and the deflection parameter of the image coordinate system are resolved by the unique orthogonal solution of the internal parameter matrix.

Further, still include: calibrating the camera using the internal parameters, the first external parameters, and the second external parameters.

Further, the first external parameter includes: a third component of the rotation matrix R and the translation matrix t; and/or, the second external parameter comprises: a first component and a second component of the translation matrix t; and/or, the intrinsic parameters include: the scale factor of the image coordinate system, the image principal point coordinates and the deflection parameter.

According to still another aspect of the present application, there is provided a calibration stand-based single-image camera calibration system, including: the calibration frame and the camera are used for photographing the calibration frame.

According to yet another aspect of the present application, there is provided a calibration stand-based single image camera calibration system, comprising: the calibration frame comprises software and a camera, wherein the camera is used for photographing the calibration frame, and the software is used for executing the calibration frame-based single-image camera calibration method.

Further, the system is installed on a vehicle or other equipment that requires calibration.

In the embodiment of the application, three linear rods with known lengths are adopted, wherein each of the three linear rods is provided with three round balls, two round balls are arranged at two ends of the linear rod, the other round ball is arranged in a certain position in the middle of the linear rod, the three round balls divide the linear rod into two sections with known lengths, and the diameters of the three round balls are known; the planes of the three straight bars are different planes. The problem that the existing calibration technology needs to calculate the orthogonal matrix by a nonlinear method for determining the internal and external parameters is solved, and the cause and effect association of the internal and external parameters is clarified through orthogonal decomposition.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application. In the drawings:

FIG. 1 is a flow chart of a calibration rig based single image camera calibration method according to an embodiment of the present application;

FIG. 2 is a schematic illustration of a proportional relationship according to an embodiment of the present application;

FIG. 3 is a scene schematic diagram of a camera calibration method according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of a calibration stand according to an embodiment of the present application.

Detailed Description

It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.

It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.

In order to obtain a clear image, the existing camera basically has an automatic focusing function, so that calibration of multiple photos is not suitable any more, in order to avoid calibration errors caused by changes of internal parameters of the camera due to focusing when the camera takes different photos, calibration of one photo becomes a necessary trend, but the existing method for determining the internal parameters of one image needs to reduce the internal parameters for calibration, so that the method is different from the actual problem, in order to accurately calculate the internal and external parameters of the camera under the condition of six internal parameters, the embodiment adopts the calibration mode shown in fig. 4 to finish camera calibration by only taking one photo, and equipment is simple and easy to obtain. Because the method only uses a single image, the camera calibration is very quick and can be applied to a high-speed real-time calibration scene. The proposed method is more flexible than the conventional methods. Compared with self-calibration, the method has better robustness. As shown in fig. 4, this embodiment designs a 3D high-precision calibration frame (or also called calibration template), that is, it is composed of three non-coplanar straight bars, each of which is divided into two segments with known distance by three spherical points, and the calibration problem can be solved as long as the camera is placed at a proper position and only one picture is taken.

The design of the camera calibration template and the camera calibration method are suitable for single image calibration of a moving camera shooting object with a calibration frame, such as mobile camera calibration under automatic driving and specific working conditions.

In this embodiment, a camera calibration stand is provided, including: the straight bar comprises three straight bars with known lengths, wherein each of the three straight bars is provided with three round balls, two of the three round balls are arranged at two ends of the straight bar, the other round ball is arranged in the middle of the straight bar at a certain position, the three round balls divide the straight bar into two sections with known lengths, and the diameters of the three round balls are known; the planes of the three straight bars are different planes.

As an alternative embodiment, the three linear bars are of the same or different lengths. The three spheres on each of the three spheres may be the same or different in diameter. For better calculation, in a preferred embodiment, all the spheres on the three linear bars have the same diameter.

According to the embodiment, only a single image is needed, iterative computation is not needed, an orthogonal matrix is not needed to be computed by a nonlinear method, real-time internal and external parameters can be rapidly computed with high precision by combining sub-pixels to obtain calibration characteristic points, and the method is suitable for processing various static or moving scenes with multiple cameras and zooming.

In this embodiment, a calibration rack-based single image camera calibration method is provided, and fig. 1 is a flowchart of a calibration rack-based single image camera calibration method according to an embodiment of the present application, and as shown in fig. 1, a flow of the method includes the following steps:

step S102, obtaining a calibration image, wherein the image is a photo obtained by photographing the camera calibration frame;

step S104, obtaining the sub-pixel center point coordinates of the spheres on the three straight bars;

step S106, constructing a first equation from world coordinates to two-dimensional pixel coordinates of the calibration image according to the collinear center line point coordinates of the sphere of each linear bar and the distance from the object point to the camera plane;

step S108, solving three first equations respectively corresponding to the three linear bars as an equation set to obtain a first external parameter of the camera;

step S110, constructing a second equation from the world coordinate corresponding to each sphere to the two-dimensional pixel coordinate of the calibration image;

step S112, solving a second equation corresponding to the nine round spheres as an equation set to obtain internal parameters of the camera;

step S114, obtaining the second external parameter according to the corresponding relation between the internal parameter and the external parameter;

and step S116, according to the external parameter matrix, performing unique orthogonal decomposition on the internal parameter matrix to obtain a scale factor and a deflection parameter of the image coordinate system.

Through the steps, nine points can exist on one graph, and corresponding internal and external parameters can be solved by using equations of the nine points.

As an optional embodiment, the camera is calibrated using the intrinsic parameter, the first extrinsic parameter, and the second extrinsic parameter.

Preferably, the first external parameter includes: a third component of the rotation matrix R and the translation moment t; and/or, the second external parameter comprises: a first component and a second component of the translation matrix t; and/or, the intrinsic parameters include: the scale factor of the image coordinate system, the image principal point coordinates and the deflection parameter. For example, the camera calibration intrinsic parameters may be five: α, β are scale factors of the image (u, v) coordinate system, (u₀,v₀) Is the image principal point coordinates, and gamma is the skew parameter.

Through the embodiment, the design of the camera space calibration frame and the invention of a new calibration method matched with the camera space calibration frame are provided. In the embodiment, the calibration of the internal and external parameters is realized by using the newly designed calibration frame and the matched calibration method. The embodiment firstly designs a 3D high-precision calibration frame, namely the calibration frame consists of three non-coplanar straight-line bars, each straight-line bar is divided into two sections with known distances by three spherical points, so that the calibration problem can be solved as long as a camera is placed at a proper position and only one photo is taken. The calibration is the camera calibration of a single image and is simple in calculation, so that the camera calibration is very quick and can be applied to a high-speed real-time calibration scene. The calibration method has more flexibility, robustness and practicability.

In this embodiment, a calibration frame-based single-image camera calibration system is provided, including: the calibration frame and the camera are used for photographing the calibration frame.

In this embodiment, a calibration frame-based single-image camera calibration system is provided, including: the calibration frame comprises software and a camera, wherein the camera is used for photographing the calibration frame, and the software is used for executing the calibration frame-based single-image camera calibration method. Fig. 3 is a schematic view of a scene of a camera calibration method according to an embodiment of the present application, and as shown in fig. 3, the system may be installed on a vehicle that needs to be calibrated, as an alternative embodiment, or may be installed on other devices that need to be calibrated.

By the embodiment, the problem that the existing calibration technology needs to calculate the orthogonal matrix by a nonlinear method for determining the internal and external parameters is solved, and the causal association of the internal and external parameters is clarified by orthogonal decomposition.

The following description of implementations of the present application refers to the accompanying drawings and the detailed description.

First, a theoretical model of camera calibration is explained. Assuming that the influence of distortion in imaging is preprocessed in advance, the imaging principle of a common camera is set as a pinhole imaging model:

adding a rotation to the camera plane coordinate systemWhen theta takes an arbitrary value, it means that the coordinate system on the camera plane can be arbitrarily selected. The rectangular coordinate system on the camera plane is selected independently of the imaging, taking into account

The extrinsic parameter matrix at this time is known asThe corresponding intrinsic parameter matrix is thusThat is, the internal and external parameter matrixes are affected when the planar coordinate systems of the cameras are selected differently. Because of the fact that

When the rectangular coordinate system of the camera plane is selected at will, the internal parameter matrix is not set asThus a three-dimensional pointProjecting onto two-dimensional image pointsThe corresponding formula is as follows:

s is a scale factor, s ═ l₃x+m₃y+n₃z+z′₀Representing the distance of the object point to the camera plane, (R, t) is the extrinsic parameter, R is a 3 × 3 rotation matrix, t is a translation matrix, K is the intrinsic parameter matrix, α, β are scale factors of the image (u, v) coordinate system, (u, v)₀,v₀) Is the image principal point coordinates, and ω, γ are the skew parameters.

Let the object space have collinear three points A, B, C, the straight line of which is marked as L₁The spatial coordinates corresponding to the three points are also denoted by the same letters. Then can be provided withLet lambda_A＝1-λ，λ_BWhen x is equal to C, x is equal to C_AA+λ_BB。

FIG. 2 is a schematic diagram of a proportional relationship according to an embodiment of the present application, as shown in FIG. 2, using the colinear of the three points in FIG. 2 and the distance from the object point to the camera plane, in combination with the similarity ratio of the similar triangles, there is a relationship:

the proportional relation of the factor point coordinate is C ═ lambda_AA+λ_BB is independent of the coordinate system, so C is still equal to lambda under the pinhole coordinate system_AA+λ_BB is that

The imaging relation at this time isHere, theIs the coordinate of the point P under the pinhole coordinate system,to correspond to the image coordinates. Therefore, it is not only easy to useMultiplying both sides by K to obtainSubstituting the formula (1.4) into the formula to obtain:

thus is provided withCombined with (1.3) formula (I):

if there are three of object spaces similar to L₁Line L of non-coplanar_iI is 1,2,3, as shown in fig. 3. Respectively recording three points on the corresponding lines as A_i,B_i,C_iAnd i is 1,2,3, and combining the formula (1.7), the equation system can be obtained:

simultaneous solution to obtain vectorThereby utilizing its die length ofCan obtain z'₀Thus, also can obtain (l)₃,m₃,n₃). Because of the vector l₁ m₁ n₁]，[l₂ m₂ n₂]And [ l₃ m₃ n₃]Two by two are perpendicular to each other, the inner product is 0, then [ l₁ m₁n₁]And [ l₂ m₂ n₂]Can be based on₃ m₃ n₃]Can be constructed to obtain an extrinsic parameter rotation matrix R and a third component z 'of t'₀。

The internal parameter matrix and the external parameter t are discussed as follows:

the imaging formula is:

is provided withBecause of the fact thatIs provided with

Thus is provided with

Because of nine points, more than nine equations can be obtained, and the ninth equation is subtracted from the first eight equations to obtain an equation set formed by eight equations:

order to

Then there are:

the intrinsic parameters of the camera can be obtained through the generalized inverse matrix, and the first two components [ x 'of the extrinsic parameter t can be obtained by substituting the intrinsic parameter result into the expression (1.10)'₀ y′₀]。

The combination formula:

can be obtained by using coordinates of two points

And then decomposing the submatrix in the internal parameters as follows:

the camera imaging model can be written as:

in the calibration method, the internal and external parameters of the camera are solved by a linear formula, so that the calculation accuracy and the calculation time are ensured, and the camera is calibrated very quickly.

The following describes a design of a camera calibration board and an implementation manner of a camera calibration method according to the present application with reference to the accompanying drawings and specific embodiments.

In this embodiment, a design of a camera calibration board and a camera calibration method are provided, fig. 3 is a scene schematic diagram of the camera calibration method according to the embodiment of the present application, and as shown in fig. 3, the method is implemented by the following steps:

the method comprises the following steps: firstly, finding three linear rods with known lengths, stringing three spheres (the radii can be the same or different) with known radii on each linear rod, stringing two spheres at two ends of each linear rod, stringing the spheres at a certain position in the middle of each linear rod, and dividing each linear rod into two sections with known distances;

step two: placing the three linear bars at proper positions in front of an automatic driving vehicle to enable the three linear bars to be respectively positioned on different planes, placing the three linear bars in a visual field range of a camera to be calibrated and measuring world coordinates of the central point of the sphere;

step three: acquiring an image of a calibration plate through a camera, extracting RGB (red, green and blue) data of the image by using corresponding software, and acquiring sub-pixel center point coordinates of spheres on three linear rods in the image;

step four: and calculating internal and external parameters of the camera according to the camera calibration method to complete camera calibration.

In this embodiment, an electronic device is provided, comprising a memory in which a computer program is stored and a processor configured to run the computer program to perform the method in the above embodiments.

These computer programs may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks, and corresponding steps may be implemented by different modules.

The programs described above may be run on a processor or may also be stored in memory (or referred to as computer-readable media), which includes both non-transitory and non-transitory, removable and non-removable media, that implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.