1. A camera calibration plate, comprising: a main calibration plate and an auxiliary calibration plate, wherein,

the pattern drawn on the main calibration plate comprises: at least one concentric circular pattern and at least one pair of rectangular patterns;

the pattern drawn on the auxiliary calibration plate comprises: at least one concentric circular pattern on each secondary calibration plate; wherein, supplementary calibration plate includes: the auxiliary calibration plate is characterized by comprising two auxiliary calibration plates which are arranged at a preset distance at intervals, wherein at least one concentric circular ring pattern is drawn on each auxiliary calibration plate, and/or the auxiliary calibration plate is folded into an auxiliary calibration plate with a certain angle, and at least one concentric circular ring pattern is respectively drawn on two folded surfaces of each auxiliary calibration plate.

2. The camera calibration plate of claim 1,

the patterns on the main calibration plate and/or the auxiliary calibration plate are distributed according to a plurality of rows, wherein the patterns on each row are circular, or the patterns on each row are rectangular.

3. The camera calibration plate of claim 2,

a first straight line is drawn between the main calibration plate and/or the auxiliary calibration plate part or all the rows, wherein the first straight line is used for spacing the adjacent rows; and/or the presence of a gas in the gas,

the patterns on the main calibration plate and/or the auxiliary calibration plate are aligned according to columns; and/or the presence of a gas in the gas,

and second straight lines are drawn between partial or all columns on the main calibration plate and/or the auxiliary calibration plate, wherein the second straight lines are used for separating adjacent columns, and the length of each second straight line can separate all elements or partial elements of the adjacent columns.

4. The camera calibration plate of claim 1,

the inner circle pattern of the concentric circles is divided into N parts with equal areas by straight lines starting from the circle center; and/or the rectangle is divided into N parts by a straight line from the center of the rectangle, wherein N is larger than or equal to 3, half of the N parts are drawn into a first color, the other half of the N parts are drawn into a second color, the first color is different from the second color, and the colors of any two adjacent parts in the N parts are different.

5. The camera calibration plate of claim 4, wherein N is 4, the rectangle is divided into four parts by two diagonal lines, or two middle lines respectively connecting middle points of opposite sides of the rectangle are divided into four parts with equal areas, wherein the rectangle comprises at least one of the following types: the type I is divided into four parts by diagonal lines, and the colors are distributed alternately; type two, divided into four parts by diagonal line, the color is opposite to type one; the type III is divided into four parts by the two middle lines, and the colors are distributed at intervals; type four, divided into four parts by the two middle lines, the color is opposite to type three.

6. A camera calibration plate according to any one of claims 1 to 5, wherein each pair of rectangular patterns is different in ordered combination as viewed from both sides of the main calibration plate towards the middle, so as to identify a pair of patterns that matches the world coordinates corresponding to its centre point.

7. A use method of a camera calibration plate is characterized in that a main calibration plate and an auxiliary calibration plate are used, the auxiliary calibration plate is folded and placed in a three-dimensional mode with the main calibration plate, and each plane in a three-dimensional space formed by two planes formed by folding the auxiliary calibration plate and one plane of the main calibration plate is provided with at least one concentric circle pattern; or, one main calibration plate and two auxiliary calibration plates are used, and each plane in a three-dimensional space formed by three calibration plates in a three-dimensional way is provided with at least one concentric circle pattern; wherein the main calibration plate is the main calibration plate of any one of claims 1 to 6, and the auxiliary calibration plate is the auxiliary calibration plate of any one of claims 1 to 6.

8. A camera-calibrated feature point extraction method, for extracting feature points from an image obtained by photographing a calibration board placed by the method of claim 7, the method comprising:

constructing an LxL grid by taking any point selected from the image as a center, and obtaining a binary matrix V according to a preset judgment standard;

searching a connected set of the binary matrix V, and arranging the connected set according to the number of elements in the filled connected set from large to small;

extracting sub-pixel boundaries marked in the image;

judging whether the connected set is a circular pattern or not, and then determining whether the pair of connected sets form a concentric pattern or not by judging whether the connected set contains another connected set of the circular pattern or not;

judging whether the connected set is a rectangular pattern;

after determining that the connected set is the outer circle of the concentric circle, the inner circle of the concentric circle and the rectangle, locating sub-pixel center coordinates of the rectangle and the concentric circle;

matching the rectangle according to the mark position on the calibration plate;

and classifying the concentric circles at least according to the matched rectangles, and fitting the classified sub-pixel boundaries of the concentric circles to obtain an elliptic boundary equation.

9. The method of claim 8, wherein matching the rectangle according to the marker location on the calibration plate comprises:

judging four types of the rectangle, and matching the rectangle according to the type of the rectangle and the mark position on the calibration board, wherein the rectangle is divided into four parts by two diagonal lines, or two middle lines respectively connected with the middle points of opposite sides of the rectangle are divided into four parts with equal areas, wherein the rectangle at least comprises four types: the type I is divided into four parts by diagonal lines, and the colors are distributed alternately; type two, divided into four parts by diagonal line, the color is opposite to type one; the type III is divided into four parts by the two middle lines, and the colors are distributed at intervals; type four, divided into four parts by the two middle lines, the color is opposite to type three.

10. The method of claim 9, wherein classifying the concentric circles according to at least the matched rectangle comprises:

classifying the concentric circles according to straight lines and the matched rectangles, wherein the straight lines comprise a first straight line and/or a second straight line, the patterns on the main calibration plate and/or the auxiliary calibration plate are distributed according to a plurality of rows, the patterns on each row are circular, or the patterns on each row are rectangular; a first straight line is drawn between the main calibration plate and/or the auxiliary calibration plate part or all the rows, and the first straight line is used for spacing the adjacent rows; and second straight lines are drawn between partial or all columns on the main calibration plate and/or the auxiliary calibration plate, wherein the second straight lines are used for separating adjacent columns, and the length of each second straight line can separate all elements or partial elements of the adjacent columns.

Background

In computer vision, three-dimensional reconstruction with two-dimensional images requires camera calibration to extract three-dimensional structural information from the two-dimensional images. The camera calibration aims at determining the internal and external parameters and the distortion coefficient of the camera, thereby laying a foundation for computer vision.

The calibration methods can be roughly classified into a camera calibration based on a three-dimensional calibration object, a camera calibration based on a two-dimensional calibration object, a camera calibration based on a one-dimensional calibration object, and a camera self-calibration method according to the dimension of the calibration object used. The camera self-calibration method has low accuracy and limited application range, the calibration methods based on two-dimension and one-dimension require a camera to take a plurality of photos, the internal parameters calculated by different photos are inconsistent due to the focusing effect of the camera in the moving process of a camera or a calibration object, and the deviation of the internal parameters of the camera can compensate the error of the external parameters of the camera, so that the calibration error is caused, and the camera self-calibration method cannot be used in the scene needing to calculate the attitude information of the camera. Most of the existing calibration methods adopt camera calibration versions which are shot completely and based on checkerboards, when shooting is incomplete, three-dimensional space coordinates cannot be extracted through two-dimensional image information, and moreover, the accuracy of extracting feature points through many existing algorithms is high, but the image coordinates and the space coordinates of the feature points are not high in matching degree and are mixed with each other, so that camera calibration is affected.

Disclosure of Invention

The embodiment of the application provides a camera calibration plate, a using method thereof and a feature point extraction method for camera calibration, and solves the problems that in the prior art, the matching degree of image coordinates and space coordinates of feature points is not high and the feature points are mixed up due to incomplete feature point extraction of the calibration plate in the camera calibration method, so that the camera calibration precision is improved.

According to an aspect of the present application, there is provided a camera calibration plate including: main calibration board and supplementary calibration board, wherein, the pattern of drawing on the main calibration board includes: at least one concentric circular pattern and at least one pair of rectangular patterns; the pattern drawn on the auxiliary calibration plate comprises: at least one concentric circular pattern on each secondary calibration plate; wherein, supplementary calibration plate includes: the auxiliary calibration plate is a folded auxiliary calibration plate with a certain angle, and at least one concentric circular ring pattern is respectively arranged on two folded surfaces of the auxiliary calibration plate.

Further, the patterns on the main calibration plate and/or the auxiliary calibration plate are distributed in multiple rows, wherein the patterns in each row are circular, or the patterns in each row are rectangular.

Furthermore, a first straight line is drawn between the main calibration plate and/or the auxiliary calibration plate part or all rows, wherein the first straight line is used for spacing adjacent rows; and/or, the patterns on the main calibration plate and/or the auxiliary calibration plate are aligned according to columns; and/or a second straight line is drawn between partial or all columns on the main calibration plate and/or the auxiliary calibration plate, wherein the second straight line is used for separating adjacent columns, and the length of the second straight line can separate all elements or partial elements of the adjacent columns.

Further, the inner circle pattern of the concentric circles is divided into N parts with equal areas by straight lines starting from the circle center; and/or the rectangle is divided into N parts by a straight line from the center of the rectangle, wherein N is more than or equal to 3, half of the N parts are drawn into a first color, the other half of the N parts are drawn into a second color, the first color is different from the second color, and the colors of any two adjacent parts in the N parts are different.

Further, N is 4, the rectangle is divided into four parts by two diagonal lines, or two middle lines respectively connecting middle points of opposite sides of the rectangle are divided into four parts with equal areas, where the rectangle includes at least one of the following types: the type I is divided into four parts by diagonal lines, and the colors are distributed alternately; type two, divided into four parts by diagonal line, the color is opposite to type one; the type III is divided into four parts by the two middle lines, and the colors are distributed at intervals; type four, divided into four parts by the two middle lines, the color is opposite to type three.

Further, each pair of rectangular patterns is ordered differently from side to side of the main calibration plate so that identification of a pair of patterns matches the world coordinates corresponding to its center point.

According to another aspect of the present application, there is provided a method for using a camera calibration board, where a main calibration board and an auxiliary calibration board are used, the auxiliary calibration board is folded and placed in a three-dimensional manner with the main calibration board, and each plane in a three-dimensional space formed by two planes formed by folding the auxiliary calibration board and one plane of the main calibration board has at least one concentric circle pattern; or, one main calibration plate and two auxiliary calibration plates are used, and each plane in a three-dimensional space formed by three calibration plates in a three-dimensional way is provided with at least one concentric circle pattern; the main calibration plate is the main calibration plate, and the auxiliary calibration plate is the auxiliary calibration plate.

According to another aspect of the present application, there is provided a camera calibration feature point extraction method, configured to extract feature points from an image obtained by photographing a calibration board placed by using the above method, the method including: constructing an LxL grid by taking any point selected from the image as a center, and obtaining a binary matrix V according to a preset judgment standard; searching a connected set of the binary matrix V, and arranging the connected set according to the number of elements in the filled connected set from large to small; extracting sub-pixel boundaries marked in the image; judging whether the connected set is a circular pattern or not, and then determining whether the pair of connected sets form a concentric pattern or not by judging whether the inside of the connected set contains another connected set of the circular pattern or not; judging whether the connected set is a rectangular pattern; after determining that the connected set is the outer circle of the concentric circle, the inner circle of the concentric circle and the rectangle, locating sub-pixel center coordinates of the rectangle and the concentric circle; matching the rectangle according to the mark position on the calibration plate; and classifying the concentric circles at least according to the matched rectangles, and fitting the classified sub-pixel boundaries of the concentric circles to obtain an elliptic boundary equation.

Further, the matching of the rectangle according to the mark position on the calibration board comprises: judging four types of the rectangle, and matching the rectangle according to the type of the rectangle and the mark position on the calibration board, wherein the rectangle is divided into four parts by two diagonal lines, or two middle lines respectively connected with the middle points of opposite sides of the rectangle are divided into four parts with equal areas, wherein the rectangle at least comprises four types: the type I is divided into four parts by diagonal lines, and the colors are distributed alternately; type two, divided into four parts by diagonal line, the color is opposite to type one; the type III is divided into four parts by the two middle lines, and the colors are distributed at intervals; type four, divided into four parts by the two middle lines, the color is opposite to type three.

Further, classifying the concentric circles according to at least the matched rectangles comprises: classifying the concentric circles according to straight lines and the matched rectangles, wherein the straight lines comprise first straight lines and/or second straight lines, the patterns on the main calibration plate and/or the auxiliary calibration plate are distributed according to a plurality of lines, the patterns on each line are circular, or the patterns on each line are rectangular; a first straight line is drawn between the main calibration plate and/or the auxiliary calibration plate part or all the rows, and the first straight line is used for spacing the adjacent rows; and second straight lines are drawn between partial or all columns on the main calibration plate and/or the auxiliary calibration plate, wherein the second straight lines are used for spacing adjacent columns, and the length of the second straight lines can be used for spacing all elements or partial elements of the adjacent columns.

In the embodiment of the present application, the camera calibration board includes: main calibration board and supplementary calibration board, wherein, the pattern of drawing on the main calibration board includes: at least one concentric ring pattern and at least one pair of rectangular patterns; at least one concentric circular pattern on each secondary calibration plate; the two auxiliary calibration plates are arranged at positions spaced apart by a predetermined distance. For convenience of operation, the two auxiliary plates can be simplified into one auxiliary calibration plate folded at a certain angle. The method and the device solve the problems that the matching degree of the image coordinates and the space coordinates of the feature points is not high and the feature points are mixed up due to incomplete extraction of the feature points of the calibration plate in the camera calibration method in the prior art, so that the camera calibration precision is improved.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application. In the drawings:

FIG. 1(a) (b) is a schematic illustration of two calibration plates according to an embodiment of the present application;

FIG. 2 is a flow chart of a method for extracting feature points for camera calibration according to an embodiment of the present application;

3(a) (b) (c) (d) are schematic diagrams of three different types of rectangular markers and four-corner division according to an embodiment of the present application;

FIG. 4(a) (b) is a schematic illustration of a camera mounting and calibration plate position according to an embodiment of the present application;

FIG. 5(a) (b) is a real object diagram of sub-pixel feature points and boundaries extracted by the present algorithm according to an embodiment of the present application; and the number of the first and second groups,

fig. 6 is a specific flowchart of camera calibration plate design and feature point extraction according to an embodiment of the present application.

Detailed Description

It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.

It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than here.

In the related art, the focal length of the camera may change continuously and unknowingly, and there is no good method for determining five intrinsic parameters of the camera based on a single common camera, a single picture and a three-dimensional calibration template when only pixel information exists. The camera calibration is carried out by solving the internal and external parameters of the camera according to the world coordinates and the image coordinates of the known characteristic points, so that the accuracy of the camera calibration is directly determined by the accuracy of extracting the characteristic points.

In this embodiment, a camera calibration board is provided, which includes: main calibration board and supplementary calibration board, wherein, the pattern of drawing on the main calibration board includes: at least one concentric circular pattern and at least one pair of rectangular patterns; at least one concentric circular pattern on each secondary calibration plate; the two calibration plates are disposed at positions spaced apart by a predetermined distance. For ease of operation, the two auxiliary plates may be reduced to one auxiliary calibration plate folded at an angle, as shown in FIG. 4.

Through above-mentioned calibration plate, the concentric ring pattern can be used for demarcating, has used at least a pair of rectangle pattern simultaneously, is favorable to confirming the plane that the concentric ring pattern belongs to. In the case of using the camera calibration plate, calibration can be completed by taking only one picture because there are at least three non-coplanar concentric circle patterns in this picture.

When the camera calibration plate is used, a main calibration plate and an auxiliary calibration plate can be used as three-dimensional structures, and a plurality of methods for manufacturing the three-dimensional structures are available, for example, the number of the auxiliary calibration plates is 1, and the number of the concentric circular patterns on the auxiliary calibration plate is at least two, wherein when the auxiliary calibration plate is folded, at least one concentric circular pattern is respectively arranged on two folded surfaces of the auxiliary calibration plate; for another example, the number of the auxiliary calibration plates is at least 2, and each auxiliary calibration plate is drawn with at least one pattern of the concentric rings. In practical use, the two modes can be flexibly selected, or more calibration plates can be used for calibration.

As an alternative embodiment, two sets of calibration plates may be used, one set being 1 secondary calibration plate and the other set being 2 secondary calibration plates, as is the case with the two examples above. Setting the two groups of calibration plates at the position spaced by the preset distance, taking a picture by a camera, taking the two groups of calibration plates, calculating the internal parameters of the camera by using the geometric measurement of the three non-coplanar concentric circles and the sub-pixel coordinates of the corresponding characteristic points, and calculating the external parameters by using the characteristic points of the main calibration plate and the matched world coordinates.

In order to make the patterns of the calibration plates easier to identify and calculate, the patterns on the main calibration plate and/or the auxiliary calibration plate are distributed in a plurality of rows, wherein the patterns of each row are circular, or the patterns of each row are rectangular. Different rows of the main calibration plates are separated by straight lines, and one of the auxiliary calibration plates is separated by straight lines of rows and columns to form concentric circles. The two types of lines are referred to as first lines and/or second lines, which are described separately below:

a first straight line is drawn between the main calibration plate and/or the auxiliary calibration plate part or all the rows, wherein the first straight line is used for spacing the adjacent rows; the same type of patterns on the main calibration plate and/or the auxiliary calibration plate are aligned according to columns;

and second straight lines are drawn between partial or all columns on the main calibration plate and/or the auxiliary calibration plate, wherein the second straight lines are used for separating adjacent columns, and the length of each second straight line can separate all elements or partial elements of the adjacent columns.

As another alternative, the inner part of the inner ring pattern of concentric circles and/or the matrix may also be treated: the inner circle pattern of the concentric circles is divided into N parts with equal areas by straight lines starting from the circle center; and/or the rectangle is divided into N parts (for example, 4) by a straight line from the center of the rectangle, wherein N is greater than or equal to 3, half of the N parts are drawn as a first color, the other half of the N parts are drawn as a second color, the first color is different from the second color, and the colors of any two adjacent parts in the N parts are different.

When the rectangle is divided into four parts, there can be two cases: the rectangle is divided into four parts by two diagonal lines, or two middle lines respectively connecting the middle points of the opposite sides of the rectangle are divided into four parts with equal areas, wherein the rectangle comprises at least one of four types: the type I is divided into four parts by diagonal lines, and the colors are distributed alternately; type two, divided into four parts by diagonal line, the color is opposite to type one; the type III is divided into four parts by the two middle lines, and the colors are distributed at intervals; type four, divided into four parts by the two middle lines, the color is opposite to type three.

In the rectangles distributed on the main calibration board and the auxiliary calibration board according to the rows, each row comprises at least two types of rectangle patterns, and each type of rectangle pattern is a pair.

The following description will be made with reference to the accompanying drawings and a preferred embodiment of a calibration board design for camera calibration.

The embodiment designs a camera calibration template based on concentric rings, and calibration by using the calibration template does not need complete shooting of calibration patterns on a calibration plate, and camera calibration can be carried out only by using a concentric ring on three different planes in an image shot by a single camera and a pair of rectangles on a bottom calibration plate. The calibration template can be directly printed by a laser printer, equipment is simple and easy to obtain, one calibration plate is controlled to be on the flat ground, the other calibration plate is folded to form a certain angle with the calibration plate and is randomly placed along with a camera, and the calibration plate is located in a three-dimensional space. In the algorithm for extracting the centers of the concentric circles and the rectangular center, the accuracy of the centers of the feature points and the matching degree between marks are high, so that the calibration result is good.

The accuracy of camera calibration depends largely on the detection of feature points, and compared with concentric circle patterns, calibration patterns based on quadrangles and triangles are more susceptible to image noise, distortion and resolution, and circular patterns have the advantage of detecting the projection center, and the projected images of concentric circles are a curve pair, and the distortion parameter is corrected by the characteristic that the feature value of the matrix before and after projection is constant, so this embodiment provides a calibration plate pattern using concentric circles as a calibration mode, wherein the calibration plate is composed of a main calibration plate and an auxiliary calibration plate, fig. 1(a) (b) are schematic diagrams of two calibration plates according to the embodiment of the present application, as shown in fig. 1, (a) in fig. 1 is the main calibration plate, (b) in fig. 1 is the auxiliary calibration plate, and (a) in fig. 1 is the calibration plate pattern composed of concentric circles, The rectangle and the straight line are formed, the inner circle pattern of each rectangle and each concentric ring is divided into four parts from a central point, the pattern center is more accurately extracted by sub-pixel angular points or saddle points in a mode of black and white blocks being alternated, the rectangle is divided into three types of patterns, the patterns are easy to distinguish and match, and the calibration can be carried out by realizing local identification. Since the embodiment recognizes the concentric circles to determine the coordinates of the two-dimensional image thereof and accurately knows the world coordinates of the concentric circles, the straight line is used as a boundary line, and different rectangles and concentric circles are matched with each other to determine which concentric circle is recognized specifically for calibration. The pattern can be printed on a laser printer and a good pattern for calibration should first provide sufficient geometric constraints and then be easily detected and identified. The present embodiment uses concentric circular planar patterns to perform the work of the present embodiment, having more features than other patterns. In addition, the technology only needs a camera to shoot a calibration plate on three different planes, and the calibration can be completed only by identifying the patterns on part of the calibration plate.

In this embodiment, a method for using a camera calibration board is provided, where a main calibration board and an auxiliary calibration board are used, the auxiliary calibration board is folded and placed in a three-dimensional manner with the main calibration board, and each plane in a three-dimensional space formed by two planes formed by folding the auxiliary calibration board and one plane of the main calibration board has at least one concentric circle pattern; or, one main calibration plate and two auxiliary calibration plates are used, and each plane in a three-dimensional space formed by three calibration plates in a three-dimensional way is provided with at least one concentric circle pattern; the main calibration plate is the main calibration plate of the camera calibration plate, and the auxiliary calibration plate is the auxiliary calibration plate of the camera calibration plate.

In this embodiment, a method for extracting feature points calibrated by a camera is provided, which is used to extract feature points from an image obtained by photographing a calibration board placed by using the method for using a camera calibration board described above, and fig. 2 is a flowchart of a method for extracting feature points calibrated by a camera according to an embodiment of the present application, and as shown in fig. 2, the method includes the following steps:

step S202, constructing an LxL grid by taking any point selected from the image as a center, and obtaining a binary matrix V according to a preset judgment standard;

step S204, searching a connected set of the binary matrix V, and arranging the connected set according to the number of elements in the filled connected set from large to small;

step S206, extracting the sub-pixel boundary marked in the image;

step S208, judging whether the connected set is a circular pattern, and determining whether the pair of connected sets form a concentric pattern by judging whether the connected set contains another connected set of the circular pattern;

step S210, judging whether the connected set is a rectangular pattern;

step S212, after determining that the connected set is the outer circle of the concentric circle, the inner circle of the concentric circle and the rectangle, positioning the sub-pixel center coordinates of the rectangle and the concentric circle;

step S214, matching the rectangle according to the mark position on the calibration plate;

and S216, classifying the concentric circles at least according to the matched rectangles, and fitting the classified sub-pixel boundaries of the concentric circles to obtain an elliptic boundary equation.

Through the steps, the characteristic points can be extracted by using a single picture or image, and compared with the condition that a plurality of pictures need to be taken to process in the prior art, a more convenient mode is provided for calibrating the camera.

Under the condition that a rectangle is arranged on the calibration board, the type of the rectangle can be matched, and the matching of the rectangle according to the mark position on the calibration board comprises the following steps: judging three types of the rectangle, and matching the rectangle according to the type of the rectangle and the mark position on the calibration board, wherein the rectangle is divided into four parts by two diagonal lines, or two middle lines respectively connected with the middle points of opposite sides of the rectangle are divided into four parts with equal areas, wherein the rectangle comprises at least one of the following types: the type I is divided into four parts by diagonal lines, and the colors are distributed alternately; type two, divided into four parts by diagonal line, the color is opposite to type one; the type III is divided into four parts by the two middle lines, and the colors are distributed at intervals; type four, divided into four parts by the two middle lines, the color is opposite to type three.

Classifying the concentric circles according to the matched rectangles comprises: classifying the concentric circles according to straight lines and the matched rectangles, wherein the straight lines comprise a first straight line and/or a second straight line, the patterns on the main calibration plate and/or the auxiliary calibration plate are distributed according to a plurality of lines, the patterns on each line are circular, or the patterns on each line are rectangular; a first straight line is drawn between the main calibration plate and/or the auxiliary calibration plate part or all the rows, and the first straight line is used for spacing the adjacent rows; and second straight lines are drawn between partial or all columns on the main calibration plate and/or the auxiliary calibration plate, wherein the second straight lines are used for separating adjacent columns, and the length of each second straight line can separate all elements or partial elements of the adjacent columns.

The implementation of the embodiment of the present application is described below with reference to a specific embodiment of a feature point extraction algorithm for camera calibration and concentric ring sub-pixel edge detection.

The accuracy of camera calibration depends largely on the detection of feature points, which are typically affected by image noise, distortion, and resolution. Due to projective transformation, the concentric circles are transformed into two ellipses through pinhole imaging projection, and the centers of the ellipses in the images are not the centers of the concentric circles. The calibration of the camera needs to accurately calculate the internal and external parameters of the camera, and the edge of the ellipse and the center of the concentric circle are quickly and accurately positioned, which plays an important role in the calibration of the camera, so that the edge and the center of the image need to be detected to reach the sub-pixel level. In view of this, this embodiment combines the Matlab source code of the sub-pixel edge detection method described in detail in document [1] (austi, Trujillo-Pino, n., Krissian, k., Alem, M., n-Flores, & Santana-CedrD. (2013). Accurate sub-pixel edge location on partial area effect. image and Vision Computing.) with an algorithm for robustly estimating the true Center position of the ellipse proposed in document [2] (Wang, x., corner, a., & Alexa, M. (2019). Center of circular after-transfer transformation.), and proposes a robustly feature point extraction algorithm. The method comprises the following specific steps:

designing and manufacturing a calibration board as shown in fig. 1, wherein the radiuses of concentric circles in the calibration board are respectively 15mm and 8mm (the two radiuses can be set as other values), the embodiment is printed by using A4 paper, the main calibration board is fixed on a flat surface, and the auxiliary calibration board is folded from the middle and fixed on the other surface;

placing the calibration plate in the visual field range of a common camera to be calibrated to acquire an image of the calibration plate;

the RGB data of the image is extracted by Matlab and the present embodiment is discussed with R values of the image colors. Let convolution kernel matrixSelecting any point (i, j) (non-image boundary point) on the image as a center to draw a nine-square grid, and setting the R value corresponding to each point in the nine-square grid as B;

let matrix B and matrix H₁The convolution of (a) represents the partial derivative in the x-axis direction at R, denoted as f_x；

Setting convolution kernel matrix by same theorySelecting any point (i, j) (non-image boundary point) on the image as a center to draw a nine-square grid, and setting the R value corresponding to each point in the nine-square grid as B;

let matrix B and matrix H₂The convolution of (d) represents the y-axis partial derivative at R, denoted as f_y；

Defining the value of J at (i, J) as J ═ abs (f)_x(i,j))+abs(f_y(i,j))；

Then, in the embodiment, an 11 × 11 grid 121 is constructed by taking any point (i, J) in the image as a center, and a binary matrix V is obtained by taking a J value corresponding to a central point of the grid 121 as a judgment standard, which is 180 larger than a point corresponding to a minimum J value in the grid;

searching a connected set in the V by using Matlab, arranging the number of elements of the connected set from large to small, filling the connected set by using an imfill function from the large connected set, and sequencing the number of the filled elements from large to small to sequentially identify the concentric circle and rectangular mark, so that the operation efficiency can be improved;

extracting sub-pixel boundaries of the image by using matlab source codes provided in the document [1 ];

judging whether the connected set in question is a great circle of concentric circles: determining two farthest points A, B on the sub-pixel boundary of the current connected set and obtaining the distance between the two points, which is marked as b; determining a maximum and minimum value point C, D of the dispersion of the connecting line from the boundary point to the two farthest points and obtaining the distance between the two points, which is marked as a; the area of the ellipse in an ideal state is pi AB, the area of the ellipse obtained by sub-pixel boundary filling is recorded as area1, the maximum pixel value of the image with the ratio of area1 to pi AB being greater than 0.95 and the drop distance of C, D on the AB straight line being less than 0.0063 times is used as a judgment condition, and if the condition is satisfied, the connected set is determined to be the outer circle of the concentric circle.

Filling the connected sets with imfill, judging whether other connected sets exist in the connected sets according to the imlabel, filling the connected sets in the connected sets with imfill, judging whether circles are contained by a method for judging large circles according to the filling area from large to small, and if the circles are contained and the ratio of the long axis of the circles to the long axis of the excircle is less than 0.8, determining that the two circles are concentric circles and storing corresponding data information.

And (3) judging whether the connected set in question is a rectangle or not, wherein A, B, C, D four points are obtained by a method similar to the circle judgment, atan2d is used for obtaining four points sorted in a counterclockwise direction, then the two points are moved in sequence according to the lengths of the two adjacent points to enable the two front points to correspond to the longest edge, the lengths of the four edges and the short edges are respectively a and b, the area of the rectangle in an ideal state is ab, the area of the rectangle filled by the sub-pixel boundary is area2, and if the ratio of the ab to the area2 is more than 0.8 and less than 1.25, and the connected set is the rectangle when the distance from the ab to the x-y-z axis in the RGB pixel space coordinate system is less than 15 and the longest edge is less than 0.1474 times of the largest pixel of the image.

And using the found geometric centers of the concentric circles and the rectangles as initial points, and then adopting a saddle point detection method to find the sub-pixel coordinates of the center of each mark.

After the rectangles are judged, three different types of patterns are determined, four vertexes of the rectangles found in the previous step are sorted according to a reverse time, fig. 3(a) (b) (c) (d) are schematic diagrams of the three different types of the rectangles marked rectangles and the division of four corner points according to the embodiment of the application, as shown in fig. 3, the midpoints of four sides are calculated, the centers are combined with the found center points of the sub-pixels of the rectangles to form triangles as shown in (a) and (b) in fig. 3, and then morphological corrosion treatment (corrosion parameter selection 2) is carried out on the triangles formed by two diagonal lines and two opposite sides in a surrounding mode, so that the points in the triangles are black as far as possible.

This time corresponds to the following two cases:

1) if the ratio of the number of the points with the pixel values smaller than 135pixel in the two triangles corresponding to the long side in (a) in fig. 3 to all the points in the two triangles is greater than 65% and the ratio of the number of the points with the pixel values smaller than 135pixel in the two triangles corresponding to the short side in (b) in fig. 3 to all the points in the two triangles is less than 65%, the rectangle is determined to be of the first type;

2) if the ratio of the number of dots having pixel values smaller than 135pixel in the two triangles corresponding to the long side in fig. 3(a) to all the dots in the two triangles is less than 65%, and the ratio of the number of dots having pixel values smaller than 135pixel in the two triangles corresponding to the short side in fig. 3(b) to all the dots in the two triangles is greater than 65%, the rectangle is determined to be of the second type.

For (c) in fig. 3, the centers of two long sides and two short sides are found, and four small rectangles are respectively formed by connecting the centers with the center, and according to the serial numbers, when the ratio of the pixel value of the small rectangle corresponding to the 1 and 3 angular points being less than 135pixel to all the points of the two rectangles is more than 65%, and the number of the points of the small rectangle corresponding to the 2 and 4 angular points being less than 135pixel is not more than 65% of all the points of the two rectangles, the rectangle is determined to be of the third type.

The stored information such as the type, center coordinates, major axis, and minor axis of the rectangle is paired.

Rectangular pairing:

for the rectangle in question, three rectangles closest to the center are found, after respective rectangle connected sets are eliminated, whether other connected sets cross the middle of the connecting line of the two centers is judged, if not, the connected sets are judged to be two paired rectangles (every two adjacent rectangles between two straight lines on a calibration plate are a pair) within a normal pixel distance, and the result is shown in (a) in fig. 5, fig. 5 is a real object diagram of sub-pixel feature points and boundaries extracted by the algorithm according to the embodiment of the application, wherein (a) in fig. 5 is the identification and classification of two calibration plate marks, and (b) in fig. 5 is the extraction of the sub-pixel boundary and the central point of a concentric circle.

Judging the position of the rectangular mark:

the type of the matching rectangle is used to determine the theoretically corresponding coordinate pair on the a4 paper.

The concentric circles are classified into three different planes.

The coordinates of the concentric circles on the main calibration plate and the corresponding theoretical coordinates are determined by the fact that the centers of the concentric circles and the centers of the rectangles paired in the previous step are within a given pixel distance range and if and only if a straight line is crossed in the middle.

That is, starting from the center of the above-found concentric circle, straight lines are drawn at 11 different angles, the length of the straight line is 0.15 times of the maximum pixel value of the image, the center of the rectangle paired with the previous step is within a given distance range, and if and only if a straight line is crossed, the concentric circle can be determined as the concentric circle on the main calibration plate, and is marked as 3, and then the coordinate of the concentric circle and the corresponding theoretical coordinate are determined.

The world coordinates corresponding to the concentric circles on the auxiliary calibration plate are not determined, and only which plane is located in is to be identified, if the middle crosses a straight line, the concentric circle on the auxiliary calibration plate is determined to be 2, if the middle has no straight line, and only the concentric circle is determined to be 1, the concentric circle on the other plane on the auxiliary calibration plate is determined to be 1, and the identification result is shown in (a) in fig. 5.

And finally, fitting the sub-pixel coordinates of the target to be measured in the discrete image by adopting a least square method (fitting the boundary of the ellipse as shown in (b) in fig. 5) to obtain an edge function of the boundary of the ellipse, so that the mean square error between the image data and the edge model data is minimum, the projected image of the concentric rings is a curve pair, and the characteristic values of matrixes before and after the projection of the curve pair are unchanged, so that the distortion parameters are corrected, and the geometric characteristics of the edges of the sub-pixels are accurately positioned.

The principle that the eigenvalues of two concentric circles are subjected to projective transformation are unchanged is as follows:

similar to an ellipse, the general form of the equation for a quadratic curve can be expressed as:

Ax²+Bxy+Cy²+Dx+Ey+F＝0 (1)

writing in matrix form:

the imaging principle of a common camera is a pinhole imaging model, so that the formula for projecting a three-dimensional point to a two-dimensional image point is as follows:

suppose Z is 0

The above formula is substituted for formula (2) to obtain:

suppose Q₁And Q₂Is obtained by projective transformation of concentric circles, so that Q₁' and Q₂' the radii of the corresponding concentric circles are R and R, respectively. Because the eigenvalues are invariant under projective transformation, there are

In practical cases, when the method in document [1] is used to detect that the edge of the concentric circle has an error due to the influence of distortion, the present embodiment can correct the edge of the concentric circle by using the characteristic that the feature value of the concentric circle is invariant under the projective transformation.

Fig. 4(a) (b) are schematic diagrams of the installation and calibration plate positions of the camera according to the embodiment of the present application, as shown in fig. 4, the main calibration plate and the auxiliary calibration plate are in three-dimensional world coordinates, and the camera takes pictures at three planes that can be taken. In practice, the calibration plate of (a) in fig. 1 may be attached to a flat ground surface, fixed, folded from the middle with one calibration plate of (b) in fig. 1, fixed to a flat surface and angled to the other calibration plate to form a three-dimensional calibration object, and/or cut in half from the middle with the calibration plate of (b) in fig. 1 to form two auxiliary calibration plates, the two auxiliary calibration plates being positioned at a predetermined distance apart to form a three-dimensional calibration object.

Fig. 5(a) and (b) are real graphs of sub-pixel feature points and boundaries extracted by the algorithm according to the embodiment of the present application, and as shown in fig. 5, fig. 5 shows the centers and boundaries of sub-pixels in the rectangle and the concentric circles identified by the feature point detection algorithm and the edge extraction algorithm proposed in this embodiment. The image resolution is 1920 multiplied by 1080, and the algorithm can be seen in the figure to extract the centers and the boundaries of the concentric circles more accurately, realize the mutual matching among different graphs and distinguish and identify the concentric circle patterns on different planes more accurately, which is enough to prove that the algorithm has high stability and strong practicability.

Fig. 6 is a specific flowchart for designing a camera calibration board and extracting feature points according to an embodiment of the present application, and as shown in fig. 6, the flowchart includes the following steps: designing a calibration template according to requirements, collecting an image of the calibration template in the visual field range of a common camera to be calibrated by the calibration template, extracting RGB data of the image by Matlab, selecting any point on the image as a center to construct a matrix of 121 grids, giving a judgment standard to obtain a binary matrix V, searching a connected set of the matrix V by Matlab, filling by an imfill function, arranging the number of elements in each filled connected set from large to small, extracting a sub-pixel boundary marked in the image, judging whether the connected set is an outer circle of a concentric circle, judging whether the connected set contains an inner circle, judging whether the connected set is a rectangle, positioning sub-pixel center coordinates of the rectangle and the concentric circle by a saddle point detection method, judging three types of the rectangle, pairing rectangles in pairs aiming at a mark position on the pre-designed calibration template, and combining a straight line and a paired rectangle, and classifying concentric circles on three different planes, and finally fitting the sub-pixel boundary of the concentric circles by using a least square method to obtain an edge function of the elliptical boundary.

In the embodiment, the calibration is carried out by using a newly designed calibration plate and a matched feature point extraction algorithm. In the embodiment, considering that the concentric circle pattern has the advantage of being capable of detecting the projection center, the projection image of the concentric circle is a curve pair, and the characteristic that the characteristic value of the matrix before and after projection is unchanged exists in the curve pair, the distortion parameter is corrected by the curve pair, and a calibration plate pattern and a characteristic point extraction algorithm for identifying and positioning by taking the concentric circle as a main calibration mode and other marks are developed. The calibration plate pattern designed by the embodiment consists of concentric rings, rectangles and straight lines, the inner circle pattern of each rectangle and each concentric ring is divided into four parts from a center point, the pattern center extraction by using sub-pixel angular points or saddle points is more accurate by using a mode of black and white blocks, the rectangle is divided into three types of patterns, and the calibration plate pattern has the advantages of easy distinguishing and paired matching.

In this embodiment, an electronic device is provided, comprising a memory in which a computer program is stored and a processor configured to run the computer program to perform the method in the above embodiments.

These computer programs may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks, and corresponding steps may be implemented by different modules.

The programs described above may be run on a processor or may be stored in memory (or referred to as computer-readable media), which includes both non-transitory and non-transitory, removable and non-removable media, that implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art to which the present application pertains. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.