1. A steel arch frame hinge hole detection method based on binocular vision is characterized by comprising the following steps:

s1, calibrating the binocular vision system to obtain the internal and external parameter matrixes and distortion coefficients of the two cameras and the rigid transformation matrix between the coordinate systems of the two cameras;

s2, collecting original images of the steel arch centering to be measured on the left side and the right side by using the binocular vision system, and performing distortion removal processing on the two original images by using the distortion coefficient obtained in the step S1 to obtain an updated image;

s3, respectively carrying out template matching on the two updated images to obtain an interested area of the hinge hole in the updated images;

s4, preprocessing the two regions of interest respectively to obtain all edge contours in the regions of interest;

s5 fitting the edge contour into an ellipse and screening, selecting the edge contour with the highest fitting degree as the hinge hole contour of the side, thereby obtaining the coordinates of the center of the hinge hole in the original image of the corresponding side;

s6, obtaining world coordinates of the circle center of the hinge hole according to the coordinates of the circle center of the hinge hole in the original image on the corresponding side of the hinge hole and the internal and external reference matrixes and the rigid body transformation matrix obtained in the step S1, and accordingly completing the measurement work of the hinge hole of the steel arch frame.

2. The binocular vision-based steel arch hinge hole detection method of claim 1, wherein in step S1, the binocular vision system is calibrated using a checkerboard.

3. The binocular vision-based steel arch hinge hole detection method of claim 1, wherein in the step S1, the distortion coefficients include a radial distortion coefficient and a tangential distortion coefficient; the rigid body transformation matrix includes a rotation transformation matrix and a translation transformation matrix.

4. The binocular vision-based steel arch hinge hole detection method of claim 1, wherein in step S3, the updated image is reduced according to an image pyramid principle, and then template matching is performed according to the reduced updated image.

5. The binocular vision-based steel arch hinge hole detection method as claimed in claim 1, wherein the left and right side regions of interest are preprocessed by the following sub-steps:

s41, removing noise of the region of interest by adopting Gaussian filtering;

s42, performing image enhancement on the region of interest without the noise to eliminate light spots formed by reflection on the surface of the steel arch centering to be detected and simultaneously increase the contrast at the edge of the hinge hole;

s43 extracts and connects the edges of the enhanced region of interest using the Canny algorithm to obtain all the edge contours within the region of interest.

6. The binocular vision-based steel arch hinge hole detection method of claim 5, wherein in step S41, the noise of the region of interest is removed by Gaussian filtering with a kernel size of 5, 7 or 9.

7. The binocular vision-based steel arch hinge hole detection method of claim 5, wherein in step S42, pixel gray values of the noise-removed region of interest are multiplied by 2.5-3.0, and pixel points having gray values greater than 255 are unified to be 255.

8. The binocular vision-based steel arch hinge hole detection method as claimed in any one of claims 1 to 7, wherein the coordinates of the center of the hinge hole in the left and right original images are obtained by respectively adopting the following sub-steps:

s51 fitting the edge contour into an ellipse by using a least square method, and primarily screening the edge contour by using the information that the length of the long axis and the short axis of the ellipse are more than 200 and less than 450;

s52, substituting all points on the preliminarily screened edge contour into the corresponding ellipse and summing to obtain the fitting degree of each edge contour;

s53, taking the edge contour with the highest fitting degree as a hinge hole contour, and taking the corresponding ellipse center coordinate as the coordinate of the center of the hinge hole in the region of interest on the corresponding side, so as to obtain the coordinate of the center of the hinge hole in the original image on the corresponding side.

9. A binocular vision system for implementing the binocular vision based steel arch hinge hole measuring method of any one of claims 1 to 8.

Background

The rock Tunnel Boring Machine (TBM) is a highly mechanized cutter head boring machine for excavating rocks at a full section under normal pressure, integrates the functions of boring, supporting, slag tapping and the like, and is important equipment in tunnel construction. The tunnel support is a necessary ring in the tunneling process and plays important roles in supporting rock walls, preventing collapse and protecting life safety of constructors. The current tunnel is strutted and is mainly adopted arc steel bow member, and cooperation stock, reinforcing bar net, reinforcing bar row are fixed, then spray the concrete, realize strengthening and strut. Because the steel arch is arc-shaped, has larger volume and is made of metal materials, the steel arch is too heavy in a tunnel with limited space, and the assembly of the steel arch becomes a great difficulty in supporting.

At present, the ring-forming assembly of the steel arch is finished by a special steel arch assembly machine, but when the two steel arches are aligned and assembled, the ring-forming assembly is still finished manually. A plurality of workers jointly carry the two steel arches to align the end faces of the two steel arches and judge whether the two steel arches are aligned by human eyes, and the method is time-consuming and labor-consuming, low in efficiency and low in precision, seriously influences the tunneling speed, cannot guarantee the construction safety and threatens the physical health of workers and people.

Disclosure of Invention

Aiming at the defects or improvement requirements of the prior art, the invention provides a steel arch hinged hole detection method based on binocular vision and application thereof, wherein the method realizes position detection of the steel arch hinged hole by using the binocular vision technology, has the advantages of high measurement speed and high precision, can help a manipulator to grab the steel arch for automatic splicing, and further achieves the purposes of saving manpower and ensuring construction safety.

In order to achieve the above object, according to one aspect of the present invention, a binocular vision-based steel arch hinge hole detection method is provided, the method comprising the following steps:

s1, calibrating the binocular vision system to obtain the internal and external parameter matrixes and distortion coefficients of the two cameras and the rigid transformation matrix between the coordinate systems of the two cameras;

s2, collecting original images of the steel arch centering to be measured on the left side and the right side by using the binocular vision system, and performing distortion removal processing on the two original images by using the distortion coefficient obtained in the step S1 to obtain an updated image;

s3, respectively carrying out template matching on the two updated images to obtain an interested area of the hinge hole in the updated images;

s4, preprocessing the two regions of interest respectively to obtain all edge contours in the regions of interest;

s5 fitting the edge contour into an ellipse and screening, selecting the edge contour with the highest fitting degree as the hinge hole contour of the side, thereby obtaining the coordinates of the center of the hinge hole in the original image of the corresponding side;

and S6, obtaining world coordinates of the circle center of the hinge hole according to the coordinates of the circle center of the hinge hole in the original image on the corresponding side and the internal and external reference matrixes and the rigid body transformation matrix obtained in the step S1, and thus finishing the measurement work of the hinge hole of the steel arch frame.

As a further preference, in step S1, the binocular vision system is calibrated using a checkerboard.

Further preferably, in step S1, the distortion coefficients include a radial distortion coefficient and a tangential distortion coefficient; the rigid body transformation matrix includes a rotation transformation matrix and a translation transformation matrix.

Further preferably, in step S3, the updated image is first reduced according to the image pyramid principle, and then template matching is performed according to the reduced updated image.

As a further preference, the following sub-steps are respectively adopted to preprocess the regions of interest on the left and right sides:

s41, removing noise of the region of interest by adopting Gaussian filtering;

s42, performing image enhancement on the region of interest without the noise to eliminate light spots formed by reflection on the surface of the steel arch centering to be detected and simultaneously increase the contrast at the edge of the hinge hole;

s43 extracts and connects the edges of the enhanced region of interest using the Canny algorithm to obtain all the edge contours within the region of interest.

As a further preference, in step S41, the noise of the region of interest is removed by gaussian filtering with a kernel size of 5, 7 or 9.

Further preferably, in step S42, the gray value of the pixel in the region of interest from which the noise is removed is multiplied by 2.5-3.0, and the pixels with the gray value greater than 255 are unified to be 255.

As a further preference, the following sub-steps are respectively adopted to obtain the coordinates of the circle centers of the hinge holes in the original images at the left side and the right side:

s51 fitting the edge contour into an ellipse by using a least square method, and primarily screening the edge contour by using the information that the length of the long axis and the short axis of the ellipse are more than 200 and less than 450;

s52, substituting all points on the preliminarily screened edge contour into the corresponding ellipse and summing to obtain the fitting degree of each edge contour;

s53, taking the edge contour with the highest fitting degree as a hinge hole contour, and taking the corresponding ellipse center coordinate as the coordinate of the center of the hinge hole in the region of interest on the corresponding side, so as to obtain the coordinate of the center of the hinge hole in the original image on the corresponding side.

According to another aspect of the present invention, there is provided a binocular vision system implementing the above method.

Generally, compared with the prior art, the above technical solution conceived by the present invention mainly has the following technical advantages:

1. the invention realizes the position detection of the steel arch hinge hole by using the binocular vision technology, has the advantages of high measurement speed and high precision, can help the manipulator to grab the steel arch for automatic splicing, further saves labor and ensures the construction safety;

2. in addition, in the process of extracting the profile of the steel arch frame hinge hole, the profile of the hinge hole is accurately obtained through a series of image preprocessing processes such as Gaussian filtering, image enhancement, edge extraction and profile screening, and is fitted into an ellipse by using a least square method.

Drawings

Fig. 1 is a flow chart of a binocular vision-based steel arch hinged hole measuring method provided by the invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

As shown in fig. 1, an embodiment of the present invention provides a binocular vision-based steel arch hinge hole detection method, including the following steps:

(1) building a binocular vision system, and placing two cameras in parallel according to the position and posture conditions of a steel arch and a manipulator, wherein the distance between the two cameras is about 300mm, the distance between the two cameras and the steel arch is 650mm, the cameras are COMS cameras with 1200 ten thousand pixels, and the focal length of a lens is 16 mm; a strip-shaped light source is respectively arranged above and below the camera, and the size of the light source is 300x30 mm;

(2) according to a Zhangyingyou calibration method, a binocular vision system is used for collecting 11-14 left and right checkerboard images, and internal and external parameter matrixes and radial distortion coefficients k of left and right cameras of the binocular vision system are obtained through calibration₁，k₂，k₃Coefficient of tangential distortion p₁，p₂And a rigid body transformation matrix of the right camera coordinate system relative to the left camera coordinate system, including a rotation transformation matrix R and a translation transformation matrix T;

(3) acquiring left and right images of a steel arch by using a binocular vision system, wherein the field of view range is about 250 multiplied by 187.5mm, a steel arch hinge hole and a certain moving range of the steel arch hinge hole can be shot, a bar-shaped light source is aimed at the steel arch to be polished, a light source controller is adjusted to enable the brightness of the light source to be proper, a left camera and a right camera are used for simultaneously acquiring the images of the steel arch, and the images are subjected to distortion removal treatment by using a radial distortion coefficient and a tangential distortion coefficient obtained through calibration, so that an updated image is obtained;

(4) firstly, according to the pyramid principle of the image, reducing the image into 1/4 of the original image, then respectively carrying out template matching on the left and right reduced updated images by using a template matching method based on gray scale, finding out the most similar parts of the left and right reduced updated images and obtaining the interested areas of the hinge holes in the left and right updated images;

(5) preprocessing an interested area, namely removing noise by Gaussian filtering with the kernel size of 5, 7 or 9, then performing image linear enhancement, multiplying all pixel gray values by 2.5-3.0, uniformly taking 255 as the gray value larger than 255, brightening the surface of a hinge hole, eliminating light spots formed by the reflection of an uneven metal surface, simultaneously increasing the contrast at the edge of a round hole, then extracting the edge, and performing closing operation on an edge binary image by using an ellipse with the size of 7-11, so that certain disconnected edges are connected into a whole outline to obtain all edge outlines in the interested area;

(6) fitting each edge profile into an ellipse by using a least square method, and primarily screening the edge profiles by using the following information: the number of points contained in the edge contour is required to be more than 600, and the lengths of the long axis and the short axis of the edge contour corresponding to the fitting ellipse are required to be more than 200 and less than 450; then substituting all the points on the preliminarily screened edge contour into respective ellipse equations and summing to obtain the ellipse fitting degree FD of each edge contour, wherein the edge contour with the best fitting degree is the hinge hole contour, and the corresponding ellipse center coordinate is the coordinate (u) of the center of the hinge hole in the region of interest_roi,v_roi) (ii) a For the elliptic formula Ax²+Bxy+Cy²+ Dx + Ey + F ═ 0, and the degree of fit FD formula is as follows:

wherein N is the number of contour points, (x)_i,y_i) A, B, C, D, E, F represents the coefficients of an elliptic equation, FD represents the degree of fit, as coordinates of points on the contour;

coordinates (u) centering the hinge hole in the region of interest_roi，v_roi) Adding the coordinate (u) of the upper left corner of the interested area in the original image₀，v₀) And obtaining the coordinates of the circle center of the hinge hole in the original image, namely:

in the formula, (u, v) is a coordinate of the circle center of the hinge hole in the original image;

(7) coordinates (u) of the center of the hinge hole in the left and right original images₁,v₁) And (u)₂,v₂) And solving the following equation set to obtain the world coordinates (X, Y, Z) of the center of the hinge hole, wherein m is¹Matrix sum m²The matrix is obtained by multiplying the internal reference matrix of the left camera and the external reference matrix of the right camera respectively,

in the formula (I), the compound is shown in the specification,is m¹The numbers in the ith row, the jth column,is m²The number of ith row and jth column in the matrix.

According to another aspect of the present invention, there is provided a binocular vision system implementing the above method. The binocular vision system comprises two cameras, two lenses, two strip-shaped light sources, a light source controller and a computer, wherein the light sources are used for polishing hinged holes of the steel arch frame, the light source controller adjusts the brightness of the light sources, and the cameras and the lenses are used for collecting images and transmitting the images to the computer for processing.

In conclusion, the steel arch hinge hole detection method and the steel arch hinge hole detection device utilize a non-contact binocular vision technology to detect the steel arch hinge hole, have the advantages of high speed, high precision, safety, reliability and the like, and are beneficial to realizing the automation of steel arch splicing.

It will be understood by those skilled in the art that the foregoing is merely a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included within the scope of the present invention.