1. A signboard recognition method for automatic driving, comprising:

acquiring a boundary frame of the signboard and a plurality of vertexes of the boundary frame;

acquiring an image comprising n pixel widths outside the boundary box according to the boundary box of the signboard, wherein n is a positive integer larger than a statistical pixel offset error value of the boundary box identified by a deep learning model;

generating a plurality of angular point blocks on the image, wherein each angular point block of the plurality of angular point blocks takes each vertex of the plurality of vertices as a center, 2m +1 pixels are side lengths, and m is a positive integer smaller than n;

obtaining edge information of each angular point block according to the gradient information of each angular point block of the plurality of angular point blocks;

acquiring an intersection point of each angular point block according to the edge information of each angular point block;

and according to the intersection point of each corner block, the boundary frame of the signboard is obtained again.

2. The method of claim 1, wherein: the method further comprises the following steps:

and if the obtained boundary box of the signboard is consistent with the obtained physical and geometric relationship of the boundary box of the signboard, determining the obtained boundary box of the signboard as the boundary box of the signboard.

3. The method according to claim 1 or 2, characterized in that: the obtaining the boundary box of the signboard again according to the intersection point of each corner block comprises:

obtaining the confidence coefficient of the intersection point of each angular point block according to the edge information of each angular point block;

and for the intersection point of each corner block, if the confidence coefficient of the intersection point is greater than a set threshold value, the boundary frame of the signboard is obtained again according to the intersection point with the confidence coefficient greater than the set threshold value.

4. The method of claim 3, wherein:

the confidence of the intersection point is the ratio of the total number of edge pixels contained on the two edges of the intersection point of the angular point blocks to the total number of pixels contained on the two edges of the intersection point of the angular point blocks.

5. The method of claim 4, wherein: the n is a positive integer of 2 times the statistical pixel shift error value of the bounding box identified by the deep learning model.

6. A signboard recognition apparatus for automatic driving, comprising:

the first acquisition module is used for acquiring a boundary box of the signboard and a plurality of vertexes of the boundary box;

the second acquisition module is used for acquiring an image comprising n pixel widths outside the boundary frame according to the boundary frame of the signboard acquired by the first acquisition module, wherein n is a positive integer larger than the statistical pixel offset error value of the boundary frame identified by the deep learning model;

a generating module, configured to generate a plurality of corner blocks on the image acquired by the second acquiring module, where each corner block of the plurality of corner blocks takes each vertex of the plurality of vertices as a center, 2m +1 pixels are side lengths, and m is a positive integer smaller than n;

the angular point block processing module is used for acquiring the edge information of each angular point block generated by the generating module according to the gradient information of each angular point block of the plurality of angular point blocks;

the intersection point module is used for acquiring the intersection point of each corner block according to the edge information of each corner block acquired by the corner block processing module;

and the boundary frame module is used for acquiring the boundary frame of the signboard again according to the intersection point of each corner block acquired by the intersection point module.

7. The apparatus of claim 6, wherein: the device further comprises:

and the boundary frame determining module is used for determining that the boundary frame of the obtained signboard is the boundary frame of the signboard if the boundary frame of the signboard obtained by the boundary frame module is consistent with the physical and geometric relationship of the boundary frame of the signboard obtained by the first obtaining module.

8. The apparatus of claim 6 or 7, wherein:

the corner block processing module is further configured to obtain a confidence level of an intersection point of each corner block generated by the generating module according to the edge information of each corner block;

and the boundary box module is also used for acquiring the boundary box of the signboard again according to the intersection point with the confidence coefficient greater than the set threshold value if the confidence coefficient of the intersection point acquired by the corner block processing module is greater than the set threshold value for the intersection point of each corner block acquired by the intersection point module.

9. An electronic device, comprising:

a processor; and

a memory having executable code stored thereon, which when executed by the processor, causes the processor to perform the method of any one of claims 1-5.

10. A non-transitory machine-readable storage medium having executable code stored thereon, which when executed by a processor of an electronic device, causes the processor to perform the method of any one of claims 1-5.

Background

With the development of the automatic driving technology, the integrated navigation system of the related art, such as the GNSS/IMU integrated navigation system, cannot provide accurate and reliable positioning results in many special scenes, and further affects subsequent planning and control. A GNSS (Global Navigation Satellite System) is a Satellite positioning System based on satellites, and an IMU Navigation System is an Inertial Navigation System based on an IMU (Inertial Measurement Unit).

The related high-precision map-based positioning technology can extract road factors (such as a signboard, a lane line and the like) obtained by a visual sensor from visual information, match the road factors with corresponding factors in a high-precision map and correct the result of combined navigation. The image is used as main visual information, after a boundary frame of a signboard on a road is identified through a deep learning model, the relative position of the self-vehicle in a high-precision map can be calculated through a 2D-3D matching technology based on EPNP (effective personal-n-Point), and special scene positioning is kept not lost.

However, the signboard recognition technology in the related art is limited by software and hardware such as a deep learning model and a camera, and cannot accurately recognize the boundary frame of the signboard, so that the obtained vertex position of the signboard has a deviation, and the high-precision map-based positioning technology in the related art also has an error.

Disclosure of Invention

In order to solve or partially solve the problems in the related art, the application provides a signboard identification method and device for automatic driving, which can accurately obtain a boundary frame of a signboard and improve the accuracy of obtaining the vertex position of the signboard.

The present application provides in a first aspect a signboard identification method for automatic driving, the method comprising:

acquiring a boundary frame of the signboard and a plurality of vertexes of the boundary frame;

acquiring an image comprising n pixel widths outside the boundary box according to the boundary box of the signboard, wherein n is a positive integer larger than a statistical pixel offset error value of the boundary box identified by a deep learning model;

generating a plurality of angular point blocks on the image, wherein each angular point block of the plurality of angular point blocks takes each vertex of the plurality of vertices as a center, 2m +1 pixels are side lengths, and m is a positive integer smaller than n;

obtaining edge information of each angular point block according to the gradient information of each angular point block of the plurality of angular point blocks;

acquiring an intersection point of each angular point block according to the edge information of each angular point block;

and according to the intersection point of each corner block, the boundary frame of the signboard is obtained again.

Preferably, the method further comprises:

and if the obtained boundary box of the signboard is consistent with the obtained physical and geometric relationship of the boundary box of the signboard, determining the obtained boundary box of the signboard as the boundary box of the signboard.

Preferably, the retrieving the bounding box of the signboard according to the intersection point of each corner block includes:

obtaining the confidence coefficient of the intersection point of each angular point block according to the edge information of each angular point block;

and for the intersection point of each corner block, if the confidence coefficient of the intersection point is greater than a set threshold value, the boundary frame of the signboard is obtained again according to the intersection point with the confidence coefficient greater than the set threshold value.

Preferably, the confidence of the intersection point is a ratio of the total number of edge pixels included on the two edges of the intersection point of the corner block to the total number of pixels included on the two edges of the intersection point of the corner block.

Preferably, n is a positive integer of 2 times the statistical pixel shift error value of the bounding box identified by the deep learning model.

A second aspect of the present application provides a signboard recognition apparatus for automatic driving, the apparatus including:

the first acquisition module is used for acquiring a boundary box of the signboard and a plurality of vertexes of the boundary box;

the second acquisition module is used for acquiring an image comprising n pixel widths outside the boundary frame according to the boundary frame of the signboard acquired by the first acquisition module, wherein n is a positive integer larger than the statistical pixel offset error value of the boundary frame identified by the deep learning model;

a generating module, configured to generate a plurality of corner blocks on the image acquired by the second acquiring module, where each corner block of the plurality of corner blocks takes each vertex of the plurality of vertices as a center, 2m +1 pixels are side lengths, and m is a positive integer smaller than n;

the angular point block processing module is used for acquiring the edge information of each angular point block generated by the generating module according to the gradient information of each angular point block of the plurality of angular point blocks;

the intersection point module is used for acquiring the intersection point of each corner block according to the edge information of each corner block acquired by the corner block processing module;

and the boundary frame module is used for acquiring the boundary frame of the signboard again according to the intersection point of each corner block acquired by the intersection point module.

Preferably, the apparatus further comprises:

and the boundary frame determining module is used for determining that the boundary frame of the obtained signboard is the boundary frame of the signboard if the boundary frame of the signboard obtained by the boundary frame module is consistent with the physical and geometric relationship of the boundary frame of the signboard obtained by the first obtaining module.

Preferably, the corner block processing module is further configured to obtain a confidence level of an intersection point of each corner block generated by the generating module according to the edge information of each corner block;

and the boundary box module is also used for acquiring the boundary box of the signboard again according to the intersection point with the confidence coefficient greater than the set threshold value if the confidence coefficient of the intersection point acquired by the corner block processing module is greater than the set threshold value for the intersection point of each corner block acquired by the intersection point module.

A third aspect of the present application provides an electronic device comprising:

a processor; and

a memory having executable code stored thereon, which when executed by the processor, causes the processor to perform the method as described above.

A fourth aspect of the present application provides a non-transitory machine-readable storage medium having stored thereon executable code, which when executed by a processor of an electronic device, causes the processor to perform a method as described above.

The technical scheme provided by the application can comprise the following beneficial effects:

according to the technical scheme, the confidence coefficient of the intersection point of each corner block is obtained according to the edge information of each corner block; for the intersection point of each angular point block, if the confidence coefficient of the intersection point is greater than the set threshold, the boundary frame of the signboard is obtained again according to the intersection point of which the confidence coefficient is greater than the set threshold; if the boundary frame of the newly acquired signboard is consistent with the physical geometric relationship of the acquired boundary frame of the signboard, the boundary frame of the newly acquired signboard is determined to be the boundary frame of the signboard, pixel-level optimization is performed on the boundary frame and the vertex of the signboard by utilizing the gradient information of the image, the physical geometric relationship of the boundary frame of the signboard and the confidence coefficient of the intersection point, so that the recognition errors caused by the situation that the signboard is too small and overlapped and other objects such as a rod or a frame shield the signboard can be avoided, the reliability of the optimization is ensured through the threshold limit of the confidence coefficient of the intersection point, the real boundary frame of the signboard can be accurately acquired, the accuracy of acquiring the vertex position of the signboard is improved, the accuracy of matching with a high-precision map is improved, and the positioning and stability of an automatic driving centimeter level are ensured.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The foregoing and other objects, features and advantages of the application will be apparent from the following more particular descriptions of exemplary embodiments of the application, as illustrated in the accompanying drawings wherein like reference numbers generally represent like parts throughout the exemplary embodiments of the application.

FIG. 1 is a flow chart diagram illustrating a signboard identification method for automatic driving according to an embodiment of the present application;

FIG. 2 is another flow chart diagram of a signboard identification method for automatic driving according to an embodiment of the present application;

FIG. 3 is another flow chart diagram of a signboard identification method for automatic driving according to an embodiment of the present application;

FIG. 4 is a diagram illustrating a bounding box of a signboard output by a deep learning model according to an embodiment of the present application;

FIG. 5 is a schematic structural diagram of a signboard identification device for automatic driving according to an embodiment of the present application;

FIG. 6 is another schematic structural diagram of a signboard identification device for automatic driving according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of an electronic device shown in an embodiment of the present application.

Detailed Description

Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. While embodiments of the present application are illustrated in the accompanying drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms "first," "second," "third," etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.

The embodiment of the application provides a signboard identification method for automatic driving, which can accurately obtain a boundary frame of a signboard and improve the accuracy of obtaining the vertex position of the signboard.

The technical solutions of the embodiments of the present application are described in detail below with reference to the accompanying drawings.

The first embodiment is as follows:

fig. 1 is a schematic flow chart of a signboard identification method for automatic driving according to an embodiment of the present application.

Referring to fig. 1, a signboard recognition method for automatic driving includes:

in step S101, a bounding box of the signboard and a plurality of vertices of the bounding box are acquired.

In one embodiment, the autonomous vehicle is provided with a camera device including, but not limited to, a monocular camera. The monocular camera is provided at a suitable position of the vehicle, for example, the monocular camera may be provided at a front windshield of the vehicle or at a rear view mirror of the vehicle, so that the monocular camera can capture an image including the road signboard during automatic driving of the vehicle.

In one embodiment, the bounding box of the signboard and a plurality of vertexes of the bounding box, which are a plurality of corner points of the bounding box, may be obtained through a deep learning model. The image containing the road signboard can be input into the deep learning model, the image containing the road signboard is recognized through the deep learning model, and a boundary box of the signboard and a plurality of vertexes of the boundary box are output; and acquiring a boundary box of the signboard output by the deep learning model and a plurality of vertexes of the boundary box.

In step S102, an image including n pixel widths outside the bounding box is obtained according to the bounding box of the signboard, where n is a positive integer greater than the statistical pixel offset error value of the bounding box identified by the deep learning model.

In one embodiment, the statistical pixel offset error value of the boundary box of the signboard is identified according to the deep learning model, n pixels are expanded to the periphery of the outer side of the boundary box of the signboard, an image including the width of n pixels at the periphery of the outer side of the boundary box is obtained, that is, n pixels are obtained at the periphery of the outer side of the boundary box, and an image of the boundary box plus n pixels is obtained. The value of n can be set according to the statistical pixel offset error value of the boundary box of the deep learning model identification signboard, and n is a positive integer with the value larger than the statistical pixel offset error value.

In step S103, a plurality of corner blocks are generated on the image, each of the plurality of corner blocks takes each of the plurality of vertices as a center, 2m +1 pixels are side lengths, and m is a positive integer smaller than n.

In one embodiment, one vertex of the image obtained in step S102 is centered on one of the vertices of the bounding box, and 2m +1 pixels are the side length, and one corner block is generated at each vertex of the vertices of the bounding box.

In one embodiment, the size of m may be set according to a statistical pixel shift error value of a bounding box of the depth learning model identification image, where the size of m is greater than the statistical pixel shift error value and less than n. A corner block with a side length of 2m +1 pixels is within an enlarged image of n pixels, so 2m +1 is also smaller than n.

In step S104, edge information of each corner block is obtained from the gradient information of each corner block of the plurality of corner blocks.

In one embodiment, the image of the plurality of corner blocks generated in step S103 is subjected to edge detection, and gradient information of each corner block is obtained through gradient calculation; and obtaining the edge information of each corner block according to the gradient information of each corner block. The edge information of the corner block is the real edges of the signboard in the corner blocks at two sides of the top point of the signboard, including the edges in the horizontal direction and the edges in the vertical direction.

In step S105, an intersection of each corner block is obtained based on the edge information of each corner block.

In an embodiment, an intersection point of the horizontal side and the vertical side, that is, an intersection point of each corner block, may be obtained according to the horizontal side and the vertical side of each corner block. The intersection point of two sides in the horizontal direction and the vertical direction of the corner block can be obtained according to the linear equation of the horizontal direction and the vertical direction of the corner block, namely the intersection point of each corner block is obtained.

In step S106, the bounding box of the signboard is retrieved according to the intersection of each corner block.

In one embodiment, the intersection point of each corner block is used as a plurality of new vertexes of the boundary box of the signboard, and the new vertexes are connected to obtain the boundary box of the signboard again.

According to the signboard identification method for automatic driving, an image comprising n pixel widths outside a boundary box is obtained according to the boundary box of the signboard, wherein n is a positive integer larger than a statistical pixel offset error value of the boundary box identified by a deep learning model, and the identified boundary box can be prevented from being in the signboard; generating a plurality of angular point blocks on the image, wherein each angular point block of the plurality of angular point blocks takes each vertex of the plurality of vertices as a center, 2m +1 pixels are used as side lengths, and m is a positive integer smaller than n; obtaining edge information of each angular point block according to the gradient information of each angular point block of the plurality of angular point blocks; acquiring an intersection point of each angular point block according to the edge information of each angular point block; according to the intersection point of each angular point block, a boundary frame of the signboard is obtained again; the boundary frame and the vertex of the signboard are optimized in pixel level by utilizing the gradient information of the image, so that the real boundary frame of the signboard can be accurately obtained, the precision of obtaining the vertex position of the signboard is improved, the precision matched with a high-precision map is improved, and the positioning and the stability of an automatic driving centimeter level are ensured.

Example two:

fig. 2 is another flow chart of a signboard identification method for automatic driving according to an embodiment of the present disclosure.

Referring to fig. 2, a signboard recognition method for automatic driving includes:

in step S201, a bounding box of the signboard and a plurality of vertices of the bounding box are acquired.

This step can be referred to the description of step S101, and is not described herein again.

In step S202, an image including n pixel widths outside the bounding box is obtained according to the bounding box of the signboard, where n is a positive integer greater than the statistical pixel offset error value of the bounding box identified by the deep learning model.

In an embodiment, as shown in fig. 4, the lower right corner of the boundary box 401 of the signboard output by the deep learning model according to the signboard image does not completely contain the whole signboard, n pixels may be expanded around the outer side of the boundary box 401 of the signboard according to the statistical pixel shift error value of the boundary box of the signboard recognized by the deep learning model, an image including n pixels around the outer side of the boundary box 401 is obtained, that is, n pixels are obtained around the outer side of the boundary box 401, and an image of the boundary box 401 plus n pixels is obtained, where the n pixels may include the signboard or may include the background. The value of n can be set according to the statistical pixel offset error value of the boundary box of the identification signboard recognized by the deep learning model, and n is a positive integer which is 2 times of the statistical pixel offset error value of the boundary box recognized by the deep learning model.

In one embodiment, the size of n may be enlarged to ensure that the subsequently extracted corner block contains the true signboard vertex. Empirically, the size of n in the embodiment of the present application is set to 20.

The signboard of the embodiment of the present invention is not limited to the actually rectangular signboard shown in fig. 4, but includes, for example, a triangular signboard, a circular signboard, and the like.

In step S203, a plurality of corner blocks are generated on the image, each of the plurality of corner blocks takes each of the plurality of vertices as a center, 2m +1 pixels are side lengths, and m is a positive integer smaller than n.

In an embodiment, with one vertex of the vertices of the bounding box as the center and 2m +1 pixels as the side length, one vertex on the image acquired in step S202 generates a corner block of a square, and for each vertex of the vertices of the bounding box, each vertex generates a corner block of a square with the same area.

In one embodiment, the size of m may be set according to a statistical pixel shift error value of a bounding box of the depth learning model identification image, where the size of m is greater than the statistical pixel shift error value and less than n. Empirically, the size of m in the embodiment of the present application is set to 12, and thus, the side length of the square is 12 × 2+1=25 pixels, and the size of the corner block of the square is 25 × 25 pixels.

In step S204, edge information of each corner block is obtained from the gradient information of each corner block of the plurality of corner blocks.

This step can be referred to the description of step S104, and is not described herein again.

In step S205, an intersection of each corner block is obtained based on the edge information of each corner block.

This step can be referred to the description of step S105, and is not described herein again.

In step S206, the bounding box of the signboard is retrieved according to the intersection of each corner block.

This step can be referred to the description of step S106, and is not described herein again.

In step S207, if the boundary box of the retrieved signboard is consistent with the physical and geometric relationship of the boundary box of the acquired signboard, the boundary box of the retrieved signboard is determined as the boundary box of the signboard.

In one embodiment, if the boundary box of the retrieved signboard is consistent with the physical geometry relationship of the boundary box of the signboard acquired in step S101, the boundary box of the retrieved signboard is finally determined as the boundary box of the signboard. For example, assume that the bounding box of the signboard acquired in step S101 is rectangular. If the boundary box of the retrieved signboard is also rectangular, and the size of the boundary box of the retrieved signboard is approximately equal to the size of the boundary box of the signboard obtained in step S101, the physical geometric relationship between the boundary box of the retrieved signboard and the boundary box of the signboard obtained in step S101 is consistent, so that the boundary box of the retrieved signboard is finally determined as the boundary box of the signboard.

In one embodiment, if the physical and geometric relationship between the boundary box of the retrieved signboard and the boundary box of the signboard obtained in step S101 is not consistent, the boundary box of the signboard obtained in step S101 is finally determined as the boundary box of the signboard.

According to the signboard identification method for automatic driving, if the boundary frame of the newly acquired signboard is consistent with the physical geometric relationship of the acquired signboard boundary frame, the newly acquired signboard boundary frame is determined to be the signboard boundary frame, pixel-level optimization is performed on the boundary frame and the vertex of the signboard by using the gradient information of the image, and the identification error caused by the fact that the signboard is shielded by other objects such as a rod or a frame due to undersize and overlapping of the signboard can be avoided by using the physical geometric relationship of the signboard boundary frame, so that the real boundary frame of the signboard can be accurately acquired, the accuracy of acquiring the position of the vertex of the signboard is improved, the accuracy of matching with a high-precision map is improved, and centimeter-level positioning and stability of automatic driving are ensured.

Example three:

fig. 3 is another flow chart of a signboard identification method for automatic driving according to an embodiment of the present disclosure.

Referring to fig. 3, a signboard recognition method for automatic driving includes:

in step S301, a bounding box of the signboard and a plurality of vertices of the bounding box are acquired.

This step can be referred to the description of step S101, and is not described herein again.

In step S302, an image including n pixel widths outside the bounding box is obtained according to the bounding box of the signboard, where n is a positive integer greater than the statistical pixel offset error value of the bounding box identified by the deep learning model.

This step can be referred to the description of step S202, and is not described herein again.

In step S303, a plurality of corner blocks are generated on the image, each of the plurality of corner blocks takes each of the plurality of vertices as a center, 2m +1 pixels are side lengths, and m is a positive integer smaller than n.

This step can be referred to the description of step S203, and is not described herein.

In step S304, edge information of each corner block is obtained from the gradient information of each corner block of the plurality of corner blocks.

This step can be referred to the description of step S104, and is not described herein again.

In step S305, an intersection of each corner block is obtained from the edge information of each corner block.

This step can be referred to the description of step S105, and is not described herein again.

In step S306, a confidence level of the intersection of each corner block is obtained according to the edge information of each corner block.

In an embodiment, the confidence of the intersection point of the two sides of each corner block may be obtained according to the intersection point of the two sides of each corner block in the horizontal direction and the vertical direction and the edge information of each corner block. The confidence of the intersection point is the ratio of the total number of edge pixels contained on the two edges of the intersection point of the angular point blocks to the total number of pixels contained on the two edges of the intersection point of the angular point blocks.

In an embodiment, different edge detection operators may be used, and therefore, the obtained edge information of the corner block may be different, the total number of edge pixels included in two edges of the intersection may be different, and the obtained confidence of the intersection of the corner block may also be different.

In step S307, for the intersection of each corner block, if the confidence of the intersection is greater than the set threshold, the boundary box of the signboard is obtained again according to the intersection whose confidence is greater than the set threshold.

In one embodiment, if the confidence of the intersection point of the corner block is greater than the set threshold, the intersection point with the confidence greater than the set threshold is re-determined as a new vertex of the boundary box of the signboard. And for the intersection point of each angular point block, if the confidence coefficient of the intersection point of each angular point block is greater than the set threshold, re-determining the intersection points with the confidence coefficients greater than the set threshold as a plurality of new vertexes of the boundary frame of the signboard, so as to re-determine the plurality of vertexes of the boundary frame of the signboard and re-obtain the boundary frame of the signboard.

In one embodiment, if the confidence of the intersection point of the corner block is less than or equal to the set threshold, the intersection point of the corner block is discarded, and the vertex of the bounding box obtained in step S101 is retained. Retrieving the bounding box of the signboard with the one or more vertices retained and the one or more vertices of the redetermined bounding box of the signboard.

In step S308, if the boundary box of the retrieved signboard is consistent with the physical and geometric relationship of the boundary box of the acquired signboard, the boundary box of the retrieved signboard is determined as the boundary box of the signboard.

This step can be referred to the description of step S207, and is not described herein again.

According to the signboard identification method for automatic driving, the confidence of the intersection point of each corner block is obtained according to the edge information of each corner block; for the intersection point of each angular point block, if the confidence coefficient of the intersection point is greater than the set threshold, the boundary frame of the signboard is obtained again according to the intersection point of which the confidence coefficient is greater than the set threshold; if the boundary frame of the newly acquired signboard is consistent with the physical geometric relationship of the acquired boundary frame of the signboard, the boundary frame of the newly acquired signboard is determined to be the boundary frame of the signboard, pixel-level optimization is performed on the boundary frame and the vertex of the signboard by utilizing the gradient information of the image, the physical geometric relationship of the boundary frame of the signboard and the confidence coefficient of the intersection point, so that the recognition errors caused by the situation that the signboard is too small and overlapped and other objects such as a rod or a frame shield the signboard can be avoided, the reliability of the optimization is ensured through the threshold limit of the confidence coefficient of the intersection point, the real boundary frame of the signboard can be accurately acquired, the accuracy of acquiring the vertex position of the signboard is improved, the accuracy of matching with a high-precision map is improved, and the positioning and stability of an automatic driving centimeter level are ensured.

Example four:

corresponding to the embodiment of the application function implementation method, the application also provides a signboard recognition device and electronic equipment for automatic driving and a corresponding embodiment.

Fig. 5 is a schematic structural diagram of a signboard identifying device for automatic driving according to an embodiment of the present application.

Referring to fig. 5, a signboard recognition apparatus for automatic driving includes a first acquisition module 501, a second acquisition module 502, a generation module 503, a corner block processing module 504, an intersection module 505, and a bounding box module 506.

The first obtaining module 501 is configured to obtain a bounding box of the signboard and a plurality of vertices of the bounding box.

In one embodiment, the autonomous vehicle is provided with a camera device including, but not limited to, a monocular camera. The monocular camera is provided at a suitable position of the vehicle, for example, the monocular camera may be provided at a front windshield of the vehicle or at a rear view mirror of the vehicle, so that the monocular camera can capture an image including the road signboard during automatic driving of the vehicle.

In an embodiment, the first obtaining module 501 may obtain, through the deep learning model, a bounding box of the signboard and a plurality of vertices of the bounding box, where the vertices of the bounding box are a plurality of corner points of the bounding box. The first obtaining module 501 may input an image including a road signboard to the deep learning model, recognize the image including the road signboard through the deep learning model, and output a bounding box of the signboard and a plurality of vertices of the bounding box; and acquiring a boundary box of the signboard output by the deep learning model and a plurality of vertexes of the boundary box.

The second obtaining module 501 is configured to obtain an image including n pixel widths outside the boundary box according to the boundary box of the signboard obtained by the first obtaining module 501, where n is a positive integer greater than a statistical pixel offset error value of the boundary box identified by the deep learning model.

In an embodiment, the second obtaining module 501 identifies a statistical pixel offset error value of a boundary box of the signboard according to the depth learning model, expands n pixels around the outer side of the boundary box of the signboard, obtains an image including n pixel widths around the outer side of the boundary box, that is, obtains n pixels around the outer side of the boundary box, and obtains an image of the boundary box plus n pixel widths. The second obtaining module 501 may set a value of n according to the statistical pixel shift error value of the boundary box of the deep learning model identification signboard, where n is a positive integer whose value is greater than the statistical pixel shift error value.

A generating module 503, configured to generate a plurality of corner blocks on the image acquired by the second acquiring module 502, where each corner block of the plurality of corner blocks takes each vertex of the plurality of vertices as a center, 2m +1 pixels are side lengths, and m is a positive integer smaller than n.

In an embodiment, the generating module 503 generates a corner block at a vertex on the image acquired by the second acquiring module 502 by taking a vertex of the plurality of vertices of the bounding box as a center and 2m +1 pixels as a side length, and for each vertex of the plurality of vertices of the bounding box, the generating module 503 generates a corner block at each vertex.

In one embodiment, the generation module 503 may set the size of m according to the statistical pixel shift error value of the bounding box of the depth learning model identification image, where the size of m is greater than the statistical pixel shift error value and less than n.

The corner block processing module 504 is configured to obtain edge information of each corner block generated by the generating module 503 according to the gradient information of each corner block of the plurality of corner blocks.

In an embodiment, the corner block processing module 504 may perform edge detection on the images of the plurality of corner blocks generated by the generating module 503, and obtain gradient information of each corner block through gradient calculation; and obtaining the edge information of each corner block according to the gradient information of each corner block. The edge information of the corner block obtained by the corner block processing module 504 is the real edges of the signboard in the corner blocks at the two sides of the signboard vertex, including the edges in the horizontal direction and the edges in the vertical direction.

An intersection module 505, configured to obtain an intersection point of each corner block according to the edge information of each corner block obtained by the corner block processing module 504.

In an embodiment, the intersection module 505 may obtain an intersection point of the horizontal side and the vertical side, that is, an intersection point of each corner block, according to the horizontal side and the vertical side of each corner block obtained by the corner block processing module 504. The intersection module 505 may obtain an intersection point of two sides of the corner block in the horizontal direction and the vertical direction according to a linear equation of the corner block in the horizontal direction and the vertical direction, that is, obtain an intersection point of each corner block.

A bounding box module 506, configured to obtain a bounding box of the signboard again according to the intersection of each corner block obtained by the intersection module 505.

In one embodiment, the bounding box module 506 may retrieve the bounding box of the signboard by connecting a plurality of new vertices with the intersection of each corner block obtained by the intersection module 505 as the new vertices of the bounding box of the signboard.

According to the technical scheme, the image with the width of n pixels outside the boundary box is obtained according to the boundary box of the signboard, wherein n is a positive integer larger than a statistical pixel offset error value of the boundary box recognized by a deep learning model, and the recognized boundary box can be prevented from being in the signboard; generating a plurality of angular point blocks on the image, wherein each angular point block of the plurality of angular point blocks takes each vertex of the plurality of vertices as a center, 2m +1 pixels are used as side lengths, and m is a positive integer smaller than n; obtaining edge information of each angular point block according to the gradient information of each angular point block of the plurality of angular point blocks; acquiring an intersection point of each angular point block according to the edge information of each angular point block; according to the intersection point of each angular point block, a boundary frame of the signboard is obtained again; the boundary frame and the vertex of the signboard are optimized in pixel level by utilizing the gradient information of the image, so that the real boundary frame of the signboard can be accurately obtained, the precision of obtaining the vertex position of the signboard is improved, the precision matched with a high-precision map is improved, and the positioning and the stability of an automatic driving centimeter level are ensured.

Example five:

fig. 6 is another schematic structural diagram of a signboard identifying device for automatic driving according to an embodiment of the present application.

Referring to fig. 6, a signboard recognition apparatus for automatic driving includes a first acquisition module 501, a second acquisition module 502, a generation module 503, a corner block processing module 504, an intersection module 505, a bounding box module 506, and a bounding box determination module 601.

The functions of the first obtaining module 501, the second obtaining module 502, the generating module 503 and the intersection module 505 can be seen in fig. 5.

An angular point block processing module 504, configured to obtain, according to gradient information of each angular point block of the multiple angular point blocks, edge information of each angular point block generated by the generating module 503; and obtaining the confidence coefficient of the intersection point of each corner block according to the edge information of each corner block.

In an embodiment, the corner block processing module 504 may perform image differentiation on an image including a plurality of corner blocks, perform edge detection on the image of the plurality of corner blocks generated by the generating module 503, and obtain gradient information of each corner block through gradient calculation; and obtaining the edge information of each corner block according to the gradient information of each corner block. The edge information of the corner block obtained by the corner block processing module 504 is the real edges of the signboard in the corner blocks at the two sides of the signboard vertex, including the edges in the horizontal direction and the edges in the vertical direction.

In an embodiment, the corner block processing module 504 may obtain a confidence level of an intersection point of two sides of each corner block according to the intersection point of two sides of each corner block in the horizontal direction and the vertical direction, and the edge information of each corner block, obtained by the intersection point module 505. The confidence of the intersection point is the ratio of the total number of edge pixels contained on the two edges of the intersection point of the angular point blocks to the total number of pixels contained on the two edges of the intersection point of the angular point blocks.

In an embodiment, the corner block processing module 504 may employ different edge detection operators, and therefore, edge information of the corner block obtained by the corner block processing module 504 may be different, total numbers of edge pixels included in two edges of the intersection may be different, and confidence degrees of the intersection of the obtained corner blocks may also be different, and in this embodiment, the corner block processing module 504 determines the confidence degree of the intersection of the corner block by using the maximum confidence degree, the minimum confidence degree, or an average value of multiple confidence degrees.

A bounding box module 506, configured to, for the intersection point of each corner block obtained by the intersection point module 505, if the confidence of the intersection point obtained by the corner block processing module 504 is greater than the set threshold, obtain the bounding box of the signboard again according to the intersection point whose confidence is greater than the set threshold.

In one embodiment, if the confidence level of the intersection point of the corner block obtained by the corner block processing module 504 is greater than the set threshold, the bounding box module 506 re-determines the intersection point with the confidence level greater than the set threshold as the new vertex of the bounding box of the signboard. For the intersection point of each corner block obtained by the intersection point module 505, if the confidence level of the intersection point of each corner block obtained by the corner block processing module 504 is greater than the set threshold, the bounding box module 506 re-determines the intersection point whose confidence level is greater than the set threshold as the new multiple vertices of the bounding box of the signboard, so as to re-determine the multiple vertices of the bounding box of the signboard, and re-obtain the bounding box of the signboard.

A bounding box determining module 601, configured to determine that the bounding box of the retrieved signboard is the bounding box of the signboard if the bounding box of the signboard retrieved by the bounding box module 506 is consistent with the physical and geometric relationship of the bounding box of the signboard retrieved by the first obtaining module 501.

In one embodiment, if the bounding box of the signboard retrieved by the bounding box module 506 is consistent with the physical geometry of the bounding box of the signboard retrieved by the first retrieving module 501, the bounding box determining module 601 finally determines the bounding box of the signboard with the retrieved bounding box of the signboard. For example, assume that the bounding box of the signboard acquired by the first acquisition module 501 is rectangular. If the bounding box of the signboard retrieved by the bounding box module 506 is also rectangular, and the size of the bounding box of the retrieved signboard is approximately equal to the size of the bounding box of the signboard retrieved by the first obtaining module 501, the physical and geometric relationship between the bounding box of the signboard retrieved by the bounding box module 506 and the bounding box of the signboard retrieved by the first obtaining module 501 are consistent, and the bounding box determining module 601 finally determines the bounding box of the signboard with the bounding box of the retrieved signboard.

According to the technical scheme shown in the embodiment of the application, the confidence coefficient of the intersection point of each corner block is obtained according to the edge information of each corner block; for the intersection point of each angular point block, if the confidence coefficient of the intersection point is greater than the set threshold, the boundary frame of the signboard is obtained again according to the intersection point of which the confidence coefficient is greater than the set threshold; if the boundary frame of the newly acquired signboard is consistent with the physical geometric relationship of the acquired boundary frame of the signboard, the boundary frame of the newly acquired signboard is determined to be the boundary frame of the signboard, pixel-level optimization is performed on the boundary frame and the vertex of the signboard by utilizing the gradient information of the image, the physical geometric relationship of the boundary frame of the signboard and the confidence coefficient of the intersection point, so that the recognition errors caused by the situation that the signboard is too small and overlapped and other objects such as a rod or a frame shield the signboard can be avoided, the reliability of the optimization is ensured through the threshold limit of the confidence coefficient of the intersection point, the real boundary frame of the signboard can be accurately acquired, the accuracy of acquiring the vertex position of the signboard is improved, the accuracy of matching with a high-precision map is improved, and the positioning and stability of an automatic driving centimeter level are ensured.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Fig. 7 is a schematic structural diagram of an electronic device shown in an embodiment of the present application.

Referring to fig. 7, the electronic device 70 includes a memory 701 and a processor 702.

The Processor 702 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 701 may include various types of storage units, such as system memory, Read Only Memory (ROM), and permanent storage. The ROM may store, among other things, static data or instructions for the processor 702 or other modules of the computer. The persistent storage device may be a read-write storage device. The persistent storage may be a non-volatile storage device that does not lose stored instructions and data even after the computer is powered off. In some embodiments, the persistent storage device employs a mass storage device (e.g., magnetic or optical disk, flash memory) as the persistent storage device. In other embodiments, the permanent storage may be a removable storage device (e.g., floppy disk, optical drive). The system memory may be a read-write memory device or a volatile read-write memory device, such as a dynamic random access memory. The system memory may store instructions and data that some or all of the processors require at runtime. In addition, memory 701 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash, programmable read only memory), magnetic and/or optical disks, among others. In some embodiments, memory 701 may include a removable storage device that is readable and/or writable, such as a Compact Disc (CD), a digital versatile disc read only (e.g., DVD-ROM, dual layer DVD-ROM), a Blu-ray disc read only, an ultra-dense disc, a flash memory card (e.g., SD card, min SD card, Micro-SD card, etc.), a magnetic floppy disk, or the like. Computer-readable storage media do not contain carrier waves or transitory electronic signals transmitted by wireless or wired means.

The memory 701 has stored thereon executable code which, when processed by the processor 702, may cause the processor 702 to perform some or all of the methods described above.

Furthermore, the method according to the present application may also be implemented as a computer program or computer program product comprising computer program code instructions for performing some or all of the steps of the above-described method of the present application.

Alternatively, the present application may also be embodied as a non-transitory machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) having stored thereon executable code (or a computer program, or computer instruction code) which, when executed by a processor of an electronic device (or electronic device, server, etc.), causes the processor to perform some or all of the various steps of the above-described methods in accordance with the present application.

Having described embodiments of the present application, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.