1. A method for detecting a passable area based on a monocular camera is characterized by comprising the following steps:

acquiring an image acquired by a monocular camera;

performing feature extraction and feature matching on the image by using a monocular SLAM feature point method to obtain a three-dimensional feature point set;

determining a target area, wherein the target area is any one area on the image;

dividing the target area to obtain a plurality of sub-areas;

acquiring a neighboring feature point set of each sub-region by using the KD-Tree;

obtaining the nearest barrier distance of each sub-region by calculating the distance from each adjacent characteristic point in the adjacent characteristic point set to the corresponding sub-region;

obtaining a cost value and a visual cost map through a cost solver according to the nearest barrier distance of each sub-area;

obtaining a passable area according to the cost value and the visual cost map and by combining a preset cost threshold;

filtering the characteristic points of the passable area by using a random sampling consistency algorithm to obtain a ground point set;

obtaining a scale factor by using a scale factor solver according to the ground point set;

and correcting the scale of the passable area according to the scale factor, and performing projection transformation on the visual cost map to obtain a passable direction.

2. The method for detecting the passable area based on the monocular camera according to claim 1, wherein the step of obtaining the nearest obstacle distance of each sub-area by calculating the distance from each neighboring feature point in the set of neighboring feature points to the corresponding sub-area comprises:

calculating the distance from each adjacent characteristic point in the adjacent characteristic point set to the corresponding sub-area;

calculating to obtain an average distance according to the distance from each adjacent characteristic point in the adjacent characteristic point set to the corresponding sub-region;

according to the probability distribution density function of each adjacent characteristic point in the adjacent characteristic point set, a first distance is obtained through Gaussian kernel density estimation calculation;

and calculating the nearest barrier distance of each sub-region according to the average distance and the first distance.

3. The method for detecting the passable area based on the monocular camera of claim 2, wherein the average distance is calculated by the following formula:

in the formula (d)_aveRepresenting the mean distance, n representing the number of neighboring feature points, d_iAnd the distance from the ith adjacent characteristic point in the adjacent characteristic point set to the corresponding sub-area is represented.

4. The method for detecting the passable area based on the monocular camera of claim 2, wherein the first distance is calculated by the following formula:

in the formula (d)_kdeDenotes the first distance, peak_i(f_h(d) ) distribution density function f representing neighboring feature points_h(d) The ith wave crest value in the corresponding density distribution image;

wherein the content of the first and second substances,in the formula (f)_h(d) Distribution density function representing neighboring feature points, n representing the number of neighboring feature points, d_iRepresenting the distance from the ith neighboring feature point in the set of neighboring feature points to the corresponding sub-region, h tableIndicating the bandwidth, and d indicates the distance value to be calculated;

wherein the content of the first and second substances,wherein K (x) represents a Gaussian kernel function,

5. the method for detecting the passable area based on the monocular camera as recited in claim 2, wherein the calculating the nearest obstacle distance of each of the sub-areas according to the average distance and the first distance is performed by the following formula:

d＝α·d_kde+(1-α)·d_ave；

wherein d represents the nearest obstacle distance, d_kdeDenotes a first distance, d_aveDenotes the average distance and alpha denotes the weighting factor.

6. The method for detecting the passable area based on the monocular camera as claimed in claim 1, wherein the solving the cost value and the visual cost map by the cost solver according to the nearest obstacle distance of each sub-area is performed by the following formula:

wherein C represents the cost value of the subarea, d represents the nearest obstacle distance, and d^*Representing the radius of influence, n representing the number of neighboring feature points, n^*Representing the boundary value of the number of adjacent feature points, and eta represents the cost scale factor.

7. The method for detecting the passable area based on the monocular camera as recited in claim 1, wherein the step of obtaining the scale factor by using a scale factor solver according to the ground point set comprises:

calculating the ground height under the normalized scale according to the ground point set;

and calculating to obtain a scale factor according to the ground height and the preset monocular camera height.

8. The method for detecting the passable area based on the monocular camera of claim 7, wherein the calculating the scale factor according to the ground height and the preset monocular camera height is performed by the following formula:

in the formula (f)_scaleRepresenting the scale factor, h representing the preset monocular camera height, and h' representing the ground height.

9. A device for detecting a passable area based on a monocular camera, comprising:

at least one processor;

at least one memory for storing at least one program;

when executed by the at least one processor, cause the at least one processor to implement the method of any one of claims 1-8.

10. Computer-readable storage medium, on which a processor-executable program is stored, which, when being executed by a processor, is adapted to carry out the method according to any one of claims 1-8.

Background

The autonomous robot needs to perform operation and firstly meets the capability of obstacle avoidance navigation planning. At present, there are various solutions based on different sensors, such as laser scanners, inertial sensors, sonar, vision, etc. The navigation obstacle avoidance method based on laser always has the problems of less laser radar information, single dimension and the like, and particularly in other scenes needing semantic analysis, such as rescue scenes, pure laser SLAM is difficult to complete tasks. The scheme of multi-sensor combination is gradually raised in the aspect of robot navigation obstacle avoidance, and although combining different types of sensors, such as ultrasonic waves, vision, infrared, laser, vision and the like, can obtain more detailed information and accurate decision, the cost is increased, and the complexity of an algorithm is improved.

For a small automobile, the obstacle avoidance difficulty is very high in the flying process due to the fact that the effective load capable of being carried by the small automobile is limited. Due to the limited battery capacity, only lightweight sensors, such as monocular cameras, can be carried, which does not affect battery life and weight limitations. For autonomous robot motion, it is crucial that the mobile robot be able to determine its position relative to the potential obstacle.

The current monocular vision obstacle avoidance scheme and the corresponding defects are as follows:

1. in the scheme, depth estimation can obtain a good effect only by acquiring a data set in advance and training a large number of data sets in advance, and the good effect can be exerted in a specific scene, but the effect is difficult to be guaranteed under different environments, and the robustness is not strong enough;

2. and (4) assisting the robot to make obstacle avoidance decisions by utilizing reinforcement learning. In the scheme, the enhanced learning utilizes monocular RGB images to effectively learn how to avoid obstacles in the simulator, and the model trained in the virtual environment can be directly transferred to a real robot, so that the method can be well applied to various new environments; but the transportability is not strong enough, and the effect is difficult to be guaranteed in a real environment which has a larger difference with a virtual environment;

3. obstacle detection is performed using a priori knowledge of the scene or obstacle. For example, in pre-creating a structured environment that is differentiated by color, color cues are used to segment obstacles from non-obstacles. However, the method has little practical significance and can only be applied to a specific scene which is set up in advance;

4. and constraining the position of the camera on the robot platform and limiting the camera to only horizontally rotate to detect the ground plane, and then deducing homography constraint conditions of the ground plane by using the characteristic point information to distinguish ground points and non-ground points. In the scheme, the final obstacle avoidance effect can be influenced by the bumping of the trolley or the movement of the camera and the accuracy degree of the characteristic points.

Disclosure of Invention

The present invention is directed to solving at least one of the problems of the prior art. Therefore, the invention provides a method and a device for detecting a passable area based on a monocular camera and a storage medium.

The technical scheme adopted by the invention is as follows:

in one aspect, an embodiment of the present invention includes a method for detecting a passable area based on a monocular camera, including:

acquiring an image acquired by a monocular camera;

performing feature extraction and feature matching on the image by using a monocular SLAM feature point method to obtain a three-dimensional feature point set;

determining a target area, wherein the target area is any one area on the image;

dividing the target area to obtain a plurality of sub-areas;

acquiring a neighboring feature point set of each sub-region by using the KD-Tree;

obtaining the nearest barrier distance of each sub-region by calculating the distance from each adjacent characteristic point in the adjacent characteristic point set to the corresponding sub-region;

obtaining a cost value and a visual cost map through a cost solver according to the nearest barrier distance of each sub-area;

obtaining a passable area according to the cost value and the visual cost map and by combining a preset cost threshold;

filtering the characteristic points of the passable area by using a random sampling consistency algorithm to obtain a ground point set;

obtaining a scale factor by using a scale factor solver according to the ground point set;

and correcting the scale of the passable area according to the scale factor, and performing projection transformation on the visual cost map to obtain a passable direction.

Further, the step of obtaining the nearest obstacle distance of each sub-region by calculating the distance from each neighboring feature point in the set of neighboring feature points to the corresponding sub-region includes:

calculating the distance from each adjacent characteristic point in the adjacent characteristic point set to the corresponding sub-area;

calculating to obtain an average distance according to the distance from each adjacent characteristic point in the adjacent characteristic point set to the corresponding sub-region;

according to the probability distribution density function of each adjacent characteristic point in the adjacent characteristic point set, a first distance is obtained through Gaussian kernel density estimation calculation;

and calculating the nearest barrier distance of each sub-region according to the average distance and the first distance.

Further, the average distance is calculated by the following formula:

in the formula (d)_aveRepresenting the mean distance, n representing the number of neighboring feature points, d_iAnd the distance from the ith adjacent characteristic point in the adjacent characteristic point set to the corresponding sub-area is represented.

Further, the first distance is calculated by the following formula:

in the formula (d)_kdeDenotes the first distance, peak_i(f_h(d) ) distribution density function f representing neighboring feature points_h(d) The ith wave crest value in the corresponding density distribution image;

wherein the content of the first and second substances,in the formula (f)_h(d) Distribution density function representing neighboring feature points, n representing the number of neighboring feature points, d_iRepresenting the distance from the ith adjacent characteristic point in the adjacent characteristic point set to the corresponding sub-area, h representing the bandwidth, and d representing the distance value to be calculated;

wherein the content of the first and second substances,wherein K (x) represents a Gaussian kernel function,

further, the calculating the nearest obstacle distance of each sub-region according to the average distance and the first distance is performed by the following formula:

d＝α·d_kde+(1-α)·d_ave；

wherein d represents the nearest obstacle distance, d_kdeDenotes a first distance, d_aveDenotes the average distance and alpha denotes the weighting factor.

Further, the solving of the cost value and the visual cost map by the cost solver according to the nearest obstacle distance of each sub-region is performed by the following formula:

wherein C represents the cost value of the subarea, d represents the nearest obstacle distance, and d^*Representing the radius of influence, n representing the number of neighboring feature points, n^*Representing boundary values of the number of adjacent feature points, ηRepresenting a cost scale factor.

Further, the step of obtaining a scale factor by using a scale factor solver according to the ground point set includes:

calculating the ground height under the normalized scale according to the ground point set;

and calculating to obtain a scale factor according to the ground height and the preset monocular camera height.

Further, the calculation of the scale factor according to the ground height and the preset monocular camera height is performed by the following formula:

in the formula (f)_scaleRepresenting the scale factor, h representing the preset monocular camera height, and h' representing the ground height.

On the other hand, the embodiment of the invention also comprises a passable area detection device based on the monocular camera, which comprises:

at least one processor;

at least one memory for storing at least one program;

when the at least one program is executed by the at least one processor, the at least one processor is caused to implement the monocular camera-based passable area detecting method.

In another aspect, the embodiment of the present invention further includes a computer-readable storage medium, on which a processor-executable program is stored, where the processor-executable program is used to implement the method for detecting a passable area based on a monocular camera when being executed by a processor.

The invention has the beneficial effects that:

the method comprises the steps of performing feature extraction and feature matching on an image acquired by a monocular camera by utilizing a monocular SLAM feature point method to obtain a three-dimensional feature point set; determining a target area, and segmenting the target area to obtain a plurality of sub-areas; then, acquiring a neighboring feature point set of each sub-region by using the KD-Tree; the distance from each adjacent characteristic point in each adjacent characteristic point set to the corresponding sub-area is calculated, and the nearest barrier distance of each sub-area is obtained; then, solving a cost value and a visual cost map through a cost solver, and combining a preset cost threshold value to obtain a passable area; filtering the characteristic points of the passable area by using a random sampling consistency algorithm to obtain a ground point set, and obtaining a scale factor by using a scale factor solver; and finally, correcting the scale of the passable area according to the scale factor, and performing projection transformation on the visual cost map to obtain the passable direction. According to the invention, the nearest barrier distance is obtained through calculation, the passable area is obtained through a cost solver, namely, the detection task of the passable area can be realized only by a low-cost monocular camera, and meanwhile, the cost value in the passable area is obtained through calculation, so that the path planning module of the following trolley is more convenient; the scale factor obtained by calculation corrects the scale of the passable area, and the problem of inconsistent scale in the monocular SLAM can be solved.

Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.

Drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

fig. 1 is a flowchart illustrating steps of a method for detecting a passable area based on a monocular camera according to an embodiment of the present invention;

fig. 2 is an overall flow chart of the method for detecting a passable area based on a monocular camera according to the embodiment of the present invention;

FIG. 3 is a schematic diagram illustrating distance calculations according to an embodiment of the present invention;

FIG. 4 is a diagram illustrating obstacle distances calculated in different ways according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of cost propagation according to an embodiment of the present invention;

FIG. 6 is a schematic diagram illustrating the solution of the scale factors according to the embodiment of the present invention;

fig. 7 is a schematic structural diagram of a trafficable area detection device based on a monocular camera according to an embodiment of the present invention.

Detailed Description

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.

In the description of the present invention, it should be understood that the orientation or positional relationship referred to in the description of the orientation, such as the upper, lower, front, rear, left, right, etc., is based on the orientation or positional relationship shown in the drawings, and is only for convenience of description and simplification of description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus, should not be construed as limiting the present invention.

In the description of the present invention, the meaning of a plurality of means is one or more, the meaning of a plurality of means is two or more, and larger, smaller, larger, etc. are understood as excluding the number, and larger, smaller, inner, etc. are understood as including the number. If the first and second are described for the purpose of distinguishing technical features, they are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.

In the description of the present invention, unless otherwise explicitly limited, terms such as arrangement, installation, connection and the like should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above terms in the present invention in combination with the specific contents of the technical solutions.

The embodiments of the present application will be further explained with reference to the drawings.

Referring to fig. 1, an embodiment of the present invention provides a method for detecting a passable area based on a monocular camera, including but not limited to the following steps:

s100, acquiring an image acquired by a monocular camera;

s200, performing feature extraction and feature matching on the image by using a monocular SLAM feature point method to obtain a three-dimensional feature point set;

s300, determining a target area, wherein the target area is any one area on the image;

s400, segmenting a target area to obtain a plurality of sub-areas;

s500, acquiring a neighboring feature point set of each sub-area by using the KD-Tree;

s600, obtaining the nearest barrier distance of each sub-region by calculating the distance from each adjacent characteristic point in each adjacent characteristic point set to the corresponding sub-region;

s700, solving a cost value and a visual cost map through a cost solver according to the nearest barrier distance of each sub-area;

s800, obtaining a passable area according to the cost value and the visual cost map and by combining a preset cost threshold;

s900, filtering the characteristic points of the passable area by using a random sampling consistency algorithm to obtain a ground point set;

s1000, obtaining a scale factor by using a scale factor solver according to the ground point set;

and S1100, correcting the scale of the passable area according to the scale factor, and performing projection transformation on the visual cost map to obtain a passable direction.

In the embodiment, in order to stably and robustly realize the passable area detection of the pure vision sensor trolley, an area division algorithm based on a monocular SLAM feature point map is provided. Calculating by using a monocular SLAM characteristic point method to obtain a three-dimensional characteristic point set, calculating the minimum distance between the current position and an adjacent characteristic point set by using a barrier distance solver in the visual field range of the camera, putting the obtained nearest barrier distance into a cost solver to calculate to obtain a corresponding cost value, and finally obtaining a passable direction by carrying out reprojection transformation on a visual cost map; meanwhile, in order to solve the problem of inconsistent scales in monocular SLAM, certain constraint is carried out on the height of a camera and a rotating plane, a scale factor is obtained through calculation, and the scales of the passable area are corrected through the scale factor; the embodiment of the invention can realize the detection task of the passable area only by a low-cost monocular camera and can be conveniently transplanted to light-weight mobile equipment. In addition, different from the conventional region detection algorithm based on deep learning or point cloud segmentation, the cost value in the passable region is obtained through calculation, and the subsequent path planning task of the trolley is more conveniently performed.

Referring to fig. 2, in this embodiment, in step S200, after the image acquired by the monocular camera is acquired, feature extraction and feature matching are performed on the image by using the monocular SLAM feature point method, so as to obtain a three-dimensional feature point setSpecifically, calibrating a monocular camera to obtain an internal reference matrix and distortion parameters of the monocular camera, extracting ORB feature points from an image according to a time sequence and obtaining an ORB descriptor, then performing feature point matching on the image to obtain image feature points, finally solving the motion of the camera between the images according to pole-pair constraint, and calculating the three-dimensional coordinates of the image feature points by utilizing triangulation measurement to obtain a three-dimensional feature point set

Regarding step S300 and step S400, in the present embodiment, the front region space S in the current image may be used as the target region, and then the front region space S may be uniformly divided into a plurality of fine sub-regions (S)₁,s₂,...,s_m,). After the segmentation is completed, step S500 is performed, that is, each sub-region is searched by using KD-TreeNearby K adjacent feature point sets For example, the target areas are allObtaining n cube small blocks after uniform segmentation, wherein one cube small block represents a sub-region, searching all adjacent characteristic points of the first cube small block accessory through KD-Tree, and recording the set of all the adjacent characteristic points as a first adjacent characteristic point set; similarly, all the adjacent feature points of the second cube small block accessory are searched through the KD-Tree, and the set of all the adjacent feature points is recorded as a second adjacent feature point set; and searching all the adjacent characteristic points near the third cube small block through the KD-Tree, and recording the set of all the adjacent characteristic points as a third adjacent characteristic point set. It is assumed that the first neighboring feature point set contains a neighboring feature points, the second neighboring feature point set contains b neighboring feature points, and the third neighboring feature point set contains c neighboring feature points. And traversing each cube small block (sub-region), and searching and obtaining an adjacent characteristic point set of each cube small block through KD-Tree.

In this embodiment, after the neighboring feature point set of each sub-region is obtained by the KD-Tree, step S600 is further performed, that is, the step of obtaining the nearest obstacle distance of each sub-region by calculating the distance from each neighboring feature point in each neighboring feature point set to the corresponding sub-region includes the following sub-steps:

s601, calculating the distance from each adjacent characteristic point in each adjacent characteristic point set to the corresponding sub-area;

s602, calculating to obtain an average distance according to the distance from each adjacent characteristic point in each adjacent characteristic point set to the corresponding sub-area;

s603, calculating to obtain a first distance through Gaussian kernel density estimation according to the probability distribution density function of each adjacent feature point in each adjacent feature point set;

and S604, calculating the nearest barrier distance of each sub-region according to the average distance and the first distance.

Specifically, the average distance is calculated by the following formula:

in the formula (d)_aveRepresenting the mean distance, n representing the number of neighboring feature points, d_iAnd the distance from the ith adjacent characteristic point in the adjacent characteristic point set to the corresponding sub-area is represented.

The first distance is calculated by the following formula:

in the formula (d)_kdeDenotes the first distance, peak_i(f_h(d) ) distribution density function f representing neighboring feature points_h(d) The ith wave crest value in the corresponding density distribution image;

wherein the content of the first and second substances,in the formula (f)_h(d) Distribution density function representing neighboring feature points, n representing the number of neighboring feature points, d_iRepresenting the distance from the ith adjacent characteristic point in the adjacent characteristic point set to the corresponding sub-area, h representing the bandwidth, and d representing the distance value to be calculated;

wherein the content of the first and second substances,wherein K (x) represents a Gaussian kernel function,

the nearest-obstacle distance is performed by the following formula:

d＝α·d_kde+(1-α)·d_ave；

wherein d represents the nearest obstacle distance, d_kdeDenotes a first distance, d_aveDenotes the average distance and alpha denotes the weighting factor.

In this embodiment, it is considered that if the average distance is directly used for calculation, the difference between different positions cannot be well expressed, and thus the accurate nearest obstacle distance cannot be obtainedAnd (5) separating. The kernel density estimation generally solves the problem of distribution density of random variables, so that the present embodiment adopts gaussian kernel density estimation to obtain the distribution of neighboring feature points near the sub-region, and performs the nearest obstacle distance solution by combining with the average distance. Similarly, taking the first cube small block (sub-region) in the target region as an example, assuming that all neighboring feature points near the first cube small block are searched through the KD-Tree, and the set of all the neighboring feature points is recorded as a first neighboring feature point set; similarly, all the adjacent feature points of the second cube small block accessory are searched through the KD-Tree, and the set of all the adjacent feature points is recorded as a second adjacent feature point set; and searching all the adjacent characteristic points near the third cube small block through the KD-Tree, and recording the set of all the adjacent characteristic points as a third adjacent characteristic point set. It is assumed that the first neighboring feature point set contains a neighboring feature points, the second neighboring feature point set contains b neighboring feature points, and the third neighboring feature point set contains c neighboring feature points. Calculating the distance from a adjacent characteristic points in the first adjacent characteristic point set to the first cube small block to obtain a distances, and obtaining a distances through a formulaAveraging the a distances, the average distance from the a neighboring feature points in the first neighboring feature point set to the first cube patch can be obtained. Then, the distribution condition of a adjacent characteristic points in the first adjacent characteristic point set is obtained by adopting Gaussian kernel density estimation, and according to the distribution condition, a formula is used forA first distance, namely a first peak distance, can be obtained; that is, the shortest distance from a neighboring feature points in the first set of neighboring feature points to the first cube patch is calculated by gaussian kernel density estimation. After obtaining these two distances, the equation d ═ α · d_kde+(1-α)·d_aveThe nearest obstacle distance of the first cube patch (first sub-region) is calculated. Similarly, b neighboring feature point arrivals in the second set of neighboring feature points are calculatedB distances can be obtained from the distance of the second cube small block through a formulaAveraging the b distances can obtain the average distance from the b adjacent feature points in the second adjacent feature point set to the second cube patch. Then, the distribution condition of b adjacent characteristic points in the second adjacent characteristic point set is obtained by adopting Gaussian kernel density estimation, and according to the distribution condition, a formula is used forThe first distance, that is, the shortest distance from b neighboring feature points in the second set of neighboring feature points to the second cube patch calculated by gaussian kernel density estimation, can be obtained. After obtaining these two distances, the equation d ═ α · d_kde+(1-α)·d_aveThe nearest obstacle distance for the second cube patch (second sub-region) is calculated. Similarly, the c distances from the c adjacent feature points in the third adjacent feature point set to the third cube small block are calculated, and the c distances are obtained through a formulaAveraging the c distances can obtain the average distance from the c neighboring feature points in the third neighboring feature point set to the third cube patch. Then, the distribution situation of c adjacent characteristic points in the third adjacent characteristic point set is obtained by adopting Gaussian kernel density estimation, and according to the distribution situation, a formula is used forObtaining a first distance, namely the shortest distance from c adjacent feature points in the third adjacent feature point set to the third cube patch, which is obtained through Gaussian kernel density estimation calculation; after obtaining these two distances, the equation d ═ α · d_kde+(1-α)·d_aveThe nearest obstacle distance of the third cube patch (third sub-region) is calculated. Each sub-area in the target area is calculated according to the calculation mode, and then the calculation is carried outThe nearest barrier distance for each sub-region is obtained.

Specifically, referring to fig. 3, (a) in fig. 3 represents a case of currently calculating a distance value of the sub-region s in the vicinity of the feature point, and (b) in fig. 3 represents a distribution case obtained by using gaussian kernel density estimation.

As can be seen from FIG. 3(b), there are two feature point clusters x near the current calculation sub-region s₁,x₂Respectively represent two obstacles O₁,O₂In order to obtain the distance information of the obstacle closest to the current calculation sub-region s, the first slope peak f (x) is taken₁) Distance x of₁Nearest obstacle distance d as current calculation sub-region s_sBy the method, a simple clustering function can be realized, and the barrier at different positions can be distinguished and processed. This example selects x in FIG. 3(b)₁And 0.3 is taken as the nearest obstacle distance value in the current scene.

Referring to FIG. 4, FIG. 4(a) shows the use of equationsThe calculated obstacle distance value is shown in fig. 4(b) using the formula d ═ α · d_kde+(1-α)·d_aveThe calculated obstacle distance value is shown in FIG. 4(c) using the formulaThe calculated average distance to the obstacle is shown in fig. 4(d) as a standard deviation of the distance.

As shown in FIGS. 4(a) and (d), the data variance in the middle of the road (at the position of 0 horizontal offset) is small, the values are relatively close, the data of the first slope peak directly taken as the distance amount has large fluctuation and error, and d is directly used_kdeOr d_aveThe principle that the distance between obstacles in the middle of the road is small and the distance between obstacles on two sides of the road is large cannot be well met, and d is used in the embodiment_kdeAnd d_aveIn combination, d is increased when the variance is smaller_aveFinally, d is obtained as shown in fig. 4(b), and this problem is solved well.

In this embodiment, after the closest obstacle distance of each sub-area is obtained through calculation, that is, after the closest obstacle distance information of each position is obtained, in order to reduce the probability that the robot collides with an obstacle and facilitate subsequent robots to perform path planning, the cost value of the area closer to the obstacle should be much higher than the cost value of the area farther from the obstacle. Therefore, in the embodiment, a cost function in the artificial potential field method is modified to a certain extent by using a common obstacle avoidance technology (artificial potential field method) in path planning. In addition, there are two problems in performing cost solution based on the feature points, one is that the feature point information on the back of the obstacle is lost, and if the cost calculation is performed directly, the algorithm regards the back of the obstacle as an idle area, and for the problem, a cost propagation mechanism is introduced in the embodiment; secondly, the number of the feature points near the center position of the free area is small, the distance information obtained by calculation may have a large error, if the distance information is not corrected, the cost value of the center free area is high, which is not logical, although the problem can be solved by increasing the KD-Tree search range, the time complexity is increased, and the real-time performance is reduced. In order to ensure real-time performance and avoid excessive errors caused by a few adjacent feature points, the embodiment tends to introduce a quantity critical value, and if the quantity of the adjacent feature points is too small, it indicates that the current area is too far away from the obstacle, and the cost value should be reduced appropriately.

Specifically, through testing, the cost solver formula is as follows:

wherein C represents the cost value of the subarea, d represents the nearest obstacle distance, and d^*Representing the radius of influence, n representing the number of neighboring feature points, n^*Representing the boundary value of the number of adjacent feature points, and eta represents the cost scale factor. In this embodiment, the radius d will be affected^*Set to 1.0, adjacent feature point number boundary value n^*Set to 100 and the cost scale factor η to 0.1.

Referring to fig. 5, in this embodiment, in order to avoid a situation where a passable region calculation is incorrect due to the feature point not being detected on the back of the obstacle, a cost propagation step is added in this embodiment, as shown in fig. 5, a ray is emitted from the camera position (origin) to a potential region of the obstacle (region with a large cost value), and the cost value of the obstacle is propagated to the back region.

In this embodiment, the distance information d of the obstacle is determined_sAnd substituting S belonging to S into a cost solver to obtain a cost value and a visual cost map, and then dividing the space by combining a preset cost threshold value to obtain a passable area. In the embodiment, the problem of inconsistent scales in the monocular SLAM is considered, and therefore, a scale factor needs to be further acquired to correct the scale of the obtained passable area. Specifically, the feature points of the passable area are filtered by using a random sampling consistency algorithm to obtain a ground point set. And then, according to the ground point set, a scale factor solver is used for obtaining a scale factor. In step S1000, that is, the step of obtaining the scale factor by using the scale factor solver according to the ground point set specifically includes:

s1001, calculating the ground height under the normalized scale according to the ground point set;

and S1002, calculating to obtain a scale factor according to the ground height and the preset monocular camera height.

Specifically, in this embodiment, according to the ground height and the preset monocular camera height, the calculation of the scale factor is performed by the following formula:

in the formula (f)_scaleRepresenting the scale factor, h representing the preset monocular camera height, and h' representing the ground height.

Referring to fig. 6, the present embodiment assumes that the monocular camera has a certain height and only performs horizontal plane rotation. Firstly, the obtained passable area and a random sample consensus (RANSAC) algorithm are used for filtering the feature points to obtain a ground point set. The ground height at the normalized scale, h' in FIG. 6, is then calculated, and the scale factor is then calculated using the preset camera height hFinally, using the scale factor f_scaleEstimating the depth of the obstacle and recovering the real size of the characteristic point; i.e. to correct the dimensions of the passable area. And finally, performing projection transformation on the visual cost map to obtain the passable direction of the passable area.

The method for detecting the passable area based on the monocular camera has the following technical effects:

the embodiment of the invention utilizes a monocular SLAM characteristic point method to carry out characteristic extraction and characteristic matching on the image collected by the monocular camera to obtain a three-dimensional characteristic point set; determining a target area, and segmenting the target area to obtain a plurality of sub-areas; then, acquiring a neighboring feature point set of each sub-region by using the KD-Tree; the distance from each adjacent characteristic point in each adjacent characteristic point set to the corresponding sub-area is calculated, and the nearest barrier distance of each sub-area is obtained; then, solving a cost value and a visual cost map through a cost solver, and combining a preset cost threshold value to obtain a passable area; filtering the characteristic points of the passable area by using a random sampling consistency algorithm to obtain a ground point set, and obtaining a scale factor by using a scale factor solver; and finally, correcting the scale of the passable area according to the scale factor, and performing projection transformation on the visual cost map to obtain the passable direction. According to the invention, the nearest barrier distance is obtained through calculation, the passable area is obtained through a cost solver, namely, the detection task of the passable area can be realized only by a low-cost monocular camera, and meanwhile, the cost value in the passable area is obtained through calculation, so that the path planning module of the following trolley is more convenient; the scale factor obtained by calculation corrects the scale of the passable area, and the problem of inconsistent scale in the monocular SLAM can be solved.

Referring to fig. 7, an embodiment of the present invention further provides a device 200 for detecting a passable area based on a monocular camera, which specifically includes:

at least one processor 210;

at least one memory 220 for storing at least one program;

when the at least one program is executed by the at least one processor 210, the at least one processor 210 is caused to implement the method as shown in fig. 1.

The memory 220, which is a non-transitory computer-readable storage medium, may be used to store non-transitory software programs and non-transitory computer-executable programs. The memory 220 may include high speed random access memory and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 220 may optionally include remote memory located remotely from processor 210, and such remote memory may be connected to processor 210 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

It will be understood that the device structure shown in fig. 7 does not constitute a limitation of device 200, and may include more or fewer components than shown, or some components may be combined, or a different arrangement of components.

In the apparatus 200 shown in fig. 7, the processor 210 may retrieve the program stored in the memory 220 and execute, but is not limited to, the steps of the embodiment shown in fig. 1.

The above-described embodiments of the apparatus 200 are merely illustrative, and the units illustrated as separate components may or may not be physically separate, may be located in one place, or may be distributed over a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purposes of the embodiments.

Embodiments of the present invention also provide a computer-readable storage medium, which stores a program executable by a processor, and the program executable by the processor is used for implementing the method shown in fig. 1 when being executed by the processor.

The embodiment of the application also discloses a computer program product or a computer program, which comprises computer instructions, and the computer instructions are stored in a computer readable storage medium. The computer instructions may be read by a processor of a computer device from a computer-readable storage medium, and executed by the processor to cause the computer device to perform the method illustrated in fig. 1.

It will be understood that all or some of the steps, systems of methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.

The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the gist of the present invention.