1. A method for counting and positioning reinforcing steel bar heads is characterized by comprising the following steps:

s11: extracting initial features of an original image;

s12: fusing the primary features of different levels to obtain fused features;

s13: performing density map estimation operation on the fusion features to obtain a density map;

s14: performing integral summation on the density map to obtain the number of the final reinforcing steel bar heads;

s15: and estimating the position of the steel bar head by using the density map.

2. The tendon head counting and positioning method according to claim 1, wherein the S15 further comprises: and processing the density map by using a local maximum method, and estimating the position of the steel bar head.

3. The method for counting and positioning the tendon tips as claimed in claim 2, wherein the step of S15 using the local maximum method further comprises:

s31: traversing a Chinese character 'tian' shape, finding the maximum value of three rows and three columns of the Chinese character 'tian' shape, judging whether the Chinese character 'tian' shape is a local maximum value, if so, returning to the position of the Chinese character 'tian' shape, and if not, recording and finding the coordinate of the maximum value in the adjacent four points;

s32: continuously comparing the next field character through the quadrant reduction range of the coordinates obtained in the S11;

s33: if the maximum value can not be found, finally, when the field character becomes a rectangle of 3 multiplied by 3, the search is completed.

4. The method for counting and positioning the tendon tips as claimed in claim 2, wherein before using the local maximum in S15, the method further comprises: filtering out the density smaller than the empirical value by adopting a pretreatment mode; and/or the presence of a gas in the gas,

before the local maximum is utilized in S15, the method further includes: and smoothing the density spectrum of the density map by adopting a preprocessing mode.

5. The method for counting and positioning the tendon tips as claimed in claim 2, wherein the step of S15, after the step of using the local maximum method, further comprises: and removing repeated block diagrams by using a non-maximum value inhibition method to obtain a final positioning frame of the steel bar head.

6. The utility model provides a reinforcing bar head count, positioning system which characterized in that includes: the device comprises a shallow layer feature extraction module, a feature fusion module, a density map estimation module, an integral summation module and a positioning module; wherein the content of the first and second substances,

the shallow feature extraction module is used for extracting preliminary features of the original image;

the density map estimation module is used for carrying out density map estimation operation on the fusion features to obtain a density map;

the integral summation module is used for carrying out integral summation on the density map to obtain the number of the final reinforcing steel bar heads;

the positioning module is used for estimating the position of the steel bar head by utilizing the density map.

7. The system of claim 6, wherein the positioning module is further configured to estimate the position of the steel bar head by processing the density map using a local maximum method.

8. The tendon head counting and positioning system of claim 7, further comprising: a preprocessing module for filtering out less than empirical densities in the density map and/or smoothing a density spectrum of the density map prior to a local maximum method in the positioning module.

9. The tendon head counting and positioning system of claim 7, wherein the positioning module is further configured to remove the repeated frame after the local maximum method by using a non-maximum suppression method to obtain a final positioning frame of the tendon head.

10. The tendon head counting and positioning system of claim 6, further comprising: and the detection network model module is used for constructing a detection network model trained end to end by utilizing the loss sum of the shallow feature extraction module, the feature fusion module and the density map estimation module, and training the whole detection network model module by utilizing the loss sum.

Background

Object detection is now widely used in many practical applications, such as automotive driving, robot vision. Dense object detection counting is a research topic in the field of object detection. The method is characterized in that the number of targets needing to be predicted is confirmed by means of computer technology through a method of analyzing a static picture or a video frame to analyze a density spectrum of the static picture or the video frame without using an anchor which is commonly used for target detection. In the industrial field, dense target detection counting plays an important role, and at present, industrial products such as: the quantity of the steel bars and the screws needs to be confirmed manually, a large amount of manpower and time cost are consumed, the process can be simplified by means of intensive target detection counting, and the efficiency and the precision are improved. In the safety field, the frequent trampling events in large-scale activities at home and abroad have already caused considerable casualties, such as the trampling event on overseas beaches in 2015, which has reached the serious casualty accident level specified in China. The research on the dense target detection counting problem can effectively reduce or avoid the occurrence of the events by accurately estimating the dense target density of the current scene and arranging corresponding security measures. Meanwhile, the dense target detection counting also has important commercial and market values and is the leading research field of the big data era. Compared with the traditional manual counting, the dense target detection counting has rapidness, accuracy and easy processing, is suitable for popular large-scale calculation nowadays, and obtains a model with good accuracy and robustness. Traditional dense target counting algorithms mainly fall into two broad categories: one is a detection based approach. Early intensive target studies focused primarily on detection-based methods. And detecting dense objects in the scene by using a sliding window detector, and counting the number of corresponding objects. Detection-based methods are mainly divided into two broad categories, one is detection based on the whole body, and the other is detection based on partial targets. The ensemble-based detection method, for example, a typical conventional method, mainly trains a classifier to detect an object by using features such as wavelets, HOG, edges, and the like extracted from the ensemble of the object. The learning algorithm mainly comprises methods such as SVM, boosting and random forest. The method based on the overall detection is mainly suitable for sparse target counting, and shielding between targets becomes more and more serious along with the increase of the density of dense targets. Taking dense population counting as an example, a partial target detection based approach is used to deal with the dense target counting problem.

Dense object detection counting studies in images such as head, shoulders, etc. count the number of dense objects. This method is slightly more efficient than the overall-based detection. The second is a regression-based approach. In any method based on detection, it is difficult to deal with the problem of severe occlusion between dense objects. Therefore, regression-based methods are increasingly being used to solve the problem of dense object counting. The main idea is to use a regression-based approach by learning a mapping of features to the number of dense objects. The method mainly comprises two steps, wherein in the first step, low-level features such as foreground features, edge features, textures and gradient features are extracted; the second step is to learn a regression model, such as linear regression, piecewise linear regression, ridge regression, and Gaussian process regression, to learn the mapping of the low-level features to the dense target numbers. Unlike the traditional detection and regression-based method, for dense target areas in the image, a better prediction result is obtained by using a density map prediction (DensityMap) method. Due to the fact that target density distribution in an image is extremely uneven, researchers utilize a Convolutional Neural Network (CNN) of a Multi-array (Multi-Column) to extract target features of different scales. The model using the network has more parameters and large calculation amount, and cannot carry out real-time dense target counting prediction.

Through retrieval, the invention patent with the Chinese patent application number of 202011355672.6 provides a steel bar identification system, a steel bar identification method and a steel bar counting acceptance system, which aim to improve and solve the defects and problems of the existing steel bar incoming number acceptance work completed by the traditional manual counting method. However, the implementation complexity of the patent is relatively high, and the calculation amount is large.

Disclosure of Invention

Aiming at the problems in the prior art, the invention provides a reinforcing steel bar head counting and positioning method and a system, which apply an intensive counting method in deep learning to the counting of reinforcing steel bars in industrial production, can provide the positions of the reinforcing steel bars while counting, replace the work of workers in the prior industrial production and improve the production efficiency.

In order to solve the technical problems, the invention is realized by the following technical scheme:

the invention provides a method for counting and positioning reinforcing steel bar heads, which comprises the following steps:

s11: extracting initial features of an original image;

s12: fusing the primary features of different levels to obtain fused features;

s13: performing density map estimation operation on the fusion features to obtain a density map;

s14: performing integral summation on the density map to obtain the number of the final reinforcing steel bar heads;

s15: estimating the position of a steel bar head by using the density map;

the S14 and S15 are not in sequence, and can be performed simultaneously.

Preferably, the models used in S11 and S12 are based on the method of generating density maps in MCNN.

Preferably, the S15 further includes: and processing the density map by using a local maximum method, and estimating the position of the steel bar head.

Preferably, the step of S15 using the local maximum method further includes:

s31: traversing a Chinese character 'tian' shape, finding the maximum value of three rows and three columns of the Chinese character 'tian' shape, judging whether the Chinese character 'tian' shape is a local maximum value, if so, returning to the position of the Chinese character 'tian' shape, and if not, recording and finding the coordinate of the maximum value in the adjacent four points;

s32: continuously comparing the next field character through the quadrant reduction range of the coordinates obtained in the S11;

s33: if the maximum value can not be found, finally, when the field character becomes a rectangle of 3 multiplied by 3, the search is completed.

Preferably, before the local maximum is utilized in S15, the method further includes: and filtering out the density smaller than the empirical value by adopting a preprocessing mode.

Preferably, before the local maximum is utilized in S15, the method further includes: and smoothing the density spectrum of the density map by adopting a preprocessing mode.

Preferably, the step S15, after the local maximum method, further includes: and removing repeated block diagrams by using a non-maximum value inhibition method to obtain a final positioning frame of the steel bar head.

The invention also provides a reinforcing steel bar head counting and positioning system, which comprises: the device comprises a shallow layer feature extraction module, a feature fusion module, a density map estimation module, an integral summation module and a positioning module; wherein the content of the first and second substances,

the shallow feature extraction module is used for extracting preliminary features of the original image;

the characteristic fusion module is used for fusing the preliminary characteristics of different levels of levels to obtain fusion characteristics;

the density map estimation module is used for carrying out density map estimation operation on the fusion features to obtain a density map;

the integral summation module is used for carrying out integral summation on the density map to obtain the number of the final reinforcing steel bar heads;

the positioning module is used for estimating the position of the steel bar head by utilizing the density map.

Preferably, the model used in the shallow feature extraction module and the feature fusion module is a density map generated based on MCNN.

Preferably, the positioning module is further configured to process the density map by using a local maximum method, and estimate a position of a steel bar head.

Preferably, the method further comprises the following steps: a preprocessing module for filtering out less than empirical densities in the density map and/or smoothing a density spectrum of the density map prior to a local maximum method in the positioning module.

Preferably, the positioning module is further configured to remove the repeated frame by using a non-maximum suppression method after the local maximum method to obtain a final positioning frame of the steel bar head.

Preferably, the method further comprises the following steps: and the detection network model module is used for constructing a detection network model trained end to end by utilizing the loss sum of the shallow feature extraction module, the feature fusion module and the density map estimation module, and training the whole detection network model module by utilizing the loss sum.

Compared with the prior art, the invention has the following advantages:

(1) according to the method and the system for counting and positioning the reinforcing steel bar heads, the intensive counting method in the deep learning is applied to the counting of the reinforcing steel bars in the industrial production, the positions of the reinforcing steel bars can be provided while the counting is carried out, a good visual effect is provided, the work of workers in the previous industrial production is replaced, the risk is reduced, and the production efficiency is improved;

(2) according to the method and the system for counting and positioning the reinforcing steel bar heads, the positioning is carried out by a local maximum searching method of the shape of Chinese character tian, so that the time complexity is reduced, and the calculated amount is small;

(3) according to the method and the system for counting and positioning the reinforcing steel bar heads, the density smaller than the empirical value is filtered in a preprocessing mode, so that the influence of noise can be eliminated, and the positioning is more accurate;

(4) according to the method and the system for counting and positioning the reinforcing steel bar heads, the density spectrum is subjected to smoothing treatment in a preprocessing mode, the problem that too many possible points of the target center appear when the peak value is searched can be effectively solved, and the positioning is more accurate.

Drawings

Embodiments of the invention are further described below with reference to the accompanying drawings:

fig. 1 is a flowchart of a method for counting and positioning a tendon head according to an embodiment of the present invention;

FIG. 2 is a diagram of a multi-tasking cascaded neural network according to one embodiment of the invention;

FIG. 3 is a flow chart of a local maximum method according to an embodiment of the present invention;

FIG. 4 is a diagram of the structure and parameters of a smoothing filter according to an embodiment of the present invention;

fig. 5 is a schematic structural view of a tendon head counting and positioning system according to an embodiment of the present invention.

Description of reference numerals: the method comprises the following steps of 1-shallow feature extraction module, 2-feature fusion module, 3-density map estimation module, 4-integral summation module and 5-positioning module.

Detailed Description

The following examples are given for the detailed implementation and specific operation of the present invention, but the scope of the present invention is not limited to the following examples.

Fig. 1 is a flowchart illustrating a method for counting and positioning a tendon head according to an embodiment of the present invention.

Referring to fig. 1, the method for counting and positioning the reinforcing steel bar heads of the present embodiment includes:

s11: extracting initial features of an original image;

s12: fusing the primary features of different levels to obtain fused features;

s13: performing density map estimation operation on the fusion features to obtain a density map;

s14: performing integral summation on the density map to obtain the number of the final reinforcing steel bar heads;

s15: and estimating the position of the steel bar head by using the density map.

The above-mentioned S14 and S15 may be performed simultaneously, not in a sequential order.

In one embodiment, the structure of the multitask cascade neural network adopted in S11, S12, and S13 is shown in fig. 2, where the parameters of each layer are as follows:

convolutional layer 1(conv 1): the 16 feature filter size is 9x 9;

convolutional layer 2(conv 2): the 32 feature filter size is 7x 7;

maximize pool 1 (maxporoling 1) 20 eigenfilter sizes 7x7 here double down-sampled;

maximize pool 2 (maxporoling 2) 40 feature filter sizes 5x5 here four times downsampled;

maximize pool 3 (maxporoling 3) 16 eigenfilter sizes 9x9 here double down-sampled;

maximize pool 4 (maxporoling 4) 32 eigenfilter sizes 7x7 here four times downsampled;

convolutional layer 3(conv 1): the 20 feature filter size is 5x 5;

convolutional layer 4(conv 2): the 10 feature filter size is 5x 5;

convolutional layer 5(conv 1): the 16 feature filter size is 7x 7;

convolutional layer 6(conv 2): the 8 feature filter size is 7x 7;

convolutional layer 7(conv 1): the 24 feature filter size is 3x 3;

convolutional layer 8(conv 2): the 32 feature filter size is 3x 3;

segmented stride deconvolution 1(FSC1) 16 features;

segmentation step deconvolution 2(FSC2) 8 features.

By adopting the network structure, shallow feature extraction is firstly carried out on an input image, and a convolution layer with the size of 9x9 and the number of 16 feature numbers is connected with a convolution network with the size of 7x7 feature numbers 32. After the shallow layer features are preliminarily extracted, a structure combining detail features and density level detection features in a cascade network structure is used for reference, one channel is adopted for extracting global features, the other channel is adopted for extracting detail features, the global features are convoluted by a filter with a large scale, the first two layers are maximized pools, and the scales are respectively 9x9 and 7x 7. The last two layers are connected for global feature extraction convolutional layers, and the filter size is set to be 7x7 which is large. Another approach to extract detail information is to first take the small scale filters 7x7 and 5x5 with the maximized pool, and then take the filter size of 5x5 for more local feature extraction. After feature extraction is completed on two channels, two layers containing global information and detail information are combined into a fused feature layer, a filter with the size of 3x3 is uniformly used for further extracting the finally obtained features, and then the original image density spectrum is restored through segmented step-by-step deconvolution up-sampling.

In a preferred embodiment, S13 is followed by: constructing a multi-task cascade neural network model capable of training end to end by utilizing a uniform loss function, and performing end to end training on the whole detection network model by utilizing the loss;

wherein the loss function is:

wherein: f_dFor density spectrum, X_iFor input image data, Θ is each parameter in the cascaded neural network, and D is the set of calibration density spectra.

In a preferred embodiment, S15 further includes: and (4) processing the density map by using a local maximum method, and estimating the position of the steel bar head.

In a preferred embodiment, the step of using the local maximum in S15 further comprises:

s31: traversing a larger Chinese character 'tian' shape, finding the maximum value of three rows and three columns of the Chinese character 'tian', judging whether the Chinese character 'tian' is a local maximum value, if so, returning the position of the Chinese character 'tian' and if not, recording and finding the coordinate of the maximum value in the adjacent four points;

s32: continuously comparing the next field character through the quadrant reduction range of the coordinates obtained in the S11;

s33: if the maximum value can not be found, finally, when the field character becomes a rectangle of 3 multiplied by 3, the search is completed. The time complexity is N, and the field word in the search method is shown in fig. 3.

Best practiceIn an embodiment, S15 is followed by: in the step, the invention designs a convolution neural network from end to end for dense steel bar target counting, and the image is input into the density map, so how to define a standard density map to express the image marking information in the data set determines the performance of the model. If the image has a steel bar target, the pixel point x where the image is located_iIt is denoted as δ (x-x)_i) A function, in a picture with N objects, representing the objects in the image as:

in order to convert a single steel bar target into a continuous density function, a steel bar central point is selected as a key point of an algorithm, and the density is regularly dispersed on the section of the steel bar target by performing two-dimensional Gaussian filtering on the key point, so that the density is convolution of an H function and a two-dimensional Gaussian filtering function.

In the preferred embodiment, because the density map predicted by the multitask cascade neural network model has noise at non-critical parts and local maximum peak values, and a method for eliminating the noise obtains empirical values according to tests, generally, peaks with target center representation meaning are all higher than 100/255, so that the local maximum values are used for filtering out the density smaller than the empirical values by adopting a preprocessing mode.

In the preferred embodiment, since the density spectrum predicted by the multitask cascade neural network model is not completely smooth, there exists density fluctuation at the peak value which is unavoidable systematically, because the density spectrum is not completely smooth, too many possible target center points will appear when the peak value is searched by using the local maximum value method, and the smoothing filter is used for preprocessing the smoothing of the density spectrum. The filter used in particular is shown in fig. 4.

In the preferred embodiment, the step of S15, after the local maximum method, further includes: and removing repeated block diagrams by using a non-maximum value inhibition method to obtain a final positioning frame of the steel bar head. In one embodiment, the frame overlap area ratio threshold is selected to be greater than 0.3, and for the score of a single frame, the integral of the density of the corresponding area of the frame is used as the score of the frame. And carrying out non-maximum suppression on the prediction result under the parameter to obtain a final prediction result.

Fig. 5 is a schematic structural view of a tendon head counting and positioning system according to an embodiment of the present invention.

Referring to fig. 5, the tendon head counting and positioning system of the present embodiment includes: the device comprises a shallow layer feature extraction module 1, a feature fusion module 2, a density map estimation module 3, an integral summation module 4 and a positioning module 5.

The shallow feature extraction module 1 is configured to extract a preliminary feature of the original image. The feature fusion module 2 is used for fusing the preliminary features of different levels to obtain fusion features. The density map estimation module 3 is used for performing density map estimation operation on the fusion features to obtain a density map. And the integral summation module 4 is used for carrying out integral summation on the density map to obtain the number of the final steel bar heads. And the positioning module 5 is used for estimating the position of the steel bar head by using the density map.

In a preferred embodiment, the positioning module is further configured to estimate the position of the steel bar head by processing the density map using a local maximum method.

In a preferred embodiment, the method further comprises: a preprocessing module for filtering out less than empirical densities in the density map and/or smoothing a density spectrum of the density map prior to a local maximum method in the positioning module.

In a preferred embodiment, the positioning module is further configured to remove the repeated frame by using a non-maximum suppression method after the local maximum method to obtain a final positioning frame of the steel bar head.

In a preferred embodiment, the method further comprises: and the detection network model module is used for constructing a detection network model trained end to end by utilizing the loss sum of the shallow feature extraction module, the feature fusion module and the density map estimation module, and training the whole detection network model module by utilizing the loss sum.

It should be noted that, the steps in the method provided by the present invention may be implemented by using corresponding modules, devices, units, and the like in the system, and those skilled in the art may refer to the technical solution of the system to implement the step flow of the method, that is, the embodiment in the system may be understood as a preferred example for implementing the method, and details are not described herein.

Those skilled in the art will appreciate that, in addition to implementing the system and its various devices provided by the present invention in purely computer readable program code means, the method steps can be fully programmed to implement the same functions by implementing the system and its various devices in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system and various devices thereof provided by the present invention can be regarded as a hardware component, and the devices included in the system and various devices thereof for realizing various functions can also be regarded as structures in the hardware component; means for performing the functions may also be regarded as structures within both software modules and hardware components for performing the methods.

The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, and not to limit the invention. Any modifications and variations within the scope of the description, which may occur to those skilled in the art, are intended to be within the scope of the invention.