1. A mining conveyor belt bulk material detection method based on deep learning is characterized by comprising the following steps:

s1: acquiring a field conveying belt material image as a sample set in real time through an explosion-proof monitoring device;

s2: screening out images with block objects in a sample set, preprocessing the screened images, then carrying out target marking on the preprocessed images, and classifying marking results into a training set, a verification set and a test set;

s3: constructing a target detection network model YOLOv3 based on deep learning;

s4: inputting the images of the training set and the verification set in the step S2 into a YOLOv3 network model for training and fine-tuning the parameters of the YOLOv3 network model until the network model reaches a convergence state, so as to obtain a trained block material detection model;

s5: inputting the test set image in the step S2 into a trained block material detection model, and if a target confidence coefficient C detected in the image_confIf the mass is more than 80%, the mass materials appear in the conveyer belt;

s6: judging whether the blocky materials are large materials or not;

s7: if a large material block is detected, the software platform records the abnormal information at the moment and controls the alarm equipment to send out an alarm signal.

2. The deep learning-based mining conveyor belt bulk material detection method according to claim 1, characterized in that: the explosion-proof monitoring device in the step S1 can meet the requirements on image quality and fluency in the underground scene of the coal mine, and the monitoring picture can completely acquire the width part of the conveying belt.

3. The deep learning-based mining conveyor belt bulk material detection method according to claim 1, characterized in that: in step S2, the image in which the sample set contains the block-like object is screened out, the screened image is subjected to detection area interception, then the image subjected to detection area interception is subjected to target rectangular frame marking by using labelimg software, the coordinate information of the target is obtained and stored in an xml file, and finally the marking result is classified into a training set, a verification set and a test set.

4. The deep learning-based mining conveyor belt bulk material detection method according to claim 1, characterized in that: in step S4, the network model reaches the convergence state according to the loss function of the training set and the verification set.

5. The deep learning-based mining conveyor belt bulk material detection method according to claim 1, characterized in that: in step S5, the target confidenceWherein the content of the first and second substances,to predict the target frame B_predAnd a real target frame B_truthThe ratio between the intersection and union of (1), P_r(object) indicates a predicted object box B_predIf there is a detection target, the value is 1, otherwise it is 0.

6. The deep learning-based mining conveyor belt bulk material detection method according to claim 5, characterized in that: predicted target frame B_predAnd a real target frame B_truthThe intersection calculation method of (2) is as follows:

predicted target frame B_predPosition and size of (x)₁,y₁,w₁,h₁) True target frame B_truthPosition and size of (x)₂,y₂,w₂,h₂) X and y respectively represent the horizontal and vertical coordinates of the center point of the target frame, w represents the width of the target frame, and h represents the height of the target frame; subscripts 1 and 2 correspond to the predicted target frame B_predAnd a real target frame B_truth；

Wherein, w_pxminTo predict the target frame B_predLower boundary of abscissa, w_pxmaxTo predict the target frame B_predThe abscissa is bounded;

wherein, w_txminAs a real target frame B_truthLower boundary of abscissa, w_txmaxReal target frame B_truthThe abscissa is bounded;

calculating w_pxminAnd w_txminMaximum value w of_x1，w_x1＝max(w_pxmin,w_txmin) (ii) a Calculating w_pxmaxAnd w_txmaxMinimum value w of_x2，w_x2＝min(w_pxmax,w_txmax) (ii) a If w_x2-w_x1<0, then B_predAnd B_truthThe abscissas of (B) do not intersect, otherwise, B is indicated_predAnd B_truthThe horizontal coordinates of the two have intersection;

wherein, w_pyminTo predict the target frame B_predLower bound of ordinate, w_pymaxTo predict the target frame B_predThe ordinate is upper bound;

wherein, w_tyminAs a real target frame B_truthLower bound of ordinate, w_tymaxReal target frame B_truthThe ordinate is upper bound;

calculating w_pyminAnd w_tyminMaximum value w of_y1，w_y1＝max(w_pymin,w_tymin) (ii) a Calculating w_pymaxAnd w_tymaxMinimum value w of_y2，w_y2＝min(w_pymax,w_tymax) (ii) a If w_y2-w_y1<0, then B_predAnd B_truthThe ordinate of (B) does not intersect, otherwise, B is indicated_predAnd B_truthThe vertical coordinates of (A) have an intersection;

further obtain B_predAnd B_truthThe intersection region of (a).

7. The deep learning-based mining conveyor belt bulk material detection method according to claim 1, characterized in that: the method for judging whether the block-shaped material is the large block-shaped material in the step S6 is as follows: calibrating the explosion-proof monitoring device to obtain the relation between the actual size of the material target and the size of the image; and then calculating the length and the width of the block-shaped materials, and judging the block-shaped materials as long as one quantity of the length and the width exceeds a preset threshold value.

8. The deep learning-based mining conveyor belt bulk material detection method according to claim 7, characterized in that: the specific steps for obtaining the relationship between the actual size of the material and the size of the image are as follows: obtaining a material with a practical size L which normally runs on the conveyor belt₁(ii) a The explosion-proof monitoring device obtains an image of the material after the material is shot, and the size of the object in the image is calculated to be M₁(ii) a In the calculation later, the image size M of the new material is calculated, and the new material can be calculated according to a formulaThe actual material size L is obtained.

9. A system for detecting the bulk materials of the mining conveying belt based on the method of claim 1 is characterized in that: the large material detection module detects large materials on the mining conveyor belt by adopting the detection steps of steps S1-S6 in claim 1 and stores the detection result; the information display module displays the video content acquired by the explosion-proof monitoring device on a software platform interface in real time, and displays whether the explosion-proof monitoring device is successfully configured, whether the data management function is normal, whether the system alarm and control function is started, and abnormal information results detected by the bulk material detection module; the data management module completes data communication, data processing, data query and data statistics through Mysql database management software; the alarm module sends out sound and light alarm information or controls the start and stop of the conveying belt according to the abnormal condition of the bulk materials; the system setting module sets the equipment name, the IP address, the detection type and the account password information of the user of the explosion-proof monitoring device and is used for monitoring the detection device and the login operation of the user.

Background

China is a country with coal as a main energy source, coal is the main energy source consumption of China and cannot be changed in a short period, and whether the coal industry is healthy and stable is related to energy safety and economic sustainable development of China. The belt conveyor is a key device for coal transportation and plays an important role in the coal transportation. Once the belt conveyor fails to operate normally, the normal production of the coal mine is seriously affected. The conveying belt is used as a core component of the whole belt conveyor system and is expensive (accounting for more than 40% of the running cost of the belt conveyor), and due to the fact that the underground environment of a mine is complex and severe, materials on the conveying belt are different in size, and the strength of the belt is poor, the accidents of deviation, slipping, belt breakage and longitudinal belt tearing are prone to occurring in production. The occurrence of these accidents not only affects the safe production, but also causes very disastrous economic losses.

Analysis of these conveyor belt accidents shows that longitudinal tear in the conveyor belt is more likely to occur in conveyor belt damage and most harmful. Wherein, the problem of most conveyer belt vertical tear is that the belt pressure is pounded and is caused because the bold material (like bold coal, bold waste rock, stock etc.) gets into fortune coal belt system to the conveyer belt. Until now, most of the safety protection research of the conveying belt focuses on the longitudinal tearing detection of the conveying belt. The conveyor belt longitudinal tearing detection can detect the early stage of the conveyor belt longitudinal tearing and respond to the early stage, so that the influence of the conveyor belt longitudinal tearing on the mine production process is reduced, but the occurrence of the conveyor belt longitudinal tearing cannot be fundamentally prevented. Therefore, starting from the reason of the longitudinal tearing accident of the conveying belt, if the large materials can be accurately identified and taken out of the system at the early stage when the large materials enter the conveying belt conveying system, the damage to the conveying belt conveying system caused by the large materials can be greatly prevented, the longitudinal tearing of the conveying belt is prevented, and the safe and stable operation of the conveying belt conveying system is guaranteed.

In addition, with the continuous advance of industry 4.0 and the continuous progress of coal integration automation, the image detection technology based on machine vision gradually becomes a new development direction. Machine vision is an advanced technology for automatically acquiring images and related processing on an object by using optical elements, image sensors and other devices, and the processing result can be used for controlling the working conditions of various mechanical devices. Compared with various traditional technologies, the machine vision technology has the advantages of non-contact, economy, flexibility, reliability and the like. If a method for detecting massive materials of a mining conveyor belt based on deep learning is researched in a belt conveyor transportation system to detect the massive materials of the belt conveyor, the intelligent monitoring function of the belt conveyor in a coal mine can be realized. Meanwhile, the added image detection function of the conveying belt reduces the labor intensity of post workers, improves the sensitivity and accuracy of the alarm device and ensures safe production; in addition, through newly adding the camera, the post workman also can direct observation transportation condition, more is favorable to observing and operating.

Disclosure of Invention

The invention provides a method and a system for detecting large materials of a mining conveying belt based on deep learning, aiming at the problem of damage of the large materials to the conveying belt conveying system.

In order to achieve the technical purpose, the invention adopts the following technical scheme:

a mining conveyor belt bulk material detection method based on deep learning comprises the following steps:

s1: acquiring a field conveying belt material image as a sample set in real time through an explosion-proof monitoring device;

s2: screening out images with block objects in a sample set, preprocessing the screened images, then carrying out target marking on the preprocessed images, and classifying marking results into a training set, a verification set and a test set;

s3: constructing a target detection network model YOLOv3 based on deep learning;

s4: inputting the images of the training set and the verification set in the step S2 into a YOLOv3 network model for training and fine-tuning the parameters of the YOLOv3 network model until the network model reaches a convergence state, so as to obtain a trained block material detection model;

s5: inputting the test set image in the step S2 into a trained block material detection model, and if a target confidence coefficient C detected in the image_co_nfIf the mass is more than 80%, the mass materials appear in the conveyer belt;

s6: judging whether the blocky materials are large materials or not;

s7: if a large material block is detected, the software platform records the abnormal information at the moment and controls the alarm equipment to send out an alarm signal.

Preferably, the explosion-proof monitoring device in the step S1 can meet the requirements on image quality and fluency in the underground coal mine scene, and the monitoring picture can completely acquire the width part of the conveyor belt.

Preferably, in step S2, the images with the block-shaped objects in the sample set are screened out, the screened images are subjected to detection area interception, then target rectangular frame labeling is performed on the images subjected to detection area interception by using labelimg software, coordinate information of the target is obtained and stored in an xml file, and finally, the labeling results are classified into a training set, a verification set and a test set.

Preferably, in step S4, the network model reaches the convergence state according to the loss function of the training set and the verification set.

Preferably, in step S5, the target confidence levelWherein the content of the first and second substances,to predict the target frame B_predAnd a real target frame B_truthThe ratio between the intersection and union of (1), P_r(object) indicates a predicted object box B_predIf there is a detection target, the value is 1, otherwise it is 0.

Preferably, the target frame B is predicted_predAnd a real target frame B_truthThe intersection calculation method of (2) is as follows:

predicted target frame B_predPosition and size of (x)₁,y₁,w₁,h₁) True target frame B_truthPosition and size of (x)₂,y₂,w₂,h₂) X and y respectively represent the horizontal and vertical coordinates of the center point of the target frame, w represents the width of the target frame, and h represents the height of the target frame; subscripts 1 and 2 correspond to the predicted target frame B_predAnd a real target frame B_truth；

Wherein, w_pxminTo predict the target frame B_predLower boundary of abscissa, w_pxmaxTo predict the target frame B_predThe abscissa is bounded;

wherein, w_txminAs a real target frame B_truthLower boundary of abscissa, w_txmaxReal target frame B_truthThe abscissa is bounded;

calculating w_pxminAnd w_txminMaximum value w of_x1，w_x1＝max(w_pxmin,w_txmin) (ii) a Calculating w_pxmaxAnd w_txmaxMinimum value w of_x2，w_x2＝min(w_pxmax,w_txmax) (ii) a If w_x2-w_x1<0, then B_predAnd B_truthThe abscissas of (B) do not intersect, otherwise, B is indicated_predAnd B_truthOfThe coordinates have intersection;

wherein, w_pyminTo predict the target frame B_predLower bound of ordinate, w_pymaxTo predict the target frame B_predThe ordinate is upper bound;

wherein, w_tyminAs a real target frame B_truthLower bound of ordinate, w_tymaxReal target frame B_truthThe ordinate is upper bound;

calculating w_pyminAnd w_tyminMaximum value w of_y1，w_y1＝max(w_pymin,w_tymin) (ii) a Calculating w_pymaxAnd w_tymaxMinimum value w of_y2，w_y2＝min(w_pymax,w_tymax) (ii) a If w_y2-w_y1<0, then B_predAnd B_truthThe ordinate of (B) does not intersect, otherwise, B is indicated_predAnd B_truthThe vertical coordinates of (A) have an intersection;

further obtain B_predAnd B_truthThe intersection region of (a).

Preferably, the method for judging whether the block-shaped material is the large block-shaped material in the step S6 is as follows: calibrating the explosion-proof monitoring device to obtain the relation between the actual size of the material target and the size of the image; and then calculating the length and the width of the block-shaped materials, and judging the block-shaped materials as long as one quantity of the length and the width exceeds a preset threshold value.

Preferably, the specific steps of obtaining the relationship between the actual size of the material and the size of the image are as follows: obtaining a material with a practical size L which normally runs on the conveyor belt₁(ii) a The explosion-proof monitoring device obtains an image of the material after the material is shot, and the size of the object in the image is calculated to be M₁(ii) a In the calculation later, the image size M of the new material is calculated, and the new material can be calculated according to a formulaThe actual material size L is obtained.

The invention also provides a system for detecting the bulk materials of the mining conveyor belt, which comprises a bulk material detection module, an information display module, a data management module, an alarm control module and a system setting module, wherein the bulk material detection module detects the bulk materials on the mining conveyor belt and stores the detection result; the information display module displays the video content acquired by the explosion-proof monitoring device on a software platform interface in real time, and displays whether the explosion-proof monitoring device is successfully configured, whether the data management function is normal, whether the system alarm and control function is started, and abnormal information results detected by the bulk material detection module; the data management module completes data communication, data processing, data query and data statistics through Mysql database management software; the alarm module sends out sound and light alarm information or controls the start and stop of the conveying belt according to the abnormal condition of the bulk materials; the system setting module sets the equipment name, the IP address, the detection type and the account password information of the user of the explosion-proof monitoring device and is used for monitoring the detection device and the login operation of the user.

Compared with the prior art, the invention has the beneficial effects that:

(1) the invention can monitor the monitored area in real time without requiring a person to visit, thereby reducing the labor cost; and when the conveyer belt in the bold material appears, can send out the police dispatch newspaper in the very first time and remind field work personnel, have promptness and validity advantage.

(2) The method overcomes the defect that the traditional image detection method is difficult to accurately distinguish the material target and the transportation background thereof due to very similar gray levels, and the method for detecting the massive materials of the mining conveyor belt based on deep learning has important social significance and economic value for preventing the longitudinal tearing and the longitudinal tearing expansion of the mining conveyor belt, maintaining the safe operation of a belt conveyor and ensuring the stable production of coal mine enterprises, and has wide market popularization prospect.

Drawings

For a clearer explanation of the embodiments or technical solutions of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for a person skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a flow chart of the detection method of the present invention;

FIG. 2(a) is a loss function of a training set of a YOLOv3 network model;

FIG. 2(b) is a LOSS function of a network model verification set of YOLOv 3;

FIG. 3 shows B in the YOLOv3 network model_predAnd B_truthThe intersection type of (c);

FIG. 4 shows B in the YOLOv3 network model_predAnd B_truthAn intersection calculation method;

FIG. 5 is a block diagram of the overall design of the detection system.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

1. A mining conveyor belt bulk material detection method based on deep learning is characterized by comprising the following steps:

s1: acquiring a field conveying belt material image as a sample set in real time through an explosion-proof monitoring device;

the selected explosion-proof monitoring device can meet different requirements on image quality and fluency in underground scenes of the coal mine, and the mounting position of the explosion-proof monitoring device meets the requirement that the monitoring picture can completely acquire the width part of the conveying belt. The specific selection and installation parameters are as follows: the resolution of the picture shot by the explosion-proof monitoring device is 1920 multiplied by 1080, most network protocols such as TCP/IP, HTTP and FTP are supported, the transmission rate is 10/100Mbps, the horizontal field angle is 89 degrees, the frame rate is 25fps, and the shutter speed is 1/3 seconds to 1/100,000 seconds. The actual width of the conveying belt is 1.2m and is made by narrow space of a coal mine working site, the explosion-proof monitoring device is arranged at a position 1.55m right above the conveying belt, and the real length of a picture shot by the explosion-proof monitoring device is about 3.1m multiplied by 1.75 m. The belt speed was 4 m/s.

S2: screening out images with block objects in a sample set, intercepting a detection area of the screened images, marking the images with a target rectangular frame by using labelimg software to obtain coordinate information of a target, storing the coordinate information in an xml file, and finally classifying marking results into a training set, a verification set and a test set. The specific process is as follows:

the screened images are taken at different acquisition times and at different conveyor belt positions;

the detection area of the image is intercepted in a rectangular area intercepting mode, the upper left corner point and the lower right corner point of the rectangular area are respectively (0,442) and (1080,1548), and the resolution of the obtained image is 1106 multiplied by 1080;

importing the image intercepted from the detection area into labellimg software, marking a block material target obviously existing in the image by drawing a rectangular frame, setting the class name of the block material target as block, and finally storing data in a PASCALVOC format to obtain an XML file storing various information such as an original image, a labeling frame and the like;

in the experiment, the number of the samples of the block material target detection data set obtained by labeling is 784, wherein the number of the samples of the training set and the number of the samples of the verification set are set to be 9:1, the number of the samples of the training set is 540, the number of the samples of the verification set is 60, and the number of the samples of the test set is 184.

S3: constructing a target detection network model YOLOv3 based on deep learning;

the detection process of constructing the YOLOv3 is that the size of an original image is firstly readjusted, and the obtained image is input in a YOLOv3 network structure; then, running a YOLOv3 network to obtain a bounding box of the position size, the confidence coefficient and the affiliated category information of the predicted target; and finally, screening the obtained bounding box, and removing redundant prediction result information to obtain a final target detection result.

S4: inputting the images of the training set and the verification set in the step S2 into a YOLOv3 network model for training and fine-tuning the parameters of the YOLOv3 network model until the network model reaches a convergence state, so as to obtain a trained block material detection model; the judgment basis of the network model reaching the convergence state is that the loss function of the training set and the verification set reaches the convergence state. The specific implementation process is as follows:

the computer used by the experimental platform is configured as AMDR7-3800XCPU, a memory 32G, an RTX2070GPU, a video memory 16G, an operating system of Windows10, a programming language of python3.8 and a deep learning framework of pytorch;

the training Epochs are set to be 100, the first 50 Epochs freeze parameters of a pre-training model, only the last classification layer is trained, the last 50 Epochs update all the parameters, the batch size of the first 50 Epochs is set to be 8, the batch size of the last 50 Epochs is 4, the first 50 Epochs set the learning rate lr to be 0.001, and the last 50 Epochs set the learning rate lr to be 0.0001;

when the YOLOv3 model is trained, as shown in fig. 2, the loss functions of the training set and the verification set are stable and have a downward trend, and finally both the loss functions can be reduced to below 5.

S5: inputting the test set image in the step S2 into a trained block material detection model, and if a target confidence coefficient C detected in the image_co_nfIf the content is more than 80%, the existence of blocky materials in the conveyer belt is indicated.

Target confidenceWherein the content of the first and second substances,to predict the target frame B_predAnd a real target frame B_truthThe ratio between the intersection and union of (1), P_r(object) indicates a predicted object box B_predWhether or notIf there is a detection target, the value is 1, otherwise it is 0.

As shown in FIG. 3, the predicted target frame B_predAnd a real target frame B_truthMay be completely contained, partially intersected, or non-intersected according to relative positions. As shown in FIG. 4, the predicted target frame B_predAnd a real target frame B_truthThe intersection calculation method of (2) is as follows:

predicted target frame B_predPosition and size of (x)₁,y₁,w₁,h₁) True target frame B_truthPosition and size of (x)₂,y₂W2, h2), wherein x and y respectively represent the abscissa and ordinate of the center point of the target frame, w represents the width of the target frame, and h represents the height of the target frame; subscripts 1 and 2 correspond to the predicted target frame B_predAnd a real target frame B_truth；

Wherein, w_pxminTo predict the target frame B_predLower boundary of abscissa, w_pxmaxTo predict the target frame B_predThe abscissa is bounded;

wherein, w_txminAs a real target frame B_truthLower boundary of abscissa, w_txmaxReal target frame B_truthThe abscissa is bounded;

calculating w_pxminAnd w_txminMaximum value w of_x1，w_x1＝max(w_pxmin,w_txmin) (ii) a Calculating w_pxmaxAnd w_txmaxMinimum value w of_x2，w_x2＝min(w_pxmax,w_txmax) (ii) a If w_x2-w_x1<0, then B_predAnd B_truthThe abscissas of (B) do not intersect, otherwise, B is indicated_predAnd B_truthThe horizontal coordinates of the two have intersection;

wherein, w_pyminTo predict the target frame B_predLower bound of ordinate, w_pymaxTo predict the target frame B_predThe ordinate is upper bound;

wherein, w_tyminAs a real target frame B_truthLower bound of ordinate, w_tymaxReal target frame B_truthThe ordinate is upper bound;

calculating w_pyminAnd w_tyminMaximum value w of_y1，w_y1＝max(w_pymin,w_tymin) (ii) a Calculating w_pymaxAnd w_tymaxMinimum value w of_y2，w_y2＝min(w_pymax,w_tymax) (ii) a If w_y2-w_y1<0, then B_predAnd B_truthThe ordinate of (B) does not intersect, otherwise, B is indicated_predAnd B_truthThe vertical coordinates of (A) have an intersection; further obtain B_predAnd B_truthThe intersection region of (a).

S6: judging whether the blocky materials are large materials or not; the method for judging whether the block-shaped material is the large block-shaped material in the step S6 is as follows: calibrating the explosion-proof monitoring device to obtain the relation between the actual size of the material target and the size of the image; and then calculating the length and the width of the block-shaped materials, and judging the block-shaped materials as long as one quantity of the length and the width exceeds a preset threshold value. The specific steps for obtaining the relationship between the actual size of the material and the size of the image are as follows: obtaining a material with a practical size L which normally runs on the conveyor belt₁(ii) a The explosion-proof monitoring device obtains an image of the material after the material is shot, and the size of the object in the image is calculated to be M₁(ii) a In the calculation later, the image size M of the new material is calculated, and the new material can be calculated according to a formulaThe actual material size L is obtained.

S7: if a large material block is detected, the software platform records the abnormal information at the moment and controls the alarm equipment to send out an alarm signal.

The invention also provides a system for detecting the massive materials of the mining conveyor belt, which comprises a massive material detection module, an information display module, a data management module, an alarm control module and a system setting module, wherein the massive material detection module detects the massive materials on the mining conveyor belt by adopting the detection steps of S1-S6 and stores the detection result; the information display module displays the video content acquired by the explosion-proof monitoring device on a software platform interface in real time, and displays whether the explosion-proof monitoring device is successfully configured, whether the data management function is normal, whether the system alarm and control function is started, and abnormal information results detected by the bulk material detection module; the data management module completes data communication, data processing, data query and data statistics through Mysql database management software; the alarm module sends out sound and light alarm information or controls the start and stop of the conveying belt according to the abnormal condition of the bulk materials; the system setting module sets the equipment name, the IP address, the detection type and the account password information of the user of the explosion-proof monitoring device and is used for monitoring the detection device and the login operation of the user.