1. A method for identifying foreign matter invasion of a subway tramcar based on deep learning is characterized by comprising the following steps: it comprises the following steps:

1) each track is provided with a camera (for example, one camera is arranged at 100 meters);

2) collecting video images;

3) identifying foreign matters;

4) identification of whether the foreign object is an animal;

5) and if the foreign matters are determined, sending out early warning in a corresponding form according to the types of the foreign matters.

2. The method for identifying the intrusion of the foreign matters into the subway tram based on the deep learning as claimed in claim 1, wherein the method comprises the following steps: the orientation of the cameras arranged on each section of track in the step 1 is consistent, and the orientation of the cameras is consistent with the driving direction of the vehicle.

3. The method for identifying the intrusion of the foreign matters into the subway tram based on the deep learning as claimed in claim 2, wherein the method comprises the following steps: and the camera in the step 1 is provided with an auxiliary lighting device.

4. The method for identifying the intrusion of the foreign matters into the subway tram based on the deep learning as claimed in claim 3, wherein the method comprises the following steps: the step 3 comprises the following steps:

s11, convolving the original image with a 2D Gaussian filtering template to eliminate noise;

s12, finding out derivatives Gx and Gy of the image gray along two directions by using a derivative operator, and solving the gradient size;

s13 calculating a gradient direction using the result of step S12;

s14, dividing the gradient direction of the edge into 0 degree, 45 degrees, 90 degrees and 135 degrees, and finding the adjacent pixel of the gradient direction of the pixel;

s15, if the gray value of a certain pixel is not the largest compared with the gradient values of two pixels in front and back in the gradient direction, the pixel value is set to 0, that is, it is not an edge, otherwise it is an edge;

s16, reading each pixel after obtaining the edge information, if the length of the edge pixel is larger than T, determining the edge pixel is a foreign matter candidate area, and recording the coordinate of the foreign matter after detecting the foreign matter candidate area;

s17, using the support vector machine to classify the foreign matters, wherein the classification comprises maintainers, vehicles and invaded foreign matters.

5. The method for identifying the intrusion of the foreign matters into the subway tram based on the deep learning as claimed in claim 4, wherein the method comprises the following steps: the step 4 comprises the following steps:

s18, if the classification is determined to be a maintainer or a vehicle in the step S17, skipping S21-S27, and if the classification is determined to be an invasive foreign matter, entering the step S21;

s21, receiving and storing the first frame of video image;

s22, receiving and storing the second frame of video image; the first frame video image and the second frame video image refer to two adjacent frame video images;

s23, comparing the second frame video image with the first frame video image, judging whether there is a changed area, if not, executing step S24, if yes, executing step S25;

s24, discarding the first frame of video image, namely discarding the previous frame of video image in the two adjacent frames of video images, and feeding back the classification of the invading foreign body as an article to a decision maker without performing subsequent operation;

s25, determining the moving pixel distance of the invaded foreign body; namely, the positions of the invading foreign matters in the two frames of video images are compared to determine the moving pixel distance of the invading foreign matters;

s26, determining the real moving distance of the invaded foreign matter based on the scale parameter, namely determining the real moving distance corresponding to the moving pixel distance according to the scale;

s27, determining the moving speed of the invading foreign matter based on the moving real distance of the invading foreign matter, namely determining the moving speed of the invading foreign matter according to the moving real distance and the time interval of two frames of video image shooting; the decision maker is fed back that the invasive foreign body is an animal and the movement direction of the invasive foreign body is predicted.

Background

In rail transit, the vehicle traveling route is completely fixed, and if foreign matters appear on the rail, the vehicle cannot avoid the foreign matters like an automobile, so that the foreign matters appearing in a certain range of the rail must be removed, and the foreign matters must be treated before the vehicle arrives in the next shift. Therefore, a foreign object intrusion identification method is brought forward, for example, a train rail foreign object intrusion detection device disclosed in chinese patent, patent No. 201721865900.8, in which: the system comprises a plurality of groups of detection groups distributed along the train track, wherein each detection group comprises two rows of laser correlation sensors which are arranged along two sides of the train track at equal intervals and matched one by one, the laser correlation sensors on one side are a1, a2, a3 and a4 … … an respectively, the laser correlation sensors on the other side are b1, b2, b3 and b4 … … bn respectively, an is matched with bn +1, the last laser correlation sensor on one side is matched with the first laser correlation sensor on the other side, and the laser correlation sensors in each group form a laser correlation network to cover the area in the group, wherein n is greater than 0. The train track foreign matter intrusion detection device has the advantages of scientific design and comprehensive coverage.

The technical scheme has the following defects that: a large number of laser sensors are required to be arranged on each section of track, so that the maintenance is difficult; in addition, the laser sensor can only sense whether the abnormality occurs and cannot sense whether the foreign object is an object or an animal.

Disclosure of Invention

Aiming at the defects of the prior art, the invention provides a method for identifying foreign matter invasion by a camera.

The technical scheme of the invention is as follows:

a method for identifying foreign matter invasion of a subway tramcar based on deep learning is characterized by comprising the following steps: it comprises the following steps:

1) each track is provided with a camera (for example, one camera is arranged at 100 meters);

2) collecting video images;

3) identifying foreign matters;

4) identification of whether the foreign object is an animal;

5) and if the foreign matters are determined, sending out early warning in a corresponding form according to the types of the foreign matters.

Specifically, in step 1, the orientations of the cameras arranged on each section of track are consistent, and the orientations of the cameras are consistent with the driving direction of the vehicle, although the tracks are not straight lines, the continuous tracking of some moving objects is facilitated by connecting the shooting ranges of the cameras into a continuous line-segment distribution.

Specifically, the camera is provided with auxiliary lighting device in step 1, and is different from traditional rail traffic, and the subway line is most located underground, and the illumination condition probably is not enough to provide the powerful condition for visual identification, so set up auxiliary lighting device, for example set up the wall lamp at the lateral wall along the subway line.

Specifically, the step 3 includes the following steps:

s11, convolving the original image with a 2D Gaussian filtering template to eliminate noise;

s12, finding out derivatives Gx and Gy of the image gray along two directions by using a derivative operator, and solving the gradient size;

s13 calculating a gradient direction using the result of step S12;

s14, dividing the gradient direction of the edge into 0 degree, 45 degrees, 90 degrees and 135 degrees, and finding the adjacent pixel of the gradient direction of the pixel;

s15, if the gray value of a certain pixel is not the largest compared with the gradient values of two pixels in front and back in the gradient direction, the pixel value is set to 0, that is, it is not an edge, otherwise it is an edge;

s16, reading each pixel after obtaining the edge information, if the length of the edge pixel is larger than T, determining the edge pixel is a foreign matter candidate area, and recording the coordinate of the foreign matter after detecting the foreign matter candidate area; some light shadows may be generated in the video image due to light, weather and the like, and a threshold value is set for preventing false alarm;

s17 uses Support Vector Machine (SVM) to classify the foreign matter, including maintainer, vehicle (subway car), intrusion foreign matter.

Specifically, the step 4 includes the following steps:

s18, if the classification is determined to be a maintainer or a vehicle in the step S17, skipping S21-S27, and if the classification is determined to be an invasive foreign matter, entering the step S21;

s21, receiving and storing the first frame of video image;

s22, receiving and storing the second frame of video image; the first frame video image and the second frame video image refer to two adjacent frame video images;

s23, comparing the second frame video image with the first frame video image, judging whether there is a changed area, if not, executing step S24, if yes, executing step S25;

s24, discarding the first frame of video image, namely discarding the previous frame of video image in the two adjacent frames of video images, and feeding back the classification of the invading foreign body as an article to a decision maker without performing subsequent operation;

s25, determining the moving pixel distance of the invaded foreign body; namely, the positions of the invading foreign matters in the two frames of video images are compared to determine the moving pixel distance of the invading foreign matters;

s26, determining the real moving distance of the invaded foreign matter based on the scale parameter, namely determining the real moving distance corresponding to the moving pixel distance according to the scale;

s27, determining the moving speed of the invading foreign matter based on the moving real distance of the invading foreign matter, namely determining the moving speed of the invading foreign matter according to the moving real distance and the time interval of two frames of video image shooting; the decision maker is fed back that the invasive foreign body is an animal and the movement direction of the invasive foreign body is predicted.

The invention has the beneficial effects that: the information source is a camera, and as a large number of cameras are inevitably arranged in most subway line nets at present, the direction of the cameras can be adjusted, so that the trouble of arranging a large number of various professional sensors and equipment is avoided, and the identification cost is reduced; effectively distinguish invasion foreign matter and be article or animal, because the subway line is a very confined space, if the animal then probably geological destruction such as cave can appear, the pipeline can also appear and be gnawed the circumstances such as gnawing, so need specially carry out the prejudgement.

Detailed Description

The following is further described in conjunction with the detailed description:

a method for identifying foreign matter invasion of a subway tramcar based on deep learning is characterized by comprising the following steps: it comprises the following steps:

1) each track is provided with a camera (for example, one camera is arranged at 100 meters); in the step 1, the orientations of the cameras arranged on each section of track are consistent, the orientations are consistent with the driving direction of the vehicle, although the tracks are not straight lines, the continuous line-segment distribution is beneficial to continuously tracking some moving objects by connecting the shooting ranges of the cameras. The camera is provided with auxiliary lighting device in step 1, and is different from traditional rail traffic, and the subway line is most located underground, and the illumination condition probably is not enough to provide the powerful condition for visual identification, so set up auxiliary lighting device, for example set up the wall lamp at the lateral wall along the subway line.

2) Collecting video images;

3) identifying foreign matters; the step 3 comprises the following steps:

s11, convolving the original image with a 2D Gaussian filtering template to eliminate noise;

s12, finding out derivatives Gx and Gy of the image gray along two directions by using a derivative operator, and solving the gradient size;

s13 calculating a gradient direction using the result of step S12;

s14, dividing the gradient direction of the edge into 0 degree, 45 degrees, 90 degrees and 135 degrees, and finding the adjacent pixel of the gradient direction of the pixel;

s15, if the gray value of a certain pixel is not the largest compared with the gradient values of two pixels in front and back in the gradient direction, the pixel value is set to 0, that is, it is not an edge, otherwise it is an edge;

s16, reading each pixel after obtaining the edge information, if the length of the edge pixel is larger than T, determining the edge pixel is a foreign matter candidate area, and recording the coordinate of the foreign matter after detecting the foreign matter candidate area; some light shadows may be generated in the video image due to light, weather and the like, and a threshold value is set for preventing false alarm;

s17 uses Support Vector Machine (SVM) to classify the foreign matter, including maintainer, vehicle (subway car), intrusion foreign matter.

4) Identification of whether the foreign object is an animal; s18, if the classification is determined to be a maintainer or a vehicle in the step S17, skipping S21-S27, and if the classification is determined to be an invasive foreign matter, entering the step S21;

s21, receiving and storing the first frame of video image;

s22, receiving and storing the second frame of video image; the first frame video image and the second frame video image refer to two adjacent frame video images;

s23, comparing the second frame video image with the first frame video image, judging whether there is a changed area, if not, executing step S24, if yes, executing step S25;

s24, discarding the first frame of video image, namely discarding the previous frame of video image in the two adjacent frames of video images, and feeding back the classification of the invading foreign body as an article to a decision maker without performing subsequent operation;

s25, determining the moving pixel distance of the invaded foreign body; namely, the positions of the invading foreign matters in the two frames of video images are compared to determine the moving pixel distance of the invading foreign matters;

s26, determining the real moving distance of the invaded foreign matter based on the scale parameter, namely determining the real moving distance corresponding to the moving pixel distance according to the scale;

s27, determining the moving speed of the invading foreign matter based on the moving real distance of the invading foreign matter, namely determining the moving speed of the invading foreign matter according to the moving real distance and the time interval of two frames of video image shooting; the decision maker is fed back that the invasive foreign body is an animal and the movement direction of the invasive foreign body is predicted.

5) And if the foreign matters are determined, sending out early warning in a corresponding form according to the types of the foreign matters.

For example, when foreign objects appear on the articles, the information is sent to the mobile phone of the maintenance personnel on the corresponding road section through the Internet so as to be conveniently removed at fixed points; when an animal appears, the system tracks the animal across shots and automatically generates a route for maintenance personnel to find and clear the animal.

The method of cross-shot tracking (this method is merely an example, and other cross-shot tracking methods are also feasible) is:

and decoding the video stream under the single lens by acquiring the video stream under the single lens to obtain a multi-frame video frame image under the single lens, wherein the multi-frame video frame image under the single lens comprises an animal target or a morphological target. And carrying out target detection tracking on the multi-frame video frame images, and determining the associated information of the animal target and the morphological target in the multi-frame video frame images. The association information between the animal target and the morphological target may be association between the animal target and the morphological target, or association between the animal target and the morphological target may be referred to as association information. Specifically, the first target detection model may be used to detect the animal target in the multi-frame video frame images to obtain an animal target boundary frame, the second target detection model may be used to detect the morphological target in the multi-frame video frame images to obtain a morphological target boundary frame, and finally the association information between the animal target and the morphological target in the multi-frame video frame images is determined according to the intersection ratio between the animal target boundary frame and the morphological target boundary frame and the track overlapping rate between the animal target and the morphological target. The first target detection model is mainly used for detecting animal targets, and the second target detection model is mainly used for detecting morphological targets. When determining the associated information of the animal target and the morphological target in the multi-frame video frame image according to the intersection ratio of the animal target boundary frame and the morphological target boundary frame and the track overlapping rate of the animal target and the morphological target, the method can be realized by the following steps: if the intersection ratio of the animal target boundary frame and the morphological target boundary frame is larger than a first preset threshold, preliminarily determining that the animal target is associated with the morphological target, then judging whether a tracking chain of the animal target and the morphological target is broken, and if the tracking chain is broken, further determining that the animal target is associated with the morphological target when the track overlapping rate of the animal target and the morphological target is larger than a second preset threshold. And if the animal target is not fractured, continuously determining the intersection ratio of the animal target boundary frame and the morphological target boundary frame until the association information of the animal target and the morphological target is determined. The first preset threshold and the second preset threshold may be set empirically.

The foregoing embodiments and description have been presented only to illustrate the principles and preferred embodiments of the invention, and various changes and modifications may be made therein without departing from the spirit and scope of the invention as hereinafter claimed.