1. A method for identifying a black smoke ship is characterized by comprising the following steps:

controlling a camera to carry out regional cruise and detecting ship information according to a preset visual field coordinate;

if the ship information is detected in the regional cruising process, tracking the ship information to obtain a tracking video;

carrying out black smoke identification on the hull information in the tracking video to obtain a black smoke ship;

and extracting the ship body code of the black smoke ship.

2. The method for identifying the black smoke ship as claimed in claim 1, wherein the controlling the camera to perform the region cruising and the detecting of the ship body information according to the preset sight coordinates comprises:

setting a visual field coordinate of a camera during cruising, wherein the visual field coordinate covers a ship sailing area;

controlling the camera to carry out regional cruise at preset time intervals according to the view coordinates;

and in the process of regional cruising, detecting the ship information by adopting a preset first neural network to the video frame information collected under the visual field coordinate.

3. The method for identifying the black smoke ship according to claim 2, wherein the detecting the ship hull information by using a preset first neural network to the video frame information collected under the visual field coordinate in the process of regional cruising comprises:

traversing video frame information collected under the view field coordinates, and segmenting the video frame information to obtain a plurality of image blocks, wherein the image blocks are rectangular and an overlapping area exists between adjacent image blocks;

carrying out ship information detection on each image block through the first neural network to generate at least one detection frame corresponding to the image block;

collecting the detection frame corresponding to each image block on the video frame information, and performing non-maximum suppression operation nms on the detection frame on the video frame information to obtain a target detection frame;

and acquiring ship information according to the target detection frame.

4. The method for identifying a black smoke vessel according to claim 3, wherein before the video frame information is divided into a plurality of image blocks, the method further comprises:

adaptively padding black edges for the video frame information to scale the video frame information to a uniform size.

5. The method for identifying the black smoke ship according to any one of claims 1 to 4, wherein if the ship hull information is detected during the regional cruising, tracking the ship hull information to obtain a tracking video comprises:

presetting a target range of ship information;

if the ship information is detected in the regional cruising process, adjusting the position of the camera according to the target range so as to keep the center point of the ship in the target range all the time;

and tracking the ship information through the adjusted camera to obtain a tracking video.

6. The method of claim 5, wherein the target range is 35% of the video frame width to the left of the video frame, 35% of the video frame width to the right of the video frame, and 50% of the video frame height to the top of the video frame.

7. The method for identifying the black smoke ship according to any one of claims 1 to 4, wherein the step of performing black smoke identification on the ship body information in the tracking video to obtain the black smoke ship comprises the following steps:

performing background filtering on each video frame information in the tracking video by adopting an improved space-time dynamic Gaussian background model, and extracting foreground information;

reconstructing the foreground information by adopting a super-resolution algorithm;

performing feature extraction on the reconstructed foreground information by adopting a dense optical flow algorithm to obtain optical flow features;

smoke extraction is carried out on the reconstructed foreground information by adopting a Local Binary Pattern (LBP) to obtain the texture characteristics of the smoke;

performing feature classification on the reconstructed foreground information by adopting a long-term and short-term memory network to obtain a plurality of spatial features;

and inputting the light stream features, the texture features and the spatial features into a Support Vector Machine (SVM) for black smoke recognition, and acquiring a black smoke ship.

8. The method for identifying a black smoke vessel as claimed in claim 7, wherein the improved spatiotemporal dynamic gaussian background model employs a dynamic learning rate;

learning video frame information before a preset frame number threshold in the tracking video according to a first learning rate, and learning video frame information after the frame number threshold according to a second learning rate;

wherein the first learning rate is greater than the second learning rate.

9. The method for identifying a black smoke vessel according to any one of claims 1 to 4, wherein the extracting of the hull code of the black smoke vessel comprises:

carrying out coordinate recognition on the black smoke ship by adopting a preset second neural network to obtain a ship body coding coordinate;

extracting a ship coding region block diagram from video frame information according to the ship coding coordinates, and correcting a character region in the ship coding region block diagram;

cutting the corrected character area to obtain a plurality of character blocks;

recognizing the character blocks by adopting a multi-layer perceptron MLP network to obtain character information corresponding to each character block;

and combining the character information to obtain the ship body code.

10. The method for identifying a black smoke ship according to claim 9, wherein the extracting of the hull coding region block map from the video frame information according to the hull coding coordinates and the correcting of the character region in the hull coding region block map comprises:

extracting a ship coding region block diagram from the video frame information according to the ship coding coordinates;

converting the ship body coding block map into a binary image;

calculating a horizontal deflection angle and a vertical deflection angle of a character area in the ship body coding block diagram according to the binary image;

and correcting the character area in the ship body coding block diagram according to the horizontal deflection angle and the vertical deflection angle.

11. The method for identifying a black smoke vessel as claimed in claim 9, wherein the cutting the corrected character region to obtain a plurality of character blocks comprises:

carrying out color space transformation on the corrected character area, and converting the character area from a color image into a gray image;

carrying out binarization processing on the gray level image to obtain a binarization image of the character area;

carrying out fuzzy degree processing on the binary image of the character area;

performing character pre-segmentation according to the image after the fuzziness processing;

performing horizontal projection and vertical projection on the character area after the character is pre-divided to respectively obtain top and bottom information and left and right information of each character;

and dividing the character area into a plurality of character blocks according to the top and bottom information and the left and right information.

12. An identification device for a black smoke ship, comprising:

the cruise module is used for controlling the camera to carry out regional cruise and detecting ship information according to preset visual field coordinates;

the tracking module is used for tracking the ship information to obtain a tracking video if the ship information is detected in the regional cruising process;

the black smoke identification module is used for carrying out black smoke identification on the ship body information in the tracking video to obtain a black smoke ship;

and the extraction module is used for extracting the ship body code of the black smoke ship.

13. A computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the method for identifying a black smoke vessel according to any one of claims 1 to 11.

14. A computer arrangement comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the method of identification of a black smoke vessel according to any one of claims 1 to 11 when executing the computer program.

Background

The pollution of ships is not a little and the problem of black smoke is particularly prominent. In 2019, China already defines a ship atmospheric pollutant discharge control area, and ships entering the control area should use fuel oil meeting the national standard requirements. However, a large number of ships are still in consideration of economic benefits, and the phenomenon that the ships emit black smoke is caused by using cheap and poor fuel which does not meet the requirements of national standards.

For the excessive emission behavior of the black smoke emitted by the ship, the ship boarding inspection mode is adopted in the prior art, equipment such as a nontransparent smokemeter and a portable Ringelmann blackness detector is used for manually detecting the content of the black smoke of the ship, the monitoring efficiency is low, the automation degree is low, and a large amount of manpower and material resources are consumed.

Disclosure of Invention

The embodiment of the invention provides a method, a device, a medium and equipment for identifying a black smoke ship, and aims to solve the problems of low automation degree and low monitoring efficiency in the process of identifying the black smoke ship in the prior art.

A method for identifying a black smoke ship comprises the following steps:

controlling a camera to carry out regional cruise and detecting ship information according to a preset visual field coordinate;

if the ship information is detected in the regional cruising process, tracking the ship information to obtain a tracking video;

carrying out black smoke identification on the hull information in the tracking video to obtain a black smoke ship;

and extracting the ship body code of the black smoke ship.

Optionally, the controlling the camera to perform area cruising and detecting ship information according to the preset visual field coordinate includes:

setting a visual field coordinate of a camera during cruising, wherein the visual field coordinate covers a ship sailing area;

controlling the camera to carry out regional cruise at preset time intervals according to the view coordinates;

and in the process of regional cruising, detecting the ship information by adopting a preset first neural network to the video frame information collected under the visual field coordinate.

Optionally, in the process of regional cruising, the detecting, by using a preset first neural network, the hull information of the video frame information collected under the visual field coordinate includes:

traversing video frame information collected under the view field coordinates, and segmenting the video frame information to obtain a plurality of image blocks, wherein the image blocks are rectangular and an overlapping area exists between adjacent image blocks;

carrying out ship information detection on each image block through the first neural network to generate at least one detection frame corresponding to the image block;

collecting the detection frame corresponding to each image block on the video frame information, and performing non-maximum suppression operation nms on the detection frame on the video frame information to obtain a target detection frame;

and acquiring ship information according to the target detection frame.

Optionally, before the video frame information is divided into a plurality of image blocks, the identifying method further includes:

and adaptively filling black edges in the video frame information so as to scale the video frame information to a uniform size.

Optionally, if hull information is detected in the area cruising process, tracking the hull information to obtain a tracking video includes:

presetting a target range of ship information;

if the ship information is detected in the regional cruising process, adjusting the position of the camera according to the target range so as to keep the center point of the ship in the target range all the time;

and tracking the ship information through the adjusted camera to obtain a tracking video.

Optionally, the target range is 35% of the video picture width to the left of the video picture, 35% of the video picture width to the right of the video picture, and 50% of the video picture height to the top of the video picture.

Optionally, the performing black smoke recognition on the hull information in the tracking video to obtain the black smoke ship includes:

performing background filtering on each video frame information in the tracking video by adopting an improved space-time dynamic Gaussian background model, and extracting foreground information;

reconstructing the foreground information by adopting a super-resolution algorithm;

performing feature extraction on the reconstructed foreground information by adopting a dense optical flow algorithm to obtain optical flow features;

smoke extraction is carried out on the reconstructed foreground information by adopting a Local Binary Pattern (LBP) to obtain the texture characteristics of the smoke;

performing feature classification on the reconstructed foreground information by adopting a long-term and short-term memory network to obtain a plurality of spatial features;

and inputting the light stream features, the texture features and the spatial features into a Support Vector Machine (SVM) for black smoke recognition, and acquiring a black smoke ship.

Optionally, the improved spatiotemporal dynamic gaussian background model employs a dynamic learning rate;

learning video frame information before a preset frame number threshold in the tracking video according to a first learning rate, and learning video frame information after the frame number threshold according to a second learning rate;

wherein the first learning rate is greater than the second learning rate.

Optionally, the extracting the hull code of the black smoke vessel comprises:

carrying out coordinate recognition on the black smoke ship by adopting a preset second neural network to obtain a ship body coding coordinate;

extracting a ship coding region block diagram from the video frame information according to the ship coding coordinates, and correcting a character region in the ship coding region block diagram;

cutting the corrected character area to obtain a plurality of character blocks;

recognizing the character blocks by adopting a multi-layer perceptron MLP network to obtain character information corresponding to each character block;

and combining the character information to obtain the ship body code.

Optionally, the extracting a hull coding block map from the video frame information according to the hull coding coordinates, and the correcting the character region in the hull coding block map includes:

extracting a ship coding region block diagram from the video frame information according to the ship coding coordinates;

converting the ship body coding block map into a binary image;

calculating a horizontal deflection angle and a vertical deflection angle of a character area in the ship body coding block diagram according to the binary image;

and correcting the character area in the ship body coding block diagram according to the horizontal deflection angle and the vertical deflection angle.

Optionally, the cutting the corrected character region to obtain a plurality of character blocks includes:

carrying out color space transformation on the corrected character area, and converting the character area from a color image into a gray image;

carrying out binarization processing on the gray level image to obtain a binarization image of the character area;

carrying out fuzzy degree processing on the binary image of the character area;

performing character pre-segmentation according to the image after the fuzziness processing;

performing horizontal projection and vertical projection on the character area after the character is pre-divided to respectively obtain top and bottom information and left and right information of each character;

and dividing the character area into a plurality of character blocks according to the top and bottom information and the left and right information.

An identification device for a black smoke vessel, the device comprising:

the cruise module is used for controlling the camera to carry out regional cruise and detecting ship body information according to a preset visual field coordinate;

the tracking module is used for tracking the ship information to obtain a tracking video if the ship information is detected in the regional cruising process;

the black smoke identification module is used for carrying out black smoke identification on the ship body information in the tracking video to obtain a black smoke ship;

and the extraction module is used for extracting the ship body code of the black smoke ship.

A computer-readable storage medium, storing a computer program which, when executed by a processor, implements the method of identifying a black smoke vessel as described above.

A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the method of identifying a black smoke vessel as described above when executing the computer program.

According to the embodiment of the invention, the camera is controlled according to the preset visual field coordinate to carry out regional cruising and ship information detection; if the ship information is detected in the regional cruising process, tracking the ship information to obtain a tracking video; carrying out black smoke identification on the hull information in the tracking video to obtain a black smoke ship; finally, extracting hull codes from the black smoke ship; therefore, automatic identification of the black smoke ship is completed, and the monitoring efficiency and accuracy of the black smoke ship are improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.

FIG. 1 is a flow chart of a method for identifying a black smoke vessel in accordance with an embodiment of the present invention;

fig. 2 is a flowchart of step S101 in the identification method of the black smoke vessel according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of an overlapping area provided in the method for identifying a black smoke vessel according to an embodiment of the present invention;

fig. 4 is a flowchart of step S102 in the identification method of the black smoke vessel according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a target range provided in the method for identifying a black smoke vessel according to an embodiment of the present invention;

fig. 6 is a flowchart of step S103 in the identification method of the black smoke vessel according to an embodiment of the present invention;

fig. 7 is a flowchart of step S104 in the identification method of the black smoke vessel according to an embodiment of the present invention;

FIG. 8 is a schematic diagram illustrating a process for correcting a block diagram of a ship body coding region provided by the identification method of a black smoke ship in an embodiment of the present invention;

fig. 9 is a schematic diagram of a segmentation process of a character area provided by the identification method of a black smoke vessel in an embodiment of the invention;

FIG. 10 is a schematic projection diagram provided by the identification method of a black smoke vessel according to an embodiment of the present invention;

FIG. 11 is a schematic view of a black smoke vessel identification device in accordance with an embodiment of the present invention;

FIG. 12 is a schematic diagram of a computing device in accordance with an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making an invasive task, are within the scope of the present invention.

The embodiment provides a method for identifying a black smoke ship, which comprises the steps of controlling a camera to carry out regional cruise and detecting ship information according to preset visual field coordinates; if the ship information is detected in the regional cruising process, tracking the ship information to obtain a tracking video; carrying out black smoke identification on the hull information in the tracking video to obtain a black smoke ship; finally, extracting hull codes from the black smoke ship; therefore, automatic identification of the black smoke ship is completed, and monitoring efficiency of the black smoke ship is improved.

The following describes in detail the method for identifying a black smoke vessel provided in this embodiment, as shown in fig. 1, the method includes:

in step S101, the camera is controlled to perform area navigation and to detect hull information according to preset sight coordinates.

Here, the visual field coordinates refer to a horizontal rotation angle P, a vertical rotation angle T, and a magnification Z of the camera head on the camera. The embodiment of the invention customizes a plurality of visual field coordinates in advance, and instructs the camera of the camera to carry out regional cruise and detect the ship information through the visual field coordinates. As shown in fig. 2, the step S101 further includes:

in step S201, the field-of-view coordinates at the time of camera cruising, which covers the ship cruising area, are set.

Different from the field of view coordinates fixed by a camera in the prior art, the embodiment of the invention covers the ship navigation area by pre-customizing a plurality of field of view coordinates, preferably 5 field of view coordinates, and stores the field of view coordinates into the database, thereby effectively preventing the loss of the field of view coordinates of the camera dome during power failure, and performing diversified cruising according to services after storing the field of view coordinates into the database, thereby realizing high customization.

In step S202, the camera is controlled to perform zone cruising at a preset time interval according to the field coordinates.

In the embodiment of the present invention, the preset time interval is preferably 6 seconds. And the camera dome camera passes through each visual field coordinate, stays at each visual field coordinate for a preset time interval, acquires video frame information and executes regional cruise.

In step S203, during the regional cruising process, a preset first neural network is used to detect the hull information of the video frame information collected under the view coordinates.

Here, the present invention detects hull information using a CNN convolutional neural network. If the ship body information is detected within the preset time interval, executing the step S102; otherwise, if the ship body information is not detected within the preset time interval, jumping to the next view field coordinate, and when the view field coordinates poll one side, continuing cruising from the beginning.

As a preferred example of the present invention, since the view range corresponding to each view coordinate is large, the video frame information acquired by the camera is generally a large-resolution image, and the ship is small relative to the view range. The problems of high detection difficulty, low detection accuracy and video card explosion generally exist when small targets are detected in a high-resolution image in the prior art. In view of this, the embodiment of the present invention adopts a blocking detection manner, and performs hull information detection after blocking the video frame information through the first neural network. The step S203 further includes:

in step S301, traversing the video frame information acquired under the view coordinates, and segmenting the video frame information to obtain a plurality of image blocks, where the image blocks are rectangles, and an overlapping region exists between adjacent image blocks, and the overlapping region is a rectangle with adjacent sides of the image blocks as side lengths.

Here, the embodiment of the present invention first divides the video frame information with large resolution, so that the video frame information becomes smaller image blocks. In order to avoid that some ships are just cut off between two image blocks, the embodiment of the invention reserves an overlapping area between adjacent image blocks. The overlapping area is a rectangle with the adjacent sides of the image blocks as the side length, and the other side length of the overlapping area is taken according to the preset proportion of the non-adjacent sides of the image blocks. For example, assuming that the preset ratio is 20%, the pixels of two adjacent image blocks are a × b, and a is taken as an adjacent edge, the size of the overlapping area may be set to a × 20% > × b. As shown in fig. 3, which is a schematic diagram of an overlapping area provided in an embodiment of the present invention, a dashed box represents one image block, and a solid box represents an adjacent image block.

Optionally, in practical applications, lengths of image frame information acquired by the cameras are different, and sizes of black edges at two ends of the image frame information are also different. In view of this, before the dividing the video frame information into a plurality of image blocks, the embodiment of the present invention may further include:

and adaptively filling black edges in the video frame information so as to scale the video frame information to a uniform size.

The method has the advantages that the black edges are filled in a self-adaptive mode, the minimum black edges are added to the original image frame information in a self-adaptive mode, image information redundancy can be effectively avoided, the first neural network is convenient to carry out block detection on the video frame information, and the speed and the precision of ship information detection are improved.

In step S302, ship hull information detection is performed on each image block through the first neural network, and at least one detection frame corresponding to the image block is generated.

Here, when the first neural network detects the hull information of each image block, each suspected hull information is marked through a detection frame.

In step S303, the detection frames corresponding to each image block are collected to the video frame information, and a non-maximum suppression operation nms is performed on the detection frames on the video frame information to obtain a target detection frame.

After the hull information detection is completed on all the image blocks, all the detection frames are uniformly put on the video frame information, and then a Non-Maximum Suppression (NMS) operation is performed on the video frame information on the whole, so that redundant detection frames collected on the video frame information are removed, and only one most representative detection frame on the same detection area is reserved.

In step S304, ship hull information is acquired according to the target detection frame.

And obtaining a target detection frame after the nms operation, wherein the target detected by the target detection frame is used as the ship information in the embodiment of the invention.

In the embodiment of the invention, the video frame information is subjected to block detection on the ship hull information through the steps S301 to S304, so that the detection accuracy and the detection efficiency of the ship hull information are greatly improved, and the consumption of memory resources is reduced.

In step S102, if hull information is detected in the area cruise process, the hull information is tracked to obtain a tracking video.

After detecting the hull information through the area cruise of the above step S101, the embodiment of the present invention tracks the hull information. Here, the smoke emission position of the ship is generally above the ship body, and when smoke is detected in a black smoke ship, sufficient space needs to be left for the smoke emission point, so that smoke can be observed more clearly. In the prior art, a camera is manually operated, and the camera is operated at a position through a background server of the camera, so that manpower is greatly consumed. In view of this, the embodiment of the present invention implements automatic tracking of the ship information by presetting the target range. Optionally, as shown in fig. 4, the step S102 further includes:

in step S401, a target range of hull information is set in advance.

Here, the embodiment of the invention sets the target range of the hull information in advance. The target range is used for framing the position of the ship information in the picture. Optionally, the target range is 35% of the video picture width to the left of the video picture, 35% of the video picture width to the right of the video picture, and 50% of the video picture height to the top of the video picture. For ease of understanding, fig. 5 is a schematic diagram of the target range provided by the embodiment of the present invention.

In step S402, if the hull information is detected during the zone cruise, the position of the camera is adjusted according to the target range so that the hull center point is always kept within the target range.

And after the ship information is detected, dynamically adjusting the position of the camera by taking the target range as an operation reference. The effect of the adjustment is that the hull center point is always kept within the target range, i.e. the hull information always falls within the target range, so as to frame the position of the hull information in the picture.

In step S403, the adjusted camera tracks the ship information to obtain a tracking video.

According to the embodiment of the invention, the target range of the ship information is set, and then the position of the camera is adjusted according to the target range, so that the central point of the ship is always kept in the target range, the automatic tracking of the ship is realized, and the manpower cost is reduced; and by reserving 50% of the height of the video picture, more space is reserved for the position of smoking, and the detection rate of black smoke is favorably improved.

In step S103, black smoke recognition is performed on the hull information in the tracking video, and a black smoke ship is acquired.

Optionally, in the embodiment of the present invention, the hull black smoke recognition includes background filtering and foreground extraction, super-resolution black smoke detail promotion, black smoke motion feature extraction, black smoke texture feature extraction, black smoke spatial feature extraction, and feature weighting fusion and inference for 6 links. As shown in fig. 6, the step S103 further includes:

in step S601, an improved spatiotemporal dynamic gaussian background model is used to perform background filtering on each video frame information in the tracking video, and foreground information is extracted.

Here, the embodiment of the present invention finds a moving region from the tracking video, sets the moving region to white, and sets other non-moving regions to black background.

In the prior art, a mixed Gaussian background modeling algorithm is mainly adopted for background filtering and foreground information extractionAnd (4) information. However, the gaussian mixture background modeling algorithm consumes a large amount of cpu resources in the calculation process, and takes a long time to extract foreground information. In view of this, the embodiment of the present invention adopts an improved spatio-temporal dynamic gaussian background modeling DSTGMM based on spatio-temporal mixed gaussian background modeling, and performs background filtering on video frame information in the tracking video through a dynamic learning rate to extract foreground information. Specifically, a frame number threshold T is set in advance₀In the tracking video, T₀The video frame information before the frame is learned according to a first learning rate, T₀And learning the video frame information after the frame according to a second learning rate, wherein the first learning rate is greater than the second learning rate, so that the contour of the ship can be found more quickly, and the detection efficiency is improved.

In step S602, the foreground information is reconstructed by using a super-resolution algorithm.

According to the embodiment of the invention, the foreground information is reconstructed by adopting a super-resolution algorithm, so that the problems of video blurring and serious noise can be effectively solved, and the false detection rate of black smoke identification is effectively reduced.

In step S603, feature extraction is performed on the reconstructed foreground information by using a dense optical flow algorithm to obtain optical flow features.

Here, in order to observe the motion trajectory of the hull information in the tracking video, the embodiment of the invention adopts dense optical flow to extract the motion features thereof.

In step S604, smoke extraction is performed on the reconstructed foreground information by using the local binary pattern LBP, so as to obtain texture features of the smoke.

Here, in order to extract the texture of smoke, the embodiment of the present invention uses a Local Binary Pattern (LBP) texture classification feature algorithm to perform extraction.

In step S605, the long and short term memory network is used to perform feature classification on the reconstructed foreground information, so as to obtain a plurality of spatial features.

In order to enable the neural network to classify the characteristics of each video frame information in each tracking video and to use the previous characteristic to infer the next event, the embodiment of the present invention uses a Long Short-Term Memory network (LSTM) to perform Long-Term Memory, performs characteristic classification on the reconstructed foreground information through the LSTM neural network, and then outputs the classified spatial characteristics.

In step S606, the optical flow features, texture features, and spatial features are input into a support vector machine SVM to perform black smoke recognition, and a black smoke ship is acquired.

The embodiment of the invention trains a Support Vector Machine (SVM) in advance, performs weighted fusion and black smoke recognition on the basis of the optical flow physical signs, the texture characteristics and the spatial characteristics through the SVM, and outputs the judgment result of whether the ship body information is a black smoke ship. According to the embodiment of the invention, the black smoke ship is obtained according to the output result of the support vector machine SVM, so that the automatic identification of the black smoke ship is realized, and the monitoring efficiency and the accuracy of the black smoke ship are improved.

After the identification of the black smoke ship is completed, the identity information of the black smoke ship, namely the ship body code, needs to be extracted.

In step S104, a hull code of the black smoke vessel is extracted.

Optionally, as a preferred example of the present invention, the hull coding extraction process is divided into four links, namely, hull coding positioning, hull coding rectification, character segmentation and character recognition. As shown in fig. 7, step S104 further includes:

in step S701, coordinate recognition is performed on the black smoke vessel by using a preset second neural network, and a vessel body code coordinate is obtained.

Here, the embodiment of the present invention extracts coordinate information of the hull code using the CNN convolutional neural network. The ship hull coding images are input into a pre-constructed CNN convolutional neural network for learning by collecting a specified number of ship hull coding images in advance, such as 5000 ship hull coding images, and the CNN convolutional neural network is trained. And finally, identifying the ship body codes in the black smoke ship by adopting the trained CNN convolutional neural network to obtain the coordinate information of the ship body codes.

Here, the second neural network may also perform the hull coding coordinate detection after the blocking and perform the adaptive black edge filling on the video frame information, which is described in the above embodiments and is not described herein again.

In step S702, a hull coding block map is extracted from the video frame information according to the hull coding coordinates, and character areas in the hull coding block map are corrected.

In the embodiment of the invention, video frame information containing ship body information is randomly selected from the tracking video, and a ship body coding block diagram is extracted from the video frame information according to the ship body coding coordinates. It should be understood that the block diagram of the hull coding area only includes the character part of the hull coding. Because the ship body coding characters obtained by the camera at different shooting visual angles are distorted, irregular and the like, in order to improve the recognition accuracy of the ship body coding, the embodiment of the invention further corrects the character areas in the ship body coding block diagram. The step S702 further includes:

in step S801, a hull coding block map is extracted from the video frame information according to the hull coding coordinates.

In step S802, the ship body coding block map is converted into a binarized image.

In step S803, a horizontal skew angle and a vertical skew angle of a character region in the hull code block map are calculated from the binarized image.

Here, in the embodiment of the present invention, a region of interest (ROI) operation is performed on the binarized image, and a character region corresponding to the hull code is outlined. And calculating the horizontal deflection angle and the vertical deflection angle of the character area. The horizontal deflection angle refers to a horizontal rotation angle of a camera on the camera, and the vertical deflection angle refers to a vertical rotation angle of the camera on the camera.

In step S804, the character areas in the ship hull coding block map are corrected according to the horizontal offset angle and the vertical offset angle.

According to the embodiment of the invention, through correction, including but not limited to operations such as stretching and zooming, character areas in the corrected ship body coding block diagram are normalized, and a rectangle with regular horizontal and vertical directions is presented.

For easy understanding, fig. 8 is a schematic diagram of a correction process of a ship hull coding block diagram according to an embodiment of the present invention. Wherein, fig. 8(a) shows a ship body coding block diagram extracted according to the coordinate information; FIG. 8(b) shows a binary image corresponding to the ship body coding block diagram; fig. 8(c) shows a character region outlined from the binarized image; FIG. 8(d) shows character areas in the ship's hull code block diagram; fig. 8(e) shows the character area after correction.

In step S703, the corrected character area is cut to obtain a plurality of character blocks.

Because the ship body code is usually a mechanical code character, in order to divide the mechanical code character as correctly as possible, the embodiment of the invention cuts the ship body code after preprocessing. Optionally, step S703 further includes:

in step S901, the corrected character region is subjected to color space conversion, and converted from a color image to a grayscale image.

In step S902, a binarization process is performed on the grayscale map to obtain a binarized image of the character region.

In step S903, the binarized image of the character region is subjected to blur degree processing.

In step S904, character pre-segmentation is performed based on the image after the blur degree processing.

For easy understanding, fig. 9 is a schematic diagram of a segmentation process of a character region according to an embodiment of the present invention. In fig. 9, 1 denotes an input corrected character area; 2, representing a gray scale image of the character area after color space conversion; 3, representing a binary image of the gray level image after binarization processing; FIG. 4 illustrates the character area after ambiguity Blur processing; and 5, a character area after character pre-segmentation. As can be seen, the character area after the character is pre-segmented has been marked with the segmentation identification.

In step S905, the character region after the character pre-segmentation is subjected to horizontal projection and vertical projection, and top and bottom information, and left and right information of each character are obtained, respectively.

In step S906, the character area is divided into a number of character blocks according to the top and bottom information and the left and right information.

Here, by projecting the histogram by the pixels, the upper and lower limits, the left and right sides of each character can be accurately found. Embodiments of the present invention find the top and bottom of each character based on horizontal projection and the left and right of each character based on vertical projection. Fig. 10 is a schematic projection diagram provided in the embodiment of the present invention. And finally, dividing a plurality of character blocks according to the top and bottom information and the left and right information of the characters. The character area is cut through the steps from S901 to S906, which is beneficial to improving the accuracy of cutting the character block and facilitating subsequent character recognition.

Alternatively, as another preferred example of the present invention, it is also possible to generate connected regions by using morphological open/close operations, and then extract connected regions using a connected tracking algorithm, each of which represents one character block.

In step S704, the multilayer perceptron MLP network is used to identify the character blocks, and character information corresponding to each character block is obtained.

The recognition stage is the last link of the mechanical coding automatic detection and recognition system, and the recognition in the embodiment of the invention is based on a single character block obtained in the previous link. Here, after comparing the multi-layer perceptron MLP and the K-neighbor classifier KNN, the embodiments of the present invention find that the more neurons of the multi-layer perceptron MLP and the K-neighbor classifier KNN are, the higher the classification performance is; however, the adjustable potential of the K-nearest neighbor classifier KNN is much smaller than that of the multi-layer perceptron MLP, so that the embodiment of the present invention preferably adopts the multi-layer perceptron MLP network to identify the segmented character blocks, and obtains the character information corresponding to each character block.

In step S705, the character information is combined to obtain the hull code.

According to the embodiment of the invention, each character in the ship body coding block diagram is independently cut, and each character is independently identified by adopting the multilayer perceptron MLP network, so that the accuracy of ship body coding identification is effectively improved.

And finally, outputting the detection result of the black smoke ship and the ship body code corresponding to the black smoke ship.

In summary, in the embodiment of the invention, the camera is controlled according to the preset visual field coordinate to perform regional cruising and ship information detection; if the ship information is detected in the regional cruising process, tracking the ship information to obtain a tracking video; carrying out black smoke identification on the hull information in the tracking video to obtain a black smoke ship; extracting a hull code of the black smoke ship; therefore, automatic identification of the black smoke ship is completed, and monitoring efficiency of the black smoke ship is improved.

It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

In one embodiment, a recognition device for a black smoke ship is provided, and the recognition device for the black smoke ship corresponds to the recognition method for the black smoke ship in the above embodiment one to one. As shown in fig. 11, the identification device for the black smoke ship includes a cruise module 10, a tracking module 20, a black smoke identification module 30, and an extraction module 40.

The functional modules are explained in detail as follows:

the cruise module 10 is used for controlling the camera to carry out regional cruise and detecting ship information according to preset visual field coordinates;

the tracking module 20 is configured to track the hull information to obtain a tracking video if the hull information is detected in the regional cruise process;

the black smoke recognition module 30 is configured to perform black smoke recognition on the hull information in the tracking video to obtain a black smoke ship;

and the extraction module 40 is used for extracting the hull codes of the black smoke ship.

The cruise module 10 comprises:

the system comprises a visual field coordinate setting unit, a navigation unit and a navigation unit, wherein the visual field coordinate setting unit is used for setting a visual field coordinate when a camera patrols the ship, and the visual field coordinate covers a ship navigation area;

the cruise unit is used for controlling the camera to carry out regional cruise at preset time intervals according to the visual field coordinates;

and the detection unit is used for detecting the ship information of the video frame information collected under the visual field coordinate by adopting a preset first neural network in the regional cruising process.

Optionally, the detection unit includes:

the dividing subunit is used for traversing the video frame information acquired under the view field coordinate, and dividing the video frame information to obtain a plurality of image blocks, wherein the image blocks are rectangular and an overlapping area exists between adjacent image blocks;

the detection subunit is used for detecting the ship body information of each image block through the first neural network to generate at least one detection frame corresponding to the image block;

the nms operation subunit is used for collecting the detection frame corresponding to each image block onto the video frame information, and performing non-maximum suppression operation nms on the detection frame on the video frame information to obtain a target detection frame;

and the acquisition subunit is used for acquiring the ship information according to the target detection frame.

Optionally, the detection unit further includes:

and the black edge filling subunit is used for adaptively filling the black edge into the video frame information so as to scale the video frame information to a uniform size.

Optionally, the tracking module 20 comprises:

a target range setting unit for setting a target range of the hull information in advance;

the adjusting unit is used for adjusting the position of the camera according to the target range if the ship information is detected in the regional cruising process, so that the center point of the ship is always kept in the target range;

and the tracking unit is used for tracking the ship information through the adjusted camera to obtain a tracking video.

Optionally, the target range is 35% of the video picture width to the left of the video picture, 35% of the video picture width to the right of the video picture, and 50% of the video picture height to the top of the video picture.

Optionally, the black smoke recognition module 30 includes:

the filtering unit is used for performing background filtering on each video frame information in the tracking video by adopting an improved space-time dynamic Gaussian background model and extracting foreground information;

the reconstruction unit is used for reconstructing the foreground information by adopting a super-resolution algorithm;

the first extraction unit is used for extracting the features of the reconstructed foreground information by adopting a dense optical flow algorithm to obtain optical flow features;

the second extraction unit is used for extracting smoke from the reconstructed foreground information by using a Local Binary Pattern (LBP) to obtain the texture characteristics of the smoke;

the third extraction unit is used for carrying out feature classification on the reconstructed foreground information by adopting a long-term and short-term memory network to obtain a plurality of spatial features;

and the black smoke recognition unit is used for inputting the optical flow characteristics, the texture characteristics and the spatial characteristics into a Support Vector Machine (SVM) to perform black smoke recognition, so as to obtain a black smoke ship.

Optionally, the improved spatiotemporal dynamic gaussian background model employs a dynamic learning rate;

learning video frame information before a preset frame number threshold in the tracking video according to a first learning rate, and learning video frame information after the frame number threshold according to a second learning rate;

wherein the first learning rate is greater than the second learning rate.

Optionally, the extraction module 40 includes:

the coordinate identification unit is used for carrying out coordinate identification on the black smoke ship by adopting a preset second neural network to obtain a ship body coding coordinate;

the correction unit is used for extracting a ship encoding block map from the video frame information according to the ship encoding coordinates and correcting a character area in the ship encoding block map;

the cutting unit is used for cutting the corrected character area to obtain a plurality of character blocks;

the character recognition unit is used for recognizing the character blocks by adopting a multi-layer perceptron MLP network to obtain character information corresponding to each character block;

and the combination unit is used for combining the character information to obtain the ship body code.

Optionally, the correction unit comprises:

the block extraction subunit is used for extracting a ship encoding block map from the video frame information according to the ship encoding coordinates;

the first binarization processing subunit is used for converting the ship coding region block map into a binarization image;

a deflection angle calculating subunit, configured to calculate, according to the binarized image, a horizontal deflection angle and a vertical deflection angle of a character region in the ship body coding region block map;

and the corrector subunit is used for correcting the character region in the ship body coding region block diagram according to the horizontal deflection angle and the vertical deflection angle.

Optionally, the cutting unit comprises:

a grayscale map conversion subunit, configured to perform color space conversion on the corrected character region, and convert the character region from a color map to a grayscale map;

the second binarization processing subunit is used for carrying out binarization processing on the gray level image to obtain a binarization image of the character area;

a Blur processing subunit, configured to perform Blur degree processing on the binarized image of the character region;

the segmentation preprocessing unit is used for performing character pre-segmentation according to the image after the fuzziness processing;

the projection subunit is used for performing horizontal projection and vertical projection on the character area after the character is pre-divided to respectively obtain top and bottom information, left and right information of each character;

and the dividing subunit is used for dividing the character area into a plurality of character blocks according to the top and bottom information and the left and right information.

For specific definition of the identification device for the black smoke vessel, reference may be made to the above definition of the identification method for the black smoke vessel, and details are not repeated here. The modules in the identification device of the black smoke ship can be completely or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 12. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through network connection. The computer program is executed by a processor to implement a method of identifying a black smoke vessel.

In one embodiment, there is provided a computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:

controlling a camera to carry out regional cruise and detecting ship information according to a preset visual field coordinate;

if the ship information is detected in the regional cruising process, tracking the ship information to obtain a tracking video;

carrying out black smoke identification on the hull information in the tracking video to obtain a black smoke ship;

and extracting the ship body code of the black smoke ship.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by the relevant hardware instructed by a computer program stored in a non-volatile computer-readable storage medium, and the computer program can include the processes of the embodiments of the methods described above when executed. Any reference to memory, storage, databases, or other media used in embodiments provided herein may include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

It will be apparent to those skilled in the art that, for convenience and simplicity of description, the foregoing functional units and modules are merely illustrated in terms of division, and in practical applications, the foregoing functional allocation may be performed by different functional units and modules as needed, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above described functions.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention, and are intended to be included within the scope of the present invention.