1. An intelligent disaster recovery system is characterized by comprising an intelligent disaster recovery device, an image acquisition device and an external display screen, wherein the intelligent disaster recovery device comprises an external interface, a communication module, a storage module, a power supply module and a calculation module, the calculation module is respectively connected with the external interface, the communication module, the storage module and the power supply module, and the intelligent disaster recovery device is connected with the image acquisition device through the external interface;

the image acquisition device is used for acquiring a video image of a position to be surveyed and sending the video image to the intelligent disaster surveying device;

the intelligent disaster exploration device is used for detecting a target to be detected in the video image of the position to be explored through the pre-stored target image through the calculation module; marking the target to be detected in the video image of the disaster position to be surveyed, and generating a marked video image; and displaying the marked video image to a user through an external display screen.

2. The intelligent disaster recovery system of claim 1, wherein said computing module comprises a central processor and an accelerator card for deep learning;

the central processing unit is used for decoding the video image of the disaster position to be surveyed to obtain a plurality of frame images, respectively carrying out target detection on the plurality of frame images through a target detection model, marking a target object by using a frame and a number when the target object is detected, operating a state identification model, carrying out logic judgment on the state of the target object, generating disaster surveying result data containing a state judgment result, and outputting the disaster surveying result data to a display screen through the peripheral interface;

the accelerator card is used for carrying out accelerated calculation on the neural network of the target detection model and the state recognition model.

3. The intelligent disaster exploration system of claim 1, wherein said peripheral interface comprises a high definition multimedia interface.

4. The intelligent disaster exploration system of claim 3, wherein said image acquisition device is an image acquisition device on an unmanned aerial vehicle, said unmanned aerial vehicle is wirelessly connected with an unmanned aerial vehicle remote control device, said intelligent disaster exploration device is connected with an unmanned aerial vehicle remote control device, said unmanned aerial vehicle remote control device is connected with said intelligent disaster exploration device through said high definition multimedia interface.

5. The intelligent disaster exploration system of claim 1, wherein said peripheral interfaces further comprise a universal serial bus interface, a video image array interface, a flexible flat cable interface.

6. The intelligent disaster exploration system of claim 5, wherein said image acquisition device comprises a vehicle-mounted camera, said vehicle-mounted camera being connected to said intelligent disaster exploration device through said universal serial bus interface.

7. The intelligent disaster recovery system according to claim 1, wherein said power supply module is provided with a power interface, and an external power source can be connected to said power interface through a usb interface to provide a 5V dc voltage input for said intelligent disaster recovery device.

8. The intelligent disaster investigation system of claim 1, wherein the intelligent disaster investigation device further comprises a storage module, and after the state of the target to be detected and the state of the target to be detected are counted into the disaster investigation result data in the file format, the storage module is configured to store the disaster investigation result data.

9. The intelligent disaster exploration system of claim 8, wherein said storage module further comprises an insertable secure digital card.

10. The intelligent disaster surveying system of claim 1, wherein said intelligent disaster surveying device further comprises a communication module.

Background

The power transmission depends on a large-scale line network, and when natural disasters occur, a power grid is easily affected to cause power failure accidents. After a disaster, efficient and rapid disaster situation survey is extremely important for emergency repair and power restoration and guarantee of safe and stable operation of a power grid.

Along with the popularization of multiple front end acquisition tools such as unmanned aerial vehicle, on-vehicle camera, the image data that gather the front end is passed back to the mode that the rear end platform carried out intelligent analysis that becomes more meticulous, has become the main mode that power line patrolled and examined daily. However, the mode is highly dependent on communication conditions and is obviously not applicable in the case of poor communication conditions in the field after disaster.

Aiming at the problem, the airborne front-end miniature image processing device of the unmanned aerial vehicle appears in the market, but is limited by computational resources of device hardware, most of the image processing device can only carry out simple pre-recognition on the image, can not carry out more precise image analysis and can not automatically arrange data; moreover, the equipment needs to be mounted on the unmanned aerial vehicle of a specific model, the weight of the unmanned aerial vehicle is increased, the flight control performance of the unmanned aerial vehicle is restricted, the cruising ability is influenced, and the equipment cannot be matched with other front-end acquisition equipment such as a vehicle-mounted camera and is single in use.

Disclosure of Invention

The embodiment of the invention provides an intelligent disaster recovery system, which aims to solve the problems of weak function, single use and low application flexibility of the existing front-end image processing device and comprises the following steps:

an intelligent disaster recovery system comprises an intelligent disaster recovery device, an image acquisition device and an external display screen, wherein the intelligent disaster recovery device comprises an external interface, a communication module, a storage module, a power supply module and a calculation module, the calculation module is respectively connected with the external interface, the communication module, the storage module and the power supply module, and the intelligent disaster recovery device is connected with the image acquisition device through the external interface;

the image acquisition device is used for acquiring a video image of a position to be surveyed and sending the video image to the intelligent disaster surveying device;

the intelligent disaster exploration device is used for detecting a target to be detected in the video image of the position to be explored through the pre-stored target image through the calculation module; marking the target to be detected in the video image of the disaster position to be surveyed, and generating a marked video image; and displaying the marked video image to a user through an external display screen.

Optionally, the computing module comprises a central processor and an accelerator card for deep learning;

the central processing unit is used for decoding the video image of the disaster position to be surveyed to obtain a plurality of frame images, respectively carrying out target detection on the plurality of frame images through a target detection model, marking a target object by using a frame and a number when the target object is detected, operating a state identification model, carrying out logic judgment on the state of the target object, generating disaster surveying result data containing a state judgment result, and outputting the disaster surveying result data to a display screen through the peripheral interface;

the accelerator card is used for carrying out accelerated calculation on the neural network of the target detection model and the state recognition model.

Optionally, the peripheral interface comprises a high definition multimedia interface.

Optionally, the image acquisition device is an image acquisition device on an unmanned aerial vehicle, the unmanned aerial vehicle is connected with an unmanned aerial vehicle remote control device in a wireless manner, the intelligent disaster exploration device is connected with the unmanned aerial vehicle remote control device, and the unmanned aerial vehicle remote control device is connected with the intelligent disaster exploration device through the high-definition multimedia interface.

Optionally, the peripheral interfaces further include a universal serial bus interface, a video image array interface, and a flexible flat cable interface.

Optionally, the image acquisition device includes a vehicle-mounted camera, and the vehicle-mounted camera is connected to the intelligent disaster investigation device through the universal serial bus interface.

Optionally, a power interface is arranged on the power supply module, and an external power supply can be switched to the power interface through a universal serial bus interface to provide 5V direct-current voltage input for the intelligent disaster recovery device.

Optionally, the intelligent disaster reconnaissance device further includes a storage module, and after the statistics of the target to be detected and the reconnaissance result data in the file format of the state of the target to be detected, the storage module is configured to store the reconnaissance result data.

Optionally, the memory module further comprises an insertable secure digital card.

Optionally, the intelligent disaster exploration device further includes a communication module.

The invention has the following advantages:

1. the invention does not relate to any transformation of a front-end image acquisition tool, is plug-and-play and has low cost;

2. the invention is provided with a high-performance computing module, and a high-accuracy target detection model and a state judgment model are carried in the computing module, so that the disaster situation of a target object can be identified and judged on site in real time and rapidly;

3. the invention can realize the real-time generation of the disaster investigation result data in the disaster investigation site without transmitting the image data back to the background for analysis, is suitable for the scene with poor communication conditions after the disaster, and improves the disaster investigation efficiency.

Drawings

In order to more clearly illustrate the technical solution of the present invention, the drawings needed to be used in the description of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

Fig. 1 is a schematic structural diagram of an intelligent disaster recovery system according to an embodiment of the present invention;

fig. 2 is a schematic structural diagram of an intelligent disaster-surveying device in cooperation with an unmanned aerial vehicle according to an embodiment of the present invention;

fig. 3 is a schematic structural diagram of an intelligent disaster-surveying device in combination with a vehicle according to an embodiment of the present invention;

fig. 4 is a schematic flow chart of a disaster surveying process of an intelligent disaster surveying device according to an embodiment of the present invention.

Detailed Description

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below. It is to be understood that the embodiments described are only a few embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The power transmission depends on a large-scale line network, and when natural disasters occur, a power grid is easily affected to cause power failure accidents. After a disaster, efficient and rapid disaster situation survey is extremely important for emergency repair and power restoration and guarantee of safe and stable operation of a power grid. With the development and popularization of unmanned aerial vehicle technology, the inspection and survey of power transmission lines by using the unmanned aerial vehicle becomes an effective scheme for replacing manual inspection, and the working mode of the conventional unmanned aerial vehicle inspection highly depends on manual operation inspection and data arrangement, so that the rapidity requirement for ensuring emergency response in the disaster exploration field is difficult to meet.

A recent similar approach: an unmanned aerial vehicle inspection system based on edge calculation and an inspection method thereof, application number 202010197746.1, publication number is CN 111522355A.

In this scheme, unmanned aerial vehicle passes through real-time transmission system and ground station real-time connection, and the unmanned aerial vehicle bottom is fixed with miniature image processing device, the camera fix in the unmanned aerial vehicle bottom and with unmanned aerial vehicle's control system electric connection, miniature image processing device includes the casing, is equipped with GPU image processor and data storage module in the casing. The unmanned aerial vehicle who equips through carrying on miniature image processing can reduce the time loss that picture wireless transmission brought, can reduce ground station image processing's pressure through the discernment in advance of GPU image processor to the picture trouble, and normal image need not to transmit to ground service station, has solved current unmanned aerial vehicle and has patrolled and examined the problem that the system is too big frequently and picture transmission is consuming time for a long time.

However, the airborne miniature image processing device of the unmanned aerial vehicle can only pre-identify the picture, and the more refined image analysis still needs to be transmitted back to the ground end for processing, so that the data cannot be automatically sorted; the airborne front-end image processing device of the unmanned aerial vehicle can restrict the flight control performance of the unmanned aerial vehicle, influence the endurance and improve the manufacturing cost; the intelligent identification of the front-end image is highly dependent on the customized products of the unmanned aerial vehicle manufacturer, and has no multiple application modes.

In order to solve the above problems, referring to fig. 1, a schematic structural diagram of an intelligent disaster recovery system according to an embodiment of the present invention is shown, where the intelligent disaster recovery system includes an intelligent disaster recovery device, an image acquisition device, and an external display screen, the intelligent disaster recovery device includes a peripheral interface, a communication module, a storage module, a power supply module, and a calculation module, the calculation module is respectively connected to the peripheral interface, the communication module, the storage module, and the power supply module, and the intelligent disaster recovery device is connected to the image acquisition device through the peripheral interface;

the image acquisition device is used for acquiring a video image of a position to be surveyed and sending the video image to the intelligent disaster surveying device;

the intelligent disaster exploration device is used for detecting a target to be detected in the video image of the position to be explored through the pre-stored target image through the calculation module; marking the target to be detected in the video image of the disaster position to be surveyed, and generating a marked video image; and displaying the marked video image to a user through an external display screen.

In an embodiment of the present application, the computing module includes a central processing unit and an accelerator card for deep learning;

the image acquisition device sends acquired video images of the positions to be surveyed to the computing module through the peripheral interface, the central processing unit decodes the video images of the positions to be surveyed to obtain a plurality of frame images, the frame images are respectively subjected to target detection through a target detection model, when a target object is detected, the target object is marked by a frame and a number, a state identification model is operated, the state of the target object is logically judged to generate disaster exploration result data containing a state judgment result, the disaster exploration result data are output to the display screen through the peripheral interface, and meanwhile, the disaster exploration result data are stored to the storage module in a file format;

the computing module integrates a target detection and automatic identification algorithm based on deep learning, performs image processing, target detection, serial number tracking and state identification on an externally input video stream, outputs a disaster investigation result, and further completes automatic arrangement and statistics of disaster investigation data.

The accelerator card is used for performing accelerated calculation on the neural network of the target detection model and the state recognition model, so that the requirements of high performance, low power consumption and low time delay of a disaster investigation application scene are met, when the video stream acquired by an image acquisition tool such as an unmanned aerial vehicle or a vehicle-mounted camera is subjected to data processing, the accelerator card can support processing of the video stream up to 1080p, has the processing capacity of 50 frames per second, and can detect and recognize the target with 60 x 60 pixels at minimum.

In an embodiment of the present application, the peripheral interface includes a high Definition Multimedia interface, which is not limited to a high Definition Multimedia interface, and a physical interface commonly used in the market at present is an hdmi (high Definition Multimedia interface), that is, a high Definition Multimedia interface, and the peripheral interface includes a high Definition Multimedia interface, which is suitable for an image capture device commonly used in the market at present. For example, unmanned aerial vehicle that generally uses in the market at present, no matter industrial level unmanned aerial vehicle or consumer-grade unmanned aerial vehicle, the last commonly used physical interface of its remote controller is HDMI, and the intelligent reconnaissance device of this application only needs can follow the video image that unmanned aerial vehicle remote control unit gathered through high definition multimedia interface, need not to do the conversion of other formats again, is applicable to the unmanned aerial vehicle type of generally using in the market at present.

Referring to fig. 2, a schematic structural diagram of an intelligent disaster recovery device collocated with an unmanned aerial vehicle according to an embodiment of the present invention is shown, where the image acquisition device is an image acquisition device on the unmanned aerial vehicle, the unmanned aerial vehicle is wirelessly connected with an unmanned aerial vehicle remote control device, the intelligent disaster recovery device is connected with an unmanned aerial vehicle remote control device, and the unmanned aerial vehicle remote control device is connected with the intelligent disaster recovery device through a high definition multimedia interface.

In an embodiment of the present application, the peripheral interfaces further include, but are not limited to, a universal serial bus interface, a video image array interface, a flexible flat cable interface, and an ethernet port, so that the intelligent disaster recovery device can be conveniently connected to different devices, and the application flexibility of the intelligent disaster recovery device is further enhanced. If the external camera is connected with the intelligent disaster exploration device through the FFC flexible flat cable or the USB interface to acquire video data; and the external display screen is connected with the disaster investigation equipment through an HDMI or VGA wire, and displays the marked and numbered target objects and the on-site disaster investigation results such as state judgment and the like in real time.

Referring to fig. 3, which is a schematic structural diagram illustrating a configuration of an intelligent disaster recovery device in combination with a vehicle according to an embodiment of the present invention, an image acquisition device includes a vehicle-mounted camera, and a universal serial bus interface of the vehicle-mounted camera is connected to the intelligent disaster recovery device.

In an embodiment of the present application, the intelligent disaster recovery device further includes a power supply module, the power supply module is provided with a power interface, and an external power source can be connected to the power interface through a usb interface to provide a 5V dc voltage input for the intelligent disaster recovery device. For example, the vehicle provides a USB interface and supplies power to the intelligent disaster exploration device through a USB-to-DC 5V line.

In an embodiment of the present application, the intelligent disaster prospecting device further includes a storage module, and an insertable secure digital card, which is used for storing the disaster prospecting result data such as the number, serial number, status, and picture of the detection targets counted in the disaster prospecting process, and forming a readable and copyable file format.

In an embodiment of the application, the intelligent disaster recovery device further comprises a communication module, and the disaster recovery result data can be sent to an emergency command center at the first time, so that an expert can rapidly make disaster assessment and schedule resources of superior departments to carry out emergency repair work.

Referring to fig. 4, a schematic flow chart of a disaster recovery process of an intelligent disaster recovery system according to an embodiment of the present invention is shown, where the disaster recovery process of the intelligent disaster recovery system includes the following steps:

step SS01, the front-end image acquisition tool acquires the video stream of the disaster investigation site;

when the unmanned aerial vehicle is used as a front-end image acquisition tool, starting the unmanned aerial vehicle to fly to a target disaster exploration site, acquiring a site video image by an airborne pan-tilt camera of the unmanned aerial vehicle, and transmitting the site video image back to an unmanned aerial vehicle remote control device through an unmanned aerial vehicle image transmission system; when the vehicle-mounted camera is used as a front-end image acquisition tool, starting a vehicle to run along a road in a disaster exploration site, and acquiring a site video image by the vehicle-mounted camera arranged at the top end of the vehicle;

step SS02, the intelligent disaster recovery device obtains video stream through the peripheral interface;

when the unmanned aerial vehicle is used as a front-end image acquisition tool, the intelligent disaster recovery device acquires a video stream returned by aerial photography of the unmanned aerial vehicle from the unmanned aerial vehicle remote control device by using an HDMI (high definition multimedia interface) line; when the vehicle-mounted camera is used as a front-end image acquisition tool, the intelligent disaster recovery device is directly connected with the vehicle-mounted camera by using FFC (flexible flat cable) to acquire a disaster recovery site video stream;

step SS03, decoding and frame taking by a computing module to obtain an image;

after the intelligent disaster exploration device obtains an exploration site video stream through an external interface, the video stream is decoded into a picture frame through a computing module, and the detection and identification of an exploration target are conveniently carried out by subsequently operating a target detection model and a state identification model;

step SS04, image preprocessing;

in order to facilitate the effective operation of the target detection model and the state recognition model, each frame of image is subjected to preprocessing operation, which comprises the following steps: adjusting the size of the image, scaling, cutting, filling and the like so that the input image meets the input requirements of a target detection model and a state recognition model;

step SS05, deep learning target detection;

inputting the preprocessed single-frame image into a target detection model, carrying out parallel computation on a neural network of the target detection model by an accelerator card so as to accelerate the whole target detection process, judging whether a target object exists, marking the target object by a square frame and a number if the target object exists, and detecting the next frame of image if the target object does not exist, until all frames of images in the video stream are detected;

step SS06, judging the target state;

extracting a target object area from the marked single-frame image, inputting a state identification model, carrying out parallel calculation on a neural network of the state identification model by an accelerator card so as to accelerate the whole state identification process, judging the state type of the target object, such as normal state, damage state or fault state, storing the target image, and continuing the state identification of the next frame image;

and step SS07, counting the overall situation of the target and outputting the disaster investigation result.

After all frame images in the video stream are detected, counting the total number of target objects and the number of each state type of the target objects according to target detection and state recognition results, outputting statistical result data into a file format, storing the file format to a storage module, and connecting the intelligent disaster exploration equipment with an external display screen through an external interface to display the whole disaster exploration result.

In conclusion, the application discloses intelligence system of prospecting a disaster, including peripheral hardware interface, communication module, storage module, power module and calculation module, can match multiple front end image acquisition instrument such as unmanned aerial vehicle, on-vehicle camera in a flexible way and use to show the display screen demonstration with prospecting result data output. The intelligent disaster recovery system is connected with a front-end image acquisition tool and an external display device through a peripheral interface, the front-end image acquisition tool is used for acquiring a video image of a position to be recovered, the video image of the position to be recovered is sent to a calculation module through the peripheral interface, the calculation module is used for performing target detection through a target detection model in the video image of the position to be recovered, a marked video image is generated, the state of a target object is logically judged through a state identification model, recovery result data containing a state judgment result are generated, and the marked video image and the recovery result data are output to a display screen through the peripheral interface. The intelligent disaster investigation system disclosed by the application has the advantages of strong functions, wide application and strong application flexibility.

Finally, it should also be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.

The above detailed description is given to the intelligent disaster recovery system provided by the present invention, and the principle and the implementation of the present invention are explained by applying specific examples, and the description of the above examples is only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.