1. The unmanned aerial vehicle cluster monitoring three-dimensional machine vision composition method is characterized by comprising the following steps:

s1, allocating a detection target area for each unmanned aerial vehicle;

s2, any unmanned aerial vehicle acquires image information of a detection target area;

s3, the unmanned aerial vehicle transmits the acquired image information to the unmanned aerial vehicle cluster three-dimensional machine vision generating module through the network;

and S4, selecting one image transmitted by any unmanned aerial vehicle to carry out image splicing according to a machine vision algorithm by the unmanned aerial vehicle cluster three-dimensional machine vision generating module.

2. The unmanned aerial vehicle cluster monitoring three-dimensional machine vision composition method of claim 1, characterized in that: in step S1, the allocation of the detection target is performed according to the monitoring scenario, the nature of the target, the monitoring requirement, the member configuration of the unmanned aerial vehicle cluster, the performance of the onboard device, and the like.

3. The unmanned aerial vehicle cluster monitoring three-dimensional machine vision composition method of claim 1, characterized in that: the method further comprises the step of setting the angle, the resolution ratio and the refresh rate parameters of the shooting monitoring image of each unmanned aerial vehicle.

4. The unmanned aerial vehicle cluster monitoring three-dimensional machine vision composition method of claim 1, characterized in that: step S3 is that according to the requirement of the three-dimensional machine vision composition of the unmanned aerial vehicle cluster, each unmanned aerial vehicle transmits the monitoring image data according to the sequence and the time delay requirement by adopting wifi, 4G and 5G wireless transmission means according to the calculation node address of data aggregation.

5. The unmanned aerial vehicle cluster monitoring three-dimensional machine vision composition method of claim 1, characterized in that: the algorithm in step S4 is a halcon machine vision algorithm package.

6. Unmanned aerial vehicle cluster monitoring three-dimensional machine vision composition system, its characterized in that includes:

the distribution system is used for distributing a detection target area for each of the unmanned aerial vehicles;

the system comprises an image acquisition system, an unmanned aerial vehicle and a control system, wherein any unmanned aerial vehicle acquires image information of a detection target area;

the unmanned aerial vehicle transmits the acquired image information to the unmanned aerial vehicle cluster three-dimensional machine vision generating module through a network;

the image generation system comprises an unmanned aerial vehicle cluster three-dimensional machine vision generation module, and is used for selecting one image transmitted by any unmanned aerial vehicle to carry out image splicing according to a machine vision algorithm packet halcon.

7. The unmanned aerial vehicle cluster monitoring three-dimensional machine vision composition method of claim 6, wherein: in the distribution system, distribution detection targets are carried out according to the conditions of monitoring scenes, the properties and monitoring requirements of the targets, member composition of the unmanned aerial vehicle cluster, the performance of carrying equipment and the like.

8. The unmanned aerial vehicle cluster monitoring three-dimensional machine vision composition system of claim 6, wherein: the method further comprises the step of setting the angle, the resolution ratio and the refresh rate parameters of the shooting monitoring image of each unmanned aerial vehicle.

9. The unmanned aerial vehicle cluster monitoring three-dimensional machine vision composition system of claim 6, wherein: the image splicing system specifically comprises the steps that according to the requirement of the visual composition of the three-dimensional machine of the unmanned aerial vehicle cluster, each unmanned aerial vehicle transmits monitoring image data according to sequence and time delay requirements by adopting wifi, 4G and 5G wireless transmission means according to the calculation node address of data aggregation.

Background

At present, monitoring and supervision based on machine vision and other applications become one of important contents established in an information-based society along with rapid penetration of an unmanned aerial vehicle platform. For example, building a three-dimensional map is one of the hotspots of digital construction of the whole society. Especially, the rapid popularization and application of the unmanned aerial vehicle surveying and mapping platform, the three-dimensional map construction based on the unmanned aerial vehicle platform machine vision is fully approved by various industries, and the three-dimensional map construction method has the remarkable advantages of low cost, high speed and accurate composition, and is incomparable to the traditional mode. The unmanned aerial vehicle platform-based machine vision plays an important role in monitoring important targets, monitoring traffic scenes, security of important activities and other occasions.

However, machine vision monitoring based on a single unmanned aerial vehicle often only can obtain a two-dimensional machine vision map, and can only judge whether there is a barrier or danger in the front, but cannot obtain more accurate and deep information such as distance and behavior prediction. The acquisition of such information requires the analysis of three-dimensional machine vision. However, the three-dimensional machine vision construction based on the single unmanned aerial vehicle machine vision requires the unmanned aerial vehicle to perform planned path measurement and perform composition calculation under the line, and a three-dimensional visual image of a monitored object cannot be obtained in real time. This may be unsatisfactory in certain situations. For example, in the field of urban traffic surveillance, real-time three-dimensional vision of scenes is desired, rather than offline. For security of important activities and the like, the accurate overall control can be ensured only by real-time dynamic three-dimensional vision of a whole scene and multiple targets.

Therefore, a method, equipment and a system for autonomously and cooperatively monitoring three-dimensional visual composition by unmanned aerial vehicle cluster are needed to be researched, multiple unmanned aerial vehicle platforms are utilized, multi-angle real-time shooting images are fused in a cooperative monitoring mode, the three-dimensional machine visual composition is generated on line, the problem that a single unmanned aerial vehicle platform cannot generate a three-dimensional machine visual diagram on line is solved, the multi-dimensional global monitoring capability of an important scene is improved, and a three-dimensional data basis is provided for accurate research and judgment, accurate command and evidence acquisition. The research has stronger practical background and multi-scene applicability and technically has stronger innovation significance, thereby having stronger theoretical and practical values.

Disclosure of Invention

In order to solve the technical problems, the invention aims to provide a method, equipment and a system for autonomous cooperative joint monitoring of three-dimensional visual composition by unmanned aerial vehicle clusters.

The technical scheme of the invention is as follows:

the unmanned aerial vehicle cluster monitoring three-dimensional machine vision composition method is characterized by comprising the following steps:

s1, allocating a detection target area for each unmanned aerial vehicle;

s2, any unmanned aerial vehicle acquires image information of a detection target area;

s3, the unmanned aerial vehicle transmits the acquired image information to the unmanned aerial vehicle cluster three-dimensional machine vision generating module through the network;

and S4, selecting one image transmitted by any unmanned aerial vehicle to carry out image splicing according to a machine vision algorithm by the unmanned aerial vehicle cluster three-dimensional machine vision generating module.

Further, in step S1, the allocation of the detection target is performed according to the monitoring scenario, the property of the target, the monitoring requirement, the member configuration of the unmanned aerial vehicle cluster, the performance of the onboard device, and the like.

Furthermore, the method also comprises the step of setting the angle, the resolution ratio and the refresh rate parameters of the shooting monitoring image of each unmanned aerial vehicle.

Further, step S3 is to transmit the monitoring image data of each unmanned aerial vehicle according to the order and the time delay requirement by using wifi, 4G, and 5G wireless transmission means according to the computing node address of data aggregation according to the requirement of the three-dimensional machine vision composition of the unmanned aerial vehicle cluster.

Further, the algorithm in step S4 is a halcon machine vision algorithm package.

Unmanned aerial vehicle cluster monitoring three-dimensional machine vision composition system, its characterized in that includes:

the distribution system is used for distributing a detection target area for each of the unmanned aerial vehicles;

the system comprises an image acquisition system, an unmanned aerial vehicle and a control system, wherein any unmanned aerial vehicle acquires image information of a detection target area;

the unmanned aerial vehicle transmits the acquired image information to the unmanned aerial vehicle cluster three-dimensional machine vision generating module through a network;

the image splicing system comprises an unmanned aerial vehicle cluster three-dimensional machine vision generating module, wherein a pair of images transmitted by any unmanned aerial vehicle are selected for image splicing according to a machine vision algorithm packet halcon.

Furthermore, in the distribution system, the distribution of the detection target is performed according to the conditions of the monitoring scene, the nature of the target, the monitoring requirement, the member composition of the unmanned aerial vehicle cluster, the performance of the carrying equipment and the like.

Furthermore, the method also comprises the step of setting the angle, the resolution ratio and the refresh rate parameters of the shooting monitoring image of each unmanned aerial vehicle.

Furthermore, the image splicing system specifically transmits the respective monitoring image data according to the sequence and time delay requirements by adopting wifi, 4G and 5G wireless transmission means according to the computing node address of data aggregation by each unmanned aerial vehicle according to the requirement of the three-dimensional machine vision composition of the unmanned aerial vehicle cluster.

By the scheme, the invention at least has the following advantages:

by utilizing the multi-unmanned aerial vehicle platform, multi-angle real-time shooting images are fused in a cooperative monitoring mode, a three-dimensional machine vision composition is generated on line, the problem that a single unmanned aerial vehicle platform cannot generate a three-dimensional machine vision pattern on line is solved, the multi-dimensional global monitoring capability of an important scene is improved, and a three-dimensional data basis is provided for accurate research and judgment, accurate command and evidence acquisition.

The foregoing description is only an overview of the technical solutions of the present invention, and in order to make the technical solutions of the present invention more clearly understood and to implement them in accordance with the contents of the description, the following detailed description is given with reference to the preferred embodiments of the present invention and the accompanying drawings.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate a certain embodiment of the present invention, and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.

FIG. 1 is a scene diagram of the present invention;

FIG. 2 is a block diagram of the method operation of the present invention;

fig. 3 is a schematic diagram of the system architecture of the present invention.

Detailed Description

The following detailed description of embodiments of the present invention is provided in connection with the accompanying drawings and examples. The following examples are intended to illustrate the invention but are not intended to limit the scope of the invention.

Referring to fig. 1 and 2, a preferred embodiment of the present invention is shown.

The specific method of the invention is as follows:

the unmanned aerial vehicle cluster monitoring three-dimensional machine vision composition method comprises the following steps:

the method comprises the steps of firstly, according to the nature and the monitoring requirement of a monitoring scene and a target, and the conditions of member composition of an unmanned aerial vehicle cluster, performance of carrying equipment and the like, distributing detection targets for each unmanned aerial vehicle, and setting the angle, the resolution ratio and the refresh rate parameters of the unmanned aerial vehicle for shooting monitoring images.

Because the heights of the targets are not uniform, each unmanned aerial vehicle is required to keep the same height relative to the ground when being distributed. The angle of the shot image is generally fixed at the same angle, the resolution can be any one of 720p, 1080p and 2180p or data of any two times, and the refresh rate is 24hz-120 hz.

And secondly, shooting images of respective monitoring targets by all the unmanned aerial vehicles according to task allocation results, and dynamically shooting and storing the images according to planning.

The unmanned aerial vehicle is generally provided with a camera and a corresponding storage chip, and the camera of the unmanned aerial vehicle is used for shooting and storing image information into the storage chip.

And thirdly, according to the requirement of the unmanned aerial vehicle cluster three-dimensional machine vision composition, each unmanned aerial vehicle transmits the monitoring image data to an unmanned aerial vehicle cluster three-dimensional machine vision generation module according to the sequence and time delay requirements by adopting wireless transmission means including wifi, 4G, 5G, an adhoc module and the like according to the calculation node address of data aggregation.

When selecting a particular picture, a picture with a higher resolution is generally selected. The unmanned aerial vehicle is generally provided with an information sending module, the information sending module adopts wireless transmission means including wifi, 4G, 5G, adhoc modules and the like, data are transmitted to an unmanned aerial vehicle cluster three-dimensional machine vision generation module, in the specific transmission process, in order to prevent information blockage, the transmission time of each unmanned aerial vehicle needs to be set, and only one unmanned aerial vehicle is in data transmission when each unmanned aerial vehicle transmits.

And fourthly, the computing node customizes and generates a three-dimensional machine vision composition of the concerned target and the whole scene according to the multi-angle shooting image converged by the unmanned aerial vehicle, the scene map, the target prior information and the like by using a three-dimensional machine vision generation algorithm.

Specifically, a halcon machine vision algorithm packet is adopted for splicing in the splicing process.

The invention can also include the fifth step and the sixth step in general, specifically:

and fifthly, according to a three-dimensional machine vision composition obtained by the aggregation of unmanned aerial vehicle cluster data, performing distance analysis of scene internal attention, performing analysis and application such as real-time three-dimensional environment perception and three-dimensional accurate recognition, and performing behavior analysis on an attention monitoring target.

And sixthly, comparing the monitoring requirements according to the analysis result of the module, namely the visual composition analysis and application of the three-dimensional machine in the unmanned aerial vehicle cluster scene, aiming at the target which needs to be further improved in accuracy and the new monitoring target which needs to be added, planning the task redistribution of member monitoring in the unmanned aerial vehicle cluster, prejudging the monitoring effect after redistribution, according with the monitoring requirements, issuing a monitoring task scheduling instruction to each member unmanned aerial vehicle, and performing the cooperative adjustment of cluster monitoring.

The method of the invention depends on hardware as follows, namely, a plurality of unmanned aerial vehicles which have the functions of camera shooting, image storage and wireless transmission.

The second is a distribution system which distributes a detection target area for each of a plurality of unmanned aerial vehicles; the system comprises an image acquisition system, an unmanned aerial vehicle and a control system, wherein any unmanned aerial vehicle acquires image information of a detection target area; the unmanned aerial vehicle transmits the acquired image information to the unmanned aerial vehicle cluster three-dimensional machine vision generating module through a network; the image generation system comprises an unmanned aerial vehicle cluster three-dimensional machine vision generation module, and is used for selecting one image transmitted by any unmanned aerial vehicle to carry out image splicing according to a machine vision algorithm packet halcon.

In the distribution system, the distribution of the detection target is performed according to the monitoring scene, the nature of the target, the monitoring requirement, the member composition of the unmanned aerial vehicle cluster, the performance of the carrying equipment and the like.

-further comprising setting the angle, resolution, refresh rate parameters of each drone taking the monitored images.

The image splicing system is specifically configured to transmit the monitoring image data of each unmanned aerial vehicle according to the sequence and the time delay requirement by adopting wifi, 4G and 5G wireless transmission means according to the calculation node address of data aggregation according to the requirement of the three-dimensional machine vision composition of the unmanned aerial vehicle cluster.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart 1 flow or flows and/or block 1 block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows of FIG. 1 and/or block diagram block or blocks of FIG. 1.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart 1 flow or flows and/or block 1 block or blocks.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, it should be noted that, for those skilled in the art, many modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.