1. An image tracking method, comprising:

acquiring coordinates of a target hotspot included in a target image, wherein the target image is an image obtained after a target area is shot;

generating a detection frame for labeling a target object where the target hotspot is located according to the coordinates of the target hotspot;

acquiring a characteristic value of the target object included in the detection frame;

and tracking the target object based on the characteristic value.

2. The method of claim 1, wherein the tracking the target object based on the feature value comprises:

calculating the pre-movement direction and the pre-movement speed of the target hotspot according to the characteristic value of the target hotspot;

and tracking the target object according to the pre-movement direction and the pre-movement speed.

3. The method of claim 2, wherein the tracking the target object further comprises at least one of:

under the condition that the target hotspot is separated from the target area, ending the tracking;

and under the condition that the staying time of the target hot spot in the target area is greater than a preset value, ending the tracking.

4. The method of claim 2, wherein prior to obtaining coordinates of a target hotspot included in a target image, the method further comprises:

determining a target exclusion area meeting a target condition in the target image;

acquiring hotspot information of other areas except the target exclusion area included in the target image;

and setting the hot spot of which the hot spot information meets the first preset condition as the target hot spot.

5. The method of claim 1, wherein prior to obtaining the coordinates of the target hotspot included in the target image, the method further comprises:

and determining the target hot spot according to the received tracking instruction.

6. An image tracking apparatus, comprising:

the coordinate acquisition module is used for acquiring the coordinates of a target hotspot in a target image, wherein the target image is an image obtained after a target area is shot;

the coordinate extension module is used for generating a detection frame for marking a target object where the target hotspot is located according to the coordinate of the target hotspot;

the characteristic extraction module is used for acquiring a characteristic value of the target object included in the detection frame;

and the tracking module is used for tracking the target object based on the characteristic value.

7. The apparatus of claim 6, wherein the tracking module comprises:

the motion prediction unit is used for calculating the pre-motion direction and the pre-motion speed of the target hotspot according to the characteristic value of the target hotspot;

and the tracking unit is used for tracking the target hot spot according to the pre-movement direction and the pre-movement speed.

8. The apparatus of claim 7, wherein the tracking unit comprises:

the first tracking subunit is used for finishing tracking under the condition that the target hotspot is separated from the target area;

and the second tracking subunit is used for finishing tracking under the condition that the staying time of the target hot spot in the target area is greater than a preset value.

9. A storage medium, in which a computer program is stored, wherein the computer program is arranged to perform the method of any of claims 1 to 5 when executed.

10. An electronic device comprising a memory and a processor, wherein the memory has stored therein a computer program, and wherein the processor is arranged to execute the computer program to perform the method of any of claims 1 to 5.

Background

At present, in order to capture the motion trail of a moving target, image tracking is often performed on the moving target. The existing moving target tracking technology is a technology for detecting, extracting, identifying and tracking a moving target, is the core of a computer vision system, and is widely applied to various fields such as military affairs, traffic, biology, medicine and the like.

The target detection mainly comprises two steps of object identification and object positioning. However, when facing a complex scene (such as a station and a square with a large number of people), the tracking technology has a low recognition rate and a low speed, and affects the actual tracking effect.

Disclosure of Invention

The embodiment of the invention provides an image tracking method, an image tracking device, a storage medium and electronic equipment, which are used for at least solving the problems of low recognition rate and low tracking efficiency in the related technology.

According to an embodiment of the present invention, there is provided an image tracking method including:

acquiring coordinates of a target hotspot included in a target image, wherein the target image is an image obtained after a target area is shot;

generating a detection frame for labeling a target object where the target hotspot is located according to the coordinates of the target hotspot;

acquiring a characteristic value of the target object included in the detection frame;

and tracking the target object based on the characteristic value.

In an exemplary embodiment, the tracking the target object based on the feature value includes:

calculating the pre-movement direction and the pre-movement speed of the target hotspot according to the characteristic value of the target hotspot;

and tracking the target hot spot according to the pre-movement direction and the pre-movement speed.

In an exemplary embodiment, said tracking said target hotspot further comprises at least one of:

under the condition that the target hotspot is separated from the target area, ending the tracking;

and under the condition that the staying time of the target hot spot in the target area is greater than a preset value, ending the tracking.

In an exemplary embodiment, before acquiring the coordinates of the target hotspot included in the target image, the method further comprises:

determining a target exclusion area meeting a target condition in the target image;

acquiring hotspot information of other areas except the target exclusion area included in the target image;

and setting the hot spot of which the hot spot information meets the first preset condition as the target hot spot.

In an exemplary embodiment, before the obtaining the coordinates of the target hotspot included in the target image, the method further includes:

and determining the target hot spot according to the received tracking instruction.

According to another embodiment of the present invention, there is provided an image tracking apparatus including:

the coordinate acquisition module is used for acquiring the coordinates of a target hotspot in a target image, wherein the target image is an image obtained after a target area is shot;

the coordinate extension module is used for generating a detection frame for marking a target object where the target hotspot is located according to the coordinate of the target hotspot;

the characteristic extraction module is used for acquiring a characteristic value of the target object included in the detection frame;

and the tracking module is used for tracking the target object based on the characteristic value.

In one exemplary embodiment, the tracking module includes:

the motion prediction unit is used for calculating the pre-motion direction and the pre-motion speed of the target hotspot according to the characteristic value of the target hotspot;

and the tracking unit is used for tracking the target hot spot according to the pre-movement direction and the pre-movement speed.

In one exemplary embodiment, the tracking unit includes:

the first tracking subunit is used for finishing tracking under the condition that the target hotspot is separated from the target area;

and the second tracking subunit is used for finishing tracking under the condition that the staying time of the target hot spot in the target area is greater than a preset value.

According to a further embodiment of the present invention, there is also provided a computer-readable storage medium having a computer program stored thereon, wherein the computer program is arranged to perform the steps of any of the above method embodiments when executed.

According to yet another embodiment of the present invention, there is also provided an electronic device, including a memory in which a computer program is stored and a processor configured to execute the computer program to perform the steps in any of the above method embodiments.

According to the invention, the target object is tracked by triggering the hot spot in thermal imaging, so that the reaction time is saved, and the characteristics of the target object are extracted according to the hot spot, so that the interference of non-abnormal objects is avoided, and the identification precision is improved.

Drawings

Fig. 1 is a block diagram of a hardware configuration of a mobile terminal of an image tracking method according to an embodiment of the present invention;

FIG. 2 is a flow chart of an image tracking method according to an embodiment of the invention;

FIG. 3 is a block diagram of an image tracking apparatus according to an embodiment of the present invention;

FIG. 4 is a block diagram of a structure of a tracking module according to an embodiment of the invention;

FIG. 5 is a block diagram of a tracking unit according to an embodiment of the present invention;

FIG. 6 is a block diagram of a preferred structure of an image tracking apparatus according to an embodiment of the present invention;

FIG. 7 is a block diagram of a preferred configuration of an image tracking apparatus according to an embodiment of the present invention;

fig. 8 is a flow chart of an embodiment of the present invention.

Detailed Description

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings in conjunction with the embodiments.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.

The method embodiments provided in the embodiments of the present application may be executed in a mobile terminal, a computer terminal, or a similar computing device. Taking an example of the method running on a mobile terminal, fig. 1 is a block diagram of a hardware structure of the mobile terminal of an image tracking method according to an embodiment of the present invention. As shown in fig. 1, the mobile terminal may include one or more (only one shown in fig. 1) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), and a memory 104 for storing data, wherein the mobile terminal may further include a transmission device 106 for communication functions and an input-output device 108. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration, and does not limit the structure of the mobile terminal. For example, the mobile terminal may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.

The memory 104 may be used to store a computer program, for example, a software program of an application software and a module, such as a computer program corresponding to the image tracking method in the embodiment of the present invention, and the processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, so as to implement the method described above. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the mobile terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission device 106 is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a Network adapter (NIC), which can be connected to other Network devices through a base station so as to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.

In the present embodiment, an image tracking method is provided, and fig. 2 is a flowchart of image tracking according to an embodiment of the present invention, as shown in fig. 2, the flowchart includes the following steps:

step S202, coordinates of a target hotspot included in a target image are obtained, wherein the target image is an image obtained after a target area is shot;

in this embodiment, the coordinates of the target hotspot are obtained to determine the accurate position of the target hotspot, so that the target hotspot can be accurately tracked.

The target hotspot may be a set of pixel points capable of indicating the temperature condition of each position of the target object in the target image, or a pixel point representing the temperature of a designated area of the target object in the target image, or other pixel points, pixel point sets, or figures capable of indicating the temperature of the target object in the target image and capable of being tracked, wherein the figure may be a polygonal point such as a dot, a triangle, or other polygonal points, or a contour delineation of the target object, or other abstract or non-abstract figures capable of indicating the target object; the target image can be a picture, a video, thermal imaging or other images containing a target object; the target object may be a human body, animal, building, vehicle or other mechanical device, etc. located in the monitored area and located within the target image.

The coordinates of the target hot spot can be acquired directly by thermal imaging equipment, or acquired when the target image is subjected to subsequent analysis processing after being acquired, or acquired in a manual labeling mode.

For example, when a thermal imaging image of a target object is acquired through a thermal imaging device, a human body is taken as a target hot spot, and then the coordinates of the human body in the thermal imaging image are determined through a coordinate determination module carried in the thermal imaging device.

It should be noted that, the shooting of the target area may be fixed shooting of the target area, that is, fixing the shooting angle and direction, and simultaneously recording and/or identifying the motion trajectory of the target object in the target image during the shooting process; after the target object is captured, the shooting angle and the shooting direction can be changed along with the motion trail of the target object, so that the target object is positioned in the center or a designated position of a shot target image, and the target object can be conveniently tracked.

Step S204, generating a detection frame for marking a target object where the target hotspot is located according to the coordinates of the target hotspot;

in this embodiment, the detection frame is generated to label the target hotspot in the target image and determine the trackable range of the target hotspot, so that the target hotspot can be locked in the tracking process, and the detection accuracy is improved.

The detection frame may be a polygonal frame such as a square frame, a triangular frame and the like with the target hotspot as the center, a circular frame, an oval frame or other curved frames with the target hotspot as the center, or the target hotspot is not in the center and is only a polygon or curved frame containing the target hotspot, and when the detection frame is the curved frame, the curved frame may be arranged along the edge of the target hotspot or be larger than the edge of the target hotspot; correspondingly, the process of generating the detection frame may be that after the coordinates of the target hotspot are determined, the coordinates are increased outwards by taking the target hotspot as a center to obtain edge points of the detection frame, and then the edge points are connected to generate the detection frame; the preset detection frame can also be directly arranged at the target hot spot.

Further, the generated detection box may (but is not limited to) include identification information of the coordinates of the target hotspot, may also include other included information, and may also not include any identification information; the identification information is included to facilitate identification of the detection frame.

For example, after the coordinates of the human body in the thermal imaging image are determined, a square detection frame centered on the human body is generated, and position coordinate information of the human body is identified on the detection frame.

It should be noted that the detection frame may be generated through an image recognition neural network, or may be generated in other manners, and the detection frame may be changed according to the size of the target hotspot, or may be directly generated according to a preset size.

Step S206, acquiring a characteristic value of the target object included in the detection frame;

in this embodiment, the feature value of the target object is obtained to determine the feature information of the target object, so as to facilitate identification and tracking of the target object in the continuous images, reduce interference of non-feature values to the identification and tracking process, and avoid losing the target object in the tracking process.

Wherein, the characteristic value can be height, width, temperature, type code of the target object, etc.; the characteristic value of the target object may be obtained by neural network recognition, or may be obtained by other methods.

For example, the human body in the detection frame is subjected to temperature identification through a thermal imaging identification neural network, so that information such as temperature characteristics and body type characteristics of the human body in the image is acquired.

And step S208, tracking the target object based on the characteristic value.

In this embodiment, tracking the target object according to the feature value enables accurate identification and tracking of the target object in a complex background environment, so that loss of the target object in the tracking process is avoided.

Through the steps, the target object is tracked through the target hotspot triggering, the problems of low identification efficiency and poor tracking effect in the related technology are solved, and the image identification and tracking efficiency is improved.

The main body of the above steps may be a base station, a terminal, etc., but is not limited thereto.

In an optional embodiment, tracking the target object based on the feature value comprises:

step S2082, calculating a pre-movement direction and a pre-movement speed of the target hotspot according to the characteristic value of the target hotspot;

and step S2084, tracking the target object according to the pre-movement direction and the pre-movement speed.

In this embodiment, the calculation of the pre-movement direction and the pre-movement speed of the target hotspot according to the feature value of the target hotspot is to predict the movement track of the target hotspot in advance, so that the target object can be tracked in advance, and the image recognition and tracking efficiency is improved.

The calculation of the pre-movement direction and the pre-movement speed of the target hotspot may be performed through a neural network, or may be performed according to a preset calculation mode.

It should be noted that, in the process of calculating the pre-movement direction and the pre-movement speed of the target hotspot, the feature value of the target hotspot needs to calculate the transformation position of the detection frame, so as to ensure that the target hotspot is always located in the detection frame.

In an optional embodiment, tracking the target hotspot further comprises at least one of:

step S20842, when the target hotspot is separated from the target area, ending the tracking;

step S20844, when the staying time of the target hotspot in the target area is larger than a preset value, ending the tracking.

In this embodiment, when the target hotspot is separated from the target area, separated from the target area for a long time, or stays still in the target area for a long time, the target object is not tracked any more, so as to save energy.

The target area is a pre-designated area according to detection requirements, and the area can be a polygonal area or a curved edge area; when the target hotspot is separated from the target area, the tracking can be temporarily not finished, meanwhile, the time for separating the target hotspot from the target area is calculated, and then, when the time for separating the target hotspot from the target area is larger than a preset value, the tracking is finished, so that omission is avoided.

For example, in the thermal imaging image, a region with a normal temperature may be set as the target region, or a region with an abnormal temperature may be set as the target region, and then the target object in the region may be tracked.

In an optional embodiment, before obtaining the coordinates of the target hotspot included in the target image, the method further comprises:

step S2002, determining a target exclusion area satisfying a target condition in the target image;

step S2004, acquiring the hot spot information of other areas except the target exclusion area included in the target image;

step S2006, the hotspot whose hotspot information meets the first preset condition is set as a target hotspot.

In this embodiment, the target exclusion area is set to exclude the interference hot spot in the target exclusion area, so as to ensure the detection accuracy.

Wherein, the target exclusion area can be set in the target image by setting a polygon; the hot spot with the hot spot information meeting the first preset condition is set as the target hot spot, and the target hot spot can be automatically locked by a computer or other computing devices or equipment; the first condition that is met may be that the temperature is greater than a preset value, or the temperature is less than the preset value, or other characteristic values meet the preset value.

For example, convex deformation or concave deformation may be set in a temperature anomaly region in a thermographic image, where the convex deformation is calculated as follows:

assuming that the vector P is (x1, y1) and Q is (x2, y2), the cross product of the vectors is defined as the signed area of the parallelogram composed of (0,0), P1, P2, and P1+ P2, i.e.: p × Q ═ x1 × y2-x2 × y1, the result is a scalar. The properties P × Q ═ - (Q × P) and P × (-Q) ═ - (P × Q) are evident.

In the algorithm described below, all points are regarded as vectors, the addition and subtraction of two points is the addition and subtraction of vectors, and the multiplication of points is regarded as a cross product of vectors.

One very important property of the cross product is that the clockwise and counterclockwise relationship between two vectors can be judged by its sign:

if P × Q >0, then P is clockwise of Q.

If P × Q <0, P is counterclockwise to Q.

If P × Q is 0, P and Q are collinear, but may be in the same direction or in opposite directions.

Therefore, if the convex quadrangle is ABCD and ABCD is clockwise, and the point to be determined is M, it needs to satisfy:

AB×AM>0；

BC×BM>0；

CD×CM>0；

DA×DM>0；

i.e. the point M is inside the convex quadrilateral.

The concave polygon is calculated as follows:

suppose the convex quadrangle is ABCD, the point to be judged is M, and the passing point M is arbitrarily made a ray L (the starting point is M, and the end point is infinity). If M is inside the convex quadrilateral, the straight line L must intersect the quadrilateral with an intersection point. If M is not inside the convex quadrilateral, L may or may not intersect the quadrilateral, either at two intersections (passing through the vertices of the quadrilateral), or at two intersections. The following were used:

and (4) passing the point M to do a ray L, judging the number of the intersection points, wherein if the intersection points are odd, the M points are in the inner part, and if the intersection points are not even, the M points are in the outer part.

In an optional embodiment, before obtaining the coordinates of the target hotspot included in the target image, the method further includes:

step S2008, determining a target hotspot according to the received tracking instruction.

In this embodiment, the target hotspot is determined through the received tracking instruction, so that a user can conveniently track the target object according to the actual situation.

For example, the target hotspot can be manually selected, a detection box is arranged on the target hotspot, and then the selected target hotspot is tracked.

Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.

In this embodiment, an image tracking apparatus is further provided, and the apparatus is used to implement the foregoing embodiments and preferred embodiments, and the description of which has been already made is omitted. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.

Fig. 3 is a block diagram of a configuration of an image tracking apparatus according to an embodiment of the present invention, as shown in fig. 3, the apparatus including:

the coordinate acquisition module 32 is configured to acquire coordinates of a target hotspot included in a target image, where the target image is an image obtained after a target area is photographed;

the coordinate extension module 34 is configured to generate a detection frame for labeling a target object where the target hotspot is located according to the coordinate of the target hotspot;

a feature extraction module 36, configured to obtain a feature value of the target object included in the detection frame;

and a tracking module 38, configured to track the target object based on the feature value.

Fig. 4 is a block diagram of a structure of the tracking module 38 according to an embodiment of the present invention, and as shown in fig. 4, the tracking module 38 includes:

the motion prediction unit 382 is configured to calculate a pre-motion direction and a pre-motion speed of the target hotspot according to the feature value of the target hotspot;

and the tracking unit 384 is used for tracking the target hot spot according to the pre-movement direction and the pre-movement speed.

Fig. 5 is a block diagram illustrating a structure of a tracking unit 384 according to an embodiment of the present invention, and as shown in fig. 5, the tracking unit 384 includes:

a first tracking subunit 3842, configured to end tracking when the target hotspot deviates from the target area;

and the second tracking subunit 3844 is configured to end tracking when the staying time of the target hotspot in the target area is greater than a preset value.

Fig. 6 is a block diagram showing a preferred structure of an image tracking apparatus according to an embodiment of the present invention, as shown in fig. 6, the apparatus further includes:

an excluded area determining module 302, configured to determine a target excluded area that meets a target condition in the target image before obtaining coordinates of a target hotspot included in the target image;

the hot spot acquisition module 304 is configured to acquire hot spot information of other areas included in the target image, except for the target exclusion area;

the first hotspot determining module 306 is configured to set a hotspot for which the hotspot information meets a first preset condition as a target hotspot.

Fig. 7 is a block diagram of a preferred structure of an image tracking apparatus according to an embodiment of the present invention, as shown in fig. 7, the apparatus further includes:

the second hot spot determining module 308 is configured to determine the target hot spot according to the received tracking instruction before acquiring the coordinates of the target hot spot included in the target image.

It should be noted that, the above modules may be implemented by software or hardware, and for the latter, the following may be implemented, but not limited to: the modules are all positioned in the same processor; alternatively, the modules are respectively located in different processors in any combination.

The present invention will be described with reference to specific examples.

Referring to fig. 8, after the target image is acquired, coordinates of a target hotspot in the target image are acquired (refer to step S801 in fig. 8), then a detection box is generated according to the coordinates of the target hotspot (refer to step S802 in fig. 8), a feature value of a target object corresponding to the target hotspot is extracted according to a preset algorithm (refer to step S803 in fig. 8), and calculates the moving speed and moving direction of the target hotspot according to the feature values (refer to step S804 in fig. 8), and then tracks the target hotspot according to the calculation (refer to step S805 in fig. 8), in the tracking process, it is determined whether the target hotspot disappears from the target area, whether the disappearing time exceeds a preset value, or whether the stationary time is greater than a preset value (refer to steps S807 and S808 in fig. 8), if the above condition is satisfied, the tracking state is maintained (see step S806 in fig. 8), and if not, the tracking is stopped.

Embodiments of the present invention also provide a storage medium having a computer program stored therein, wherein the computer program is arranged to perform the steps of any of the above method embodiments when executed.

In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to: various media capable of storing computer programs, such as a usb disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.

Embodiments of the present invention also provide an electronic device comprising a memory having a computer program stored therein and a processor arranged to run the computer program to perform the steps of any of the above method embodiments.

In an exemplary embodiment, the electronic apparatus may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.

For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary embodiments, and details of this embodiment are not repeated herein.

It will be apparent to those skilled in the art that the various modules or steps of the invention described above may be implemented using a general purpose computing device, they may be centralized on a single computing device or distributed across a network of computing devices, and they may be implemented using program code executable by the computing devices, such that they may be stored in a memory device and executed by the computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into various integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the principle of the present invention should be included in the protection scope of the present invention.