1. A pedestrian abnormal behavior detection method is characterized by comprising the following steps:

respectively carrying out target detection on multiple frames of images shot by the camera device to obtain a detection area corresponding to a target pedestrian in each frame of image;

respectively carrying out optical flow detection on the detection area of each frame of image to obtain optical flow information of the detection area of each frame of image, wherein the multi-frame image comprises a target frame image and at least one frame of sample frame image;

analyzing the optical flow information of the target frame image using the optical flow information of the at least one frame sample frame image; and

determining whether the target pedestrian has abnormal behavior based on the result of the analysis;

wherein, before the analyzing the optical flow information of the target frame image by using the optical flow information of the at least one frame sample frame image, the method further comprises:

judging whether the detection area is smaller than a preset minimum size;

if so, amplifying the image in the detection area to a preset standard size, and taking the amplified image as the image in the detection area;

if not, calculating the size ratio between the size of the detection area and the preset standard size, and correspondingly taking the product of the optical flow displacement of each optical flow point in the detection area and the size ratio as the optical flow displacement of the optical flow point.

2. The method of claim 1, wherein the determining whether the detection area is smaller than a preset minimum size comprises:

acquiring a first length of a diagonal line of the detection area and a second length of the diagonal line with a preset minimum size;

judging whether the first length is smaller than the second length;

if so, determining that the detection area is smaller than the preset minimum size;

if not, determining that the detection area is not smaller than the preset minimum size.

3. The method of claim 1, wherein the calculating of the size ratio between the size of the detection area and the preset standard size comprises:

acquiring a first length of a diagonal line of the detection area and a third length of the diagonal line of the preset standard size;

taking a ratio of the third length to the first length as the dimension ratio.

4. The method of claim 1, wherein said analyzing said optical flow information of said target frame image using said optical flow information of said at least one frame sample frame image comprises:

obtaining an optical flow threshold of the target frame image based on the optical flow information of the at least one frame of sample frame image;

comparing the optical flow information of the target frame image to the optical flow threshold;

the determining whether the target pedestrian has abnormal behavior based on the result of the analysis includes:

and if the optical flow information of the target frame image and the optical flow threshold value meet a preset relation condition, determining that the target pedestrian has abnormal behaviors.

5. The method of claim 4, wherein the optical flow threshold comprises at least one of: an optical flow displacement threshold and an optical flow ratio threshold; the preset relation conditions comprise: a second optical flow displacement of the detection area of the target frame image is greater than or equal to a first preset multiple of the optical flow displacement threshold; and/or the optical flow ratio of the target frame image is greater than or equal to a second preset multiple of the optical flow ratio threshold.

6. The method of claim 4, wherein the optical flow threshold comprises at least one of: an optical flow displacement threshold and an optical flow ratio threshold;

the obtaining an optical flow threshold of the target frame image based on the optical flow information of the at least one frame sample frame image comprises:

acquiring first optical flow displacement of a detection area of each frame of sample frame image, and determining and obtaining an optical flow displacement threshold of the target frame image based on the first optical flow displacement; and/or

Counting the number of first optical flow points of which the optical flow direction change is larger than a preset angle in a detection area of each frame of sample frame image, and determining and obtaining an optical flow ratio threshold of the target frame image based on the number of the first optical flow points;

the comparing the optical flow information of the target frame image to the optical flow threshold comprises:

acquiring a second optical flow displacement of a detection area of the target frame image, and comparing the second optical flow displacement with the optical flow displacement threshold; and/or the presence of a gas in the gas,

and counting the number of second optical flow points of which the optical flow direction change is larger than a preset angle in the detection area of the target frame image, obtaining the optical flow ratio of the target frame image based on the number of the second optical flow points, and comparing the optical flow ratio with the optical flow ratio threshold.

7. The method according to claim 1, wherein the at least one frame sample frame image is several frame images before the target frame image;

the target detection is respectively carried out on the multi-frame images shot by the camera device, and the detection area corresponding to the target pedestrian in each frame of image is obtained by the following steps:

detecting each frame of image by using a preset head-shoulder frame detection model to obtain a head-shoulder frame corresponding to the target pedestrian in each frame of image;

and obtaining the detection area corresponding to the target pedestrian from the head-shoulder frame corresponding to the target pedestrian.

8. An image processing apparatus comprising a memory and a processor coupled to each other;

the processor is configured to execute the program instructions stored by the memory to implement the method of any of claims 1 to 7.

9. The apparatus according to claim 8, further comprising an image pickup device for picking up a plurality of frames of images in time series.

10. A storage device storing program instructions executable by a processor to perform the method of any one of claims 1 to 7.

Background

Along with the monitoring cameras are distributed and controlled in all corners of a city more and more densely, the intelligent monitoring system also continuously expands the service functions of the intelligent monitoring system, so that monitoring personnel are gradually replaced to complete a large amount of repeated monitoring work.

The service function of detecting the abnormal behavior of the pedestrian is particularly important in the crowded areas such as prisons, stations, squares and markets. For example, when abnormal behaviors such as fighting and fighting occur, related departments can be timely involved, so that the loss of life and property is reduced, the social stability and the regional safety are facilitated, and the method has great application value and wide application prospect. Therefore, how to accurately detect whether the pedestrian has abnormal behaviors becomes a problem to be solved urgently in the intelligent monitoring service.

Disclosure of Invention

The technical problem mainly solved by the application is to provide a pedestrian abnormal behavior detection method, an image processing device and a storage device, which can accurately detect the abnormal behavior of a pedestrian.

In order to solve the above problems, a first aspect of the present application provides a method for detecting abnormal behaviors of pedestrians, including performing target detection on multiple frames of images captured by an image capture device, respectively, to obtain a detection region corresponding to a target pedestrian in each frame of image; respectively carrying out optical flow detection on the detection area of each frame of image to obtain optical flow information of the detection area of each frame of image, wherein the multi-frame image comprises a target frame image and at least one frame of sample frame image; analyzing optical flow information of a target frame image by using the optical flow information of at least one frame of sample frame image; and determining whether the target pedestrian has abnormal behavior based on the result of the analysis.

In order to solve the above problem, a second aspect of the present application provides an image processing apparatus including a memory and a processor coupled to each other; the processor is adapted to execute the program instructions stored by the memory to implement the method of the first aspect described above.

In order to solve the above problem, a third aspect of the present application provides a storage device storing program instructions executable by a processor, the program instructions being for implementing the method of the first aspect.

In the above scheme, the target detection is performed on the multi-frame images including the target frame image and the at least one frame of sample frame image, which are obtained by the image pickup device, respectively, so as to obtain the detection area corresponding to the target pedestrian in each frame of image, thereby detecting the optical flow detection on the detection area, further obtaining the optical flow information of the target frame image and the at least one frame of sample frame image, further analyzing the optical flow information of the target frame image by using the optical flow information of the sample frame image, and further finding the change of the optical flow information of the target frame image compared with the sample frame image, so that the optical flow information of the sample frame image, which is obtained by shooting the target pedestrian, can be used as the current behavior judgment standard of the target pedestrian to judge the optical flow information of the target frame image by taking the optical flow information of the sample frame image, regardless of the scene of the target pedestrian, so as to accurately detect whether the target pedestrian has abnormal behavior.

Drawings

FIG. 1 is a schematic flow chart of an embodiment of a method for detecting abnormal behavior of a pedestrian according to the present application;

FIG. 2 is a schematic flow chart of another embodiment of the method for detecting abnormal behavior of pedestrians according to the present application;

FIG. 3 is a flowchart illustrating an embodiment of step S21 in FIG. 2;

FIG. 4 is a flowchart illustrating an embodiment of step S23 in FIG. 2;

FIG. 5 is a flowchart illustrating an embodiment of step S11 in FIG. 1;

FIG. 6 is a schematic flow chart diagram illustrating a method for detecting abnormal pedestrian behavior according to yet another embodiment of the present application;

FIG. 7 is a flowchart illustrating an embodiment of step S61 in FIG. 6;

FIG. 8 is a flowchart illustrating an embodiment of step S62 in FIG. 6;

FIG. 9 is a schematic flow chart illustrating another embodiment of step S61 in FIG. 6;

FIG. 10 is a schematic flow chart diagram illustrating a method for detecting abnormal pedestrian behavior according to yet another embodiment of the present application;

FIG. 11 is a block diagram of an embodiment of an image processing apparatus according to the present application;

FIG. 12 is a block diagram of an embodiment of a memory device according to the present application.

Detailed Description

The following describes in detail the embodiments of the present application with reference to the drawings attached hereto.

In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular system structures, interfaces, techniques, etc. in order to provide a thorough understanding of the present application.

The terms "system" and "network" are often used interchangeably herein. The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship. Further, the term "plurality" herein means two or more than two.

Referring to fig. 1, fig. 1 is a schematic flow chart of an embodiment of a method for detecting abnormal behavior of a pedestrian according to the present application. Specifically, the method may include:

step S11: and respectively carrying out target detection on the multi-frame images shot by the camera device to obtain a detection area corresponding to the target pedestrian in each frame of image.

The camera device may be a night vision camera, an infrared camera, or the like. Different types of image pickup devices can be selected according to different application scenes. For example, for a place with a dark environment and poor lighting, the camera device can be a night vision camera or an infrared camera; aiming at indoor places with bright light, the camera device can be a common digital camera or a network camera; and for the outdoor non-sheltered scene, the camera can be a waterproof camera, and this embodiment does not make specific limitations.

The multi-frame image may be a 2-frame image, a 3-frame image, a 4-frame image, etc., and this embodiment is not an example here.

The detection area may be a rectangle, and the rectangle surrounds the pedestrian in each frame of image. Alternatively, the detection area may be an irregular figure, and in an implementation scenario, the detection area may be a contour area of a pedestrian in order to more accurately perform optical flow detection on the detection area of the multi-frame image, so as to more accurately obtain optical flow information of a target pedestrian in the multi-frame image.

The determination of the Detection area can be realized by a Pedestrian Detection (Pedestrian Detection) technology, which judges whether a Pedestrian exists in an image or a video sequence by using a computer vision technology and gives accurate positioning. The existing pedestrian detection methods are mainly three, which are a global feature-based method, a human body part-based method, and a stereoscopic vision-based method, such as a Haar wavelet feature-based method, an HOG (Histogram of Oriented Gradients) feature-based method, and a contour template-based method, which are representative methods based on the global feature. The pedestrian detection technology is the prior art in the technical field of computer vision, and the present embodiment is not described herein again.

Step S12: and respectively carrying out optical flow detection on the detection area of each frame of image to obtain optical flow information of the detection area of each frame of image, wherein the multi-frame image comprises a target frame image and at least one frame of sample frame image.

Optical Flow (Optical Flow) is a concept in object motion detection in the visual domain, and is used to describe the motion of an observed object, surface, or edge caused by motion relative to an observer. The optical flow refers to the speed of the motion of the image surface. The reason why the object is found by the human eye while moving is that when the object moves, a series of continuously changing images are formed on the retina of the human eye, and the changing information continuously flows through the retina of the glasses at different times as if an optical flow passes, which is called optical flow.

Optical flow detection plays an important role in the fields of pattern recognition, computer vision, and other image processing. Specifically, optical flow detection may be used to detect motion, object cuts, computation of collision time and object inflation, motion compensated encoding, or stereo measurements through object surfaces and edges, among others.

Methods related to optical flow detection currently include gradient-based methods, matching-based methods, energy-based methods, and the like. The following are typical: the Horn-hill Method (Horn-Schunck Method), the Lucas-Kanade Method (Lucas-Kanade Method), and the like. The optical flow detection method is the prior art in the field, and the description of the embodiment is omitted here.

In one implementation scenario, in order to quantitatively represent the position information of the optical flow point, the optical flow information includes coordinate information of at least one optical flow point of the detection area on a multi-dimensional coordinate axis, for example, the optical flow information of the optical flow point 1 in the previous frame image is represented as coordinate information (X) on a two-dimensional coordinate axis_1,t，Y_1,t) Or optical flow information of the optical flow point 2 in the previous frame image is represented as coordinate information (X) on a two-dimensional coordinate axis_2,t，Y_2,t) Or optical flow information of the optical flow point 1 in the subsequent frame image is represented as coordinate information (X) on a two-dimensional coordinate axis_1,t+1，Y_1,t+1) Or optical flow information of the optical flow point 2 in the subsequent frame image is represented as coordinate information (X) on a two-dimensional coordinate axis_2,t+1，Y_2,t+1) Etc., which will not be described in detail in this embodiment.

In an implementation scenario, in order to adaptively perform the detection of the abnormal behavior of the pedestrian on the target frame image, at least one frame of sample frame image is a plurality of frames of image before the target frame image, for example, ten frames of image before the target frame image, fifteen frames of image before the target frame image, and the like, which is not illustrated here.

Step S13: and analyzing the optical flow information of the target frame image by using the optical flow information of at least one frame of sample frame image.

Analyzing optical flow information of the target frame image based on optical flow information of at least one frame of sample frame image. In one implementation scenario, changes between the target frame image and the sample image optical flow information may be discovered based on optical flow information of at least one frame of the sample frame image.

Step S14: based on the result of the analysis, it is determined whether there is abnormal behavior in the target pedestrian.

And determining whether the target pedestrian has abnormal behaviors or not based on the analysis of the optical flow information of the target frame image and the at least one frame of sample frame image. In one implementation scenario, a change between the optical flow information of the target frame image and the optical flow information of the sample frame image can be found based on the optical flow information of at least one frame of the sample frame image, so that whether the target pedestrian has abnormal behavior or not is determined based on the change between the optical flow information of the target frame image and the optical flow information of the sample frame image.

In the above manner, the multi-frame images including the target frame image and the at least one frame of sample frame image, which are captured by the imaging device, are respectively subjected to target detection, so that the detection area corresponding to the target pedestrian in each frame of image is obtained, and then the optical flow detection is performed on the detection area, so that the optical flow information of the target frame image and the optical flow information of the at least one frame of sample frame image are obtained, and therefore, the optical flow information of the target frame image is analyzed by using the optical flow information of the sample frame image, and further, the change of the optical flow information of the target frame image compared with the sample frame image can be found, so that no matter the scene where the target pedestrian is located, the optical flow information of the sample frame image captured by the target pedestrian can be used as the current behavior judgment standard of the target pedestrian for performing behavior judgment on the optical flow information of the target frame image, so as to accurately detect whether the target pedestrian has abnormal behavior.

Referring to fig. 2, fig. 2 is a schematic flow chart of another embodiment of the method for detecting abnormal pedestrian behavior according to the present application. Before the step S13, the method may further include:

step S21: and judging whether the detection area is smaller than a preset minimum size. If so, go to step S22, otherwise go to step S23.

The preset minimum size is the minimum size that can contain optical flow information required for abnormal behavior detection. When the detection area is smaller than the preset minimum size, the optical flow information in the detection area is less, which may result in insufficient use for subsequent detection of abnormal behavior. The preset minimum size is a preset size which is uniformly set, or can also be a preset size which is set by a user in a self-defined way. In one implementation scenario, the detection area is a rectangular detection area, and the preset minimum size is also a rectangle. Referring to fig. 3, the step S21 may specifically include:

step S211: a first length of a diagonal line of the detection area and a second length of the diagonal line of the preset minimum size are acquired.

In one implementation scenario, the first length of the diagonal line may be determined by coordinate information of vertices of a rectangle corresponding to the detection region, for example, the coordinate information of four vertices corresponding to the detection region is (X) respectively_A，Y_A)、(X_A+30，Y_A)、(X_A+30，Y_A+40)、(X_A，Y_A+40) Then, according to the pythagorean theorem, the first length of the diagonal line of the detection area may be determined to be 50, and similarly, the second length of the diagonal line with the preset minimum size may be determined, which is not illustrated in this embodiment.

Step S212: and judging whether the first length is smaller than the second length. If so, go to step S213, otherwise go to step S214.

Comparing whether the first length is less than the second length.

Step S213: and determining that the detection area is smaller than a preset minimum size.

If the first length is smaller than the second length, the detection area can be determined to be smaller than the preset minimum size.

Step S214: determining that the detection area is not smaller than a preset minimum size.

If the first length is not less than the second length, it may be determined that the detection area is not less than a preset minimum size.

Step S22: and amplifying the image in the detection area to a preset standard size, and taking the amplified image as the image in the detection area.

When the detection area is smaller than the preset minimum size, it means that if the optical flow detection is performed on the image in the detection area at this time, the detected optical flow information may be less, and thus, the subsequent detection of the abnormal behavior may not be sufficient. At this time, the image in the detection area may be enlarged to a preset standard size, and the enlarged image may be used as the image in the detection area.

Step S23: and calculating the size ratio between the size of the detection area and a preset standard size, and correspondingly taking the product of the optical flow displacement of each optical flow point in the detection area and the size ratio as the optical flow displacement of the optical flow point.

If the detection area is not smaller than the preset minimum size, it means that if the optical flow detection is performed on the image in the detection area at this time, the detected optical flow information is sufficient for the subsequent detection of the abnormal behavior. At this time, a size ratio between the size of the detection area and a preset standard size may be further calculated, and a product of the optical flow displacement of each optical flow point in the detection area and the size ratio may be taken as the optical flow displacement of the corresponding optical flow point. Specifically, the preset standard size may be divided by the size of the detection area to obtain a size ratio, for example, if the size ratio is 1.2, it indicates that the preset standard size is larger than the detection area, and the optical flow displacement of each optical flow point in the detection area is multiplied by the size ratio 1.2 to obtain the optical flow displacement corresponding to the optical flow point; or, for example, if the size ratio is 0.8, it indicates that the preset standard size is smaller than the detection area, the optical flow displacement of each optical flow point in the detection area is multiplied by the size ratio 10.8 to obtain the optical flow displacement of the corresponding optical flow point, and the optical flow displacements of the optical flow points in the detection areas with different sizes can be normalized by the above method, so that the influence of the difference between the optical flow displacements between the near view and the far view in the image on the abnormal behavior detection is eliminated.

In step S22, if the detection area is smaller than the preset minimum size, the image in the detection area is enlarged to the preset standard size, and the enlarged image is used as the image in the detection area, where the size ratio between the size of the detection area and the preset standard size is 1, which is equivalent to that the optical flow displacement of the optical flow point is not necessarily scaled.

In an implementation scenario, in step S23, in order to quickly and conveniently calculate the size ratio between the size of the detection area and the preset standard size, referring to fig. 4, step S23 may specifically include the following steps:

step S231: a first length of a diagonal line of the detection area and a third length of the diagonal line of a preset standard size are acquired.

In one implementation scenario, the first length of the diagonal line may be determined by coordinate information of vertices of a rectangle corresponding to the detection region, for example, the coordinate information of four vertices corresponding to the detection region is (X) respectively_A，Y_A)、(X_A+30，Y_A)、(X_A+30，Y_A+40)、(X_A，Y_A+40) Then the first length of the diagonal of the detection area can be determined to be 50 according to the pythagorean theorem. Similarly, the third length of the diagonal line of the preset standard size can be calculated by analogy, and this embodiment is not exemplified here.

Step S232: the ratio of the third length to the first length is taken as the dimension ratio.

The ratio of the third length to the first length is taken as the dimension ratio.

By the method, the optical flow displacement of each optical flow point in the detection areas with different sizes can be normalized, so that the influence of the difference between the optical flow displacement between the near view and the distant view in the image on the abnormal behavior detection is eliminated.

Referring to fig. 5, fig. 5 is a flowchart illustrating an embodiment of step S11 in fig. 1. Specifically, step S11 in the above embodiment may include the following steps:

step S111: and detecting each frame of image by using a preset head-shoulder frame detection model to obtain a head-shoulder frame corresponding to the target pedestrian in each frame of image.

The preset head and shoulder detection model may be obtained in advance through deep learning training, and the embodiment is not particularly limited herein.

And detecting each frame of image through a preset head and shoulder detection model to obtain a head and shoulder frame corresponding to the target pedestrian in each frame of image.

Step S112: and obtaining a detection area corresponding to the target pedestrian from the head and shoulder frame corresponding to the target pedestrian.

The detection region corresponding to the target pedestrian can be obtained based on the head-shoulder frame corresponding to the target pedestrian. In one implementation scenario, the head and shoulder frames may be extended according to a predetermined ratio and a predetermined direction to obtain the detection area. For example, the head and shoulder frames are uniformly pulled down by a certain ratio, such as 1.2 times, and the like.

In the above manner, the image is detected by presetting the head and shoulder detection model, and the head and shoulder area is a relatively stable part in the image, so that the identification accuracy of the target pedestrian can be improved by presetting the head and shoulder detection model to detect the image, the influence of the motion of other objects on the determination of the detection area can be avoided by utilizing the head and shoulder frame, the accuracy of the obtained detection area can be improved, and the accuracy of the subsequent abnormal behavior detection based on the optical flow information in the detection area is improved.

Referring to fig. 6, fig. 6 is a schematic flowchart illustrating a method for detecting abnormal pedestrian behavior according to another embodiment of the present application.

Specifically, the step S13 may include the following steps:

step S61: and obtaining an optical flow threshold value of the target frame image based on the optical flow information of at least one frame of sample frame image.

And obtaining an optical flow threshold value which is subsequently used for detecting whether the pedestrian has abnormal behaviors in the target frame image based on the optical flow information of the at least one frame of sample frame image. In one implementation scenario, in order to detect whether the pedestrian has abnormal behavior from multiple dimensions, thereby making the detection result of the abnormal behavior more accurate, the optical flow threshold may include at least one of an optical flow displacement threshold, an optical flow ratio threshold.

Specifically, the optical-flow displacement threshold of the target frame image may be obtained by acquiring a first optical-flow displacement of the detection area of each frame sample image, and determining based on the first optical-flow displacement. In one implementation scenario, the optical flow displacement of the current frame may be obtained based on the optical flow information of the optical flow point in the current frame and the optical flow information of the corresponding optical flow point in the previous frame, and the size ratio in the above embodiment, and specifically, may be obtained by coordinate information in the optical flow information, such as the optical flow point1 the coordinate information in the current frame image is (X)_1,t，Y_1,t) The coordinate information in the previous frame image is (X)_1,t-1，Y_1,t-1) Then the optical flow displacement of optical flow point 1 can be represented as a vectorFurther, the magnitude of the optical flow displacement of the optical flow point 1 can be expressed in the modulo of the vector asThe product between the above value and the size ratio can finally be taken as the optical flow displacement. By analogy, the optical flow displacement and the optical flow direction change of other optical flow points can be calculated, and this embodiment is not exemplified.

In addition, the optical flow rate threshold of the target frame image can be obtained by counting the number of first optical flow points in the detection area of each frame sample image, wherein the optical flow direction change is larger than a preset angle, and determining the optical flow rate threshold based on the number of the first optical flow points. In one implementation scenario, the optical flow direction change of the current frame may be obtained based on the optical flow information of the optical flow point in the current frame and the optical flow information of the corresponding optical flow point in the previous frame and the next frame, and specifically, may be obtained by coordinate information in the optical flow information, for example, the coordinate information of the optical flow point 1 in the current frame image is (X)_1,t，Y_1,t) The coordinate information in the subsequent frame image is (X)_1,t+1，Y_1,t+1) The coordinate information in the previous frame image is (X)_1,t-1，Y_1,t-1) So that the optical flow represented by the optical flow point 1 in the current frame and the previous frame is shifted byThe optical flow point 1 is displaced by the optical flow represented in the current frame and the next frameSo that the change of the optical flow direction of the optical flow point 1 in the current frame can be obtained asThe preset angle may be uniformly preset, or may be set by a user according to an actual application in a self-defined manner, and the preset angle may be 100 degrees, 120 degrees, and the like, which is not limited in this embodiment.

Step S62: the optical flow information of the target frame image is compared with an optical flow threshold value.

In one implementation scenario, to detect whether there is abnormal behavior of a pedestrian from multiple dimensions, the optical flow threshold may include at least one of an optical flow displacement threshold, an optical flow ratio threshold. In this case, the step S62 may be specifically implemented by acquiring a second optical flow displacement of the detection area of the target frame image, and comparing the second optical flow displacement with an optical flow displacement threshold value; alternatively, the step S62 may be implemented by counting the number of second optical flow points in the detection area of the target frame image, where the change in the optical flow direction is greater than a preset angle, obtaining the optical flow ratio of the target frame image based on the number of second optical flow points, and comparing the optical flow ratio with the optical flow ratio threshold; alternatively, the step S62 may be implemented by first acquiring a second optical flow displacement of the detection area of the target frame image, comparing the second optical flow displacement with the optical flow displacement threshold, counting the number of second optical flow points in the detection area of the target frame image, the number of which optical flow direction changes by more than a predetermined angle, obtaining an optical flow ratio of the target frame image based on the number of second optical flow points, and comparing the optical flow ratio with the optical flow ratio threshold. Specifically, the specific manner of calculating the second optical-flow displacement of the detection area of the target frame image may refer to the above-described embodiment of calculating the optical-flow displacement threshold in step S61, and the specific manner of calculating the optical-flow direction change in the detection area of the target frame image may refer to the above-described embodiment of calculating the optical-flow direction change in the detection area of the sample frame image in step S61, which is not illustrated here.

The step S14 may include:

step S63: and if the optical flow information of the target frame image and the optical flow threshold value meet the preset relation condition, determining that the target pedestrian has abnormal behaviors.

In one implementation scenario, to detect whether there is abnormal behavior of a pedestrian from multiple dimensions, the optical flow threshold may include at least one of an optical flow displacement threshold, an optical flow ratio threshold. At this time, the preset relation condition includes: the second optical flow displacement is greater than or equal to a first preset multiple of the optical flow displacement threshold; and/or the optical flow ratio is greater than or equal to a second preset multiple of the optical flow ratio threshold. The first preset multiple may be 1.3, 1.5, 1.7, 1.9, 2.1, etc., and the embodiment is not limited in this respect. The second preset multiple may be 1.3, 1.5, 1.7, 1.9, 2.1, etc., and the embodiment is not limited herein.

In the above manner, the optical flow threshold is obtained through the optical flow information of a plurality of frame sample frame images before the target frame image, and by comparing the optical flow information of the target frame image with the optical flow threshold, if the optical flow information of the target frame image and the optical flow threshold satisfy the preset relation condition, the target pedestrian can be determined to have abnormal behaviors.

Referring to fig. 7, fig. 7 is a flowchart illustrating an embodiment of step S61 in fig. 6. In order to reduce the amount of calculation for subsequently determining whether there is an abnormal behavior by using the optical flow information, the step S61 of "acquiring the first optical flow displacement of the detection area of each frame of the sample frame image" may be specifically implemented by:

step S611: and determining a first optical flow point of which the optical flow displacement in the detection area of each frame of sample frame image meets a preset displacement condition.

The preset displacement condition includes a preset number of maximum optical flow displacements in the detection area, where the preset number may be 10% of the number of all optical flow points in the detection area, and the preset displacement condition is 10% of the maximum optical flow displacements in the detection area. For example, the sample frame image f_tThe detection area in (1) contains 3000 optical flow points in total, then optical flow displacement is calculated according to optical flow information of the 3000 optical flow points, the 3000 optical flow points are sorted in the descending order of the optical flow displacement, the first 10% of the optical flow points are taken as first optical flow points, and the first optical flow points meeting the preset displacement condition are taken as other sample frame images in the same wayAnd determining the first optical flow point in the detection area of each frame of sample image, wherein the optical flow displacement meets the preset displacement condition.

Step S612: and obtaining a first optical flow displacement of the detection area corresponding to the sample frame image based on the optical flow displacement of the first optical flow point.

In one implementation scenario, the average value of the optical flow displacements of the first optical flow points that have been determined in step S611 described above may be used as the first optical flow displacement of the detection area of the sample frame image. For example, the sample frame image f_tThe total number of the first optical flow points in the detection area is 300, and the optical flow displacements of the 300 first optical flow points are respectively D_1,t、D_2,t、D_3,t……D_300,tSo that the average value of the optical flow displacements of the 300 first optical flow points can be calculatedThe average value is used as a sample frame image f_tThe first optical flow displacement of the detection area in (1). Of course, in another implementation scenario, the sum of the optical flow displacements of the first optical flow points of the sample frame image may be used as the first optical flow displacement of the detection area of the sample frame image. For example, the sample frame image f_tThe total number of the first optical flow points in the detection area is 300, and the optical flow displacements of the 300 first optical flow points are respectively D_1,t、D_2,t、D_3,t……D_300,tSo that the sum of the 300 first luminous flux point luminous flux displacements can be calculatedThe embodiment is not particularly limited herein.

Further, the "determining the optical-flow displacement threshold value of the target frame image based on the first optical-flow displacement" in the above step S61 may include:

step S613: and taking the average value of the first optical flow displacement of the at least one frame of sample frame image as the optical flow displacement threshold value of the target frame image.

Calculating all first optical flow displacements based on the calculated first optical flow displacements of all sample frame imagesAnd the average value of the optical flow displacement is used as an optical flow displacement threshold value for detecting the abnormal pedestrian behaviors in the target frame image. For example, taking consecutive 10 frame sample frame images before the target frame image, the first optical flow displacement is D_f1、D_f2、D_f3……D_f10Then the optical flow displacement threshold can be expressed asWhen the number of the sample frame images taken is other values, the number may be sequentially calculated, and this embodiment is not illustrated here.

Referring to fig. 8, fig. 8 is a flowchart illustrating an embodiment of step S62 in fig. 6. Similarly, in order to reduce the amount of calculation for subsequently determining whether there is an abnormal behavior using the optical flow information, "acquiring the second optical flow displacement of the detection area of the target frame image" in the above step S62 may include:

step S621: and determining a second optical flow point in the detection area of the target frame image, wherein the optical flow displacement meets a preset displacement condition.

The preset displacement condition includes a preset number of maximum optical flow displacements in the detection area, where the preset number may be 10% of the number of all optical flow points in the detection area, and the preset displacement condition is 10% of the maximum optical flow displacements in the detection area. For example, the target frame image f_t+1When 3000 optical flow points are included in the detection area, the optical flow displacement is calculated according to the optical flow information of the 3000 optical flow points, the 3000 optical flow points are sorted in the descending order of the optical flow displacement, and the first 10% of the optical flow points are taken as the second optical flow points.

Step S622: and obtaining a second optical flow displacement of the detection area of the target frame image based on the optical flow displacement of the second optical flow point.

In one implementation scenario, when the first optical flow displacement of the detection area in the sample frame image is obtained by calculating an average of the optical flow displacements of the first optical flow points, the average of the optical flow displacements of the second optical flow points of the target frame image may be taken as the second optical flow displacement of the detection area of the target frame image. For example, the target frame image f_t+1Second light in the detection areaThe total number of flow points is 300, and the optical flow displacement of 300 second optical flow points is D_1,t+1、D_2,t+1、D_3,t+1……D_300,t+1So that the average value of the optical flow displacements of the 300 second optical flow points can be calculatedUsing the average value as the target frame image f_t+1The second optical flow displacement of the detection area in (1). In another implementation scenario, when the first optical flow displacement of the detection area in the sample frame image is obtained by calculating the sum of the optical flow displacements of the first optical flow points, the sum of the optical flow displacements of the second optical flow points of the target frame image may be taken as the second optical flow displacement of the detection area of the target frame image. For example, the target frame image f_t+1The total number of the second optical flow points in the detection area is 300, and the optical flow displacements of the 300 second optical flow points are respectively D_1,t+1、D_2,t+1、D_3,t+1……D_300,t+1So that the sum of the 300 second luminous flux point luminous flux displacements can be calculatedAnd displaced as a second optical flow.

Referring to fig. 9, fig. 9 is a schematic flowchart illustrating another embodiment of step S61 in fig. 6. In one implementation scenario, in order to reduce the calculation amount of the optical flow direction, in the above step S61, "count the number of first optical flow points in which the optical flow direction changes by more than a preset angle in the detection area of each frame of sample frame image" may count the number of first optical flow points in which the optical flow direction is more than the preset angle based on the first optical flow points in which the optical flow displacement satisfies the preset displacement condition in the detection area of each frame of sample frame image has been determined in the above step S611. Specifically, the "determining the optical flow ratio threshold value of the target frame image based on the number of the first optical flow points" in the above-described step S61 may include:

step S91: and acquiring the point ratio between the first optical flow point number of each frame of sample frame image and the total optical flow point number of the detection area.

Counting to obtain the detection area of each frame of sample frame imageThe number of the first optical flow points with the optical flow direction change larger than the preset angle can be calculated, so that the point ratio between the number of the first optical flow points of each frame of sample image and the total number of the optical flow points of the detection area can be calculated. For example, when the first optical flow point is acquired based on the fact that the optical flow displacement in the detection area of each frame of the sample frame image has been determined to satisfy the preset displacement condition in step S611 described above, for example, the sample frame image f_tTaking 300 optical flow points satisfying a preset condition as first optical flow points, and further counting the number of optical flow directions in the 300 first optical flow points, for example, 200, to determine the sample frame image f_tIs compared with the total number of optical flow points of the detection area_tIs 200/300. Similarly, the same reasoning can be repeated to obtain other sample frame images f through calculation respectively_t-1、f_t-2、f_t-3……f_t-nIs compared with the total number of optical flow points of the detection area_t-1、P_t-2、P_t-3……P_t-n。

Step S92: and taking the average value of the point ratio of at least one frame of sample frame image as the optical flow ratio threshold value of the target frame image.

And taking the average value of the point ratios of all the sample frame images as an optical flow rate threshold value for detecting the abnormal behavior of the pedestrian in the target frame image. For example, the other sample frame images f are calculated respectively_t-1、f_t-2、f_t-3……f_t-nIs compared with the total number of optical flow points of the detection area_t-1、P_t-2、P_t-3……P_t-nThen, the dot ratio of all sample frame images is averaged as the optical flow ratio threshold

Correspondingly, in an implementation scenario, in order to reduce the calculation amount of the optical flow direction, the above-mentioned "count the number of second optical flow points whose optical flow direction changes by more than a preset angle in the detection area of the target frame image" in step S62 may be based on the second optical flow points whose optical flow displacement satisfies the preset displacement condition in the detection area of the target frame image determined in step S621, and count the number in which the optical flow direction changes by more than the preset angle.

The "obtaining the optical flow ratio of the target frame image based on the number of the second optical flow points" in the above-described step S62 may include: and taking the ratio of the second optical flow point number of the target frame image to the total optical flow point number of the detection area as the optical flow ratio of the target frame image.

Specifically, the number of second optical flow points in which the change in the optical flow direction in the detection area of the target frame image is larger than a preset angle may be obtained, so that the point ratio between the number of second optical flow points of the target frame image and the total number of optical flow points of the detection area may be calculated. For example, when the second optical flow point is the second optical flow point whose optical flow displacement satisfies the preset displacement condition in the detection area based on the target frame image determined in the above-described step S621, for example, the target frame image f_t+1Taking 300 optical flow points satisfying a preset condition as second optical flow points, and further counting the number of optical flow directions in the 300 second optical flow points larger than a preset angle, for example, 100, the target frame image f can be determined_t+1Is compared with the total number of optical flow points of the detection area_t+1Is 100/300.

Referring to fig. 10, fig. 10 is a schematic flow chart of a pedestrian abnormal behavior detection method according to another embodiment of the present application. Fig. 10 is a flowchart illustrating a method for detecting abnormal pedestrian behavior according to the present application, wherein the steps in any of the above embodiments may be referred to in the detailed description. Specifically, the method may include:

step S1001: and detecting each frame of image by using a preset head-shoulder frame detection model to obtain a head-shoulder frame corresponding to the target pedestrian in each frame of image.

The preset head and shoulder detection model may be obtained in advance through deep learning training, and the embodiment is not particularly limited herein.

Step S1002: and obtaining a detection area corresponding to the target pedestrian from the head and shoulder frame corresponding to the target pedestrian.

The detection region corresponding to the target pedestrian can be obtained based on the head-shoulder frame corresponding to the target pedestrian. In one implementation scenario, the head and shoulder frames may be extended according to a predetermined ratio and a predetermined direction to obtain the detection area. For example, the head and shoulder frames are uniformly pulled down by a certain ratio, such as 1.2 times, and the like.

Step S1003: and judging whether the detection area is smaller than a preset minimum size. If so, go to step S1004, otherwise go to step S1005.

The preset minimum size is the minimum size that can contain optical flow information required for abnormal behavior detection. When the detection area is smaller than the preset minimum size, the optical flow information in the detection area is less, which may result in insufficient use for subsequent detection of abnormal behavior. The preset minimum size is a preset size which is uniformly set, or can also be a preset size which is set by a user in a self-defined way. In one implementation scenario, the detection area is a rectangular detection area, and the preset minimum size is also a rectangle.

Step S1004: and amplifying the image in the detection area to a preset standard size, and taking the amplified image as the image in the detection area.

When the detection area is smaller than the preset minimum size, it means that if the optical flow detection is performed on the image in the detection area at this time, the detected optical flow information may be less, and thus, the subsequent detection of the abnormal behavior may not be sufficient. At this time, the image in the detection area may be enlarged to a preset standard size, and the enlarged image may be used as the image in the detection area.

Step S1005: and calculating the size ratio between the size of the detection area and a preset standard size, and correspondingly taking the product of the optical flow displacement of each optical flow point in the detection area and the size ratio as the optical flow displacement of the optical flow point.

If the detection area is not smaller than the preset minimum size, it means that if the optical flow detection is performed on the image in the detection area at this time, the detected optical flow information is sufficient for the subsequent detection of the abnormal behavior. At this time, a size ratio between the size of the detection area and a preset standard size may be further calculated, and a product of the optical flow displacement of each optical flow point in the detection area and the size ratio may be taken as the optical flow displacement of the corresponding optical flow point. Specifically, the preset standard size may be divided by the size of the detection area to obtain a size ratio, for example, if the size ratio is 1.2, it indicates that the preset standard size is larger than the detection area, and the optical flow displacement of each optical flow point in the detection area is multiplied by the size ratio 1.2 to obtain the optical flow displacement corresponding to the optical flow point; or, for example, if the size ratio is 0.8, it indicates that the preset standard size is smaller than the detection area, the optical flow displacement of each optical flow point in the detection area is multiplied by the size ratio 10.8 to obtain the optical flow displacement of the corresponding optical flow point, and the optical flow displacements of the optical flow points in the detection areas with different sizes can be normalized by the above method, so that the influence of the difference between the optical flow displacements between the near view and the far view in the image on the abnormal behavior detection is eliminated.

Step S1006: and respectively carrying out optical flow detection on the detection area of each frame of image to obtain optical flow information of the detection area of each frame of image, wherein the multi-frame image comprises a target frame image and at least one frame of sample frame image, and the at least one frame of sample frame image is a plurality of frame images before the target frame image.

Methods related to optical flow detection currently include gradient-based methods, matching-based methods, energy-based methods, and the like. The following are typical: the Horn-hill Method (Horn-Schunck Method), the Lucas-Kanade Method (Lucas-Kanade Method), and the like. The optical flow detection method is the prior art in the field, and the description of the embodiment is omitted here. At least one frame of sample frame image is several frame images before the target frame image, for example, 10 frame images before the target frame image, or 15 frame images before the target frame image, and the like, and the embodiment is not limited in particular herein.

Step S1007: and determining a first optical flow point of which the optical flow displacement in the detection area of each frame of sample frame image meets a preset displacement condition.

The preset displacement condition includes a preset number of maximum optical flow displacements in the detection area, where the preset number may be 10% of the number of all optical flow points in the detection area, and the preset displacement condition is 10% of the maximum optical flow displacements in the detection area.

Step S1008: the average value of the optical flow displacements of the first optical flow points of the sample frame image is taken as the first optical flow displacement of the detection area of the sample frame image.

For example, the sample frame image f_tThe total number of the first optical flow points in the detection area is 300, and the optical flow displacements of the 300 first optical flow points are respectively D_1,t、D_2,t、D_3,t……D_300,tSo that the average value of the optical flow displacements of the 300 first optical flow points can be calculatedThe average value is used as a sample frame image f_tThe first optical flow displacement of the detection area in (1).

Step S1009: and taking the average value of the first optical flow displacement of the at least one frame of sample frame image as the optical flow displacement threshold value of the target frame image.

For example, taking consecutive 10 frame sample frame images before the target frame image, the first optical flow displacement is D_f1、D_f2、D_f3……D_f10Then the optical flow displacement threshold can be expressed as

Step S1010: and determining a second optical flow point in the detection area of the target frame image, wherein the optical flow displacement meets a preset displacement condition.

The preset displacement condition includes a preset number of maximum optical flow displacements in the detection area, where the preset number may be 10% of the number of all optical flow points in the detection area, and the preset displacement condition is 10% of the maximum optical flow displacements in the detection area.

Step S1011: and taking the average value of the optical flow displacement of the second optical flow point of the target frame image as the second optical flow displacement of the detection area of the target frame image.

For example, the target frame image f_t+1The total number of the second optical flow points in the detection area is 300, and the optical flow displacements of the 300 second optical flow points are respectively D_1,t+1、D_2,t+1、D_3,t+1……D_300,t+1So that the average value of the optical flow displacements of the 300 second optical flow points can be calculatedUsing the average value as the target frame image f_t+1The second optical flow displacement of the detection area in (1).

In an implementation scenario, the steps S107 to S109 and the steps S1010 to S1011 may be executed sequentially or in parallel, and this embodiment is not limited specifically herein.

Step S1012: and judging that the second optical flow displacement is greater than or equal to a first preset multiple of the optical flow displacement threshold, if so, executing step S1013, otherwise, executing step S1018.

The first preset multiple may be 1.3, 1.5, 1.7, 1.9, 2.1, etc., and the embodiment is not limited in this respect.

Step S1013: counting the number of first optical flow points of which the direction change of the optical flow in the detection area of each frame of sample frame image is larger than a preset angle, and acquiring the point ratio between the number of the first optical flow points of each frame of sample frame image and the total number of the optical flow points of the detection area.

For example, the sample frame image f_tTaking 300 optical flow points satisfying a preset condition as first optical flow points, and further counting the number of optical flow directions in the 300 first optical flow points, for example, 200, to determine the sample frame image f_tIs compared with the total number of optical flow points of the detection area_tIs 200/300. Similarly, the same reasoning can be repeated to obtain other sample frame images f through calculation respectively_t-1、f_t-2、f_t-3……f_t-nIs compared with the total number of optical flow points of the detection area_t-1、P_t-2、P_t-3……P_t-n。

Step S1014: and taking the average value of the point ratio of at least one frame of sample frame image as the optical flow ratio threshold value of the target frame image.

For example, respectively calculatedOther sample frame image f_t-1、f_t-2、f_t-3……f_t-nIs compared with the total number of optical flow points of the detection area_t-1、P_t-2、P_t-3……P_t-nThen, the dot ratio of all sample frame images is averaged as the optical flow ratio threshold

Step S1015: and counting the number of second optical flow points of which the optical flow direction change is larger than a preset angle in the detection area of the target frame image, and taking the ratio of the number of the second optical flow points of the target frame image to the number of total optical flow points of the detection area as the optical flow ratio of the target frame image.

For example, the target frame image f_t+1Taking 300 optical flow points satisfying a preset condition as second optical flow points, and further counting the number of optical flow directions in the 300 second optical flow points larger than a preset angle, for example, 100, the target frame image f can be determined_t+1Is compared with the total number of optical flow points of the detection area_t+1Is 100/300.

In an implementation scenario, the steps S1013 to S1014 and the step S1015 may be executed successively or in parallel, and this embodiment is not limited in this respect.

Step S1016: and judging that the optical flow ratio is greater than or equal to a second preset multiple of the optical flow ratio threshold, if so, executing step S1017, otherwise, executing step S1018.

The second preset multiple may be 1.3, 1.5, 1.7, 1.9, 2.1, etc., and the embodiment is not limited herein.

Step S1017: and determining that the pedestrian has abnormal behaviors.

If the determination results in step S1012 and step S1016 are both yes, it can be determined that the pedestrian has an abnormal behavior. In an implementation scene, alarm information can be sent out at the moment, specifically, acousto-optic alarm information can be sent out, so that monitoring personnel are prompted to have pedestrian abnormal behaviors, the monitoring personnel can intervene and intervene in time, or the monitoring personnel inform security personnel to intervene and intervene in time, life and property losses are reduced, and security and safety are maintained stably.

Step S1018: it is determined that the pedestrian does not have abnormal behavior.

If at least one of the determination results of the step S1012 and the step S1016 is negative, it may be determined that the pedestrian has no abnormal behavior.

Referring to fig. 11, fig. 11 is a block diagram of an image processing apparatus 1100 according to an embodiment of the present disclosure. The image processing apparatus 1100 comprises a memory 1110 and a processor 1120 which are coupled to each other, and the processor 1120 is configured to execute program instructions stored in the memory 1110 to implement the steps of the pedestrian abnormal behavior detection method in any one of the embodiments.

Specifically, the processor 1120 is configured to control itself and the memory 1110 to implement the pedestrian abnormal behavior detection method in any of the above embodiments, and the processor 1120 is further configured to control itself and the memory 1110 to implement the optical flow autocorrelation determination method of the object in any of the above embodiments. Processor 1120 may also be referred to as a CPU (Central Processing Unit). Processor 1120 may be an integrated circuit chip having signal processing capabilities. The Processor 1120 may also be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. In addition, the processor 1120 may be commonly implemented by a plurality of circuit forming chips.

In this embodiment, the processor 1120 is further configured to perform target detection on multiple frames of images captured by the imaging device respectively to obtain a detection area corresponding to a target pedestrian in each frame of image, the processor 1120 is further configured to perform optical flow detection on the detection area of each frame of image respectively to obtain optical flow information of the detection area of each frame of image, where the multiple frames of images include the target frame of image and at least one frame of sample frame of image, the processor 1120 is further configured to analyze the optical flow information of the target frame of image by using the optical flow information of the at least one frame of sample frame of image, and the processor 1120 is further configured to determine whether an abnormal behavior exists in the target pedestrian based on a result of the analysis.

In the above manner, the multi-frame images including the target frame image and the at least one frame of sample frame image, which are captured by the imaging device, are respectively subjected to target detection, so that the detection area corresponding to the target pedestrian in each frame of image is obtained, and then the optical flow detection is performed on the detection area, so that the optical flow information of the target frame image and the optical flow information of the at least one frame of sample frame image are obtained, and therefore, the optical flow information of the target frame image is analyzed by using the optical flow information of the sample frame image, and further, the change of the optical flow information of the target frame image compared with the sample frame image can be found, so that no matter the scene where the target pedestrian is located, the optical flow information of the sample frame image captured by the target pedestrian can be used as the current behavior judgment standard of the target pedestrian for performing behavior judgment on the optical flow information of the target.

In some embodiments, the processor 1120 is further configured to derive an optical flow threshold of the target frame image based on the optical flow information of the at least one frame of sample frame image, the processor 1120 is further configured to compare the optical flow information of the target frame image with the optical flow threshold, and the processor 1120 is further configured to determine that the target pedestrian has abnormal behavior if a preset relation condition is satisfied between the optical flow information of the target frame image and the optical flow threshold.

In some embodiments, the optical flow threshold comprises at least one of: the processor 1120 is further configured to obtain a first optical flow displacement of the detection area of each frame of the sample frame image and determine to obtain an optical flow displacement threshold of the target frame image based on the first optical flow displacement, the processor 1120 is further configured to count a number of first optical flow points in the detection area of each frame of the sample frame image, where a change in optical flow direction is greater than a preset angle, and determine to obtain an optical flow ratio threshold of the target frame image based on the number of the first optical flow points, the processor 1120 is further configured to obtain a second optical flow displacement of the detection area of the target frame image, compare the second optical flow displacement with the optical flow displacement threshold, the processor 1120 is further configured to count a number of second optical flow points in the detection area of the target frame image, where a change in optical flow direction is greater than the preset angle, and obtain an optical flow ratio of the target frame image based on the number of the second optical flow points, compare the optical flow ratio with the optical flow ratio threshold, the preset relation conditions comprise: the second optical flow displacement is greater than or equal to a first preset multiple of the optical flow displacement threshold; and/or the optical flow ratio is greater than or equal to a second preset multiple of the optical flow ratio threshold.

In some embodiments, the processor 1120 is further configured to determine a first optical flow point in the detection area of each frame of the sample frame image, where the optical flow displacement satisfies a preset displacement condition, the processor 1120 is further configured to derive a first optical flow displacement of the detection area of the corresponding sample frame image based on the optical flow displacement of the first optical flow point, the processor 1120 is further configured to determine a second optical flow point in the detection area of the target frame image, where the optical flow displacement satisfies the preset displacement condition, and the processor 1120 is further configured to derive a second optical flow displacement of the detection area of the target frame image based on the optical flow displacement of the second optical flow point.

In some embodiments, the preset displacement condition includes a preset number of optical flow displacements in the detection area that is the largest, the processor 1120 is further configured to use an average of the optical flow displacements of the first optical flow points of the sample frame images as a first optical flow displacement of the detection area of the sample frame images, the processor 1120 is further configured to use an average of the first optical flow displacements of at least one frame of the sample frame images as an optical flow displacement threshold of the target frame images, and the processor 1120 is further configured to use an average of the optical flow displacements of the second optical flow points of the target frame images as a second optical flow displacement of the detection area of the target frame images.

In some embodiments, the processor 1120 is further configured to obtain a point ratio between a first optical flow point of each frame of the sample frame image and a total optical flow point of the detection area, the processor 1120 is further configured to use an average of the point ratios of at least one frame of the sample frame image as an optical flow ratio threshold of the target frame image, and the processor 1120 is further configured to use a ratio between a second optical flow point of the target frame image and the total optical flow point of the detection area as an optical flow ratio of the target frame image.

In some embodiments, the processor 1120 is further configured to determine whether the detection area is smaller than a preset minimum size, when the detection area is determined to be smaller than the preset minimum size, the processor 1120 is further configured to enlarge an image in the detection area to a preset standard size and use the enlarged image as the image in the detection area, and when the detection area is determined not to be smaller than the preset minimum size, the processor 1120 is further configured to calculate a size ratio between the size of the detection area and the preset standard size, and to correspond a product of an optical flow displacement of each optical flow point in the detection area and the size ratio to an optical flow displacement of the optical flow point.

In some embodiments, the processor 1120 is further configured to obtain a first length of a diagonal line of the detection region and a second length of the diagonal line of the preset minimum size, the processor 1120 is further configured to determine whether the first length is smaller than the second length, the processor 1120 is further configured to determine that the detection region is smaller than the preset minimum size when the first length is determined to be smaller than the second length, and the processor 1120 is further configured to determine that the detection region is not smaller than the preset minimum size when the first length is determined to be not smaller than the second length.

In some embodiments, the processor 1120 is further configured to obtain a first length of a diagonal line of the detection area and a third length of the diagonal line of a preset standard size, and the processor 1120 is further configured to use the third length and the first length as a size ratio.

In some embodiments, at least one frame of the sample frame images is several frames of images before the target frame image, the processor 1120 is further configured to detect each frame of image by using a preset head-shoulder frame detection model to obtain a head-shoulder frame corresponding to the target pedestrian in each frame of image, and the processor 1120 is further configured to obtain a detection region corresponding to the target pedestrian from the head-shoulder frame corresponding to the target pedestrian.

In some embodiments, the image processing apparatus 1100 further includes an image pickup device 1130 for capturing a plurality of frames of images in time sequence.

Referring to fig. 12, fig. 12 is a schematic diagram of a memory device 1200 according to an embodiment of the present application. The memory device 1200 of the present application stores program instructions 1210 that can be executed by a processor, and the program instructions 1210 are used to implement the steps of any one of the above-described embodiments of the method for detecting abnormal behaviors of pedestrians.

In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a module or a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) or a processor (processor) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.