1. A micro-expression compression method based on three-branch decision and optical flow filtering mechanism is characterized in that: the method comprises the following steps:

s1, selecting the microexpression data set A ═ V₁,V₂,V₃,…V_tPerforming image completion, size unification and image graying pretreatment;

s2-use MTCNN multitask cascade neural network pair V₁,V₂,V₃,…V_tPositioning and cutting a face area of the video clip picture, and unifying the size of the picture;

s3, for each video V_i＝{v₁,v₂,…,v_tV for every two consecutive video segments_iAnd v_i+1All have o_iGenerating, video V_iConversion to optical flow set O_i＝{o₁,o₂,…,o_t-1}；

S4 for O_i＝{o₁,o₂,…,o_t-1Acquiring the lateral displacement of each optical flowAnd longitudinal displacementCalculating the intensity of each optical flow by the following expression, wherein W represents a horizontal pixel, and H represents a vertical pixel size;

s5 for current optical flow o_iObtaining average pixel intensity under current optical flowThe expression is as follows:

s6 applying the weighted function to each optical flow o_iAnd carrying out weighted assignment, wherein the assignment expression is as follows:

s7 repeating S3-S6 to perform optical flow processing weighting on each video clip set, and obtaining each video set V_iCorresponding optical flow weighting set omega_i；

S8 defining three threshold values (alpha, beta) by comparing omega_iThe set is subjected to branch screening, and the rule is defined as:

when in useTime, light flow o_iDivision into NEG_(α,β)(O) a set of domains;

when in useTime, light flow o_iDivision into BND_(α,β)(O) a set of domains;

when in useTime, light flow o_iDivided into POS_(α,β)(O) a set of domains;

s9, acquiring a video clip set according to the optical flow branching rule, wherein the rule is defined as:

when o is_i∈POS_(α,β)(O) time, video clip v_i+1Is defined as v_i+1∈POS_(α,β)(V)；

When o is_i∈BND_(α,β)(O) time, video clip v_i+1Is defined as v_i+1∈BND_(α,β)(V)；

When o is_i∈NEG_α(O) time, video clip v_i+1Is defined as v_i+1∈NEG_(α,β)(V)；

S10 collecting BND according to light flow_(α,β)(V)∪POS_(α,β)(V) updating the video segment set, and performing video segment reordering integration based on the time sequence to obtain a new video segment sequence set V;

s11: defining a convergence coefficient eta, and repeating the steps S3-S10 for the times of reaching the convergence coefficient or NEG_(α,β)(O) the domain set data remains 0 in the self-growth threshold state, stopping the iteration;

s12: obtaining a high-quality semantic video clip set, and obtaining a high-quality video clip set V;

s13, extracting the texture feature based on the dynamic video of the video V based on the optical flow filtration to obtain the video feature values on three planes of a space plane XY and space-time planes XT, YT, and the expression is shown as follows:

and S14, training the video features acquired in the S13 by adopting a classifier, and acquiring a micro expression recognition model for final recognition of micro expressions.

Background

A micro-expression is a subtle, involuntary facial expression that is usually affected by complex environmental, human, etc. factors and is produced in an involuntary situation. Micro-expressions are imperceptible in general, compared to macro-expressions, and in fact, due to the physical characteristics of humans, such involuntary facial expressions are represented by an extremely rapid and subtle facial movement. The exfiltration of micro-expressions expresses the real emotion that people suppress and try to hide, and the current micro-expressions are mainly composed of several basic emotions, including happiness, anger, disgust, fear, surprise and others. Micro-expressions are a responsive action ascribed to physiology and thus reveal a real psychological state that is not controllable.

Through the analysis of the micro expression related data, the video contains a large number of images with lower semantic information, and the images are not expressed and have little morphological and semantic change. According to the analysis of the micro-expression video data, the area where the micro-expression occurs is mainly concentrated from the starting frame to the offset frame, and the peak of the semantics is reached in the top frame. Therefore, unprocessed data contains a large amount of low quality and unbalanced data.

Disclosure of Invention

The invention provides a micro expression compression method based on three-branch decision and an optical flow filtering mechanism, which defines a weighting function by using an optical flow attribute and provides a micro expression research method with redundancy removal and video compression functions based on a rough set probability decision method.

The invention is realized by the following technical scheme:

a micro-expression compression method based on three-branch decision and optical flow filtering mechanism comprises the following steps:

s1, selecting the microexpression data set A ═ V₁,V₂,V₃,…V_tPreprocessing such as image completion, size unification, image graying and the like;

s2, adopting MTCNN multitask cascade neural network pair V₁,V₂,V₃,…V_tPositioning and cutting a face area of the video clip picture, and unifying the size of the picture;

s3, for each video V_i＝{v₁,v₂,…,v_tFor every two consecutive video segments v_iAnd v_i+1All have o_iGenerating, video V_iConversion to optical flow set O_i＝{o₁,o₂,…,o_t-1}；

S4 for O_i＝{o₁,o₂,…,o_t-1Acquiring the lateral displacement of each optical flowAnd longitudinal displacementCalculating the intensity of each optical flow by the following expression, wherein W represents a horizontal pixel, and H represents a vertical pixel size;

s5 for current optical flow o_iObtaining average pixel intensity under current optical flowThe expression is as follows:

s6 applying the weighted function to each optical flow o_iAnd carrying out weighted assignment, wherein the assignment expression is as follows:

s7 repeating S3-S6 to perform optical flow processing weighting on each video clip set, and obtaining each video set V_iCorresponding optical flow weighting set omega_i；

S8 defining three threshold values (alpha, beta) by comparing omega_iThe set is subjected to branch screening, and the rule is defined as:

when in useTime, light flow o_iDivision into NEG_(α,β)(O) a set of domains;

when in useTime, light flow o_iDivision into BND_(α,β)(O) a set of domains;

when in useTime, light flow o_iDivided into POS_(α,β)(O) a set of domains;

s9, acquiring a video clip set according to the optical flow branching rule, wherein the rule is defined as:

when o is_i∈POS_(α,β)(O) time, video clip v_i+1Is defined as v_i+1∈POS_(α,β)(V)；

When o is_i∈BND_(α,β)(O) time, video clip v_i+1Is defined as v_i+1∈BND_(α,β)(V)；

When o is_i∈NEG_α(O) time, video clip v_i+1Is defined as v_i+1∈NEG_(α,β)(V)；

S10 collecting BND according to light flow_(α,β)(V)∪POS_(α,β)(V) updating the video segment set, and performing video segment reordering integration based on the time sequence to obtain a new video segment sequence set V;

s11: defining a convergence coefficient eta, and repeating the steps S3-S10 for the number of times to reach the convergence coefficient or NEG_(α,β)(O) the domain set data remains 0 in the self-growth threshold state, stopping the iteration;

s12: obtaining a high-quality semantic video clip set, and obtaining a high-quality video clip set V;

s13, extracting the texture feature based on the dynamic video of the video V based on the optical flow filtration to obtain the video feature values on three planes of a space plane XY and space-time planes XT, YT, and the expression is shown as follows:

and S14, training the video features acquired in the S13 by adopting a classifier, and acquiring a micro expression recognition model for final recognition of micro expressions.

Compared with the prior art, the invention has the following advantages:

1. the invention introduces the rough set probability decision into the field of micro expression, and carries out information decision through the rough set, thereby expanding the new direction of micro expression research.

2. According to the method, the redundancy of the picture information is removed according to the optical flow change weight, the video clip information is effectively compressed, and the semantic expression among the information is improved.

Drawings

FIG. 1 is a basic flow diagram of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings and embodiments. The embodiments described herein are only for explaining the technical solution of the present invention and are not limited to the present invention.

The invention will be further explained by the following embodiments, as shown in the basic flow chart of the micro-expression compression method based on the three-branch decision and optical flow filtering mechanism shown in fig. 1.

1. Taking a micro-expression CASMIEII data set as experimental data, wherein the experimental data comprises 26 participants, and 256 micro-expression video files V ═ V { (V)₁,V₂,V₃,…V₂₅₆And 5, the micro-emotions labels comprise happy, disgusting, fear, sadness and other 5 kinds of emotions labels. And defining an initialization iteration threshold eta of 5, a statistical number S of 0, and a threshold (alpha, beta) of (0.35, 0.6).

2. For video files V₁＝{v₁,v₂,…,v₂₉₀Anda unit configured to obtain an optical flow set O ═ O according to an optical flow extraction rule, the optical flow set being composed of 290 video frame pictures₁,o₂,…,o₂₈₉289 optical flows are used for expressing semantic change relations among pictures.

3. Using the optical flow weighting function pair defined by S4-S6, O ═ { O ═ O₁,o₂,…,o₂₈₉Carrying out weight calculation to obtain an optical flow weight set

4. For theThe optical flow filtering is performed according to the threshold value (α, β) ═ 0.35, 0.6.

5. Traverse ω (O), according to step S8, whenTime, light flow o_iDivision into NEG_(α,β)(O) a set of domains, wherein,time division into optical flows o_iDivision into BND_(α,β)(O) sets of fields, whereas the optical flow O_iDivided into POS_(α,β)(O)。

6. According to step S9, the video frames are classified into POS according to optical flow classification_(α,β)(V)，BND_(α,β)(V) and NEG_(α,β)(V)。

7. And (5) repeating the steps 2-6, wherein the iteration threshold alpha is increased by 0.02 and S +1 every time the iteration threshold alpha is finished, and stopping until the iteration times S is more than or equal to eta for the statistical times.

8. Merging POS_(α,β)(V) and BND_(α,β)(V) and reordering to complete the video V₁Compressed into V₁′＝{v₁′,v₂′,…,v′₂₀₇In which v is₁′,v′₂,…,v′₂₀₇The reordered pictures are numbered.

9. And similarly, filtering other videos of the video V according to the steps 2-8 to obtain a new compressed video file V' ═ V₁′,V₂′,V₃′,…V′₂₅₆}。

10. Extraction of V' feature H_α,βThe method is used for classification and identification, and the effect of obtaining the identification rate is about 51%.

The foregoing merely represents preferred embodiments of the invention, which are described in some detail and detail, and therefore should not be construed as limiting the scope of the invention. It should be noted that, for those skilled in the art, various changes, modifications and substitutions can be made without departing from the spirit of the present invention, and these are all within the scope of the present invention. Therefore, the protection scope of the present patent should be subject to the appended claims.