1. The brain muscle coupling method based on the time delay scale long-short term memory network and the transfer entropy comprises the following steps:

step (1): collecting sample data of human electroencephalogram signals and electromyogram signals, and then carrying out wavelet threshold denoising on the collected data;

step (2): marking a cell unit for processing current information as a cell unit at the time of t in a long-short term memory network, then marking a cell unit which is before the current cell unit as a cell unit at the time of t-1, and then marking a cell unit which is before the cell unit at the time of t-1 as a cell unit at the time of t-2;

adding a time delay connection on the basis of the long-short term memory network, and connecting the output of the t-2 moment cell unit to the t moment cell unit; the cell unit at the t moment for processing the current moment information can not only consider the data characteristic of the previous moment, but also learn the output layer characteristic of the cell unit at the t-2 moment, and the model constructed in the step is marked as a long-short term memory network with the time delay scale of 1;

and (3): marking the model constructed in the step (2) as a network with the time delay scale of 1; in a network with the time delay scale of 1, marking a previous cell unit of a cell unit at the time of t-2 as a cell unit at the time of t-3, and connecting the output of the cell unit at the time of t-3 to the cell unit at the time of t, so that the cell unit at the time of t for processing the current time information can learn the characteristics of an output layer of the cell unit at the time of t-3, and marking the model as a long-short term memory network with the time delay scale of 2;

then, in a network with the time delay scale of 2, marking a previous cell unit of a cell unit at the time of t-3 as a cell unit at the time of t-4, and connecting the output of the cell unit at the time of t-4 to the cell unit at the time of t, so that the cell unit at the time of t can learn the characteristics of an output layer of the cell unit at the time of t-4, and marking the model as a long-short term memory network with the time delay scale of 3;

finally, in the network with the time delay scale of 3, the previous cell unit of the cell unit at the time of t-4 is marked as a cell unit at the time of t-5, the output of the cell unit at the time of t-5 is connected to the cell unit at the time of t, so that the cell unit at the time of t can learn the characteristics of an output layer of the cell unit at the time of t-5, and the model is marked as a long-short term memory network with the time delay scale of 4;

and (4): respectively inputting the electroencephalogram signal and the electromyogram signal into a long-short term memory network with four delay scales, and selecting a model with the minimum error according to three evaluation indexes of average absolute percentage error, root mean square error and Nash coefficient, namely an optimal delay scale model;

fourier transformation is carried out on the time sequence characteristics extracted by the long-term and short-term memory network with four time delay scales, four frequency domain characteristic sequences are obtained through conversion, the transfer entropy is obtained for the brain and muscle electrical signals at the same frequency point in the four obtained frequency domain characteristic sequences, four transfer entropy results are obtained, and the average value is obtained to serve as the reference coupling strength;

and (5): and (4) calculating the transfer entropy of the sequence extracted by the optimal time delay scale model obtained in the step (4) at the same frequency, and calculating the area between the transfer entropy and the reference coupling strength in the random frequency band, so as to obtain the coupling strength of the random frequency band.

Background

Artificial intelligence is known as the third technical revolution in human history, and along with the development of technology, the demands of people on intelligent medical treatment, intelligent home and novel human-computer interaction are increasing day by day. How to sense action behaviors and decode the movement intention of human bodies has become a popular research direction in the fields of pattern recognition, modern medical treatment, intelligent wearing and the like.

In a traditional human behavior perception and identification method such as a computer vision technology, the identification and prediction effect is easily influenced by the intensity of light and obstacles. Although the infrared detection technology can sense human body movement in a dark environment, the detection range of the infrared technology is easily limited and expensive. The Wifi signal detection technology is easily interfered by electromagnetic field in environment, and the applicable scene is very limited. Researchers have recognized that there is a need to find a more accurate and reliable method for analyzing human behavior patterns to follow the future development direction.

The electroencephalogram signals generated by the neuron cells in the cerebral cortex and the electromyogram signals generated by the muscle cells contain rich physiological information, so that the action mode of a human body is explored on the basis of the electroencephalogram signals and the electromyogram signals, the movement intention is decoded from the perspective of the physiological mechanism of the human body, and the method is more accurate and reliable compared with the traditional method.

The motor control theory describes the generation process of human behavior and movement, and indicates that the motor system of the human body is composed of two major systems of neuron cells and muscle tissues in cerebral cortex. When the brain controls the human body to complete a certain movement, the central nervous system of the human body conducts the potential information generated by the neurons in the brain downwards, and the potential information is finally converted into the kinetic energy of limbs to complete the movement. Taking the example of the brain controlling the palm to pick up an object, when a person wants to grasp an object, a motion control command is first issued from the cerebral cortex, and the command is propagated down the motion conduction path to the nerve endings and muscles of the palm portion of the person. When the palm touches an object, the tactile sensation generated by the palm part is transmitted to the cerebral cortex, spinal cord, cerebellum and other parts along the sensory conduction path, and the brain analyzes the sensory signals and then controls the subsequent activities of the human body. This process of information interaction between the brain and muscles through synchronous oscillations is called cortical muscle functional coupling.

When a human body executes a certain action, the electroencephalogram signal and the electromyogram signal acquired by using the synchronization method can reflect synchronous interaction information between the brain and the muscle, for example, the synchronous discharge phenomenon when the cerebral cortex and the muscle interact can be researched by performing coherence analysis on the electroencephalogram signal and the electromyogram signal on a frequency domain. The degree of discharge changes with the progress of the movement, and factors such as relaxation of muscle tissue, mental activity of the brain, magnitude of the movement, and external environmental disturbance affect the coherence between the cortex and the muscle. The study of cortical muscle coupling is beneficial to revealing the working rule of the central nervous system of a human body and the information flow mode between the insides of the motion systems, further understanding the domination mode of the human brain to limbs and the feedback mode of the limbs to the control instructions of the human brain, and further, the study has deeper significance in revealing the motion mechanism of the human body and providing experimental support for the fields of artificial limb control, medical rehabilitation, virtual reality technology and the like.

The existing brain muscle coupling analysis mainly calculates the coupling degree between the electroencephalogram signal and the electromyogram signal in a time domain or a frequency domain, and the calculated coupling degree is possibly far smaller than the actual coupling degree at this time due to different energy ranges of the electroencephalogram signal and the electromyogram signal. Therefore, the invention introduces a long-short term memory network model and the transfer entropy to be combined to calculate the brain muscle coupling degree.

Disclosure of Invention

The electroencephalogram signal and the electromyogram signal are physiological signals containing rich human body information, and the key for realizing decoding of human body movement intentions is to use the human body physiological signals to carry out brain muscle coupling analysis. The invention provides a method for measuring the brain muscle coupling degree by combining a long-short term memory network model with increased time delay scales and transfer entropy, which carries out Fourier transformation on time sequence characteristics extracted by long-short term memory networks with different time delay scales and converts the time sequence characteristics into frequency domain characteristics. And solving the transfer entropy of the frequency domain characteristics of the brain electromyographic signals at the same frequency point, and solving the mean value of the transfer entropy as the reference coupling strength. And calculating the transfer entropy of the sequence extracted by the optimal time delay scale model at the same frequency, and recording the area between the transfer entropy and the reference coupling strength in the random frequency band as the coupling strength of the frequency band.

In order to achieve the above object, the method of the present invention mainly comprises the following steps:

step (1): collecting sample data of human electroencephalogram signals and electromyogram signals, and then carrying out wavelet threshold denoising on the collected data.

Step (2): in a traditional long-short term memory network, a cell unit for processing current information is marked as a cell unit at time t, then a cell unit before the current cell unit is marked as a cell unit at time t-1, and then a cell unit before the cell unit at time t-1 is marked as a cell unit at time t-2. And adding a delay connection on the basis of the traditional long-short term memory network, and connecting the output of the cell unit at the t-2 moment to the cell unit at the t moment. Therefore, the t-time cell unit processing the current time information can not only consider the data characteristics of the previous time, but also learn the output layer characteristics of the t-2-time cell unit. Because the electroencephalogram signal and the electromyogram signal are time sequence sequences related to time, the electroencephalogram and electromyogram characteristics can be effectively prevented from being forgotten in the transmission process by adding the time delay connection.

And (3): and (3) recording the model constructed in the step (2) as a network with the time delay scale of 1. In the network with the time delay scale of 1, a cell unit before a cell unit at the time of t-2 is marked as a cell unit at the time of t-3, and the output of the cell unit at the time of t-3 is connected to the cell unit at the time of t, so that the cell unit at the time of t, which processes the current time information, can learn the characteristics of an output layer of the cell unit at the time of t-3, and the model is marked as the network with the time delay scale of 2.

In the network with the time delay scale of 2, a cell unit before a cell unit at the time t-3 is marked as a cell unit at the time t-4, and the output of the cell unit at the time t-4 is connected to the cell unit at the time t, so that the cell unit at the time t can learn the characteristics of an output layer of the cell unit at the time t-4, and the model is marked as the network with the time delay scale of 3.

In the network with the time delay scale of 3, a cell unit before a cell unit at the time of t-4 is marked as a cell unit at the time of t-5, the output of the cell unit at the time of t-5 is connected to the cell unit at the time of t, so that the cell unit at the time of t can learn the characteristics of an output layer of the cell unit at the time of t-5, and the model is marked as the network with the time delay scale of 4. Four long-short term memory networks with delay scales of 1, 2, 3 and 4 are constructed.

And (4): respectively inputting the EEG signal and the EMG signal into a long-short term memory network with four time delay scales, and selecting a model with the minimum error according to three evaluation indexes of average absolute percentage error, root mean square error and Nash coefficient, namely the model with the optimal time delay scale

Fourier transformation is carried out on the time sequence characteristics extracted by the long-term and short-term memory network with the four time delay scales, four frequency domain characteristic sequences are obtained through conversion, the transfer entropy is obtained for the brain and muscle electrical signals at the same frequency point in the four obtained frequency domain characteristic sequences, four transfer entropy results are obtained, and the average value is obtained and used as the reference coupling strength.

And (5): calculating the transfer entropy T of the sequence extracted by the optimal time delay scale model obtained in the step (4) at the same frequency₁Then calculating the frequency f₁～f₂Between T₁With reference to the coupling strengthThe area of (a) is the coupling strength of the frequency band.

The brain muscle coupling detection model based on the long-short term memory network with the increased time delay scale and the transfer entropy respectively calculates the brain muscle coupling characteristics with the four time delay scales, the transfer entropy is used for representing the coupling strength, and the area between the average value of the four coupling strengths and the optimal characteristic is used for representing the variation of the coupling strength.

The invention has the beneficial effects that:

the human brain electrical signal and the human myoelectrical signal are non-stationary signals, and different individuals have great difference, so that when the traditional machine learning classification method is used, uniform representative characteristics are difficult to find. And deep learning becomes a tool very suitable for extracting the physiological signal features due to the strong feature extraction capability of the deep learning.

The invention provides a long-short term memory network for increasing time delay scale and increasing time delay scale aiming at the characteristic that electroencephalogram signals and electromyogram signals are time sequences with close influence on front and back information, cell output at long and remote moments is transmitted to a cell unit at the current moment through the time delay scale, and an adaptive selector is designed through average absolute percentage error, root mean square error and Nash coefficient to select the optimal time delay scale.

The traditional coherence method can not express the information flowing direction, the time sequence characteristics extracted by the improved long-short term memory network with the increased time delay scale are converted into a frequency domain, and then the transfer entropy area between the electroencephalogram and the electromyogram is calculated by utilizing the transfer entropy to express the coupling degree between the signals.

Drawings

FIG. 1 is a flow chart of an embodiment of the present invention;

FIG. 2 is a diagram of a model architecture of a long-short term memory network with added delay metrics according to the present invention;

FIG. 3 is a diagram of the overall process of the brain muscle coupling analysis according to the present invention;

FIGS. 4(a) - (c) are graphs showing the results of analysis of three movements of the present invention, i.e., extending wrist and making fist.

Detailed Description

Aiming at the current results of domestic scholars exploring cerebral cortical areas and muscle tissues respectively, the method mainly researches the condition that cerebral movement areas are activated when a human body executes different movement tasks by means of a nuclear magnetic resonance imaging technology, or researches the movement function areas of the human body by means of electromyographic signals, machine learning and other modes. The exploration scheme of combining the cortex of the moving area with the moving muscle group of the human body is not effective enough. Although the traditional technologies such as nuclear magnetic resonance imaging and magnetoencephalography are accurate in positioning, the traditional technologies are not easy to detect synchronously with electromyographic signals. And when the invasive method is adopted to collect the physiological signals of the human body, the injury to the subject is large, and the research cases are very limited. Meanwhile, most of the current methods for exploring the coupling phenomenon between the cortex and the muscle still utilize the traditional coherence or causal test method, the problem that the frequency of electroencephalogram and myoelectricity acquired by experiments is unequal by using the coherence method can be met, and the granger causal test method has the defects of strict time selection and poor robustness. In addition, most scholars study the coupling relationship between the electroencephalogram signal and the electromyogram signal on the basis of the original time series, and do not consider the directionality.

Aiming at the problems, the invention designs a brain-muscle electrical coupling detection model based on a long-short term memory network with increased time delay scale and transfer entropy, and analyzes the functional coupling change of the human body when the human body executes action instructions and information interaction between the brain and limb muscles. The brain cortex and muscle signals are subjected to cooperative analysis, a long-term and short-term memory network for increasing the time delay scale is used for extracting a characteristic sequence of the signals, and then the coupling area is calculated by using the transfer entropy, so that the coupling relation between the cortex and the muscle of a human body under different behavior actions is explored, the movement intention of the human body is decoded, and the method for researching the movement mechanism of the human body is expanded.

As shown in fig. 1, the present example includes the following steps:

the method comprises the following steps: acquiring electroencephalogram signals and electromyogram signals when a human carries out different hand motions, and carrying out wavelet denoising treatment, wherein the specific process comprises the following steps:

(1) the electroencephalogram signals and the electromyogram signals of seven healthy male subjects and seven healthy female subjects are respectively collected, and the fact that all people have no history of neurologic and motor function diseases is guaranteed before experiments. The specific operation of collecting the experiment was informed in advance before the experiment began, and the subjects were signed on an informed protocol. Obtaining the electromyographic signals by using TrignoTMWireless EMG equipment and corresponding EMG Works 4.0 acquisition software, wherein the selected muscles are extensor muscles, flexor carpi radialis muscles, flexor digitorum profundus muscles and flexor carpi ulnaris muscles. The electroencephalogram signals are acquired by a g.MOBllab + MP-2015 acquisition instrument, and 32 channels of Fp1, Fp2, AFz, F7, F3, Fz, F4, F8, FC5, FC1, FCz, FC2, FC6, T7, C3, Cz, C4, T8, TP9, CP5, CP1, CP2, CP6, TP10, P7, P3, Pz, P4, P8, O1, O2 and lz are selected in experiments. The position of the discharge electrode is swung according to the international uniform extension 10-20 electrode system. In the process of carrying out the experiment, male and female testees repeatedly make three actions of wrist bending, wrist stretching and fist making along with the animation on the computer screen according to the requirements, wherein each action is repeated for 10 times, the interval between each action is 5 minutes, the interval between each action is 20 minutes, and 3 times are repeated.

(2) Denoising the brain and electromyographic signals by using wavelet threshold denoising, and removing most of high-frequency-band noise interference.

Step (2): as shown in fig. 2, which is a model structure diagram of a long-short term memory network with a time delay added thereto, in a conventional long-short term memory network, a cell unit for processing current information is denoted as a cell unit at time t, a cell unit before the current cell unit is denoted as a cell unit at time t-1, and a cell unit before the cell unit at time t-1 is denoted as a cell unit at time t-2. And adding a delay connection on the basis of the traditional long-short term memory network, and connecting the output of the cell unit at the t-2 moment to the cell unit at the t moment. Therefore, the t-time cell unit processing the current time information can not only consider the data characteristics of the previous time, but also learn the output layer characteristics of the t-2-time cell unit. Because the electroencephalogram signal and the electromyogram signal are time sequence sequences related to time, the electroencephalogram and electromyogram characteristics can be effectively prevented from being forgotten in the transmission process by adding the time delay connection.

Long-short term memory networks were first proposed in 1997 and were mainly used to predict time-related sequences. Long and short term memory networks generally perform better than recurrent neural networks, which are no longer built up of simple stacks of repeating modules, but instead incorporate gates to handle different cell units. In the 2009 handwritten recognition contest, long and short term memory networks achieved the best performance over other neural networks, and also over other types of neural network models in natural language processing. The long-short term memory network is a nonlinear model and is characterized in that a plurality of cell units can be relied on to sense past state characteristics in a time sequence, thereby improving the accuracy of a traditional model. The core of the long-short term memory network is a memory block thereof, which comprises a memory unit and three gate valves used as forgetting, inputting and outputting. The long-term and short-term memory network adds and deletes the information of the cells through a gate control unit, and the gate is composed of a Sigmoid function and multiplication operation and can determine whether the information passes through the cells. The output of the gate is composed of 0 and 1, output 0 indicates that information cannot pass, and output 1 indicates that information can pass.

1. Forgetting door

The long-short term memory network firstly needs to decide which information can pass through the cell unit and which information needs to be blocked, and the decision is completed by a Sigmoid function of a forgetting gate. The forgetting gate can determine whether past information has an effect on the state of the current cell. The calculation formula is shown as formula (1):

f_t＝σ(W_xfx_t+W_hfh_t-1+W_cfc_t-1+b_f) (1)

wherein, W_xf，W_hfAnd W_cfIs the weight of the gate and cell unit that was forgotten at the current and previous time, b_fIs the offset.

2. Input gate

In the process of passing back the information at the past time, some information needs to be replaced, and the function is realized by an input door. First, it is determined whether the output of Sigmoid is 0 or 1, and if 0, replacement is not necessary, and if 1, replacement is necessary. The replacement values are then generated from the Tanh calculation and the new values generated by the current layer may be added to the cell unit. The input gate can be viewed as determining the effect of the cell state on the current cell unit at the previous point in time. The calculation formula is shown as formula (2)

i_t＝σ(W_xix_t+W_hih_t-1+W_cic_t-1+b_i) (2)

Wherein, c_t-1Is a candidate state value at a previous point in time. W_xi，W_hiAnd W_ciWeights representing the forgetting gate, candidate states at the current and previous time points, b_iIs the offset.

3. Output gate

The output gate functions to control the output of information, the result of which is indicative of the current cell state. Firstly, obtaining initial parameters by using Sigmoid function, and then, obtaining c according to Tanh function_tNormalizing to be between-1 and 1, and multiplying the result by the Sigmoid result to obtain a calculation result of the whole model. The calculation formula is shown in formula (3).

o_t＝σ(W_xox_t+W_hoh_t-1+W_coc_t-1+b_o) (3)

Wherein, W_xo，W_hoAnd W_coIs the weight of the output gate and the memory cell for the current time and the previous time, b_oIs the offset.

And (3): and (3) recording the model constructed in the step (2) as a network with the time delay scale of 1. In the network with the time delay scale of 1, a cell unit before a cell unit at the time of t-2 is marked as a cell unit at the time of t-3, and the output of the cell unit at the time of t-3 is connected to the cell unit at the time of t, so that the cell unit at the time of t, which processes the current time information, can learn the characteristics of an output layer of the cell unit at the time of t-3, and the model is marked as the network with the time delay scale of 2.

Then, in the network with the time delay scale of 2, the previous cell unit of the cell unit at the time t-3 is marked as a cell unit at the time t-4, the output of the cell unit at the time t-4 is connected to the cell unit at the time t, so that the cell unit at the time t can learn the output layer characteristics of the cell unit at the time t-4, and the model is marked as the network with the time delay scale of 3.

And finally, in the network with the time delay scale of 3, marking the previous cell unit of the cell unit at the time t-4 as a cell unit at the time t-5, and connecting the output of the cell unit at the time t-5 to the cell unit at the time t, so that the cell unit at the time t can learn the characteristics of an output layer of the cell unit at the time t-5, and marking the model as the network with the time delay scale of 4. Four long-short term memory networks with delay scales of 1, 2, 3 and 4 are constructed.

And (4): and (3) respectively inputting the electroencephalogram signals and the electromyogram signals into the long-term and short-term memory networks with four time delay scales constructed in the step (3), and selecting a model with the minimum error according to three evaluation indexes, namely the average absolute percentage error, the root mean square error and the Nash coefficient, wherein the model is called as an optimal time delay scale model, and is shown as an integral process diagram of the brain muscle coupling analysis in fig. 3.

And performing Fourier transform on the time sequence characteristics extracted by the long-term and short-term memory networks with the four time delay scales, and converting to obtain four frequency domain characteristic sequences. And solving the transfer entropy of the brain electromyographic signals at the same frequency point in the obtained four frequency domain characteristic sequences to obtain four transfer entropy results, and solving the average value as the reference coupling strength. The reference coupling strength is obtained specifically as follows:

let two time sequences of electroencephalogram signal and electromyogram signal be X ═ X₁,x₂,x₃…x_TY and Y ═ Y₁,y₂,y₃…y_TWhere T is the signal length, x₁，y₁First measured values, x, of two sequences, respectively₂,y₂Respectively taking the second measurement values of the two sequences, and solving the entropy rate of the frequency domain characteristics of the brain electromyographic signals at the same frequency point:

where n represents a reference judgment point and τ represents a judgment time.

The transfer entropy TE of Y to X is defined as shown in equation (6).

The transfer entropy TE of X to Y is defined as shown in equation (7).

The formula can be written as formula (8) and formula (9) using a conditional probability formula.

And (3) solving the transfer entropy from the electroencephalogram to the electromyogram and from the electromyogram to the electroencephalogram by using the formulas (8) and (9), and solving the mean value of the transfer entropy of the four frequency domain characteristic sequences as reference coupling strength.

And (5): calculating the transfer entropy T of the sequence extracted by the optimal time delay scale model obtained in the step (4) at the same frequency₁Then calculating a random frequency f₁～f₂Between T₁The area between the reference coupling strength and the reference coupling strength is the coupling strength of the frequency band. As shown in fig. 4(a) - (c), the results of analysis are shown for three movements of analyzing, extending wrist and clenching fist.