1. A correlation filtering target tracking method based on double space-time constraints is characterized by comprising the following steps:

function for constructing correlation filtering model based on double space-time constraintsAcquiring a correlation filter;

in a current image frame, a rectangular area is intercepted by taking a tracked target position as a center, HOG characteristics of the area are extracted to be used as training samples, and a tracker is initialized;

constructing a regression weight graph, inhibiting interference sample response, and updating an adaptive adjustment factor;

inputting the extracted training sample characteristics into a model, rapidly solving the model in a Fourier domain by adopting an ADMM algorithm, and updating a relevant filter;

taking a target position in a current frame as a rectangular center, intercepting a rectangular region with different scaling factors from a next frame image as a search region, extracting HOG characteristics of the search region, carrying out Fourier transform on the HOG characteristics, carrying out fast matching calculation with an updated related filter, obtaining a filter response image, obtaining a maximum response position in the response image as a tracked target position by adopting a Newton iteration method, and taking a corresponding scaling factor as a currently estimated scaling factor of the target;

the extraction of HOG features, the updating of adjustment factors, the updating of filters, the tracking of target positions and the estimation of scaling factors are repeated to continuously track the target.

2. The method for tracking correlation filtering targets based on double space-time constraints according to claim 1, wherein the function of the filtering model comprises: setting upWhen w is_rAnd when the middle element is 1, adding time constraint to the regression quantity of the corresponding sample, and when the sample is strong interference, taking 0 as the corresponding element.

3. The method for tracking correlation filtering targets based on double space-time constraints according to claim 1, wherein the function of the filtering model comprises: setting upAndwith A (p)_sr) Dynamically adjusting the spatio-temporal constraint strength of the regression quantity if p_srSmaller, the A (p) is decreased_sr) To decrease the constraint strength, otherwise increase A (p)_sr)。

Background

The visual target tracking is an important component of computer vision, and is one of key technologies in the fields of video monitoring, intelligent traffic, robot vision, automatic driving, accurate guidance and the like. And given the position information of the target in the initial frame, and accurately estimating the motion track of the target in the subsequent frame. Because the target may have factors such as illumination change, motion blur, deformation, scale change, rotation, and occlusion during the operation process, it is very difficult to accurately track the target in a complex scene.

The target tracking method based on the relevant filtering is a target tracking method which has the advantages of good effect, high speed and wide application. And obtaining densely sampled samples by circularly shifting a rectangular frame with the target as the center, and obtaining the correlation filter by solving a linear regression objective function with a regular constraint term.

Most of the currently known correlation filtering models directly constrain the filter, and neglect the constrained samples. Good samples and bad samples have the same weight in the model, and the model loses the ability to select sample information.

Disclosure of Invention

In order to solve the problems of time consumption and low tracking precision in the prior art, the invention provides a double-space-time constraint-based related filtering target tracking method, which provides a core technical support for video tracking-based computer vision application.

Function for constructing correlation filtering model based on double space-time constraints

Acquiring a correlation filter;

in the current image frame, taking the tracked target position as the center, intercepting a rectangular area, extracting HOG characteristics of the area as a training sample, and initializing a tracker;

constructing a regression weight graph, inhibiting interference sample response, and updating an adaptive adjustment factor;

inputting the extracted training sample characteristics into a model, rapidly solving the model in a Fourier domain by adopting an ADMM algorithm, and updating a relevant filter;

taking a target position in a current frame as a rectangular center, intercepting a rectangular region with different scaling factors from a next frame image to be used as a search region, extracting HOG characteristics of the search region, carrying out Fourier transform on the HOG characteristics, then carrying out fast matching calculation with an updated related filter to obtain a filter response image, and obtaining a maximum response position in the response image by adopting a Newton iteration method to be used as a tracked target position, wherein a corresponding scaling factor is used as a currently estimated scale scaling factor of a target;

and repeating the steps of extracting HOG characteristics, updating the adjustment factor, updating the filter, tracking the position of the target and estimating the scaling factor so as to continuously track the target.

Further, settingWhen w is_rWhen the middle element is 1, adding time constraint corresponding to the regression quantity of the sample; when the sample is strong interference, the corresponding element takes 0, and the regression response of the corresponding sample is suppressed.

Further, settingAndwith A (p)_sr) Dynamically adjusting the spatio-temporal constraint strength of the regression quantity if p_srIf smaller, indicating that the target confidence is not high, A (p) is decreased_sr) To reduce the constraint strength, reduce the influence of the current sample on the filter, otherwise increase A (p)_sr)。

The invention has the beneficial effects that: the double space-time constraint related filtering model is provided, different constraints are added to different samples, the model is more stable, a more robust related filter can be learned, fast and high-precision tracking is realized, a core technical support is provided for computer vision application based on video tracking, the result is more accurate, the increased operation amount is extremely small, calculation in a Fourier domain is realized, and the operation speed is high.

Drawings

Fig. 1 is a flow chart of the method and fig. 2 is an objective function of the filter model.

Detailed Description

The technical scheme of the invention is specifically explained in the following by combining the attached drawings.

The method carries out space-time constraint on the filter and the regression quantity, solves the model through an objective function of an optimized model to obtain an updated related filter, adds a binary mask of 0-1 to the regression quantity to realize the 'off-on' operation of the time constraint of the sample, and adds different constraints to different samples.

The specific implementation steps are shown in figure 1: establishing an objective function of a filter model, as shown in FIG. 2Where denotes cyclic convolution, denotes product by element,is the c-th channel HOG feature of the training sample, f is the filter to be learned, w_cIs a filter weight map which gives f^cThe corresponding target region is given a low weight (1e-3) and the background region is given a high weight (1e5) to suppress the background, w_rIs a regression weight graphControlling the constraint mode of regression quantity of each sample, c is the serial number of the characteristic channel, t-1 and t represent the serial number of the image in the video, rho is the constraint factor, in this embodiment, 15 is taken, y is the expected output, and is a two-dimensional Gaussian distribution, A (p)_sr) Is the adaptive adjustment factor.

The second term of the objective function is the space constraint of the filter, and the boundary problem in the relevant filtering is solved; the third term is a filter time constraint term, so that the filter is updated online; w in the fourth item_rCarrying out space constraint on the regression quantity when w_rWhen the middle element takes 0, the regression response of the corresponding sample is inhibited, and when w is_rWhen the middle element is 1, the regression quantity of the corresponding sample is added with time constraint, so that the learned filter has more robustness.

In the current image frame, a rectangular image region centered on the target position is cut out, the area of the rectangular region is 5 times of the target area, and the HOG (histogram of ordered gradient) features of the image region are extracted as training samples.

Construction of a regression weight graph w_rUpdating the adaptive adjustment factor A (p)_sr) When w is_rWhen the middle element is 1, the regression quantity of the corresponding sample is added with time constraint, so that the learned filter has more robustness; when the sample is strongly interfering, it is always desirable to suppress the interfering sample, when w_rThe element corresponding to the medium interference sample should be 0, and regression response of the corresponding sample is suppressed; the interference samples are not targets, but have higher regression responses, which may interfere with target tracking.

w_rIs calculated asWherein (i, j) represents a two-dimensional position marker, v_maxIs the maximum value in v, w and h are the width and height of the object in the image,&and (4) representation and operation.

Adaptive adjustment factor A (p)_sr) Is the peak side lobe ratioFunction of (2)WhereinIs a filter response plot, max (v) is the maximum value of v, μ (v) is the mean of v, σ (v) is the standard deviation of v; p is a radical of_meanIs p in the history frame_srIs a scaling factor, this embodiment takes 0.6 and 7, respectively, and τ is a bias factor, takes 0.5.

A(p_sr) Dynamically adjusting the spatio-temporal constraint strength of the regressors when p is_srSmaller, indicating low confidence in the target, turn down A (p)_sr) The constraint strength is reduced, and the influence of the current sample on the filter is reduced; when p is_srLarger, indicating high confidence in the target, increasing A (p)_sr) Making the model a more robust filter.

The model is rapidly solved through an ADMM algorithm, an objective function is a convex optimization problem, and an auxiliary variable h-f is defined_tLet us orderIs equal to y^stThe augmented Lagrangian form of the objective function isWherein s is Lagrange multiplier, eta is penalty factor, and controls convergence rate of the augmented Lagrange method to makeIs converted intoAnd (5) iteratively solving the three subproblems of f, h and p to obtain an optimal solution.

Sub-problem form The Parseval's theorem, f can be converted into Fourier domain to solve by using the Parseval theoremWherein the superscript ^ represents the discrete Fourier transform, orderRepresenting the eigenvector taken out along the direction of the eigen-channel at the ith position,is the corresponding desired output, the sub-problem f is converted into M × N independent sub-problems, where M × N is the feature size, and the sub-problem f is converted intoTo pairDerivative, make the reciprocal zero to obtainIs closed toAndwherein I is an identity matrix, and is converted by Sherman-Morrison formulaFurther reducing the calculation amount to obtainThe matrix multiplication combination law is utilized, only vector inner product and addition operation are included, and the calculation amount is reduced.

Solving of each element of the subproblem h is independent, and the derivative reciprocal is zero to obtain the derivative of each channelClosed solutionThe division number represents that the division is carried out according to elements, and after each iteration, the updating mode of the penalty factor eta is eta⁽ⁱ⁺¹⁾＝min(η^max,εη⁽ⁱ⁾) Wherein eta^maxThe maximum value is represented, epsilon is a scale factor, 100 and 10 are respectively taken in the embodiment, 1 is taken as the initial value of eta, and the final f is obtained through iterative solution in a similar manner.

Taking the target position in the current frame as a rectangular center, intercepting a rectangular image area with different scaling factors from the next frame image as a search area, and extracting the HOG characteristic z of the search area_sAnd s e {1,2,3,4,5} represents five scaling factors, and fast matching calculation is carried out on the Fourier domain and the search region characteristics by utilizing the learned filter to obtain a filter response graph.

Finding out the maximum response position in the response graph as the tracked target position by a Newton iteration method, wherein the scale scaling factor corresponding to the maximum response is the currently estimated scale scaling factor of the target, and obtaining the filterTo z_sFourier transform to obtainBy passingAnd obtaining a filtering response graph, wherein the length and width of the grid are 4 because the HOG features are extracted on the original pixel scale by taking the grid as a unit, and the obtained response graph is only the response on the grid point and is not the response on each pixel point.

AdoptInterpolation mode to obtain sub-pixel level position responseFinding the position of the maximum value of the response graph, taking the position as initial estimation, finding the maximum response position by using a Newton iteration method, and experiments show that convergence can be achieved only by a few iterative algorithms, the optimal target position is found, and the size scaling factor corresponding to the maximum response is found on the 5 scale scaling factors and is used as the size scaling factor of the currently estimated target.

The target can be continuously tracked by repeating the steps.

The above-described embodiments are not intended to limit the present invention, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present invention are included in the scope of the present invention.