1. A non-contact vibration measurement method, comprising:

acquiring a first image sequence; the first image sequence comprises a plurality of frames of images, and each frame of image in the first image sequence comprises a structure to be measured;

performing Gaussian smoothing and edge extraction on one frame in the first image sequence to obtain a binary edge mask;

carrying out gridding division on the binary edge mask;

acquiring a second image sequence; the second image sequence is a subsequence in the first image sequence, and the time length of the second image sequence is determined by the historical fundamental frequency of the structure to be measured;

determining a gradient of the reference image; the reference image is a mean image of the second image sequence;

selecting partial information in each frame of the first image sequence by using the binary edge mask to obtain a third image sequence;

selecting partial information in the gradient of the reference image by using the binary edge mask to obtain gradient information;

determining displacement information of each grid in the binary edge mask according to the third image sequence and the gradient information by using a least square method;

carrying out structural modal identification on the displacement information to obtain structural modal parameters;

and determining a vibration measurement result of the structure to be measured according to the structural modal parameters.

2. The non-contact vibration measurement method according to claim 1, wherein said acquiring a first sequence of images comprises:

shooting and sampling the structure to be measured by using a set sampling frequency to obtain the first image sequence;

graying and image deflash are carried out on each frame in the first image sequence.

3. The non-contact vibration measurement method according to claim 2, wherein the image deflash includes:

performing histogram relation linear fitting on any two adjacent frames in the first image sequence;

and calibrating the brightness level of the later frame of the two frames according to the result of the linear fitting of the histogram relation.

4. The method of claim 1, wherein the determining displacement information for each grid in the binary edge mask from the third image sequence and the gradient information using a least squares method comprises:

wherein, Grid_iRepresenting the ith mesh in the binary edge mask,representing Grid of a Grid_iA displacement of the middle coordinate (x, y) at time t, I (x, y, t) representing the value at the frame coordinate (x, y) of the third image sequence corresponding to time t, I₀(x, y) represents the reference image,representing the gradient of the reference image.

5. The method of claim 1, wherein the structural mode identification of the displacement information to obtain structural mode parameters is performed by a stochastic subspace method, a peak picking method, or a frequency decomposition method.

6. The method of claim 1, wherein determining the vibration measurement of the structure to be measured from the structure modal parameters comprises:

determining a relative error value between the structure modal parameter and the natural frequency of the structure to be measured;

or

Determining a modal shape according to the structural modal parameters, and determining an MAC value between the modal shape and a historical modal shape of the structure to be measured;

and the relative error value and the MAC value are vibration measurement results of the structure to be measured.

7. The non-contact vibration measurement method according to claim 1, further comprising:

respectively determining interested areas in each frame image of the first image sequence, wherein only the interested areas in each frame image of the first image sequence are subjected to Gaussian smoothing and edge extraction.

8. The non-contact vibration measurement method according to any one of claims 1 to 7, further comprising:

receiving the first image sequence through a streaming media protocol;

caching the first image sequence by using a video data cache pool;

and caching the displacement information by using a displacement identification cache pool.

9. A computer device comprising a memory for storing at least one program and a processor for loading the at least one program to perform the non-contact vibration measurement method of any one of claims 1-8.

10. A storage medium in which a program executable by a processor is stored, wherein the program executable by the processor is adapted to perform the non-contact vibration measurement method according to any one of claims 1 to 8 when executed by the processor.

Background

The contact type vibration measurement and the non-contact type vibration measurement can be applied to the field of engineering structure vibration measurement. The contact type vibration measurement is that contact sensors such as a strain gauge, a displacement meter and an accelerometer are installed on the surface of a structure to be measured, but some structures to be measured such as large buildings exist, sensors or cables cannot be arranged due to the structure of the contact type vibration measurement or the engineering difficulty, or the sensors can be installed at partial positions of the structure to be measured, so that the contact type vibration measurement technology cannot be applied, or the acquired information is too sparse, and the contact type vibration measurement technology cannot be supported to be used.

The existing non-contact sensor uses active optical devices such as a laser Doppler vibration meter and an electronic speckle interferometer to realize non-contact measurement of a structure, and vibration conditions are analyzed according to measurement results. However, these instruments have the disadvantages of sensitivity to the testing environment, complex device layout, long global testing time, and the like, and the use cost is high.

Disclosure of Invention

In view of at least one of the above-mentioned technical problems, it is an object of the present invention to provide a non-contact vibration measuring method, a computer device, and a storage medium.

In one aspect, embodiments of the present invention include a method for non-contact vibration measurement, comprising:

acquiring a first image sequence; the first image sequence comprises a plurality of frames of images, and each frame of image in the first image sequence comprises a structure to be measured;

performing Gaussian smoothing and edge extraction on one frame in the first image sequence to obtain a binary edge mask;

carrying out gridding division on the binary edge mask;

acquiring a second image sequence; the second image sequence is a subsequence in the first image sequence, and the time length of the second image sequence is determined by the historical fundamental frequency of the structure to be measured;

determining a gradient of the reference image; the reference image is a mean image of the second image sequence;

selecting partial information in each frame of the first image sequence by using the binary edge mask to obtain a third image sequence;

selecting partial information in the gradient of the reference image by using the binary edge mask to obtain gradient information;

determining displacement information of each grid in the binary edge mask according to the third image sequence and the gradient information by using a least square method;

carrying out structural modal identification on the displacement information to obtain structural modal parameters;

and determining a vibration measurement result of the structure to be measured according to the structural modal parameters.

Further, the acquiring the first sequence of images comprises:

shooting and sampling the structure to be measured by using a set sampling frequency to obtain the first image sequence;

graying and image deflash are carried out on each frame in the first image sequence.

Further, the image deflash comprises:

performing histogram relation linear fitting on any two adjacent frames in the first image sequence;

and calibrating the brightness level of the later frame of the two frames according to the result of the linear fitting of the histogram relation.

Further, the determining, by using a least square method, displacement information of each grid in the binary edge mask according to the third image sequence and the gradient information uses a formula including:

wherein, Grid_iRepresenting the ith mesh in the binary edge mask,representing Grid of a Grid_iA displacement of the middle coordinate (x, y) at time t, I (x, y, t) representing the value at the frame coordinate (x, y) of the third image sequence corresponding to time t, I₀(x, y) represents the reference image,representing the gradient of the reference image.

Further, the structural modal identification of the displacement information to obtain structural modal parameters is performed by a random subspace method, a peak value selection method or a frequency decomposition method.

Further, the determining a vibration measurement result of the structure to be measured according to the structure modal parameters includes:

determining a relative error value between the structure modal parameter and the natural frequency of the structure to be measured;

or

Determining a modal shape according to the structural modal parameters, and determining an MAC value between the modal shape and a historical modal shape of the structure to be measured;

and the relative error value and the MAC value are vibration measurement results of the structure to be measured.

Further, the non-contact vibration measurement method further includes:

respectively determining interested areas in each frame image of the first image sequence, wherein only the interested areas in each frame image of the first image sequence are subjected to Gaussian smoothing and edge extraction.

Further, the non-contact vibration measurement method further includes:

receiving the first image sequence through a streaming media protocol;

caching the first image sequence by using a video data cache pool;

and caching the displacement information by using a displacement identification cache pool.

In another aspect, embodiments of the present invention further include a computer apparatus including a memory for storing at least one program and a processor for loading the at least one program to perform the non-contact vibration measurement method in embodiments.

In another aspect, embodiments of the present invention also include a storage medium in which a processor-executable program is stored, the processor-executable program being configured to perform the non-contact vibration measurement method in the embodiments when executed by a processor.

The invention has the beneficial effects that: the vibration measurement method in the embodiment is based on the principle of an optical flow method, carries out displacement solution on a first image sequence obtained by shooting a structure to be measured, realizes non-contact structural modal analysis, can overcome the defects of the existing contact vibration measurement technology, has the advantages of strong adaptability to a test environment, suitability for off-line measurement and on-line measurement and the like compared with the existing non-contact vibration measurement, can be realized only by algorithm programming of a computer, and has low requirements on device hardware and low use cost.

Drawings

FIGS. 1 and 2 are diagrams of the architecture of the non-contact vibration measurement method in the embodiment;

FIG. 3 is a flowchart of a non-contact vibration measurement method in an embodiment;

FIG. 4 is a schematic diagram of the step of identifying the displacement of the structure in the non-contact vibration measurement method in the embodiment;

FIG. 5 is a schematic view of a four-layer frame and arranged sensors as a structure to be measured in an embodiment;

FIG. 6 is a graph comparing the measurement results of the template matching method in the example and the vibration measuring method in the present example;

fig. 7 is a graph comparing the measurement results of the LVDT lateral displacement sensor used in the example and the vibration measuring method in the present example.

Detailed Description

The structure of the non-contact vibration measurement method in this embodiment is shown in fig. 1 and 2. Fig. 1 shows an offline measurement mode, and fig. 2 shows an online measurement mode.

Referring to fig. 1 and 2, both the off-line measurement mode and the on-line measurement mode include steps of identifying structure displacement, identifying structure modality, historical data comparison, and the like, and optionally adding a step of defining a region of interest. The main difference between the off-line measurement mode and the on-line measurement mode is the manner in which the first image sequence is acquired.

In the off-line measurement mode, the video data obtained by shooting the structure to be measured can be uploaded completely, and frame extraction is carried out on the video data, so that a first image sequence is obtained.

In the online measurement mode, video data can be obtained by shooting a structure to be measured by using a video acquisition module such as a camera, the video data is acquired through streaming media protocols such as rtsp and rtp, the video data is cached in a video data cache pool, when a first image sequence needs to be obtained, the video data is read from the video data cache pool, and frame extraction is performed on the video data, so that the first image sequence is obtained. After the step of identifying the structure displacement is executed to obtain the displacement information, the displacement information can be cached in a displacement identification cache pool, and when the subsequent steps such as identifying the structure mode are executed, the displacement information is read from the displacement identification cache pool for processing. When the video data cache pool is full, the collection of the video data can be stopped, the user is reminded that the cache is insufficient, or the subsequent video data is abandoned to be cached, and the subsequent video data is written in when the cache is vacant. When the displacement identification cache pool is full, caching of displacement information may be stopped. By setting the video data cache pool and the displacement identification cache pool, online processing of video data can be realized, and thus the non-contact vibration measurement method in the embodiment is applied to occasions with high real-time requirements.

In this embodiment, referring to fig. 3, the non-contact vibration measurement method includes the following steps:

s1, acquiring a first image sequence;

s2, performing Gaussian smoothing and edge extraction on one frame in the first image sequence to obtain a binary edge mask;

s3, carrying out gridding division on the binary edge mask;

s4, acquiring a second image sequence;

s5, determining the gradient of the reference image; the reference image is a mean image of the second image sequence;

s6, selecting partial information in each frame of the first image sequence by using a binary edge mask to obtain a third image sequence;

s7, selecting partial information in the gradient of the reference image by using a binary edge mask to obtain gradient information;

s8, determining displacement information of each grid in the binary edge mask according to the third image sequence and the gradient information by using a least square method;

s9, carrying out structural modal identification on the displacement information to obtain structural modal parameters;

and S10, determining a vibration measurement result of the structure to be measured according to the structural modal parameters.

Wherein, steps S1-S8 are the steps of identifying the structure displacement in fig. 1 and 2, step S9 is the step of identifying the structure mode in fig. 1 and 2, and step S10 is the step of comparing the history data in fig. 1 and 2.

The principle of steps S1-S8 is shown in fig. 4, in which step numbers such as S1 indicate the processing of the corresponding step, and images before and after the step numbers indicate the object to be processed and the processing result of the step. For example, of the two images connected in step S2, the preceding image represents one frame in the first image sequence to be subjected to the gaussian smoothing and edge extraction processing in step S2, and the following image represents the binary edge mask obtained through the processing in step S2.

In step S1, the first image sequence is obtained by the offline measurement mode shown in fig. 1 or the online measurement mode shown in fig. 2. The first image sequence comprises a plurality of frames of images, each frame of images comprising a structure to be measured. Wherein the structure to be measured can be a bridge, a building or other measuring object.

In this embodiment, the steps S2 to S8 may be executed to process the entire related frame in the first image sequence, or the region of interest may be selected in each frame of the first image sequence, and only the region of interest in the related frame is processed while the steps S2 to S8 are executed, and the other regions outside the region of interest are not processed. For example, after the region of interest is selected, in performing step S2, only the region of interest in one frame of the first image sequence is subjected to the gaussian smoothing and edge extraction processing, and the region other than the region of interest is not subjected to the gaussian smoothing and edge extraction processing.

In performing step S1, the sampling frequency of the camera may be set to f_sSampling frequency f_sIs determined by the historical modal data of the structure to be measured, the sampling frequency f_sIs more than twice of the Nyquist frequency of the structure to be measured, thereby avoiding aliasing effect, false vibration frequency identification and the like. When the camera is used, the camera can be calibrated by utilizing a preset calibration object in a shooting scene to obtain a camera matrix P, and the camera matrix P can be used for recovering the real displacement of the structure subsequently.

In performing step S1, the frames in the first image sequence may be grayed out and image deflash. The graying of each frame in the first image sequence is to convert each frame in the first image sequence into a Lab color mode and extract a brightness channel as a grayscale image, because the structure to be measured usually has obvious geometric shape information, the graying can avoid that the color information contained in each frame in the first image sequence reduces the calculation cost of the subsequent flow, and improve the calculation efficiency. The image deflash is carried out on each frame in the first image sequence, the brightness information between any two adjacent frames in the first image sequence is considered to be in a linear relation, and the subsequent frame image in any two frames can be calibrated by fitting the linear relation of the histograms of any two adjacent frames. Because the assumption of the optical flow method is invalid under the scene with violent light change, a stroboscopic phenomenon can be generated by artificial light sources such as fluorescent lamps, the brightness level of the first image sequence can be calibrated through image de-flashing, and the problem caused by light source flashing or light source change is solved.

In step S2, a more stable frame I 'is selected in the first image sequence'₀(x,y) To l'₀(x, y) Gaussian smoothing, specifically I ″)₀(x,y)＝K_σ*I′₀(x, y) wherein I ″)₀(x, y) is the result of Gaussian smoothing, K_σThe gaussian convolution kernel used for gaussian smoothing, σ denotes the standard deviation. For one frame I 'in the first image sequence'₀Gaussian smoothing of (x, y) can reduce I'₀And (x, y) high-frequency noise improves the signal-to-noise ratio and reduces meaningless high-frequency information for subsequent edge identification.

In step S2, Canny and Sobel operators are adopted to carry out smoothing on the image I ″)₀(x, y) extracting the edge to obtain a binary edge mask I_edge. Because the structure to be measured usually has regular edge features, the edge on the structure to be measured can be selected by using an edge extraction algorithm, so that the calculation cost of subsequent steps can be further reduced.

In step S3, binary edge mask I_edgeOn-demand partitioning into n appropriately sized Grid grids_iI is 1,2, …, n, and the displacement of the grid area is adopted subsequently due to the underqualification of the displacement identificationAs a result of the identification.

In step S4, the historical fundamental frequency f of the structure to be measured is used_baseDetermining a suitable time length T to satisfyThen, a subsequence G ═ { I' (x, y, t) te [ t ] is selected from the first image sequence₀,t₀+ T) }, where T is₀Is the starting time point of the subsequence G, the time length of the subsequence G is T, and then a Gaussian convolution kernel function K is used_σAnd performing Gaussian smoothing on each frame in the subsequence G to obtain a second image sequence G' ═ { K_σ*I′(x,y,t)|t∈[t₀,t₀+T)}。

In step S5, a mean image of the second image sequence G' is calculated as the reference image I₀(x, y), calculating a reference image I₀Gradient of (x, y)

In step S6, binary edge mask I is used_edgeSelecting partial information in each frame of the grayed first image sequence I '(x, y, t), specifically I (x, y, t) ═ bitwise _ AND (I' (x, y, t), I_edge) Wherein I (x, y, t) is a binary edge mask I_edgeThe result obtained by extracting partial information in each frame of the grayed first image sequence I' (x, y, t) is discarded information at a position where the value in I (x, y, t) is 0. I (x, y, t), (x, y) e Grid in each Grid_iIs a third image sequence.

In step S7, binary edge mask I is used_edgeSelecting a reference image I₀Gradient of (x, y)The obtained result is gradient information, and can still be usedRepresenting gradient information.

In step S8, a least squares method is used to generate a gradient from the third image sequence I (x, y, t) and the gradient informationDetermining a binary edge mask I_edgeDisplacement information of each grid. The method specifically comprises the following steps:

wherein, Grid_iRepresenting the ith mesh in a binary edge mask,representing Grid of a Grid_iThe middle coordinate (x, y) is a displacement at time t, and I (x, y, t) represents the value at the frame coordinate (x, y) corresponding to time t in the third image sequence.

In the present embodiment, a least squares method is used, based on the third image sequence I (x, y, t) and the gradient informationDetermining a binary edge mask I_edgeThe principle of the displacement information of each grid is as follows: an optical flow method assuming conservation of luminance is applied, and thus there is I (x + u (x, y, t), y + v (x, y, t), t ═ I (x, y,0) ═ I₀(x, y), where u (x, y, t) represents the lateral displacement at time (x, y) t and v (x, y, t) represents the vertical displacement at time (x, y) t; for convenience, the above formula may be expressed as I (x + u (x, t), t ═ I (x, t) ═ I (t) — I (x, t) — I (x, t) using vectors₀(x) (ii) a Where x is (x, y), u (x, t) is (u (x, y, t), v (x, y, t)), and where y is defined as x + u (x, t), then I (y, t) is I₀(y-u (x, t)), since it can be assumed that the displacement is relatively small in the structural vibration, u (x, t) is u (y, t), and I (y, t) is I (y, t) substituted into the above formula₀(y-u (y, t)), there is Taylor expansion:the above formula arrangement can be:

the above formula is an underdetermined equation system, in order to solve two unknowns in u (y, t), it can be assumed that u (y, t) is constant in y neighborhood y e V (y), then u (y, t) can be solved by using least square method, the system adopts preset region of interest or Grid division, the displacement calculation is limited in one Grid, and for the ith Grid or region Grid_iAnd (4) displacement calculation formula:

the displacement calculation formula is the displacement calculation formula used in step S8.

Obtaining displacement information through steps S1-S8Thereafter, step S9 may be executed to analyze the displacement information using a structural mode analysis method such as a random subspace method (SSI), a peak selection method, or a frequency decomposition method (FDD)And identifying the structural mode to obtain structural mode parameters.

In step S10, a relative error value between the structural modal parameters obtained by performing step S9 and the natural frequency f of the structure to be measured may be calculated, to serve as the vibration measurement result for the structure to be measured.

In step S10, the modal shape may be calculated from the structural modal parametersDetermining modal shapeAnd the MAC value between the historical modal shape phi of the structure to be measured, wherein the calculation mode of the MAC value is thatAnd taking the MAC value obtained by calculation as a vibration measurement result of the structure to be measured.

With the four-layer frame shown in fig. 5 as the structure to be measured, the vibration measurement method in the present embodiment is applied to obtain a vibration measurement result. Meanwhile, sensors are installed at 8 positions such as ROI1-8 of the four-layer frame in fig. 5 to obtain measurement results by applying the existing template matching method, and the result of applying the vibration measurement method in the present embodiment is compared with the result of applying the existing template matching method as shown in fig. 6. Fig. 6 shows a comparison graph of the measurement results of 8 positions, in which a curve denoted by Template Matching represents the result of the Template Matching method, a curve denoted by Ours represents the result of the vibration measurement method in the present embodiment, and the abscissa in each comparison graph is the number of frames (time) and the ordinate is the displacement.

In order to verify the measurement accuracy of the displacement result in fig. 6, an LVDT lateral displacement sensor is installed at the upper right corner of the frame in fig. 5 for comparison verification, and the results of measuring the same position of the four-layer frame in three ways, namely, using the LVDT lateral displacement sensor and using the vibration measurement method in the present embodiment, are shown in fig. 7. In fig. 7, the abscissa is the number of frames (time) and the ordinate is the displacement, wherein the curve denoted by LVDT represents the results measured using the LVDT lateral displacement sensor, and the curve denoted by Optical Flow represents the results of the vibration measurement method in the present embodiment.

Since the LVDT lateral displacement sensor can directly measure the displacement of the four-layer frame, the measurement result of the LVDT lateral displacement sensor can be regarded as an accurate measurement result. As can be seen from fig. 7, the measurement result of the vibration measurement method in the present embodiment is closer to the measurement result of the LVDT lateral displacement sensor than the measurement result of the template matching method, and thus it is proved that the vibration measurement method in the present embodiment has higher measurement accuracy than the existing method.

The non-contact vibration measurement method in the present embodiment may be performed by writing a computer program that executes the non-contact vibration measurement method in the present embodiment, writing the computer program into a computer device or a storage medium, and when the computer program is read out and run.

It should be noted that, unless otherwise specified, when a feature is referred to as being "fixed" or "connected" to another feature, it may be directly fixed or connected to the other feature or indirectly fixed or connected to the other feature. Furthermore, the descriptions of upper, lower, left, right, etc. used in the present disclosure are only relative to the mutual positional relationship of the constituent parts of the present disclosure in the drawings. As used in this disclosure, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. In addition, unless defined otherwise, all technical and scientific terms used in this example have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the description of the embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this embodiment, the term "and/or" includes any combination of one or more of the associated listed items.

It will be understood that, although the terms first, second, third, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element of the same type from another. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of the present disclosure. The use of any and all examples, or exemplary language ("e.g.," such as "or the like") provided with this embodiment is intended merely to better illuminate embodiments of the invention and does not pose a limitation on the scope of the invention unless otherwise claimed.

It should be recognized that embodiments of the present invention can be realized and implemented by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer readable memory. The methods may be implemented in a computer program using standard programming techniques, including a non-transitory computer-readable storage medium configured with the computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner, according to the methods and figures described in the detailed description. Each program may be implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Furthermore, the program can be run on a programmed application specific integrated circuit for this purpose.

Further, operations of processes described in this embodiment can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described in this embodiment (or variations and/or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) collectively executed on one or more processors, by hardware, or combinations thereof. The computer program includes a plurality of instructions executable by one or more processors.

Further, the method may be implemented in any type of computing platform operatively connected to a suitable interface, including but not limited to a personal computer, mini computer, mainframe, workstation, networked or distributed computing environment, separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, and the like. Aspects of the invention may be embodied in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optically read and/or write storage medium, RAM, ROM, or the like, such that it may be read by a programmable computer, which when read by the storage medium or device, is operative to configure and operate the computer to perform the procedures described herein. Further, the machine-readable code, or portions thereof, may be transmitted over a wired or wireless network. The invention described in this embodiment includes these and other different types of non-transitory computer-readable storage media when such media include instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. The invention also includes the computer itself when programmed according to the methods and techniques described herein.

A computer program can be applied to input data to perform the functions described in the present embodiment to convert the input data to generate output data that is stored to a non-volatile memory. The output information may also be applied to one or more output devices, such as a display. In a preferred embodiment of the invention, the transformed data represents physical and tangible objects, including particular visual depictions of physical and tangible objects produced on a display.

The above description is only a preferred embodiment of the present invention, and the present invention is not limited to the above embodiment, and any modifications, equivalent substitutions, improvements, etc. within the spirit and principle of the present invention should be included in the protection scope of the present invention as long as the technical effects of the present invention are achieved by the same means. The invention is capable of other modifications and variations in its technical solution and/or its implementation, within the scope of protection of the invention.