1. A NSST-based contact element distribution and morphology information fusion method is characterized by comprising the following steps:

1) acquiring an element distribution image of the surface of the electrical contact through a scanning electron microscope and an X-ray energy spectrometer, and acquiring a micro-topography image of the surface of the electrical contact through a machine vision system;

2) by image preprocessing, the image resolution and the edge information are improved, and an element distribution image and a surface micro-topography image are displayed in the same proportion;

3) registering the element distribution image and the surface micro-topography image by adopting a B-spline-based registration method;

4) and performing composite expansion by adopting a two-dimensional affine system based on an NSST image fusion technology, and obtaining relevant information from a source image by applying a weighted energy fusion rule to obtain a final electrical contact element distribution and morphology information fusion image.

2. The NSST-based contact element distribution and topography information fusion method according to claim 1, wherein the step 2) of improving the image resolution and the edge information comprises:

the enhancement of the fine structure edge of the electrical contact image is realized by adopting Gaussian smoothing and edge sharpening, and two images are displayed in the same proportion by adopting an image-centered zooming technology.

3. The NSST-based contact element distribution and topography information fusion method according to claim 1, wherein the B-spline-based registration method in step 3) registers the element distribution image and the surface micro topography image as:

by means of deformation in the positive image with deformation coefficients, a set of deformation coefficients is selected through mutual information values among source image pixels to optimally register the moving image pixels, and the specific process is as follows:

301) moving images according to the map DM ═ Δ Mov → Δ Stat, scanning electron microscope and X-ray spectrometer from the field Δ MovInto a fixed surface topography imageField Δ Stat of moving imagesChange to being in a fixed imageIn the domain of (2), the moving image is in the form of a deformation of D_M(y | δ), where δ represents a set of transformation constraints;

302) according to moving imagesAnd fixing the imageThe transform coefficient is selected by the optimal mutual information value therebetween, and an objective function for selecting the transform coefficient is determined by the following equation:

O(δ)＝MaxMI(I_Stat,I_Mov) (1)

the objective function is calculated according to the following equation:

equation (2) characterizes a deformation coefficient selected according to a low variation between a moving image and a fixed image when the moving image is changed to DM (y | δ);

303) optimal B-spline registration is accomplished based on mutual information between moving and fixed images, represented by equation (3):

MI(I_Mov,I_Stat)＝I_E(Mov)+I_E(Stat)-I_E(Mov,Stat) (3)

wherein, I_E(Mov)，I_E(Stat) and I_E(Mov, Stat) characterizes the entropy of moving images, the entropy of fixed images and the joint entropy of images, MI (I)_Mov，I_Stat) Is mutual information between moving images and fixed images.

4. The NSST-based contact element distribution and topography information fusion method according to claim 3, wherein step 3) further comprises: the optimal transformation coefficient based on the B-spline registration technology is effectively selected by using a WOA algorithm, a possible transformation coefficient set is selected in each iteration process, and the registered output image reflects mutual information to the maximum extent; after the optimal transform coefficients are identified, the images are effectively registered, ready for image fusion.

5. The NSST-based contact element distribution and morphology information fusion method according to claim 1, wherein the composite expansion in step 4) using a two-dimensional affine system based on NSST image fusion technology is given by:

in the formula: AM represents an anisotropic matrix, controls Shearlet scale, S represents a shear matrix, and I is associated with scale transformation; i. j and k respectively represent a scale parameter, a translation parameter and a displacement parameter; xx represents an unknown number, k represents a displacement parameter psi^i，j，k(x) An overall functional expression representing wavelet transforms corresponding to different values of i, j, k,. phi. (S)^JIⁱx-m) is the wavelet transform after the shift,represents a set of integers;

the clipping matrices S (| detS | ═ 1) and AM are invertible matrices of size 2 × 2, if d > 0 and S ∈ R, AM and S are:

6. the NSST-based contact element distribution and topography information fusion method according to claim 1, wherein the NSST discrete equation in step 4) is as follows:

whereinξ₁Fourier transform of not equal to 0, psiLet psi₁∈C^∞(R) and psi₂∈C^∞(R) is a tightly supported wavelet,. psi₀∈C^∞(R) is a wavelet transform function basis.

7. The NSST-based contact element distribution and morphology information fusion method according to claim 1, wherein the weighted energy fusion rule in step 4) is as follows:

401) the local energy of the moving and static images is calculated,

where En (p, q) is the local energy at location (p, q), and Cof (p, q) represents the wavelet coefficients at location (p, q), for moving and still images, respectivelyAndthe local energy of a rectangular window of size 3x3 is calculated, and the local energy of moving and still images is evaluated by the following equations (8) to (9): p and q respectively represent the horizontal coordinate and the vertical coordinate of a pixel in an image;

402) the weights of the fusion motion and static coefficients are calculated by equations (10) to (11):

403) the fused pixel is calculated according to the following equation (12):

I_(F)(p,q)＝I_(Mov)(p,q)×ω^(Mov)+I_(Stat)(p,q)×ω^(Stat) (15)

wherein, I_(F)(p, q) denotes a fusion coefficient, I_(Mov)(p,q)，I_(Stat)(p, q) represents a fusion coefficient of the moving image and the fixed image, ω, respectively^(Mov)，ω^(Stat)Weights Mn representing fusion coefficients of moving and fixed images, respectively^(Mov)(p, q) is the local energy of the moving image, En^(Stat)(p, q) is the local energy of the still image.

Background

In the electrical industry, electrical contacts are important electrical components for connecting or disconnecting an electrical circuit between electrical devices. Ablation and wear of the contact surfaces of the electrical contacts can lead to premature failure of the equipment, affecting the overall operating system, and therefore it is necessary to ensure reliability of the contact during operation. Under the action of thermal expansion, ablation substances are deposited at the edge of the ablation area, resulting in an increase in the ablation area. At the same time, as the degree of ablation damage increases, the surface composition also has a different distribution in the different contact zones. In order to more accurately locate the eroded regions, truly restoring the contact pattern of the electrical contacts, it is necessary to delineate the eroded regions in the image. With the development and progress of imaging equipment, the relay contact can directly acquire information containing accurate three-dimensional topography of the surface of the contact; meanwhile, information containing the distribution of the elements on the surface of the contact, such as SEM & EDS, can also be collected.

However, evaluating electrical contacts by viewing a single modal image requires spatial imagination and subjective experience, so that multi-modal methods are used to obtain contact surface information, and for this purpose it becomes important to discover the spatial relationship between such information. The image fusion is widely applied in the fields of computer vision, medical research, material mechanics, remote sensing and the like. But observing a single mode image makes it difficult to fully evaluate the electrical contact surface.

Disclosure of Invention

Aiming at the problem that the surface of the electrical contact is difficult to be completely evaluated by observing a single-mode image, the invention aims to provide a contact element distribution and morphology information fusion method based on NSST (non-subsampled contourlet transform), and all relevant information can be effectively deployed into a single image from multiple modes.

In order to solve the technical problems, the technical scheme adopted by the invention is as follows:

the invention provides a NSST-based contact element distribution and morphology information fusion method, which comprises the following steps:

1) acquiring an element distribution image of the surface of the electrical contact through a scanning electron microscope and an X-ray energy spectrometer, and acquiring a micro-topography image of the surface of the electrical contact through a machine vision system;

2) by image preprocessing, the image resolution and the edge information are improved, and an element distribution image and a surface micro-topography image are displayed in the same proportion;

3) registering the element distribution image and the surface micro-topography image by adopting a B-spline-based registration method;

4) and performing composite expansion by adopting a two-dimensional affine system based on an NSST image fusion technology, and obtaining relevant information from a source image by applying a weighted energy fusion rule to obtain a final electrical contact element distribution and morphology information fusion image.

The improvement of the image resolution and the edge information in the step 2) comprises the following steps:

the enhancement of the fine structure edge of the electrical contact image is realized by adopting Gaussian smoothing and edge sharpening, and two images are displayed in the same proportion by adopting an image-centered zooming technology.

The registration method based on the B-spline in the step 3) is used for registering the element distribution image and the surface micro-topography image as follows:

by means of deformation in the positive image with deformation coefficients, a set of deformation coefficients is selected through mutual information values among source image pixels to optimally register the moving image pixels, and the specific process is as follows:

301) moving images according to the map DM ═ Δ Mov → Δ Stat, scanning electron microscope and X-ray spectrometer from the field Δ MovImage of Δ Mov → R entry into fixed surface topographyΔ Stat → field of R Δ Stat, moving imageChange to being in a fixed imageIn the domain of (2), the moving image is in the form of a deformation of D_M(y | δ), where δ represents a set of transformation constraints;

302) according to moving imagesAnd fixing the imageThe transform coefficient is selected by the optimal mutual information value therebetween, and an objective function for selecting the transform coefficient is determined by the following equation:

O(δ)＝MaxMI(I_Stat,I_Mov) (1)

the objective function is calculated according to the following equation:

equation (2) characterizes a deformation coefficient selected according to a low variation between a moving image and a fixed image when the moving image is changed to DM (y | δ);

303) optimal B-spline registration is accomplished based on mutual information between moving and fixed images, represented by equation (3):

MI(I_Mov,I_Stat)＝I_E(Mov)+I_E(Stat)-I_E(Mov,Stat) (3)

wherein, I_E(Mov)，I_E(Stat) and I_E(Mov, Stat) characterizes the entropy of moving images, the entropy of fixed images and the joint entropy of images, MI (I)_Mov，I_Stat) Is mutual information between moving images and fixed images.

Step 3) also includes: the optimal transformation coefficient based on the B-spline registration technology is effectively selected by using a WOA algorithm, a possible transformation coefficient set is selected in each iteration process, and the registered output image reflects mutual information to the maximum extent; after the optimal transform coefficients are identified, the images are effectively registered, ready for image fusion.

The composite expansion in step 4) using the two-dimensional affine system based on the NSST image fusion technology is given by the following formula:

in the formula: AM represents an anisotropic matrix, controls Shearlet scale, S represents a shear matrix, and I is associated with scale transformation; i. j and k respectively represent a scale parameter, a translation parameter and a displacement parameter; xx represents an unknown number, k represents a whole function expression of displacement parameters psi i, j, k (x), represents wavelet transformation corresponding to different values of i, j and k, and psi (S)^JIⁱx-m) is the wavelet transform after the shift,represents a set of integers;

the clipping matrices S (| detS | ═ 1) and AM are invertible matrices of size 2 × 2, if d > 0 and S ∈ R, AM and S are:

the discrete equation for NSST in step 4) is as follows:

whereinFourier transform of psiLet psi₁∈C^∞(R) and psi₂∈C^∞(R) is a tightly supported wavelet,. psi₀∈C^∞(R) is a wavelet transform function basis.

The weighted energy fusion rule in the step 4) is as follows:

401) the local energy of the moving and static images is calculated,

where En (p, q) is the local energy at location (p, q), and Cof (p, q) represents the wavelet coefficients at location (p, q), for moving and still images, respectivelyAndthe local energy of a rectangular window of size 3x3 is calculated, and the local energy of moving and still images is evaluated by the following equations (8) to (9): p and q respectively represent the horizontal coordinate and the vertical coordinate of a pixel in an image;

402) the weights of the fusion motion and static coefficients are calculated by equations (10) to (11):

403) the fused pixel is calculated according to the following equation (12):

I_(F)(p,q)＝I_(Mov)(p,q)×ω^(Mov)+I_(Stat)(p,q)×ω^(Stat) (15)

wherein, I_(F)(p, q) denotes a fusion coefficient, I_(Mov)(p,q)，I_(Stat)(p, q) represents a fusion coefficient of the moving image and the fixed image, ω, respectively^(Mov)，ω^(Stat)Weights, En, representing fusion coefficients of moving and fixed images, respectively^(Mov)(p, q) is the local energy of the moving image, En^(Stat)(p, q) is the local energy of the still image.

The invention has the following beneficial effects and advantages:

1. the NSST-based element distribution and morphology information fusion method provided by the invention solves the problem that the contact surface is difficult to be completely evaluated when a single-mode image is observed, can accurately fuse information of different image technologies into the same mode, can obviously improve the accuracy of target positioning, has clear outline of each part, has good subjective visual effect, can capture the detailed characteristics of the image edge comprehensively, and can observe the contact structure and element distribution more conveniently and more accurately.

2. The method of the invention is convenient for accurately positioning the component distribution of the contact surface and analyzing the relationship between the surface structure and the component distribution.

Drawings

FIG. 1 is a flow chart of a NSST-based contact element distribution and topography information fusion method according to the present invention;

FIG. 2 is a diagram of the distribution of surface elements of a contact according to the present invention;

FIG. 3 is a surface microstructure of a contact;

FIG. 4 is a flow chart of weighted energy calculation;

FIG. 5 is a graph of coefficients with a window size of 3x3 for local energy computation;

FIG. 6 is a superimposed image before contact element distribution and topography information are fused;

FIG. 7 is an image of the method of the present invention after the contact element distribution and the topography information are fused.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

As shown in fig. 1, an embodiment of the present invention provides a NSST-based contact element distribution and topography information fusion method, including the following steps:

1) acquiring an element distribution image (shown in figure 2) of the surface of the electric contact through a scanning electron microscope and an X-ray energy spectrometer (SEM & EDS), and acquiring a micro-topography image (shown in figure 3) of the surface of the electric contact through a machine vision system;

2) by image preprocessing, the image resolution and the edge information are improved, and an element distribution image and a surface micro-topography image are displayed in the same proportion;

3) adjusting transformation coefficients through Whale Optimization Algorithm (WOA), and registering the element distribution image and the surface micro-topography image by adopting a B-spline-based registration method;

4) and acquiring related information from the source image by adopting an image fusion technology based on NSST and applying a weighted energy fusion rule to obtain a final electrical contact element distribution and morphology information fusion image.

The improvement of the image resolution and the edge information in the step 2) comprises the following steps:

the enhancement of the fine structure edge of the electrical contact image is realized by adopting Gaussian smoothing and edge sharpening, and two images are displayed in the same proportion by adopting an image-centered zooming technology.

And 3) in B spline registration, the deformation in the image can be easily corrected by means of the deformation coefficient. In optimal B-spline based registration techniques, moving image pixels are optimally registered by appropriately selecting a set of deformation coefficients using the mutual information values of the other source image pixels.

The specific process of step 3) is as follows:

the registering method based on the B-spline in the step 3) is used for registering the element distribution image and the surface micro-topography image as follows:

by means of deformation in the positive image with deformation coefficients, a set of deformation coefficients is selected through mutual information values among source image pixels to optimally register the moving image pixels, and the specific process is as follows:

301) moving images according to the map DM ═ Δ Mov → Δ Stat, scanning electron microscope and X-ray spectrometer from the field Δ MovImage of Δ Mov → R entry into fixed surface topographyΔ Stat → field of R Δ Stat, moving imageChange to being in a fixed imageIn the moving image distortion form D_M(y | δ), where δ represents a set of transformation constraints;

302) according to moving imagesAnd fixing the imageThe transform coefficient is selected by the optimal mutual information value therebetween, and an objective function for selecting the transform coefficient is determined by the following equation:

O(δ)＝MaxMI(I_Stat,I_Mov) (1)

the objective function is calculated according to the following equation:

equation (2) characterizes a deformation coefficient selected according to a low variation between a moving image and a fixed image when the moving image is changed to DM (y | δ);

303) optimal B-spline registration is accomplished based on mutual information between moving and fixed images, represented by equation (3):

MI(I_Mov,I_Stat)＝I_E(Mov)+I_E(Stat)-I_E(Mov,Stat) (3)

wherein, I_E(Mov)，I_E(Stat) and I_E(Mov, Stat) characterizes the entropy of moving images, the entropy of fixed images and the joint entropy of images, MI (I)_Mov，I_Stat) Is mutual information between moving images and fixed images.

And 3) effectively selecting the optimal transformation coefficient based on the B spline registration technology by utilizing a WOA algorithm. A B-spline-based registration technology selects a possible transformation coefficient set in each iteration process, and a registered output image reflects mutual information to the maximum extent. After the optimal transform coefficients are identified, the images are effectively registered and prepared for the fusion step.

The composite expansion of the two-dimensional affine system based on the NSST image fusion technology in the step 4) is derived by the following processes:

in the formula: AM represents an anisotropic matrix, controls Shearlet scale, S represents a shear matrix, and I is associated with scale transformation; i. j and k respectively represent a scale parameter, a translation parameter and a displacement parameter; x represents an unknown number, k represents a displacement parameter psi i, j, k (x) and an integral function expression, represents wavelet transformation corresponding to different values of i, j and k, and psi (S)^JIⁱx-m) is the wavelet transform after the shift,represents a set of integers;

the clipping matrices S (| detS | ═ 1) and AM are invertible matrices of size 2 × 2, if d > 0 and S ∈ R, AM and S are:

the discrete equation for NSST in step 4) is as follows:

the Shearlet function is:

whereinFourier transform of psiLet psi₁∈C^∞(R) and psi₂∈C^∞(R) is a tightly-supported wavelet,supp denotes the argument of the function, # denotes the basic wavelet and denotes # denotes₀∈C^∞(R) andin addition, assume that:

omega is weight, i is greater than or equal to 0 and psi₂Comprises the following steps:

the following equations (7) and (8) can be summarized:

the weighted energy calculation flowchart in step 4) is shown in FIG. 4

The weighted energy fusion rule in the step 4) comprises the following specific steps:

401) the local energy of the moving and static images is calculated,

where En (p, q) is the local energy at location (p, q). Cof (p, q) represents the wavelet coefficient at position (p, q). For moving and static images respectivelyAndthe local energy of a rectangular window of size 3x3 was calculated, and a graph of the window size 3x3 coefficients used for the local energy calculation is shown in fig. 5. The local energy of the moving and still images is evaluated by the following equations (11) (12); p and q are horizontal and vertical coordinates respectively representing pixels in the image;

402) the weights Eqs for the fused motion and static coefficients are calculated from equations (10) (11).

403) The fused pixel is calculated according to the following equation (12):

I_(F)(p,q)＝I_(Mov)(P,q)×ω^(Mov)+I_(Stat)(P,q)×ω^(stat) (15)

wherein, I_(F)(p, q) denotes a fusion coefficient, I_(Mov)(p,q)，I_(Stat)(p, q) represents a fusion coefficient of the moving image and the fixed image, ω, respectively^(Mov)，ω^(Stat)Weights, En, representing fusion coefficients of moving and fixed images, respectively^(Mov)(p, q) is the local energy of the moving image, En^(Stat)(p, q) is the local energy of the still image.

Fig. 6 is an image directly superimposed without registration and fusion, and the difference between the two images in the spatial position can be directly and obviously observed, and the difference has obvious displacement and difference on the scale.

Fig. 7 shows the fusion effect achieved by the SEM scanned image and the surface element distribution image after registration, and it can be found that the fusion effect is achieved by the SEM scanned image and the surface element distribution image both locally and globally. The fusion algorithm can effectively extract the information of SEM scanning results and surface element distribution images, surface height information which is not contained in some surface element distribution images is reflected in the fusion images, the outlines of all parts are clear, the subjective visual effect is good, and the edge detail features of the images are captured comprehensively.

Besides subjective visual evaluation, there are also some common objective evaluation indexes for the fusion result. For objective evaluation of the algorithm, Average Gradient (AG), Standard Deviation (SD), Mutual Information (MI) and Structural Similarity (SSIM) are respectively introduced as evaluation indexes of the fused image. AG and SD are 7.138 and 42.894, respectively, which correspond to the contact ablation profile, edge definition on subjective visual effects. MI and SSIM are 7.680 and 0.639, respectively, which shows that the proposed method can well retain the original information in the original source image and the image distortion is small.

TABLE 1 Objective evaluation index of the results of the experiment fusion

The above results show that: the invention can effectively fuse the contact element distribution and the morphology information.

The invention solves the problem that the contact surface is difficult to be completely evaluated by observing a single-mode image, can accurately fuse information of different image technologies into the same mode, can obviously improve the accuracy of target positioning, has clear outline of each part, has good subjective visual effect, can comprehensively capture the edge detail characteristics of the image, and can more conveniently and more accurately observe the contact structure and element distribution. The component distribution of the contact surface can be conveniently and accurately positioned subsequently, and the relationship between the surface structure and the component distribution can be analyzed.

The invention has been explained by applying specific examples to the principle and implementation of the invention, and the above description of the embodiments is only used to help understanding the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed.