1. An image registration method, comprising:

acquiring a target image and an image to be registered;

generating a reference image in a preset shape according to the target image, and acquiring attribute information of a first pixel point in the reference image, wherein the attribute information of the first pixel point is used for indicating whether the first pixel point is located in the target image;

and obtaining registration parameters between the target image and the image to be registered based on the image information of the reference image and the image to be registered and the attribute information.

2. The method according to claim 1, wherein the obtaining of the registration parameter between the target image and the image to be registered based on the image information of the reference image and the image to be registered and the attribute information comprises:

extracting first feature points in the reference image and determining first feature representations of the first feature points based on the attribute information; and the number of the first and second groups,

extracting a second feature point and a second feature representation thereof in the image to be registered;

and obtaining a registration parameter between the target image and the image to be registered based on the first feature point and the first feature representation and the second feature point and the second feature representation.

3. The method of claim 2, wherein said determining a first feature representation of the first feature point based on the attribute information comprises:

selecting a preset number of groups of pixel point pairs in an image area of the reference image; wherein the image region contains the first feature point;

obtaining a first pixel comparison value of the pixel point pair based on the attribute information of the pixel point pair;

and obtaining a first characteristic representation of the first characteristic point by using the first pixel comparison value of the preset number of groups of pixel point pairs.

4. The method of claim 3, wherein obtaining the first pixel alignment value of the pixel point pair based on the attribute information of the pixel point pair comprises:

setting a first pixel comparison value of the pixel point pair as a preset invalid character under the condition that the attribute information of the pixel point pair indicates that the pixel point pair is not located in the target image;

and/or obtaining a first pixel comparison value of the pixel point pair based on the size relationship between the pixel values of the pixel point pair under the condition that the attribute information of the pixel point pair is located in the target image.

5. The method according to any one of claims 2 to 4, characterized in that, prior to said determining a first feature representation of said first feature point based on said attribute information, the method further comprises:

generating a template image with the same size as the reference image based on the attribute information; the pixel value of each second pixel point in the template image represents whether the corresponding first pixel point in the reference image is located in the target image or not;

the determining a first feature representation of the first feature point based on the attribute information includes:

based on the template image, a first feature representation of the first feature point in the reference image is determined.

6. The method according to claim 5, wherein prior to said determining a first feature representation of said first feature point in said reference image based on said template image, the method further comprises:

respectively adopting at least one scaling ratio to scale the reference image to obtain at least one scaled reference image, and obtaining at least one scaled template image based on the at least one scaling ratio; wherein the at least one scaled template image is the same size as the at least one scaled reference image, respectively;

the determining, based on the template image, a first feature representation of the first feature point in the reference image comprises:

determining a first feature representation of a first feature point in an unscaled reference image based on an unscaled template image; and the number of the first and second groups,

a first feature representation of a first feature point in a scaled reference image is determined based on a template image of the same size as the scaled reference image.

7. The method of claim 6, wherein said deriving at least one scaled template image based on said at least one scaling comprises:

respectively adopting the at least one scaling to scale the template image which is not scaled to obtain at least one first candidate image;

and respectively determining the pixel values of corresponding second pixel points in the template image with the same size as the first candidate image based on the size relation between the pixel value of each third pixel point in the first candidate image and a preset pixel value aiming at each first candidate image.

8. The method according to claim 7, wherein the determining the pixel values of the corresponding second pixels in the template image with the same size as the first candidate image based on the magnitude relationship between the pixel value of each third pixel in the first candidate image and a preset pixel value respectively comprises:

determining the pixel value of a corresponding second pixel point in the template image with the same size as the first candidate image as a first numerical value under the condition that the pixel value of the third pixel point is larger than the preset pixel value, wherein the first numerical value is used for indicating that the corresponding first pixel point in the reference image is located in the target image;

and/or determining the pixel value of a corresponding second pixel point in the template image with the same size as the first candidate image as a second numerical value under the condition that the pixel value of the third pixel point is not larger than the preset pixel value, wherein the second numerical value is used for indicating that the corresponding first pixel point in the reference image is not located in the target image.

9. The method of claim 7, wherein said deriving at least one scaled template image based on said at least one scaling comprises:

respectively zooming the target image by adopting the at least one zoom ratio to obtain at least one second candidate image;

and generating a reference image of the preset shape according to each second candidate image, and generating a template image with the same size as the reference image.

10. The method according to any one of claims 2 to 9, wherein the first feature representation comprises a first pixel alignment value of a preset number of groups of pixel point pairs, the preset number of groups of pixel point pairs are included in an image area including the first feature point in the reference image, and the presence of the attribute information of the pixel point pairs indicates that the pixel point pairs are not located in the target image, the first pixel alignment value of the pixel point pairs is a preset invalid character;

the obtaining of the registration parameter between the target image and the image to be registered based on the first feature point and the first feature representation and the second feature point and the second feature representation includes:

obtaining feature similarity between the first feature point and the second feature point based on a first pixel comparison value in the first feature representation except the preset invalid character and a second pixel comparison value in the second feature representation;

taking the first characteristic point and the second characteristic point with the characteristic similarity meeting a preset condition as a characteristic point pair;

and acquiring registration parameters between the target image and the image to be registered based on the characteristic point pairs.

11. The method according to claim 10, wherein the second feature representation comprises the preset number of second pixel alignment values; the obtaining of the feature similarity between the first feature point and the second feature point based on the first pixel comparison value in the first feature representation except for the preset invalid character and the second pixel comparison value in the second feature representation includes:

counting the number of the matching of a first pixel comparison value except the preset invalid character in the first feature representation and a second pixel comparison value in the second feature representation;

and obtaining the feature similarity between the first feature point and the second feature point based on the matching number.

12. The method according to any one of claims 1 to 11, wherein after the obtaining of the registration parameters between the target image and the image to be registered based on the image information of the reference image and the image to be registered and the attribute information, the method further comprises:

determining a fourth pixel point corresponding to the first pixel point in the target image in the image to be registered by using the registration parameter;

and checking whether the registration parameters are accurate or not based on the pixel value difference between the first pixel point and the fourth pixel point corresponding to the first pixel point.

13. The method of claim 12, wherein the checking whether the registration parameter is accurate based on a difference in pixel values between the first pixel point and a corresponding fourth pixel point comprises:

verifying the registration parameters as inaccurate if the pixel value difference is greater than a preset difference threshold;

and under the condition that the pixel value difference is not larger than the preset difference threshold value, verifying that the registration parameter is accurate.

14. An image registration apparatus, comprising:

the image acquisition module is used for acquiring a target image and an image to be registered;

the image generation module is used for generating a reference image with a preset shape according to the target image;

the attribute acquisition module is used for acquiring attribute information of a first pixel point in the reference image, wherein the attribute information of the first pixel point is used for indicating whether the first pixel point is located in the target image;

and the parameter acquisition module is used for obtaining registration parameters between the target image and the image to be registered based on the image information of the reference image and the image to be registered and the attribute information.

15. An electronic device comprising a memory and a processor coupled to each other, the processor being configured to execute program instructions stored in the memory to implement the image registration method of any one of claims 1 to 13.

16. A computer readable storage medium having stored thereon program instructions which, when executed by a processor, implement the image registration method of any of claims 1 to 13.

Background

With the development of electronic information technology, Augmented Reality (AR), Virtual Reality (VR), and the like become application hotspots in the field of computer vision, and the surrounding environment can be digitized by using a camera as an input device and processing with an image algorithm, so that the use experience of interaction with a real environment is obtained.

The image registration is a research focus in the field of computer vision such as AR, VR and the like, and the registration parameters between the image to be registered and the target image shot by the camera can be obtained through the image registration technology, so that the registration position of the target image in the image to be registered can be obtained through the registration parameters subsequently. However, existing image registration techniques are not able to register images of arbitrary shape. In view of this, how to realize image registration in an arbitrary shape becomes an urgent problem to be solved.

Disclosure of Invention

The application provides an image registration method and a related device and equipment.

A first aspect of the present application provides an image registration method, including: acquiring a target image and a to-be-registered image with a preset shape; generating a reference image in a preset shape according to the target image, and acquiring attribute information of a first pixel point in the reference image, wherein the attribute information of the first pixel point is used for indicating whether the first pixel point is located in the target image; and obtaining registration parameters between the target image and the image to be registered based on the image information and the attribute information of the reference image and the image to be registered.

Therefore, a target image and an image to be registered are obtained, on the basis, a reference image in a preset shape is generated according to the target image, attribute information of a first pixel point in the reference image is obtained, and the attribute information of the first pixel point is used for indicating whether the first pixel point is located in the target image, so that a registration parameter between the target image and the image to be registered is obtained based on the image information and the attribute information of the reference image and the image to be registered, and then the registration between the target image and the image to be registered can be realized according to the reference image generated by the target image, so that the image registration in any shape can be realized.

The method for obtaining the registration parameters between the target image and the image to be registered based on the image information and the attribute information of the reference image and the image to be registered comprises the following steps: extracting first feature points in the reference image, and determining first feature representation of the first feature points based on the attribute information; extracting a second feature point and a second feature representation thereof in the image to be registered; and obtaining a registration parameter between the target image and the image to be registered based on the first characteristic point and the first characteristic representation and the second characteristic point and the second characteristic representation.

Therefore, the first feature representation of the first feature point can be determined based on whether the first pixel point in the reference image is located in the target image or not by extracting the first feature point in the reference image and determining the first feature representation of the first feature point based on the attribute information, so that the accuracy of the first feature representation can be improved.

Wherein determining a first feature representation of the first feature point based on the attribute information comprises: selecting a preset number of groups of pixel point pairs in an image area of a reference image; wherein the image area comprises a first feature point; obtaining a first pixel comparison value of the pixel point pair based on the attribute information of the pixel point pair; and obtaining a first characteristic representation of the first characteristic point by using the first pixel comparison value of the preset number of groups of pixel point pairs.

Therefore, a preset number of groups of pixel point pairs are selected in an image area of the reference image, the image area contains the first feature point, and the first pixel comparison value of the pixel point pairs is obtained based on the attribute information of the pixel point pairs, so that the accuracy of the first pixel comparison value can be improved.

Obtaining a first pixel comparison value of the pixel point pair based on the attribute information of the pixel point pair, including: under the condition that the attribute information of the pixel point pair indicates that the pixel point pair is not located in the target image, setting a first pixel comparison value of the pixel point pair as a preset invalid character; and/or obtaining a first pixel comparison value of the pixel point pair based on the magnitude relation between the pixel values of the pixel point pair under the condition that the attribute information of the pixel point pair is located in the target image.

Therefore, when the attribute information of the pixel point pair indicates that the pixel point pair is not located in the target image, the first pixel comparison value of the pixel point pair is set as the preset invalid character, and when the attribute information of the pixel point pair indicates that the pixel point pair is located in the target image, the first pixel comparison value of the pixel point pair is obtained based on the magnitude relation between the pixel values of the pixel point pair, so that the accuracy of the first pixel comparison value can be improved.

Wherein, prior to determining the first feature representation of the first feature point based on the attribute information, the method further comprises: generating a template image having the same size as the reference image based on the attribute information; the pixel value of each second pixel point in the template image indicates whether the corresponding first pixel point in the reference image is located in the target image; determining a first feature representation of the first feature point based on the attribute information, comprising: based on the template image, a first feature representation of a first feature point in the reference image is determined.

Therefore, the template image with the same size as the reference image is generated based on the attribute information, the pixel value of each second pixel point in the template image indicates whether the corresponding first pixel point in the reference image is located in the target image, and on the basis, the first feature representation of the first feature point in the reference image is determined through the template image, so that the attribute information for indicating whether the first pixel point is located in the target image can be recorded through the template image, and the complexity of extracting the first feature representation can be reduced.

Wherein, prior to determining the first feature representation of the first feature point in the reference image based on the template image, the method further comprises: respectively adopting at least one scaling ratio to scale the reference image to obtain at least one scaled reference image, and obtaining at least one scaled template image based on the at least one scaling ratio; wherein the at least one scaled template image is the same size as the at least one scaled reference image, respectively; determining a first feature representation of a first feature point in a reference image based on a template image, comprising: determining a first feature representation of a first feature point in an unscaled reference image based on an unscaled template image; and determining a first feature representation of the first feature point in the scaled reference image based on the template image of the same size as the scaled reference image.

Thus, by scaling the reference image with at least one scaling ratio, respectively, at least one scaled reference image is obtained, and based on at least one scaling, obtaining at least one scaled template image, and the at least one scaled template image is respectively the same size as the at least one scaled reference image, on the basis of the template image, determining a first feature representation of a first feature point in the reference image without zooming, and determining the first feature representation of the first feature point in the scaled reference image based on the template image with the same size as the scaled reference image, which can be beneficial to extracting the first feature point and the first feature representation thereof in the reference image with different scales through at least one scaling, thereby being beneficial to improving the accuracy of the registration parameters.

Wherein deriving at least one scaled template image based on at least one scaling comprises: respectively adopting at least one scaling ratio to scale the template image which is not scaled to obtain at least one first candidate image; and respectively determining the pixel values of corresponding second pixel points in the template image with the same size as the first candidate image based on the size relation between the pixel value of each third pixel point in the first candidate image and a preset pixel value aiming at each first candidate image.

Therefore, the non-scaled template image is scaled by adopting at least one scaling ratio respectively to obtain at least one first candidate image, and for each first candidate image, the pixel values of the corresponding second pixel points in the template image with the same size as the first candidate image are determined respectively based on the size relationship between the pixel values of the third pixel points in the first candidate image and the preset pixel values, so that the non-scaled template image can be directly scaled by the at least one scaling ratio and subjected to correlation processing to obtain the at least one scaled template image, and the complexity of obtaining the at least one scaled template image can be reduced.

The method for determining the pixel values of the corresponding second pixel points in the template image with the same size as the first candidate image based on the size relationship between the pixel value of each third pixel point in the first candidate image and the preset pixel value comprises the following steps: determining the pixel value of a corresponding second pixel point in the template image with the same size as the first candidate image as a first numerical value under the condition that the pixel value of the third pixel point is larger than a preset pixel value, wherein the first numerical value is used for indicating that the corresponding first pixel point in the reference image is positioned in the target image; and/or determining the pixel value of a corresponding second pixel point in the template image with the same size as the first candidate image as a second numerical value under the condition that the pixel value of the third pixel point is not larger than the preset pixel value, wherein the second numerical value is used for indicating that the corresponding first pixel point in the reference image is not located in the target image.

Therefore, under the condition that the pixel value of the third pixel point is larger than the preset pixel value, the pixel value of the corresponding second pixel point in the template image with the same size as the first candidate image is determined as a first numerical value, and the first numerical value is used for indicating that the corresponding first pixel point in the reference image is positioned in the target image, and determining the pixel value of the corresponding second pixel point in the template image with the same size as the first candidate image as a second value under the condition that the pixel value of the third pixel point is not more than the preset pixel value, and the second numerical value is used for representing that the corresponding first pixel point in the reference image is not positioned in the target image, so that the pixel value of the second pixel point can be directly determined by judging the size relationship between the pixel value of the third pixel point and the preset pixel value, the scaled template image is obtained, and the complexity of obtaining at least one scaled template image can be favorably reduced.

Wherein deriving at least one scaled template image based on at least one scaling comprises: respectively zooming the target image by adopting at least one zoom ratio to obtain at least one second candidate image; and respectively generating a reference image with a preset shape according to each second candidate image, and generating a template image with the same size as the reference image.

Therefore, the target image is zoomed by adopting at least one zoom scale respectively to obtain at least one second candidate image, a reference image with a preset shape is generated according to each second candidate image respectively, and a template image with the same size as the reference image is generated, so that the accuracy of the at least one zoomed template image can be improved.

The first feature representation comprises a first pixel comparison value of a preset number of groups of pixel point pairs, the preset number of groups of pixel point pairs are contained in an image area containing a first feature point in a reference image, and the first pixel comparison value of the pixel point pair is a preset invalid character under the condition that attribute information of the pixel point pair exists and represents that the pixel point pair is not located in a target image; obtaining a registration parameter between the target image and the image to be registered based on the first feature point and the first feature representation and the second feature point and the second feature representation, wherein the registration parameter comprises: obtaining feature similarity between the first feature point and the second feature point based on a first pixel comparison value in the first feature representation except for the preset invalid character and a second pixel comparison value in the second feature representation; taking the first characteristic point and the second characteristic point with the characteristic similarity meeting the preset condition as a characteristic point pair; and acquiring registration parameters between the target image and the image to be registered based on the characteristic point pairs.

Therefore, in the case where the first feature representation includes a first pixel alignment value of a predetermined number of groups of pixel point pairs, the predetermined number of groups of pixel point pairs are included in an image region including the first feature point in the reference image, and attribute information of the pixel point pairs exists and indicates that the pixel point pairs are not located in the target image, the first pixel alignment value of the pixel point pair is a predetermined invalid character, on the basis of which the feature similarity between the first feature point and the second feature point is obtained based on the first pixel alignment value except the predetermined invalid character in the first feature representation and the second pixel alignment value in the second feature representation, and the first feature point and the second feature point whose feature similarities satisfy the predetermined condition are used as the feature point pair, so that the registration parameter between the target image and the image to be registered is obtained based on the feature point pair, so that the feature similarity between the first feature point and the second feature point can be calculated, the first pixel comparison value except the preset invalid character is eliminated, namely the influence of the pixel points which are not positioned in the target image on the feature similarity can be eliminated, so that the accuracy of the registration parameters can be improved.

The second feature representation comprises a preset number of second pixel comparison values; obtaining feature similarity between the first feature point and the second feature point based on a first pixel comparison value in the first feature representation except for the preset invalid character and a second pixel comparison value in the second feature representation, including: counting the number of the first pixel comparison values except for the preset invalid characters in the first feature representation and the number of the second pixel comparison values in the second feature representation; and obtaining the feature similarity between the first feature point and the second feature point based on the matching number.

Therefore, the second feature representation comprises a preset number of second pixel comparison values, on the basis, the number of the first pixel comparison values except for the preset invalid characters in the first feature representation matched with the second pixel comparison values in the second feature representation is counted, so that the influence of pixel points which are not located in the target image on the number counting can be favorably eliminated, the feature similarity between the first feature points and the second feature points is obtained based on the number of the matching, and the accuracy of the feature similarity can be favorably improved.

After obtaining the registration parameters between the target image and the image to be registered based on the image information and the attribute information of the reference image and the image to be registered, the method further comprises the following steps: determining a fourth pixel point corresponding to the first pixel point in the target image in the image to be registered by using the registration parameter; and checking whether the registration parameters are accurate or not based on the pixel value difference between the first pixel point and the fourth pixel point corresponding to the first pixel point.

Therefore, the fourth pixel point corresponding to the first pixel point located in the target image is determined in the image to be registered by using the registration parameter, so that whether the registration parameter is accurate or not is verified based on the pixel value difference between the first pixel point and the corresponding fourth pixel point, and the reliability of the registration parameter can be improved.

Wherein, based on the pixel value difference between the first pixel point and the corresponding fourth pixel point, whether the registration parameter is accurate is checked, including: under the condition that the difference of the pixel values is larger than a preset difference threshold value, checking the registration parameters to be inaccurate; and in the case that the pixel value difference is not larger than a preset difference threshold value, verifying the registration parameters to be accurate.

Therefore, when the pixel value difference is greater than the preset difference threshold, the registration parameter is checked to be inaccurate, and when the pixel value difference is not greater than the preset difference threshold, the registration parameter is checked to be accurate, that is, whether the registration parameter is accurate can be checked by judging the size relationship between the pixel value difference and the preset difference threshold, so that the complexity of checking the registration parameter can be reduced.

A second aspect of the present application provides an image registration apparatus, comprising: the system comprises an image acquisition module, an image generation module, an attribute acquisition module and a parameter acquisition module, wherein the image acquisition module is used for acquiring a target image and an image to be registered; the image generation module is used for generating a reference image with a preset shape according to the target image; the attribute acquisition module is used for acquiring attribute information of a first pixel point in the reference image, wherein the attribute information of the first pixel point is used for indicating whether the first pixel point is positioned in the target image; and the parameter acquisition module is used for obtaining registration parameters between the target image and the image to be registered based on the image information and the attribute information of the reference image and the image to be registered.

A third aspect of the present application provides an electronic device, comprising a memory and a processor coupled to each other, wherein the processor is configured to execute program instructions stored in the memory to implement the image registration method in the first aspect.

A fourth aspect of the present application provides a computer-readable storage medium having stored thereon program instructions that, when executed by a processor, implement the image registration method of the first aspect described above.

According to the scheme, the target image and the image to be registered are obtained, on the basis, the reference image in the preset shape is generated according to the target image, the attribute information of the first pixel point in the reference image is obtained, the attribute information of the first pixel point is used for indicating whether the first pixel point is located in the target image, therefore, the registration parameter between the target image and the image to be registered is obtained based on the image information and the attribute information of the reference image and the image to be registered, and then the registration between the target image and the image to be registered can be achieved according to the reference image generated by the target image, and therefore the image registration in any shape can be achieved.

Drawings

FIG. 1 is a schematic flow chart diagram of an embodiment of an image registration method of the present application;

FIG. 2 is a schematic diagram of an embodiment of a reference image;

FIG. 3 is a schematic view of an embodiment of a deflection angle acquisition mode;

FIG. 4 is a flowchart illustrating an embodiment of step S13 in FIG. 1;

FIG. 5 is a schematic diagram of obtaining a first characterization representation;

FIG. 6 is a schematic flow chart illustrating another embodiment of step S13 in FIG. 1;

FIG. 7 is a schematic flow chart diagram of another embodiment of an image registration method of the present application;

FIG. 8 is a block diagram of an embodiment of an image registration apparatus according to the present application;

FIG. 9 is a block diagram of an embodiment of an electronic device of the present application;

FIG. 10 is a block diagram of an embodiment of a computer-readable storage medium of the present application.

Detailed Description

The following describes in detail the embodiments of the present application with reference to the drawings attached hereto.

In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular system structures, interfaces, techniques, etc. in order to provide a thorough understanding of the present application.

The terms "system" and "network" are often used interchangeably herein. The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship. Further, the term "plurality" herein means two or more than two.

Referring to fig. 1, fig. 1 is a schematic flowchart illustrating an embodiment of an image registration method according to the present application.

Specifically, the method may include the steps of:

step S11: and acquiring a target image and an image to be registered.

In one implementation scenario, the image to be registered may be an image captured by a camera. For example, in application scenarios such as AR and VR, the image to be registered may be an image captured by an electronic device such as a mobile phone, a tablet computer, and smart glasses; alternatively, in a video monitoring scene, the image to be registered may be an image captured by a monitoring camera, which is not limited herein. Other scenarios may be analogized, and are not exemplified here. Furthermore, the shape of the image to be registered may be rectangular, i.e. the image to be registered may be a rectangular image.

In another implementation scenario, the shape of the target image may include, but is not limited to: triangular, circular, trapezoidal, etc., without limitation. In addition, the target image may also be an irregular shape, for example, the target image may be a human face image, and the edge of the target image is a human face contour; or, the target image may also be an animal image, and the edge of the target image is an animal contour, and so on in other cases, which is not illustrated here. It should be noted that the embodiments of the present disclosure and the following embodiments are not limited to specific shapes of the target image, and the target image may have any shape.

Step S12: and generating a reference image with a preset shape according to the target image, and acquiring attribute information of a first pixel point in the reference image.

In one implementation scenario, a preset shape may be adopted to fit the target image, so as to obtain a reference image which includes the target image and is in the preset shape. Referring to fig. 2, fig. 2 is a schematic diagram of an embodiment of a reference image. As shown in fig. 2, for example, when the target image is a circle and the preset shape is a rectangle, a circular circumscribed rectangle may be obtained, and the circle in the circumscribed rectangle is the target image, and the pixel point between the circle and the circumscribed rectangle may be any pixel value, so as to obtain the reference image, and if the area between the circle and the circumscribed rectangle may be uniformly filled with black, or the area between the circle and the circumscribed rectangle may be uniformly filled with white, the method is not limited herein. With reference to fig. 2, when the target image is a circle and the predetermined shape is a rectangle, a rectangle including the circle and not tangent to the circle may be obtained, the circle in the rectangle is the target image, and the pixel point between the circle and the rectangle may be any pixel value, so as to obtain the reference image, i.e., the rectangle including the circle, which may not be limited to the circumscribed rectangle. In the case that the target image is in another shape, or in the case that the preset shape is in another shape, the analogy can be made, and the examples are not repeated here.

In one implementation scenario, the preset shape may be set to be the same as the shape of the image to be registered. For example, in the case where the image to be registered is a rectangular image, the preset shape may be set to be rectangular. Other cases may be analogized, and no one example is given here.

In the embodiment of the present disclosure, the attribute information of the first pixel point is used to indicate whether the first pixel point is located in the target image. Specifically, the attribute information may specifically include an attribute value of the first pixel, and when the attribute value is a first numerical value (e.g., 1), it indicates that the first pixel is located in the target image, and when the attribute value is a second numerical value (e.g., 0), it indicates that the first pixel is not located in the target image. Referring to fig. 2, the first pixel points in the circular area in the reference image are located in the target image, so the attribute values of the first pixel points can be set to a first value (e.g., 1), and the first pixel points in the reference image between the circular area and the rectangular area are not located in the target image, so the attribute values of the first pixel points can be set to a second value (e.g., 0). Other cases may be analogized, and no one example is given here.

Step S13: and obtaining registration parameters between the target image and the image to be registered based on the image information and the attribute information of the reference image and the image to be registered.

In an implementation scenario, a first feature point in a reference image may be extracted, a first feature representation of the first feature point may be determined based on attribute information, and a second feature point and a second feature representation thereof in an image to be registered may be extracted, on the basis of which, a registration parameter between a target image and the image to be registered may be obtained based on the first feature point and the first feature representation, and the second feature point and the second feature representation. The specific process may refer to the related description in the following disclosed embodiments, and is not repeated herein. That is, the registration parameters between the target image and the image to be registered can be obtained by extracting the feature points and feature representations thereof in the reference image and the image to be registered, respectively.

In a specific implementation scenario, the registration parameters between the target image and the image to be registered are obtained by respectively extracting the feature points and the feature representations thereof in the reference image and the image to be registered, and may be specifically applied to a case where the image size of the target image is larger than a preset size, that is, the registration mode may be specifically applied to a case where the image size of the target image is relatively large, in which case, the first feature points that are relatively rich may be extracted, and the accuracy of the first feature representation is relatively high, so that the accuracy of the registration parameters may be improved.

In another specific implementation scenario, a Feature extraction method, such as orb (organized FAST and rolling brief), SIFT (Scale-innovative Feature Transform), and the like, may be specifically used to extract a first Feature point and a first Feature representation thereof in the reference image, and extract a second Feature point and a second Feature representation thereof in the image to be registered, and the Feature extraction method may be specifically selected according to actual needs, which is not limited herein. It should be noted that the ORB may be used to quickly create feature representations (such as feature vectors) for feature points in an image, and these feature representations may be used to identify objects in the image, and the specific extraction process is not described herein again; in addition, SIFT has scale invariance, and can detect key points in an image, which is a local feature description, and the specific extraction process is not repeated herein.

In another implementation scenario, a first deflection angle of the first image region with respect to the reference image may be determined, a second deflection angle of the second image region with respect to the image to be registered is determined, at least a part of pixel points of the first image region are in the reference image, and at least a part of pixel points of the second image region are in the image to be registered, and on this basis, a candidate registration parameter may be obtained based on the first deflection angle and the second deflection angle, and a final registration parameter between the target image and the image to be registered may be obtained based on the candidate registration parameter. It should be noted that the first deflection angle is a directional included angle between a connection line between the centroid of the first image region and the center of the first image region and the preset direction, and the second deflection angle is a directional included angle between a connection line between the centroid of the second image region and the center of the second image region and the preset direction. That is, in addition to acquiring the registration parameters by extracting the feature points and their feature representations as described above, the registration parameters may also be determined by acquiring the deflection angles.

In a specific implementation scenario, the determining of the registration parameter by obtaining the deflection angle may be specifically applied to a case that the image size of the target image is not larger than a preset size, that is, the registration method may be specifically applied to a case that the image size of the target image is relatively small, in this case, since the extractable first feature point is relatively poor and accuracy of the first feature representation cannot be ensured, and the influence of the image size on the deflection angle is relatively small or even negligible, the determining of the registration parameter by obtaining the deflection angle may be beneficial to improving the accuracy of the registration parameter.

In another specific implementation scenario, please refer to fig. 3 in combination, and fig. 3 is a schematic diagram of an embodiment of the deflection angle obtaining manner. As shown in fig. 3, a solid rectangle represents a reference image (or an image to be registered), a dashed rectangle in the solid rectangle represents a first image region (or a second image region), P is a centroid of the first image region (or the second image region), a rectangular coordinate system is established with a center of the first image region as a coordinate origin O, a connection line between the centroid P of the first image region (or the second image region) and the center of the first image region (or the second image region) is OP, a predetermined direction may be an x-axis of the rectangular coordinate system, and the directed angle may be an angle θ from the predetermined direction to a counterclockwise direction of the connection line. Other cases may be analogized, and no one example is given here.

In yet another embodiment, please continue with FIG. 3, the centroid (c)_x,c_y) Can be expressed as:

in the above formula (1), (x, y) represents an offset of a certain pixel point in the first image region (or the second image region) with respect to the center of the first image region (or the second image region), I (x, y) represents a pixel value of the pixel point, and Σ represents a summation coincidence, where the summation range is a pixel point in the first image region (or the second image region). It should be noted that, in the case that the attribute information of a certain pixel point in the first image region (or the second image region) indicates that the certain pixel point is not located in the target image, that is, in the case that the attribute value of a certain pixel point in the first image region (or the second image region) is the second value, the pixel point may be removed in the summation process, that is, in the centroid calculation process, the pixel point is not considered.

In another specific implementation scenario, an estimated region of the reference image in the image to be registered may be obtained, and an estimated scaling may be obtained based on the estimated region and the size of the reference image, so that the candidate registration parameter may be obtained based on the estimated scaling, the first deflection angle, and the second deflection angle. Specifically, an angular difference between the first deflection angle and the second deflection angle may be obtained, so that the candidate registration parameter may be obtained based on the angular difference and the estimated scaling. For convenience of description, the pre-estimated scaling is denoted as s, the angle difference is denoted as α, the center of the second image region is denoted as (u, v), and the center of the reference image is denoted as (O)_x,O_y) Then the candidate registration parameter H can be expressed as:

in yet another specific implementation scenario, a preset number of second image regions at different positions may be selected in the registration process, so that a preset number of candidate registration parameters may be obtained through the foregoing process, and on this basis, for each candidate registration parameter, the candidate registration parameter may be utilized to determine the similarity between the image to be registered and the target image, so as to obtain a first scoring result of the candidate registration parameter, and based on the first scoring result of the preset number of candidate registration parameters, a final registration parameter is obtained. Specifically, the similarity between the image to be registered and the target image may be obtained by using the candidate registration parameters based on a method such as SSD (Sum of Squared Differences), NCC (Normalized Cross Correlation), and the like. In addition, when the similarity is greater than a preset similarity threshold, the first scoring result may be considered to meet a preset condition, and the candidate registration parameter is taken as a final registration parameter, or when the similarity is not greater than the preset similarity threshold, the first scoring result may be considered to not meet the preset condition, and the candidate registration parameter may be rejected.

In another specific implementation scenario, in order to improve the accuracy of the final registration parameter, a candidate registration parameter of which the first scoring result meets a preset condition may be further selected as a rough registration parameter, the rough registration parameter is optimized by using a preset optimization manner (e.g., Levenberg-Marquard, Gauss-Newton), and then the optimized rough registration parameter is used to determine the similarity between the image to be registered and the target image to obtain a second scoring result of the rough registration parameter, so that the rough registration parameter may be selected as the final registration parameter based on the second scoring result of the rough registration parameter. In addition, the second scoring result may be obtained by using a method such as NCC, SSD, etc., and specific reference may be made to the foregoing related description, which is not described herein again. After obtaining the second scoring result of the optimized rough-selected registration parameter, the one with the best second scoring result may be selected as the final registration parameter.

In an implementation scenario, after obtaining the registration parameters, the registration parameters may be used to process the target image, so as to obtain a registration position of the target image in the image to be registered.

According to the scheme, the target image and the image to be registered are obtained, on the basis, the reference image in the preset shape is generated according to the target image, the attribute information of the first pixel point in the reference image is obtained, the attribute information of the first pixel point is used for indicating whether the first pixel point is located in the target image, therefore, the registration parameter between the target image and the image to be registered is obtained based on the image information and the attribute information of the reference image and the image to be registered, and then the registration between the target image and the image to be registered can be achieved according to the reference image generated by the target image, and therefore the image registration in any shape can be achieved.

Referring to fig. 4, fig. 4 is a schematic flowchart illustrating an embodiment of step S13 in fig. 1. Specifically, the method may include the steps of:

step S41: first feature points in the reference image are extracted, and a first feature representation of the first feature points is determined based on the attribute information.

In one implementation scenario, as described in the foregoing disclosure, the first feature point in the reference image may be extracted by using an extraction method such as ORB, SIFT, or the like. The specific extraction process may refer to technical details of the extraction methods such as ORB and SIFT, which are not described herein again.

In an implementation scenario, a preset number of groups of pixel point pairs may be selected from an image region of a reference image, the image region includes a first feature point, and a first pixel comparison value of the pixel point pairs is obtained based on attribute information of the pixel point pairs, so that a first feature representation of the first feature point may be obtained by using the first pixel comparison value of the preset number of groups of pixel point pairs, which may be beneficial to improving accuracy of the first feature representation.

In a specific implementation scenario, the image region including the first feature point may include, but is not limited to, a circular region, and the like. Further, in order to improve the accuracy of the first feature representation, the center of the image area may be the first feature point. For example, in the case where the image area is a circular area, the center of the circular area may be the first feature point. Other cases may be analogized, and no one example is given here.

In another specific implementation scenario, the preset number may be set according to the actual application requirement. For example, in a case where the requirement on the accuracy of the registration parameter is high, the requirement on the accuracy of the first feature representation may be correspondingly increased, in this case, the preset number may be set to be slightly larger, for example, may be set to be 256, 512, and the like, and is not limited herein; conversely, in the case that the accuracy requirement on the registration parameter is relatively loose, the accuracy requirement on the first feature representation may be correspondingly reduced, in which case, the preset number may be set to be slightly smaller, for example, may be set to 128, 64, and so on, and is not limited herein.

In yet another specific implementation scenario, in a case where the attribute information of the pixel point pair indicates that the pixel point pair is not located in the target image, the first pixel alignment value of the pixel point pair may be set as a preset invalid character, and in a case where the attribute information of the pixel point pair indicates that the pixel point pair is located in the target image, the first pixel alignment value of the pixel point pair may be obtained based on a size relationship between the pixel point pairs. It should be noted that the attribute information of the pixel point pair includes attribute information of each first pixel point in the pixel point pair.

In yet another specific implementation scenario, please refer to fig. 5 in combination, and fig. 5 is a schematic diagram of obtaining the first feature representation. As shown in fig. 5, a circular area represents an image area including a first feature point, each square area in the circular area represents a pixel point in the image area, wherein a direction area filled with a grid line represents the first feature point, pixel point pairs filled with other same shadows and connected by a dotted line are a preset number of pixel point pairs selected in the image area, for convenience of description, fig. 5 schematically represents 4 pixel point pairs, and a pixel point on an arrow side of the dotted line in the pixel point pairs may be marked as a pixel point on an arrow side of the dotted lineA, the pixel point at one side of the dotted dot can be marked as B, so that the pixel point pair filled with the right oblique line can be marked as P₁(A, B), the pixel point pairs filled with points are marked as P₂(A, B), the pixel point pairs filled with transverse lines are marked as P₃(A, B), pixel point pairs filled with left oblique lines are marked as P₄(A, B). If any one of the pixel point a and the pixel point B in the pixel point pair is not located in the target image, the first pixel comparison value of the pixel point pair may be set as a preset invalid character (e.g., a character, # or the like), otherwise, the first pixel comparison value of the pixel point pair may be obtained based on a magnitude relationship between the pixel value of the pixel point a and the pixel value of the pixel point B. For example, in the case where the pixel value of the pixel point a is greater than the pixel value of the pixel point B, the first pixel alignment value of the pixel point pair may be set to a first value (e.g., 1), and in the case where the pixel value of the pixel point a is not greater than the pixel value of the pixel point B, the first pixel alignment value of the pixel point pair may be set to a second value (e.g., 0). For example, for a pixel point pair P₁(A, B) the pixel value of the pixel A is greater than the pixel value of the pixel B, so that the pixel point can be paired with the P₁The first pixel alignment value of (A, B) is set to 1, and for a pixel point pair P₂(A, B) the pixel value of the pixel A is not greater than the pixel value of the pixel B, so that the pixel point can be paired with the P₂The first pixel alignment value of (A, B) is set to 0, and P is set for the pixel point pair₃(A, B) the pixel value of the pixel A is greater than the pixel value of the pixel B, so that the pixel point can be paired with the P₃The first pixel alignment value of (A, B) is set to 1, and P is set for the pixel point pair₄(A, B) the pixel value of the pixel A is greater than the pixel value of the pixel B, so that the pixel point can be paired with the P₄The first pixel alignment value of (A, B) is set to 1, so the first feature representation of the first feature point shown in FIG. 5 can be described as [ 1011 ]]. Other cases may be analogized, and no one example is given here.

Step S42: and extracting a second feature point and a second feature representation thereof in the image to be registered.

In one implementation scenario, as described in the foregoing disclosure, the second feature point and the second feature representation thereof in the image to be registered may be extracted by using an extraction method such as ORB, SIFT, or the like. The specific process may refer to technical details of the extraction methods such as ORB and SIFT, which are not described herein again.

It should be noted that the process of acquiring the second feature representation may be similar to the process of acquiring the first feature representation. That is, a preset number of groups of pixel point pairs may be selected from an image region including second feature points in the image to be registered, and a second pixel comparison value of the pixel point pairs may be obtained, so that a second feature representation of the second feature points may be obtained by using the second pixel comparison values of the preset number of groups of pixel point pairs. For a specific process, reference may be made to fig. 5 and the foregoing related description, which are not repeated herein.

Step S43: and obtaining a registration parameter between the target image and the image to be registered based on the first characteristic point and the first characteristic representation and the second characteristic point and the second characteristic representation.

As mentioned above, the first feature represents a first pixel comparison value including a predetermined number of groups of pixel point pairs, the predetermined number of groups of pixel point pairs are included in an image region including the first feature point in the reference image, and the first pixel comparison value of the pixel point pair is a predetermined invalid character when the attribute information of the pixel point pair exists and represents that the pixel point pair is not located in the target image. In this case, the feature similarity between the first feature point and the second feature point may be obtained based on a first pixel comparison value in the first feature representation except for the preset invalid character and a second pixel comparison value in the second feature representation, and on this basis, the first feature point and the second feature point whose feature similarity satisfies a preset condition may be used as a feature point pair, and the registration parameter between the target image and the image to be registered may be obtained based on the feature point pair. Therefore, in the process of calculating the feature similarity between the first feature point and the second feature point, the first pixel comparison value except for the preset invalid character can be eliminated, namely, the influence of the pixel points which are not located in the target image on the feature similarity can be eliminated, so that the accuracy of the registration parameters can be improved.

In a specific implementation scenario, as described above, the second feature representation includes a preset number of second pixel comparison values, and then the number of matches between the first pixel comparison value in the first feature representation except for the preset invalid character and the second pixel comparison value in the second feature representation may be counted, and based on the number of matches, the feature similarity between the first feature point and the second feature point is obtained. Continuing with fig. 5, taking the first feature expression [ 1011 ] as an example, in the case of the second feature expression [ 1010 ], the first feature expression and the second feature expression have 3 pairs of matched pixel alignment values, and since there is no preset invalid character in the first feature expression, the feature similarity between the two can be considered to be 3/4-75%; alternatively, when the first feature representation is [ 101 # ] as an example (# representing preset invalid characters), and the second feature representation is [ 1010 ], the first feature representation and the second feature representation have 3 pairs of matched pixel alignment values, and the first feature representation has 1 preset invalid character, so that the feature similarity between the two can be considered to be 3/3-100%. Other cases may be analogized, and no one example is given here.

In another specific implementation scenario, the preset condition may be that the feature similarity is greater than a preset similarity threshold, and the preset similarity threshold may be set according to an actual application situation. For example, in the case that the requirement on the accuracy of the registration parameter is high, the preset similarity threshold may be set to be slightly larger, such as may be set to 90%, 95%, and so on, whereas in the case that the requirement on the accuracy of the registration parameter is relatively loose, the preset similarity threshold may be set to be slightly smaller, such as may be set to 80%, 85%, and so on, which is not limited herein.

In yet another specific implementation scenario, after obtaining the sets of feature point pairs, the sets of feature point pairs may be processed in a RANdom SAmple Consensus (RANdom SAmple Consensus) manner to obtain the registration parameters. The specific processing procedure may refer to the details of the relevant technology of random consistent sampling, and is not described herein again.

Different from the foregoing embodiment, the first feature point in the reference image is extracted, and the first feature representation of the first feature point is determined based on the attribute information, so that the first feature representation of the first feature point can be determined based on whether the first pixel point in the reference image is located in the target image, which is favorable for improving the accuracy of the first feature representation, meanwhile, the second feature point and the second feature representation thereof in the image to be registered are extracted, and the registration parameter between the target image and the image to be registered is obtained based on the first feature point, the first feature representation, the second feature point and the second feature representation, which is favorable for improving the accuracy of the registration parameter.

Referring to fig. 6, fig. 6 is a schematic flowchart illustrating another embodiment of step S13 in fig. 1. Specifically, the method may include the steps of:

step S61: based on the attribute information, a template image of the same size as the reference image is generated.

In the embodiment of the present disclosure, the pixel value of each second pixel point in the template image indicates whether the corresponding first pixel point in the reference image is located in the target image. Specifically, a first numerical value (e.g., 1) may be used to indicate that a corresponding first pixel point in the reference image is located in the target image, and a second numerical value (e.g., 0) may be used to indicate that a corresponding first pixel point in the reference image is not located in the target image. It should be noted that, in the embodiments of the present disclosure, the term "corresponding to" indicates that the images are located at the same position in the image. For example, a first coordinate system is established by taking the upper-left pixel point of the template image as the origin of coordinates, the horizontal rightward direction of the origin is taken as the positive direction of the x-axis of the first coordinate system, the vertical downward direction of the origin is taken as the positive direction of the y-axis of the first coordinate system, a second coordinate system is established by taking the upper-left pixel point of the reference image as the origin, the horizontal rightward direction of the origin is taken as the positive direction of the x-axis of the second coordinate system, the vertical downward direction of the origin is taken as the positive direction of the y-axis of the second coordinate system, and a second pixel point (u) in the template image is established (u is the positive direction of the y-axis of the second coordinate system)₀,v₀) And a first pixel point (u) in the reference image₀,v₀) And correspondingly.

Step S62: based on the template image, a first feature representation of a first feature point in the reference image is determined.

In an implementation scenario, a preset number of pixel point pairs may be selected from an image region including a first feature point in a reference image, and whether the pixel point pair is located in a target image is determined based on a template image, specifically, whether the pixel point pair is located in the target image is determined by using a pixel value of a pixel point corresponding to the pixel point pair in the template image. For example, the pixel values of the pixel points corresponding to the pixel points in the template image are all the first values (e.g., 1), and it may be determined that the pixel points are located in the target image, whereas the pixel values of the pixel points corresponding to the pixel points in the template image are all the second values (e.g., 0), and it may be determined that the pixel points are not located in the target image. On this basis, when the pixel point pairs are all located in the target image, the first pixel comparison value of the pixel point pair may be obtained based on the magnitude relationship between the pixel values of the pixel point pair, and when any one of the pixel point pairs is not located in the target image, the first pixel comparison value of the pixel point pair may be set as a preset invalid character, and finally, the first feature representation of the first feature point may be obtained by using the first pixel comparison values of a preset number of groups of pixel point pairs. Reference may be made to the related description in the foregoing embodiments, which are not repeated herein.

In another implementation scenario, the reference image may be further scaled by at least one scaling ratio to obtain at least one scaled reference image, and the at least one scaled template image is obtained based on the at least one scaling ratio, and the at least one scaled template image is the same size as the at least one scaled reference image, respectively. Therefore, the first feature point and the first feature representation thereof in the reference image with different scales can be extracted and obtained through at least one scaling, and the accuracy of the registration parameter can be improved.

In a specific implementation scenario, at least one scaling factor may be set according to the actual application requirement. For example, the at least one scaling may include, but is not limited to: 0.2, 0.4, 0.6, 0.8, etc., without limitation.

In another specific implementation scenario, the non-scaled template image is scaled by at least one scaling ratio to obtain at least one first candidate image, and for each first candidate image, the pixel values of corresponding second pixel points in the template image with the same size as the first candidate image are determined based on the size relationship between the pixel values of the third pixel points in the first candidate image and the preset pixel values. Therefore, the non-zoomed template image can be directly zoomed through at least one zoom scale and subjected to correlation processing to obtain at least one zoomed template image, which is beneficial to reducing the complexity of obtaining the at least one zoomed template image.

Specifically, the first candidate image corresponding to the scaling ratio may be obtained based on a preset image interpolation manner (e.g., bilinear interpolation). For a specific interpolation process, technical details such as bilinear interpolation may be referred to, and are not described herein.

In addition, under the condition that the pixel value of the third pixel point is greater than the preset pixel value, the pixel value of the corresponding second pixel point in the template image with the same size as the first candidate image is determined as a first numerical value, and under the condition that the pixel value of the third pixel point is not greater than the preset pixel value, the pixel value of the corresponding second pixel point in the template image with the same size as the first candidate image is determined as a second numerical value. The preset pixel value may be set according to practical applications, for example, may be set to 0.5, and is not limited herein.

In yet another specific implementation scenario, different from the manner of obtaining the scaled template image, the target image may be scaled by using at least one scaling ratio to obtain at least one second candidate image, and a reference image including a preset shape is generated according to each second candidate image, and a template image having the same size as the reference image is generated. As such, the accuracy of the at least one scaled template image can be advantageously improved.

Specifically, taking at least one scaling ratio including 0.2, 0.4, 0.6, and 0.8 as an example, the target image a may be scaled by the above scaling ratios to obtain the second candidate image 01 scaled by the scaling ratio 0.2, the second candidate image 02 scaled by the scaling ratio 0.4, the second candidate image 03 scaled by the scaling ratio 0.6, and the second candidate image 04 scaled by the scaling ratio 0.8, respectively. On this basis, for the second candidate image 01, a reference image including the second candidate image 01 may be generated in the manner as described in the foregoing disclosed embodiment, and the template image 01 including the reference image may be generated in the manner as described in the foregoing disclosed embodiment. By analogy, the template image 02 corresponding to the second candidate image 02, the template image 03 corresponding to the second candidate image 03, and the template image 04 corresponding to the second candidate image 04 can be generated. Other cases may be analogized, and no one example is given here.

In yet another specific implementation scenario, a specific process of determining the first feature representation of the first feature point in the non-scaled reference image based on the non-scaled template image, and a specific process of determining the first feature representation of the first feature point in the scaled reference image based on the template image with the same size as the scaled reference image may refer to the foregoing related description, and are not described herein again.

Step S63: and extracting a second feature point and a second feature representation thereof in the image to be registered.

Reference may be made to the related description in the foregoing embodiments, which are not repeated herein.

Step S64: and obtaining a registration parameter between the target image and the image to be registered based on the first characteristic point and the first characteristic representation and the second characteristic point and the second characteristic representation.

Reference may be made to the related description in the foregoing embodiments, which are not repeated herein.

Different from the foregoing embodiment, the template image with the same size as the reference image is generated based on the attribute information, and the pixel value of each second pixel point in the template image indicates whether the corresponding first pixel point in the reference image is located in the target image, on this basis, the first feature representation of the first feature point in the reference image is determined through the template image, so that the attribute information for indicating whether the first pixel point is located in the target image can be recorded through the template image, and the complexity of extracting the first feature representation can be favorably reduced.

Referring to fig. 7, fig. 7 is a flowchart illustrating an image registration method according to another embodiment of the present application. Specifically, the method may include the steps of:

step S71: and acquiring a target image and an image to be registered.

Reference may be made to the related description in the foregoing embodiments, which are not repeated herein.

Step S72: and generating a reference image with a preset shape according to the target image, and acquiring attribute information of a first pixel point in the reference image.

In the embodiment of the present disclosure, the attribute information of the first pixel point is used to indicate whether the first pixel point is located in the target image. Reference may be made to the related description in the foregoing embodiments, which are not repeated herein.

Step S73: obtaining registration parameters between the target image and the image to be registered based on the image information and the attribute information of the reference image and the image to be registered

Reference may be made to the related description in the foregoing embodiments, which are not repeated herein.

Step S74: and determining a fourth pixel point corresponding to the first pixel point in the target image in the image to be registered by using the registration parameter.

Specifically, the coordinate information of the first pixel point Q located in the target image may be converted by using the registration parameter H to obtain the coordinate information of a fourth pixel point corresponding to the first pixel point Q in the image to be registered, and on this basis, the pixel value of the fourth pixel point may be obtained. That is to say, in the process of verifying the registration parameters, pixel points in the reference image that are not located in the target image need to be excluded, so that the accuracy of verification can be improved.

In an implementation scenario, as described in the foregoing disclosure, a template image with the same size as the reference image may be generated based on the attribute information, and a pixel value of each second pixel point in the template image indicates whether a corresponding first pixel point in the reference image is located in the target image. On the basis, the first pixel point of the reference image located in the target image can be determined based on the template image. Reference may be made to the related description in the foregoing embodiments, which are not repeated herein.

In another implementation scenario, for convenience of description, the coordinate information of the first pixel point Q may be denoted as Q (x, y), so that the registration parameter H may be multiplied by Q (x, y) to obtain the coordinate information Q '(x, y) of the fourth pixel point Q' corresponding to the first pixel point Q in the image to be registered.

Step S75: and checking whether the registration parameters are accurate or not based on the pixel value difference between the first pixel point and the fourth pixel point corresponding to the first pixel point.

In an implementation scenario, for each first pixel point, a difference between a pixel value of the first pixel point and a pixel value of a fourth pixel point corresponding to the first pixel point may be obtained, and a sum of squares of the differences corresponding to all the first pixel points is obtained to obtain a pixel value difference.

In another implementation scenario, the registration parameter may be checked as inaccurate if the pixel value difference is greater than a preset difference threshold, whereas the registration parameter may be checked as accurate if the pixel value difference is not greater than the preset difference threshold. Specifically, the preset difference threshold may be set according to an actual application, for example, in a case where the requirement on the accuracy of the registration parameter is high, the preset difference threshold may be set to be slightly smaller, whereas in a case where the requirement on the accuracy of the registration parameter is relatively loose, the preset difference threshold may be set to be slightly larger, and a specific value of the preset difference threshold is not limited herein.

Different from the foregoing embodiment, the fourth pixel point corresponding to the first pixel point located in the target image is determined in the image to be registered by using the registration parameter, so that whether the registration parameter is accurate or not is verified based on the pixel value difference between the first pixel point and the fourth pixel point corresponding to the first pixel point, and the reliability of the registration parameter can be improved.

Referring to fig. 8, fig. 8 is a schematic diagram of an embodiment of an image registration apparatus 80 according to the present application. The image registration apparatus 80 includes: the system comprises an image acquisition module 81, an image generation module 82, an attribute acquisition module 83 and a parameter acquisition module 84, wherein the image acquisition module 81 is used for acquiring a target image and an image to be registered; an image generating module 82, configured to generate a reference image with a preset shape according to the target image; the attribute obtaining module 83 is configured to obtain attribute information of a first pixel point in the reference image, where the attribute information of the first pixel point is used to indicate whether the first pixel point is located in the target image; and a parameter obtaining module 84, configured to obtain a registration parameter between the target image and the image to be registered based on the image information and the attribute information of the reference image and the image to be registered.

According to the scheme, the target image and the image to be registered are obtained, on the basis, the reference image in the preset shape is generated, the attribute information of the first pixel point in the reference image is obtained, the attribute information of the first pixel point is used for indicating whether the first pixel point is located in the target image or not, therefore, the registration parameter between the target image and the image to be registered is obtained based on the image information and the attribute information of the reference image and the image to be registered, and then the registration between the target image and the image to be registered can be achieved by generating the reference image which comprises the target image and has the same shape as the image to be registered, and therefore the image registration in any shape can be achieved.

In some disclosed embodiments, the parameter obtaining module 84 includes a first feature extraction sub-module, configured to extract a first feature point in the reference image and determine a first feature representation of the first feature point based on the attribute information, the parameter obtaining module 84 includes a second feature extraction sub-module, configured to extract a second feature point and a second feature representation thereof in the image to be registered, and the parameter obtaining module 84 includes a registration parameter obtaining sub-module, configured to obtain a registration parameter between the target image and the image to be registered based on the first feature point and the first feature representation and the second feature point and the second feature representation.

Different from the foregoing embodiment, the first feature point in the reference image is extracted, and the first feature representation of the first feature point is determined based on the attribute information, so that the first feature representation of the first feature point can be determined based on whether the first pixel point in the reference image is located in the target image, which is favorable for improving the accuracy of the first feature representation, meanwhile, the second feature point and the second feature representation thereof in the image to be registered are extracted, and the registration parameter between the target image and the image to be registered is obtained based on the first feature point, the first feature representation, the second feature point and the second feature representation, which is favorable for improving the accuracy of the registration parameter.

In some disclosed embodiments, the first feature extraction submodule includes a pixel point pair obtaining unit configured to select a preset number of groups of pixel point pairs in an image region of the reference image, where the image region includes a first feature point, the first feature extraction submodule includes a pixel comparison value obtaining unit configured to obtain a first pixel comparison value of the pixel point pair based on attribute information of the pixel point pair, and the first feature extraction submodule feature representation obtaining unit is configured to obtain a first feature representation of the first feature point by using the first pixel comparison value of the preset number of groups of pixel point pairs.

Different from the foregoing embodiment, a preset number of groups of pixel point pairs are selected from an image region of a reference image, the image region includes a first feature point, and a first pixel comparison value of the pixel point pairs is obtained based on attribute information of the pixel point pairs, so that the accuracy of the first pixel comparison value can be improved.

In some disclosed embodiments, the pixel comparison value obtaining unit is specifically configured to set a first pixel comparison value of a pixel point pair as a preset invalid character when the attribute information of the pixel point pair indicates that the pixel point pair is not located in the target image; the pixel comparison value obtaining unit is specifically configured to, when the attribute information of the pixel point pair indicates that the pixel point pair is located in the target image, obtain a first pixel comparison value of the pixel point pair based on a size relationship between pixel values of the pixel point pair.

Different from the foregoing embodiment, when the attribute information of the pixel point pair indicates that the pixel point pair is not located in the target image, the first pixel comparison value of the pixel point pair is set as the preset invalid character, and when the attribute information of the pixel point pair indicates that the pixel point pair is located in the target image, the first pixel comparison value of the pixel point pair is obtained based on the magnitude relationship between the pixel values of the pixel point pair, so that the accuracy of the first pixel comparison value can be improved.

In some disclosed embodiments, the parameter obtaining module 84 further includes a template image generating sub-module for generating a template image having the same size as the reference image based on the attribute information; the first feature extraction submodule is specifically configured to determine a first feature representation of a first feature point in the reference image based on the template image.

Different from the foregoing embodiment, the template image with the same size as the reference image is generated based on the attribute information, and the pixel value of each second pixel point in the template image indicates whether the corresponding first pixel point in the reference image is located in the target image, on this basis, the first feature representation of the first feature point in the reference image is determined through the template image, so that the attribute information for indicating whether the first pixel point is located in the target image can be recorded through the template image, and the complexity of extracting the first feature representation can be favorably reduced.

In some disclosed embodiments, the parameter obtaining module 84 further includes an image scaling sub-module, configured to scale the reference image with at least one scaling ratio, respectively, to obtain at least one scaled reference image, and obtain at least one scaled template image based on the at least one scaling ratio; the first feature extraction submodule is specifically used for determining a first feature representation of a first feature point in the non-scaled reference image based on the non-scaled template image; and determining a first feature representation of the first feature point in the scaled reference image based on the template image of the same size as the scaled reference image.

In distinction to the previous embodiment, by scaling the reference image with at least one scaling ratio, respectively, at least one scaled reference image is obtained, and based on at least one scaling, obtaining at least one scaled template image, and the at least one scaled template image is respectively the same size as the at least one scaled reference image, on the basis of the template image, determining a first feature representation of a first feature point in the reference image without zooming, and determining the first feature representation of the first feature point in the scaled reference image based on the template image with the same size as the scaled reference image, which can be beneficial to extracting the first feature point and the first feature representation thereof in the reference image with different scales through at least one scaling, thereby being beneficial to improving the accuracy of the registration parameters.

In some disclosed embodiments, the image scaling sub-module includes a first scaling unit configured to scale the template image that is not scaled by at least one scaling ratio, respectively, to obtain at least one first candidate image, and the image scaling sub-module includes a determining unit configured to determine, for each first candidate image, pixel values of corresponding second pixel points in the template image having the same size as the first candidate image, respectively, based on a size relationship between pixel values of respective third pixel points in the first candidate image and a preset pixel value.

Different from the foregoing embodiment, at least one first candidate image is obtained by respectively scaling the non-scaled template image with at least one scaling ratio, and for each first candidate image, the pixel values of the corresponding second pixel points in the template image with the same size as the first candidate image are respectively determined based on the size relationship between the pixel values of the third pixel points in the first candidate image and the preset pixel values, so that the non-scaled template image can be directly scaled with at least one scaling ratio and subjected to correlation processing to obtain at least one scaled template image, which can be beneficial to reducing the complexity of obtaining at least one scaled template image.

In some disclosed embodiments, the determining unit is specifically configured to determine, as a first numerical value, a pixel value of a second pixel point corresponding to a template image of the same size as the first candidate image when the pixel value of the third pixel point is greater than a preset pixel value, where the first numerical value is used to indicate that the corresponding first pixel point in the reference image is located in the target image; and/or determining the pixel value of a corresponding second pixel point in the template image with the same size as the first candidate image as a second numerical value under the condition that the pixel value of the third pixel point is not larger than the preset pixel value, wherein the second numerical value is used for indicating that the corresponding first pixel point in the reference image is not located in the target image.

Different from the foregoing embodiment, in a case that the pixel value of the third pixel is greater than the preset pixel value, the pixel value of the second pixel corresponding to the template image with the same size as the first candidate image is determined as a first value, and the first value is used to indicate that the corresponding first pixel in the reference image is located in the target image, and determining the pixel value of the corresponding second pixel point in the template image with the same size as the first candidate image as a second value under the condition that the pixel value of the third pixel point is not more than the preset pixel value, and the second numerical value is used for representing that the corresponding first pixel point in the reference image is not positioned in the target image, so that the pixel value of the second pixel point can be directly determined by judging the size relationship between the pixel value of the third pixel point and the preset pixel value, the scaled template image is obtained, and the complexity of obtaining at least one scaled template image can be favorably reduced.

In some disclosed embodiments, the image scaling sub-module includes a second scaling unit configured to scale the target image by at least one scaling ratio, respectively, to obtain at least one second candidate image, and the image scaling sub-module includes a generating unit configured to generate a reference image of a preset shape according to each second candidate image, respectively, and generate a template image having the same size as the reference image.

Different from the foregoing embodiment, the target image is scaled by using at least one scaling ratio to obtain at least one second candidate image, and a reference image of a preset shape is generated according to each second candidate image, and a template image having the same size as the reference image is generated, so that the accuracy of the at least one scaled template image can be improved.

In some disclosed embodiments, the first feature representation includes a first pixel comparison value of a preset number of groups of pixel point pairs, the preset number of groups of pixel point pairs are included in an image region including a first feature point in the reference image, and the attribute information of the pixel point pairs exists and indicates that the pixel point pairs are not located in the target image, the first pixel comparison value of the pixel point pairs is a preset invalid character, the registration parameter obtaining sub-module includes a feature similarity obtaining unit configured to obtain a feature similarity between the first feature point and a second feature point based on the first pixel comparison value in the first feature representation except for the preset invalid character and a second pixel comparison value in the second feature representation, the registration parameter obtaining sub-module includes a feature point pair obtaining unit configured to obtain, as the feature point pairs, the first feature point and the second feature point whose feature similarities satisfy a preset condition, the registration parameter obtaining sub-module includes a registration parameter obtaining unit, and the method is used for acquiring registration parameters between the target image and the image to be registered based on the characteristic point pairs.

Different from the foregoing embodiment, the first feature representation includes a first pixel comparison value of a predetermined number of groups of pixel point pairs, the predetermined number of groups of pixel point pairs are included in an image region including the first feature point in the reference image, and the attribute information of the pixel point pairs exists and indicates that the first pixel comparison value of the pixel point pair is not located in the target image, on this basis, the feature similarity between the first feature point and the second feature point is obtained based on the first pixel comparison value of the first feature representation except for the predetermined invalid character and the second pixel comparison value of the second feature representation, and the first feature point and the second feature point whose feature similarities satisfy the predetermined condition are used as the feature point pair, so that the registration parameter between the target image and the image to be registered is obtained based on the feature point pair, so that in the process of calculating the feature similarity between the first feature point and the second feature point, the first pixel comparison value except the preset invalid character is eliminated, namely the influence of the pixel points which are not positioned in the target image on the feature similarity can be eliminated, so that the accuracy of the registration parameters can be improved.

In some disclosed embodiments, the second feature representation includes a preset number of second pixel comparison values, and the feature similarity obtaining unit is specifically configured to count the number of matches between the first pixel comparison values in the first feature representation except for the preset invalid characters and the second pixel comparison values in the second feature representation; and obtaining the feature similarity between the first feature point and the second feature point based on the number of matching

Different from the foregoing embodiment, the second feature representation includes a preset number of second pixel comparison values, on this basis, the number of matches between the first pixel comparison value in the first feature representation except for the preset invalid character and the second pixel comparison value in the second feature representation is counted, so that the influence of pixel points not located in the target image on the number statistics can be favorably eliminated, and based on the number of matches, the feature similarity between the first feature point and the second feature point is obtained, so that the accuracy of the feature similarity can be favorably improved.

In some disclosed embodiments, the image registration apparatus 80 further includes a corresponding pixel determining module, configured to determine, by using the registration parameter, a fourth pixel point corresponding to the first pixel point located in the target image in the image to be registered, and the image registration apparatus 80 further includes a registration parameter checking module, configured to check whether the registration parameter is accurate based on a pixel value difference between the first pixel point and the corresponding fourth pixel point.

Different from the foregoing embodiment, the fourth pixel point corresponding to the first pixel point located in the target image is determined in the image to be registered by using the registration parameter, so that whether the registration parameter is accurate or not is verified based on the pixel value difference between the first pixel point and the fourth pixel point corresponding to the first pixel point, and the reliability of the registration parameter can be improved.

In some disclosed embodiments, the registration parameter verification module comprises a first verification sub-module for verifying that the registration parameter is inaccurate if the pixel value difference is greater than a preset difference threshold, and the registration parameter verification module comprises a second verification sub-module for verifying that the registration parameter is accurate if the pixel value difference is not greater than the preset difference threshold.

Different from the foregoing embodiment, when the pixel value difference is greater than the preset difference threshold, the registration parameter is checked to be inaccurate, and when the pixel value difference is not greater than the preset difference threshold, the registration parameter is checked to be accurate, that is, whether the registration parameter is accurate can be checked by judging the magnitude relationship between the pixel value difference and the preset difference threshold, so that the complexity of checking the registration parameter can be reduced.

Referring to fig. 9, fig. 9 is a schematic block diagram of an embodiment of an electronic device 90 according to the present application. The electronic device 90 comprises a memory 91 and a processor 92 coupled to each other, the processor 92 being configured to execute program instructions stored in the memory 91 to implement the steps of any of the above-described embodiments of the image registration method. In one particular implementation scenario, the electronic device 90 may include, but is not limited to: a microcomputer, a server, and the electronic device 90 may also include a mobile device such as a notebook computer, a tablet computer, and the like, which is not limited herein.

In particular, the processor 92 is configured to control itself and the memory 91 to implement the steps of any of the above-described embodiments of the image registration method. The processor 92 may also be referred to as a CPU (Central Processing Unit). The processor 92 may be an integrated circuit chip having signal processing capabilities. The Processor 92 may also be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. In addition, the processor 92 may be collectively implemented by an integrated circuit chip.

By the scheme, image registration in any shape can be realized.

Referring to fig. 10, fig. 10 is a block diagram illustrating an embodiment of a computer-readable storage medium 100 according to the present application. The computer readable storage medium 100 stores program instructions 101 executable by a processor, the program instructions 101 for implementing the steps of any of the image registration method embodiments described above.

By the scheme, image registration in any shape can be realized.

In some embodiments, functions of or modules included in the apparatus provided in the embodiments of the present disclosure may be used to execute the method described in the above method embodiments, and specific implementation thereof may refer to the description of the above method embodiments, and for brevity, will not be described again here.

The foregoing description of the various embodiments is intended to highlight various differences between the embodiments, and the same or similar parts may be referred to each other, and for brevity, will not be described again herein.

In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a module or a unit is merely one type of logical division, and an actual implementation may have another division, for example, a unit or a component may be combined or integrated with another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some interfaces, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on network elements. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) or a processor (processor) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.