1. An image registration method, comprising:

acquiring a preset number of images to be registered;

acquiring the image category of each image to be registered;

according to the image category of each image to be registered, sequencing the images to be registered in the preset number from front to back according to a preset image registration category sequence to obtain an image sequence to be registered consisting of the images to be registered in the preset number;

and performing image registration based on the image sequence to be registered.

2. The method according to claim 1, wherein the step of obtaining the image category of each image to be registered comprises:

using the image sequence to be recognized formed by the preset number of images to be registered as the input of a pre-trained image sequence category recognition model, and recognizing the image sequence category corresponding to the image sequence to be recognized through the image sequence category recognition model;

and determining the image category of each image in the image sequence to be identified according to the image sequence category corresponding to the image sequence to be identified.

3. The method according to claim 2, wherein the step of using the image sequence to be recognized composed of the preset number of images to be registered as an input of a pre-trained image sequence category recognition model, and recognizing the image sequence category corresponding to the image sequence to be recognized by the image sequence category recognition model further comprises:

determining that the image category of the image in the image sequence to be recognized is abnormal according to the image sequence category corresponding to the image sequence to be recognized;

after the step of obtaining the image category of each image to be registered, the method further comprises:

and if the image type of the images in the image sequence to be identified is determined to be abnormal, outputting prompt information of the error image type to be registered, and returning to the step of acquiring the preset number of images to be registered.

4. The method according to claim 2 or 3, wherein the step of using the image sequence to be recognized composed of the preset number of images to be registered as an input of a pre-trained image sequence category recognition model to recognize the image sequence category corresponding to the image sequence to be recognized by the image sequence category recognition model comprises:

determining the first image sequence categories with the same number according to the arrangement number of the image categories, and determining the first image sequence categories which are in one-to-one correspondence with each arrangement sequence of the image categories;

acquiring a plurality of image sequences corresponding to the first image sequence categories according to the arrangement sequence of the image categories corresponding to the first image sequence categories, and constructing a training sample corresponding to the first image sequence categories based on each image sequence;

and training the image sequence class recognition model based on the constructed training sample.

5. The method according to claim 4, wherein the step of obtaining a plurality of image sequences corresponding to each of the first image sequence categories according to the arrangement order of the image categories corresponding to the first image sequence categories, and constructing a training sample corresponding to the first image sequence category based on each of the image sequences comprises:

acquiring a plurality of images for each of said image categories;

enumerating image combinations obtained by respectively selecting one image from a plurality of images of each image category;

and for each first image sequence type, sequencing the images in each image combination according to the arrangement sequence of the image types corresponding to the first image sequence type to obtain a training sample corresponding to the first image sequence type.

6. The method according to claim 4, wherein the step of determining the same number of first image sequence categories according to the number of the arrangement of the image categories, and determining the first image sequence categories in one-to-one correspondence with each arrangement order of the image categories, further comprises:

determining a second image sequence category different from each first image sequence category, and determining the arrangement sequence of the image categories corresponding to the second image sequence category as the arrangement sequence at least comprising repeated image categories;

constructing a training sample corresponding to the second image sequence type according to at least two images randomly selected from a plurality of images of any one image type and images selected from a plurality of images of the rest image types;

the step of training the image sequence class recognition model based on the constructed training samples comprises:

and training the image sequence type recognition model based on the training samples corresponding to the first image sequence types and the training samples corresponding to the second image sequence types.

7. An image registration apparatus, comprising:

the system comprises a to-be-registered image acquisition module, a registration module and a registration module, wherein the to-be-registered image acquisition module is used for acquiring a preset number of to-be-registered images;

the image type acquisition module is used for acquiring the image type of each image to be registered;

the sorting module is used for sorting the images to be registered in the preset number from front to back according to the image category of each image to be registered and a preset image registration category sequence to obtain an image sequence to be registered consisting of the images to be registered in the preset number;

and the registration module is used for carrying out image registration based on the image sequence to be registered.

8. The apparatus of claim 7, wherein the image category acquisition module is further configured to:

using the image sequence to be recognized formed by the preset number of images to be registered as the input of a pre-trained image sequence category recognition model, and recognizing the image sequence category corresponding to the image sequence to be recognized through the image sequence category recognition model;

and determining the image category of each image in the image sequence to be identified according to the image sequence category corresponding to the image sequence to be identified.

9. The apparatus of claim 8, wherein the image category obtaining module is further configured to:

determining that the image category of the image in the image sequence to be recognized is abnormal according to the image sequence category corresponding to the image sequence to be recognized;

the device further comprises:

and the exception handling module is used for outputting prompt information of wrong image types to be registered if the image types of the images in the image sequence to be identified are determined to be abnormal, and returning to the image acquisition module to be registered.

10. An electronic device comprising a memory, a processor and program code stored on the memory and executable on the processor, wherein the processor implements the image registration method of any one of claims 1 to 6 when executing the program code.

11. A computer-readable storage medium, on which a program code is stored, characterized in that the program code realizes the steps of the image registration method of any one of claims 1 to 6 when executed by a processor.

Background

With the increasingly wide application of the internet, the network security requirement is also continuously raised, and many network platforms require users to perform some operations on the network platform after identity registration and authentication. For example, a takeaway delivery platform needs a user who is registered as a takeaway rider to upload photos of the user and a certificate in a certain order, and then a registration application compares and identifies the photos sequentially uploaded by the user, and the user may be successfully registered after the comparison and identification are passed. In the prior art, an image comparison and identification module of a registration application has strict requirements on the type and the sequence of photos uploaded by a user, registration failure often occurs due to wrong sequence of photos uploaded by the user, and the user needs to repeatedly execute photo uploading operation.

Therefore, the problem of low registration efficiency exists in the identity registration system in the prior art.

Disclosure of Invention

The embodiment of the application provides an image registration method which is beneficial to improving the registration efficiency of an identity registration system.

In order to solve the above problem, in a first aspect, an embodiment of the present application provides an image registration method, including:

acquiring a preset number of images to be registered;

acquiring the image category of each image to be registered;

according to the image category of each image to be registered, sequencing the images to be registered in the preset number from front to back according to a preset image registration category sequence to obtain an image sequence to be registered consisting of the images to be registered in the preset number;

and performing image registration based on the image sequence to be registered.

In a second aspect, an embodiment of the present application provides an image registration apparatus, including:

the system comprises a to-be-registered image acquisition module, a registration module and a registration module, wherein the to-be-registered image acquisition module is used for acquiring a preset number of to-be-registered images;

the image type acquisition module is used for acquiring the image type of each image to be registered;

the sorting module is used for sorting the images to be registered in the preset number from front to back according to the image category of each image to be registered and a preset image registration category sequence to obtain an image sequence to be registered consisting of the images to be registered in the preset number;

and the registration module is used for carrying out image registration based on the image sequence to be registered.

In a third aspect, an embodiment of the present application further discloses an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the image registration method according to the embodiment of the present application is implemented.

In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, performs the steps of the image registration method disclosed in the present application.

According to the image registration method disclosed by the embodiment of the application, the images to be registered in the preset number are obtained, the image types of the images to be registered are obtained, then the images to be registered in the preset number are sequenced from front to back according to the image types of the images to be registered, the image sequences to be registered consisting of the images to be registered in the preset number are obtained, and finally, the image registration is carried out based on the image sequences to be registered, so that the image registration efficiency can be improved.

The foregoing description is only an overview of the technical solutions of the present application, and the present application can be implemented according to the content of the description in order to make the technical means of the present application more clearly understood, and the following detailed description of the present application is given in order to make the above and other objects, features, and advantages of the present application more clearly understandable.

Drawings

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Fig. 1 is a flowchart of an image registration method according to a first embodiment of the present application;

fig. 2a is one of schematic diagrams of images to be registered in each category of an image registration method according to a first embodiment of the present application;

FIG. 2b is a second schematic diagram of images to be registered in each category of the image registration method according to the first embodiment of the present application;

fig. 2c is a third schematic diagram of images to be registered of each category of the image registration method according to the first embodiment of the present application;

FIG. 3 is a schematic diagram of model input in an image registration method according to a first embodiment of the present application;

FIG. 4 is a flowchart of an image registration method according to a second embodiment of the present application;

FIG. 5 is a schematic structural diagram of an image registration apparatus according to a third embodiment of the present application;

FIG. 6 is a second schematic structural diagram of an image registration apparatus according to a third embodiment of the present application;

FIG. 7 schematically shows a block diagram of an electronic device for performing a method according to the present application; and

fig. 8 schematically shows a storage unit for holding or carrying program code implementing a method according to the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Example one

As shown in fig. 1, an image registration method disclosed in an embodiment of the present application includes: step 110 to step 140.

Step 110, obtaining a preset number of images to be registered.

In the user identity registration process, in order to perform identity verification, many application scenarios require a user to perform face images, document images, and images including faces and documents. For example, during the registration process of logistics distribution personnel (such as take-away jockeys, mail deliverers, couriers and the like), images of face-held certificates (such as fig. 2a), certificate face images (such as fig. 2b) and face images (such as fig. 2c) of the front faces of the distribution personnel need to be uploaded; in the process of company registration application, the face image of the front face of a legal person, the image of the front face of a certificate and the image of a handheld certificate of the face of the front face need to be uploaded.

In the similar application scenes, some registered applications guide the registered user to collect a preset number of images to be registered (such as collecting three images of a takeaway rider, including a front face image, a certificate front face image and an image of a front face hand-held certificate) in real time by calling an interface of the image collecting device, and some registered applications guide the registered user to upload and collect the preset number of images to be registered in real time by calling an interface of image management software (such as a system photo album).

The registration application may acquire the preset number of images to be registered through the image acquisition device, or acquire the preset number of images to be registered uploaded by the user by calling an image management software interface, where the acquired images to be registered are represented as: pic1, pic2, and pic 3.

In the embodiment of the present application, a specific implementation manner of obtaining an image to be registered is not limited.

And step 120, acquiring the image category of each image to be registered.

In the prior art, a registered application guides a registered user to sequentially collect or upload images of different image categories according to an interface of a preset image recognition engine. For example, in a take-away rider registration process, the registration application will guide the user to collect in turn: the certificate image, the certificate front image and the front face image are held by the front face. When the registration application collects the images in sequence, the collected images to be registered are sequentially marked with image categories and are respectively stored. After acquiring a preset number of images to be registered (such as images pic1, pic2 and pic3), the image comparison and identification module of the registration application compares every two images to be registered in sequence according to the acquisition sequence of the images to be registered, and executes user registration according to the comparison result. For example, pairwise comparisons can be made by holding the face H1 of the document in hand in an image of the document against a face (e.g., fig. 2a), the face H2 on the document, the face H3 in a document front image (e.g., fig. 2b), and the face H4 in a front face image (e.g., fig. 2 c). And if the two comparison results meet the preset condition, determining that the image registration is passed.

In the prior art, an image comparison and identification module sets different confidence threshold values for comparison results of two images to be registered in different image categories. For example, the pairwise comparison categories and sequences of the images preset by the image comparison identification module are respectively as follows: the certificate image and the certificate front image are held by the face of the front face, and the confidence threshold is T12; the certificate image and the face image are held by the face of the front face, and the confidence threshold value is T13; the confidence threshold value is T23 when the certificate front face image and the front face image are shown, wherein the confidence threshold values T12, T23 and T13 correspond to different values.

As the registration application defaults that the user collects or uploads the images to be registered according to the guiding sequence, the registration application defaults that the image types of the images to be registered collected or uploaded in sequence are consistent with the image types of the images to be registered input in sequence by the image comparison and identification module, and the images to be registered collected or uploaded in sequence are input to the image comparison and identification module for registration and identification. However, if the user does not upload or collect the images to be registered of the corresponding categories according to the guiding sequence, the image comparison and identification module may compare the content with the images of which the types are not matched, and the content of the images to be registered of different image categories is greatly different, so that the comparison result does not meet the preset confidence level threshold, thereby affecting the image registration result and causing the registration failure. The user is required to upload or collect the image to be registered repeatedly, so that the image registration efficiency is reduced.

In some embodiments of the present application, the image content is identified by an image identification technology, and the image category of each image to be registered is determined. For example, several images of each image category may be acquired in advance, and the image category identification model of each image category may be trained based on the several images of each image category. In a specific application, for each acquired image to be registered (such as images to be registered pic1, pic2 and pic3), the image category identification model of each image category is respectively input to determine the image category of each image to be registered.

In some preferred embodiments of the present application, a preset number of acquired images to be registered (for example, the images pic1, pic2 and pic3 to be registered) are used as an image sequence and input to a pre-trained image sequence class identification model to determine the image class of each image to be registered in the image sequence. For example, the acquiring of the image category of each image to be registered includes: using the image sequence to be recognized formed by the preset number of images to be registered as the input of a pre-trained image sequence category recognition model, and recognizing the image sequence category corresponding to the image sequence to be recognized through the image sequence category recognition model; and determining the image category of each image in the image sequence to be identified according to the image sequence category corresponding to the image sequence to be identified. The following describes in detail a specific technical solution for performing combined recognition on a preset number of images to be registered based on an image sequence to obtain an image category of each of the images to be registered.

In some embodiments of the present application, the preset number of images to be registered may be arranged in any order to form an image sequence to be recognized. If the acquired images pic1, pic2 and pic3 to be registered are arranged into the image sequence to be identified: pic1 pic2 pic 3. And taking the image sequence to be recognized formed by the preset number of images to be registered as the input of a pre-trained image sequence category recognition model, wherein the image sequence to be recognized comprises the following steps: sequentially splicing the images to be registered in the image sequence to be recognized into one image (as shown in fig. 3), and inputting the spliced image into the pre-trained image sequence category recognition model; or, respectively extracting preset image features of each image to be registered in the image column to be recognized, splicing the preset image features of each image to be registered in sequence, and inputting the spliced preset image features to the pre-trained image sequence category recognition model.

For a specific implementation of image stitching, reference is made to the prior art, and details are not described in the embodiment of the present application. The preset image features may be texture features, color features, or other image features, which is not limited in this application. For a specific implementation of image feature stitching, reference is made to the prior art, and details are not described in this embodiment.

In specific implementation, the image sequence to be recognized, which is composed of the preset number of images to be registered, is input to a pre-trained image sequence category recognition model in an image splicing mode or a splicing characteristic mode, and is determined according to the input requirement of the image sequence category recognition model. Taking the input data of the image sequence category identification model as an image, correspondingly, images to be registered in the image sequence to be identified need to be sequentially spliced into one image (as shown in fig. 3), and the spliced image is input into the pre-trained image sequence category identification model.

The pre-trained image sequence category identification model outputs probability values of the input images matched with the preset image sequence categories through feature extraction and mapping processing of the input images, and each probability value indicates whether the input image belongs to the image sequence category. Taking the preset image sequence category comprising six first image sequence categories, which are respectively denoted as C1, C2, C3, C4, C5 and C6 as an example, the image sequence category identification model outputs respective probability values corresponding to the six image sequence categories, and then the first image sequence category (for example, C1) corresponding to the maximum probability value can be used as the image sequence category matched with the input image.

In some embodiments of the present application, determining an image category of each image in the image sequence to be recognized according to an image sequence category corresponding to the image sequence to be recognized includes: and determining the image category of each image in the image sequence to be identified according to the preset corresponding relation between the arrangement sequence of the image categories and the first image sequence category. For example, in some embodiments of the present application, an arrangement order of image categories corresponding to each of the first image sequence categories is pre-established. The image categories include: for example, the front face image, the certificate front image and the front face image are taken by hand, and it is assumed that the arrangement sequence of the graphic categories corresponding to the first image sequence category C1 is preset from front to back: the positive face hand-held certificate image, the certificate positive image and the positive face image can be further characterized in that the arrangement sequence of the graphic categories corresponding to the first image sequence category C1 is as follows from front to back: a first, front face hand-held certificate image; the second picture is a certificate front image; and the third, a frontal face image. Thus, the image of each image in the image sequence to be recognized can be determined.

In some embodiments of the present application, before the step of using an image sequence to be recognized, which is composed of the preset number of images to be registered, as an input of a pre-trained image sequence category recognition model, and recognizing, by the image sequence category recognition model, an image sequence category corresponding to the image sequence to be recognized, the method further includes: determining the first image sequence categories with the same number according to the arrangement number of the image categories, and determining the first image sequence categories which are in one-to-one correspondence with each arrangement sequence of the image categories; acquiring a plurality of image sequences corresponding to the first image sequence categories according to the arrangement sequence of the image categories corresponding to the first image sequence categories, and constructing a training sample corresponding to the first image sequence categories based on each image sequence; and training the image sequence class recognition model based on the constructed training sample. That is, before using the image sequence class identification model, the image sequence class identification model needs to be trained first.

In order to facilitate the reader to more clearly understand the beneficial effects brought by the technical scheme of the present application, the following describes in detail the technical means adopted in the technical scheme in combination with a training process illustrating an image sequence class recognition model.

Taking an application scenario in which a takeaway rider performs image registration as an example, the registration application generally requires the takeaway rider to sequentially acquire a front face handheld certificate image, a certificate front face image, and a front face image. When the image category of the image to be registered is determined, if each image to be registered is respectively input into a pre-trained image category identification model with a single image category for identification, and the image category of each image to be registered is respectively determined, because the image contents of the front face hand-held certificate image and the front face image are very similar, false identification is easily caused. Therefore, in a preferred embodiment of the present application, a to-be-recognized image sequence is formed by the acquired preset number of to-be-registered face images according to any order, and the preset number of to-be-registered face images are subjected to combined recognition through an image sequence class recognition model, so as to reduce the confusion-prone feature and reduce the false recognition rate. And further determining the image category of each image to be registered in the image sequence to be identified by identifying the combination sequence of the image sequence to be identified. Correspondingly, when the image sequence class identification model is trained, a plurality of training samples corresponding to each image sequence class need to be constructed.

In some embodiments of the present application, different combination orders of all preset image categories of the image to be registered are respectively used as a first image sequence category. That is, the arrangement types of all the preset image categories are used as the number of the first image sequence categories, and each first image sequence category corresponds to one arrangement of all the preset image categories. The images to be registered of the takeaway riders still include: for convenience of description, the front face image represents the front face handheld certificate image by the symbol "P1", the certificate front image by the symbol "P2", and the front face image by the symbol "P3", then six kinds of arrangement of the three image categories can be obtained according to the arrangement formula a (3,3), for example, the three kinds are respectively represented as: P1P 2P 3, P1P 3P 2, P2P 1P 3, P2P 3P 1, P3P 1P 2, P3P 2P 1. Accordingly, it may be determined that the first image sequence categories include six kinds, and each of the first image sequence categories corresponds to one arrangement kind of the three kinds of image categories. For example, six first image sequence categories are respectively represented by symbols: c1, C2, C3, C4, C5 and C6, the arrangement order of the image categories corresponding to the first image sequence category C1 may be set to P1 (i.e., a positive face hand-held document image) P2 (i.e., a document positive image) P3 (i.e., a positive face image); setting the arrangement order of the image categories corresponding to the first image sequence category C2 as P1P 3P 2; … …

In some embodiments of the present application, before training the image sequence class identification model, a plurality of training samples corresponding to each first image sequence class need to be constructed, where a sample label of each training sample is a class identifier of the first image sequence class; the sample data is an image sequence, and the arrangement order of the images in the image sequence is matched with the arrangement order of the image category corresponding to the first image sequence category. For example, in the sample data of all training samples corresponding to the first image sequence category C1, the image category of the first image is P1 (i.e., positive face hand-held certificate image), the image category of the second image is P2 (i.e., certificate positive image), and the image category of the third image is P3 (i.e., positive face image).

In some embodiments of the present application, the training samples may be constructed by directly acquiring image sequences corresponding to various first image sequence categories. For example, the images of the above three image categories of each rider are acquired in different orders, respectively, resulting in several image sequences. However, it is time consuming to acquire the image sequence, and a large number of training samples cannot be acquired.

In some embodiments of the present application, a plurality of images of each image category may also be obtained first, and then training samples are constructed in a manner of generating image sequences corresponding to various first image sequence categories by performing image combination and sorting. Based on the requirement that an image sequence to be recognized is formed by the acquired face images to be registered in the preset number according to any sequence and combined recognition is carried out, when training samples of the model are constructed, in order to ensure the distribution balance of the training samples of each first image sequence category, namely the training samples of the combination sequence of various image categories, and improve the model recognition accuracy, in some embodiments of the application, the images of various image categories are regularly combined to generate new samples so as to increase the number of the training samples and the distribution balance of the samples.

For example, in some embodiments of the present application, the step of obtaining a plurality of image sequences corresponding to the first image sequence categories according to the arrangement order of the image categories corresponding to the first image sequence categories, and constructing a training sample corresponding to the first image sequence category based on each of the image sequences includes: acquiring a plurality of images for each of said image categories; enumerating image combinations obtained by respectively selecting one image from a plurality of images of each image category; and for each first image sequence type, sequencing the images in each image combination according to the arrangement sequence of the image types corresponding to the first image sequence type to obtain a training sample corresponding to the first image sequence type.

Still in the image category: for example, the front face hand-held certificate image, the certificate front image and the front face image are represented by a symbol "P1", the certificate front image is represented by a symbol "P2", and the front face image is represented by a symbol "P3", and a plurality of front face hand-held certificate images, a plurality of certificate front images and a plurality of front face images need to be acquired first. In specific implementation, some types of images are difficult to acquire, for example, front face images, and due to unbalanced samples, overfitting is likely to occur in the model training process, so that in some embodiments of the present application, a "new sample" is regenerated by using a sample combination method.

The following describes in detail the technical scheme of generating training samples by combining images of the ith image type of the jth rider with the symbol Pi _ j as an example. Where i and j are positive integers, in particular, for the present embodiment, assuming that there are more than 3 kinds of images of a 100-bit rider as original samples, the value of j ranges from 1 to 100, i can take values of 1, 2 and 3, P1_1 represents the type 1 image of rider 1 (i.e., the face image of a front face, and a document image), P2_1 represents the type 2 image of rider 1 (i.e., the document face image), and P3_1 represents the type 3 image of rider 1 (i.e., the face image of a front face). Several images of each category of image may be taken for each rider, such as 10 images of category 1 (i.e., a front face handful of documents image), 10 images of category 2 (i.e., a document front image), and 10 images of category 3 (i.e., a front face image) for rider 1. Several images of each image category of the other riders were acquired in the same way. Finally, several images for each image category may be obtained.

In order to increase the number of samples and further improve the recognition accuracy of the trained model, a plurality of images of each image category are combined to generate a plurality of image sequences. For example, selecting one image P1_1 from the 1 st image category, selecting one image P2_1 from the 2 nd image category, and each image from the 3 rd image category to form an image sequence, if N images are included in the 3 rd image category, N image sequences corresponding to the first image sequence category C1 (i.e., the corresponding image categories are arranged in the order of P1P 2P 3) will be obtained. Selecting a picture P1_1 from the 1 st picture category, selecting a picture P2_2 from the 2 nd picture category, and composing a picture sequence from each picture in the 3 rd picture category, respectively, will result in N picture sequences corresponding to the first picture sequence category C1 (i.e., the corresponding picture categories are arranged in the order of P1P 2P 3). According to the measuring method, a plurality of image sequences corresponding to the first image sequence category are obtained. In the same way, several image sequences corresponding to each first image sequence category can be obtained.

The method covers all permutations of selecting one image from 3 image categories respectively to form image combination.

For the image sequence corresponding to each first image sequence category, taking each image sequence as sample data of one training sample, taking the category identification of the first image sequence category corresponding to the image sequence as a sample label of the training sample, and constructing one piece of training data, so as to obtain a plurality of training samples corresponding to the first image sequence category.

After determining a plurality of training samples corresponding to each first image sequence class, starting to train the image sequence class identification model based on the constructed training samples. In some embodiments of the present application, the image sequence class identification model is implemented using a ResNet50 network structure. In other embodiments of the present application, the image sequence category identification model may also adopt other multi-classification network mechanisms, and the present application does not limit the specific network structure adopted by the image sequence category identification model, as long as it is satisfied that the multi-classification result of the input data can be output.

For a specific implementation of training the image sequence class identification model based on the constructed training sample, reference is made to the prior art, and details are not repeated in the embodiment of the present application.

Step 130, according to the image category of each image to be registered, sorting the images to be registered in the preset number from front to back according to a preset image registration category sequence to obtain an image sequence to be registered, wherein the image sequence to be registered is composed of the images to be registered in the preset number.

After the image category of each image to be registered is determined, for example, the image categories of the images to be registered pic1 pic2 pic3 are, in order: the face image P1, the face image P3, and the certificate face image P2, and the preset image registration category sequence (i.e., the default image registration category sequence of the image comparison recognition module in the registration application) is: and (3) carrying the certificate image P1, the certificate front image P2 and the front face image P3 by the front face, rearranging the images pic1, pic2 and pic3 to be registered to obtain an image sequence pic1 pic3 pic2 to be registered, namely adjusting the arrangement sequence of the acquired images to be registered to be consistent with the input requirement of the image comparison and identification module.

And 140, registering the image based on the image sequence to be registered.

And finally, inputting the image sequence to be registered obtained after the sequence is adjusted to the image comparison and identification module, and comparing and identifying the image to be registered through the image comparison and identification module so as to complete image registration.

According to the image registration method disclosed by the embodiment of the application, the images to be registered in the preset number are obtained, the image types of the images to be registered are obtained, then the images to be registered in the preset number are sequenced from front to back according to the image types of the images to be registered, the image sequences to be registered consisting of the images to be registered in the preset number are obtained, and finally, the image registration is carried out based on the image sequences to be registered, so that the image registration efficiency can be improved.

Specifically, in the image registration method disclosed in the embodiment of the present application, the category identification and the sequence correction operation of the image to be registered are added in the image registration link in the identity registration system in the prior art, so that the situation of the abnormal category of the uploaded image is automatically corrected, and the automatic throughput rate is improved. The method has no strict requirement on the sequence of collecting or uploading the images to be registered by the user, and improves the usability of the registration system. A large amount of experimental data show that the automatic registration success rate is improved by 5 percent compared with the prior art when the image registration method disclosed by the embodiment of the application is adopted for carrying out the registration of the rider. On the other hand, in the training process, the sample combination is adopted as a training sample unit, so that the problem of partial data type shortage in the initial stage of business development is solved, samples are enriched, and the accuracy of image category identification is improved.

Example two

In the image registration method disclosed in the embodiment of the present application, in the training process of the image sequence category identification model, the image sequence category further includes: a second image sequence category. After the step of determining the same number of first image sequence categories according to the number of the image categories, and determining the first image sequence categories corresponding to each sort of the image categories, the method further includes: and determining a second image sequence category different from each first image sequence category, and determining the arrangement sequence of the image categories corresponding to the second image sequence category as the arrangement sequence at least comprising repeated image categories. The second image sequence category corresponds to an abnormal image sequence, for example, at least two images in the image sequence have the same category. As described in embodiment one, the second image sequence category may be represented by the symbol C7. Specifically, the image category described in the first embodiment, for example, the image sequence composed of the front face handheld certificate image P1, the front face image P3, and the front face handheld certificate image P1 belongs to the second image sequence category; and an image sequence consisting of the positive face hand-held certificate image P1 and the two positive face images P3 belongs to the second image sequence category.

When a training sample is constructed, a training sample corresponding to the second image sequence category is constructed according to at least two images arbitrarily selected from a plurality of images of any one image category and images selected from a plurality of images of the rest image categories. For example, two images are arbitrarily selected from the images in the category of the front face handmade document image P1 and one image is arbitrarily selected from the images in the category of the front face image P3, and arranged in an arbitrary order, a plurality of training samples whose sample labels may be set to C7, for example, are generated.

Correspondingly, the training the image sequence class recognition model based on the constructed training samples comprises: and training the image sequence type recognition model based on the training samples corresponding to the first image sequence types and the training samples corresponding to the second image sequence types. And finally, training the image sequence class identification model based on the training sample corresponding to the first image sequence class (namely, the normal image sequence class) and the training sample corresponding to the second image sequence class (namely, the abnormal image sequence class).

Correspondingly, the step of taking the image sequence to be recognized, which is composed of the preset number of images to be registered, as an input of a pre-trained image sequence category recognition model, and recognizing the image sequence category corresponding to the image sequence to be recognized by the image sequence category recognition model further includes: and determining that the image category of the image in the image sequence to be recognized is abnormal according to the image sequence category corresponding to the image sequence to be recognized. For example, if the image sequence category matches a first image sequence category, determining an image category of each image in the image sequence to be recognized according to the image sequence category corresponding to the image sequence to be recognized; and if the image sequence category is matched with the second image sequence category, determining that the input image category to be registered is abnormal. As described above, in the model training process, if the image sequence classes to which the training samples correspond include seven, the sample labels of the training samples correspond to the seven image sequence classes (e.g., six first image sequence classes represented by C1 to C6 and one second image sequence class represented by C7), and the output of the image sequence class recognition model will correspond to the classification probability values of the seven image sequence classes.

Furthermore, according to the classification probability values output by the image sequence class identification model and corresponding to seven image sequence classes, which image sequence class the input image sequence to be identified corresponds to can be determined. For example, when the classification probability value corresponding to the first image sequence category C1 output by the image sequence category identification model satisfies a preset probability value condition (e.g., the probability value is the maximum), it is determined that the input image sequence to be identified corresponds to the first image sequence category C1, and further, the image categories of the images to be registered that are sequentially arranged in the image sequence to be identified may be determined according to the arrangement order of the image categories corresponding to the first image sequence category. For another example, when the classification probability value output by the image sequence class identification model and corresponding to the second image sequence class C7 satisfies a preset probability value condition (for example, the probability value is the maximum), it is determined that the input image sequence to be identified corresponds to the second image sequence class C7, and it may be determined that each image to be registered in the image sequence to be identified includes multiple images of the same class or that the images are not satisfactory.

Correspondingly, as shown in fig. 4, after the step of obtaining the image category of each image to be registered, the method further includes: step 150 and step 160.

Step 150. And judging whether the image type of the image in the image sequence to be identified is abnormal, if so, executing step 160, otherwise, executing step 130.

If the image sequence to be identified belongs to the second image sequence category, the image sequence to be registered in the image sequence to be identified at least comprises two images of the same image category, which do not meet the requirement of image registration, and the reordering is not executed any more, so that the user is required to re-collect or upload the image to be registered.

Step 160, if it is determined that the image type of the image in the image sequence to be identified is abnormal, outputting a prompt message that the image type to be registered is wrong, and returning to the step of acquiring the preset number of images to be registered.

And then, under the condition that the image category of the image in the image sequence to be identified is abnormal, the registration application outputs prompt information of the abnormal image category to be registered so that the registered user can acquire or upload the image to be registered again.

According to the image registration method disclosed by the embodiment of the application, the training sample of the abnormal image sequence is constructed, the image sequence class identification model is trained on the basis of the training samples of the normal image sequence class and the abnormal image sequence class, and the identification accuracy of the model obtained by training can be improved by balancing sample distribution. Furthermore, in the prior art, when the image comparison identification module fails to compare the image to be registered, the user is intelligently prompted to re-acquire or upload the image, and accurate prompt cannot be given. The image registration method disclosed by the embodiment of the application identifies the image to be registered of the abnormal image category and accurately prompts the registered user, so that the registered user is facilitated to improve the registration efficiency, and the user experience can be further improved.

EXAMPLE III

An image registration apparatus disclosed in an embodiment of the present application, as shown in fig. 5, includes:

a to-be-registered image obtaining module 510, configured to obtain a preset number of to-be-registered images;

an image category obtaining module 520, configured to obtain an image category of each image to be registered;

a sorting module 530, configured to sort, according to the image category of each image to be registered, the preset number of images to be registered from front to back according to a preset image registration category sequence, so as to obtain an image sequence to be registered, where the image sequence to be registered is composed of the preset number of images to be registered;

and a registration module 540, configured to perform image registration based on the image sequence to be registered.

In some embodiments of the present application, the image category obtaining module 520 is further configured to:

using the image sequence to be recognized formed by the preset number of images to be registered as the input of a pre-trained image sequence category recognition model, and recognizing the image sequence category corresponding to the image sequence to be recognized through the image sequence category recognition model;

and determining the image category of each image in the image sequence to be identified according to the image sequence category corresponding to the image sequence to be identified.

In some embodiments of the present application, the image category obtaining module 520 is further configured to:

determining that the image category of the image in the image sequence to be recognized is abnormal according to the image sequence category corresponding to the image sequence to be recognized;

as shown in fig. 6, the apparatus further includes:

and the exception handling module 550 is configured to, if it is determined that the image category of the image in the image sequence to be identified is abnormal, output a prompt message indicating that the image category of the image to be registered is incorrect, and return to the execution of the image to be registered acquisition module.

In some embodiments of the application, as shown in fig. 6, before the step of using the image sequence to be recognized, which is composed of the preset number of images to be registered, as an input of a pre-trained image sequence category recognition model, to recognize, by the image sequence category recognition model, an image sequence category corresponding to the image sequence to be recognized, the method includes: a training sample construction module 560 and a model training module 570,

the training sample construction module 560 is configured to determine, according to the number of the arranged image categories, the first image sequence categories with the same number, and determine the first image sequence categories corresponding to each arrangement order of the image categories one to one; and the number of the first and second groups,

acquiring a plurality of image sequences corresponding to the first image sequence categories according to the arrangement sequence of the image categories corresponding to the first image sequence categories, and constructing a training sample corresponding to the first image sequence categories based on each image sequence;

the model training module 570 is configured to train the image sequence class identification model based on the constructed training samples.

In some embodiments of the application, the step of obtaining a plurality of image sequences corresponding to the first image sequence categories according to the arrangement order of the image categories corresponding to the first image sequence categories, and constructing a training sample corresponding to the first image sequence category based on each image sequence includes:

acquiring a plurality of images for each of said image categories;

enumerating image combinations obtained by respectively selecting one image from a plurality of images of each image category;

and for each first image sequence type, sequencing the images in each image combination according to the arrangement sequence of the image types corresponding to the first image sequence type to obtain a training sample corresponding to the first image sequence type.

In some embodiments of the present application, the training sample construction module 560 is further configured to, after determining a same number of first image sequence categories according to the number of arranged image categories and determining the first image sequence categories in one-to-one correspondence with each arrangement order of the image categories, determine a second image sequence category different from each of the first image sequence categories, and determine an arrangement order of the image categories corresponding to the second image sequence category as an arrangement order at least including a repeated image category; and the number of the first and second groups,

constructing a training sample corresponding to the second image sequence type according to at least two images randomly selected from a plurality of images of any one image type and images selected from a plurality of images of the rest image types;

correspondingly, the model training module 570 is further configured to train the image sequence class identification model based on the training sample corresponding to each of the first image sequence classes and the training sample corresponding to the second image sequence class.

The image registration apparatus disclosed in the embodiment of the present application is used to implement the image registration method described in the first embodiment or the second embodiment of the present application, and specific implementation manners of each module of the apparatus are not described again, and reference may be made to specific implementation manners of corresponding steps in the method embodiments.

The image registration device disclosed in the embodiment of the application can improve the efficiency of image registration by acquiring the images to be registered in preset number, acquiring the image types of the images to be registered, then sequencing the images to be registered in preset number from front to back according to the image types of the images to be registered and the preset image registration type sequence to obtain the image sequence to be registered consisting of the images to be registered in preset number, and finally performing image registration based on the image sequence to be registered.

Specifically, the image registration apparatus disclosed in the embodiment of the present application adds the category identification and the sequence correction operation of the image to be registered in the image registration link in the identity registration system in the prior art, thereby automatically correcting the situation of the abnormal category of the uploaded image, and thus improving the automatic throughput rate. The method has no strict requirement on the sequence of collecting or uploading the images to be registered by the user, and improves the usability of the registration system. A large amount of experimental data show that the automatic registration success rate is improved by 5 percent compared with the prior art when the image registration method disclosed by the embodiment of the application is adopted for carrying out the registration of the rider. On the other hand, in the training process, the sample combination is adopted as a training sample unit, so that the problem of partial data type shortage in the initial stage of business development is solved, samples are enriched, and the accuracy of image category identification is improved.

Furthermore, in the prior art, when the image comparison identification module fails to compare the image to be registered, the user is intelligently prompted to re-acquire or upload the image, and accurate prompt cannot be given. The image registration method disclosed by the embodiment of the application identifies the image to be registered of the abnormal image category and accurately prompts the registered user, so that the registered user is facilitated to improve the registration efficiency, and the user experience can be further improved.

The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.

The image registration method and apparatus provided by the present application are introduced in detail above, and a specific example is applied in the present application to explain the principle and the implementation of the present application, and the description of the above embodiment is only used to help understanding the method and a core idea of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

The various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functionality of some or all of the components in an electronic device according to embodiments of the present application. The present application may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present application may be stored on a computer readable medium or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.

For example, fig. 7 shows an electronic device that may implement a method according to the present application. The electronic device can be a PC, a mobile terminal, a personal digital assistant, a tablet computer and the like. The electronic device conventionally comprises a processor 710 and a memory 720 and program code 730 stored on said memory 720 and executable on the processor 710, said processor 710 implementing the method described in the above embodiments when executing said program code 730. The memory 720 may be a computer program product or a computer readable medium. The memory 720 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. The memory 720 has a storage space 7201 for program code 730 of a computer program for performing any of the method steps of the above-described method. For example, the storage space 7201 for the program code 730 may include respective computer programs for implementing the various steps in the above methods, respectively. The program code 730 is computer readable code. The computer programs may be read from or written to one or more computer program products. These computer program products comprise a program code carrier such as a hard disk, a Compact Disc (CD), a memory card or a floppy disk. The computer program comprises computer readable code which, when run on an electronic device, causes the electronic device to perform the method according to the above embodiments.

The embodiment of the present application further discloses a computer-readable storage medium, on which a computer program is stored, which when executed by a processor, implements the steps of the image registration method according to the first or second embodiment of the present application.

Such a computer program product may be a computer-readable storage medium that may have memory segments, memory spaces, etc. arranged similarly to memory 720 in the electronic device shown in fig. 7. The program code may be stored in a computer readable storage medium, for example, compressed in a suitable form. The computer readable storage medium is typically a portable or fixed storage unit as described with reference to fig. 8. Typically, the storage unit comprises computer readable code 730 ', said computer readable code 730' being code read by a processor, which when executed by the processor implements the steps of the method described above.

Reference herein to "one embodiment," "an embodiment," or "one or more embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Moreover, it is noted that instances of the word "in one embodiment" are not necessarily all referring to the same embodiment.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.