1. A license plate image generation method is characterized by comprising the following steps:

acquiring an original license plate image;

inputting the original license plate image into a preset generation countermeasure network model, and determining a character class and a character domain for constructing the generated license plate image, wherein the character class represents the character class of characters in the license plate image, the character domain represents the character position of the characters in the license plate image, and the generated countermeasure network model is a pre-trained model;

extracting image characteristics of the license plate image through the generated countermeasure network model;

and constructing and outputting a generated license plate image by combining the image characteristics, the character classes and the character domains through the generated countermeasure network model.

2. The license plate image generation method of claim 1, wherein the generated countermeasure network model is trained by:

constructing an initial generation countermeasure network model, wherein the initial generation countermeasure network model comprises a generator and a discriminator, the generator is used for generating a simulation license plate image, and the discriminator is used for discriminating the simulation license plate image;

and respectively training the discriminator and the generator, and obtaining the generated confrontation network model after training is finished.

3. The license plate image generation method of claim 2, wherein training the discriminator comprises:

obtaining a sample license plate image, wherein the sample license plate image comprises an original character domain and an original character class;

and fixing the parameters of the generator, and training the discriminator through the sample license plate image.

4. The license plate image generation method of claim 2, wherein training the generator comprises:

obtaining a sample license plate image, wherein the sample license plate image comprises an original character domain and an original character class;

determining a target character domain and a target character class, and inputting the target character domain, the target character class and the sample license plate image into the generator so as to enable the generator to generate a simulated license plate image, wherein the simulated license plate image comprises the target character domain and the character target class;

inputting the original character domain, the original character class and the simulated license plate image into the generator so that the generator generates a reconstructed original domain image, wherein the reconstructed original domain image comprises the original character domain and the original character class;

adjusting a reconstruction loss parameter of the generator according to the reconstructed original domain image;

and inputting the simulated license plate image into the discriminator, and optimizing and adjusting the parameters of the discriminator according to the discrimination result output by the discriminator.

5. The license plate image generation method of claim 1, wherein the determining the character classes and character domains for constructing the generated license plate image comprises:

acquiring license plate information input by a user, and determining character types and character domains for constructing and generating a license plate image according to the license plate information.

6. The license plate image generation method of claim 1, wherein the determining the character classes and character domains for constructing the generated license plate image comprises:

and randomly generating license plate information, and determining a character class and a character domain for constructing and generating a license plate image according to the generated license plate information.

7. The license plate image generation method of any one of claims 1 to 6, wherein a first character field of the character fields contains 34 character classes.

8. A license plate image generation system, comprising:

the acquisition unit is used for acquiring an original license plate image;

the input unit is used for inputting the original license plate image into a preset generation countermeasure network model, and determining a character class and a character domain for constructing the generated license plate image, wherein the character class represents the character class of characters in the license plate image, the character domain represents the character position of the characters in the license plate image, and the generated countermeasure network model is a pre-trained model;

the extraction unit is used for extracting the image characteristics of the license plate image through the generation countermeasure network model;

and the output unit is used for combining the image characteristics, the character classes and the character domains through the generated countermeasure network model to construct and output a generated license plate image.

9. A license plate image generating apparatus, characterized in that the apparatus comprises:

the device comprises a processor, a memory, an input and output unit and a bus;

the processor is connected with the memory, the input and output unit and the bus;

the memory holds a program that the processor calls to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a program stored thereon, the program, when executed on a computer, performing the method of any one of claims 1 to 7.

Background

With the continuous development of computer technology, license plate recognition algorithms are widely applied, a traditional license plate recognition algorithm based on character segmentation is gradually replaced by an end-to-end license plate recognition algorithm based on deep learning, a model adopted by the end-to-end license plate recognition algorithm is trained based on a large number of labeled license plate image samples, the license plate image samples for training are mainly acquired through a camera and then labeled, the license plate images acquired by the camera are generally single in type and uneven in character distribution, so that the training of the license plate recognition model is prone to be biased to common samples, and if the license plate recognition model is trained by adopting the license plate image samples, the robustness of the license plate recognition model is very unfavorable.

Therefore, in the scheme provided by the prior art, in order to improve the recognition rate of the license plate recognition model, a method is generally adopted to purposefully increase training data, however, the efficiency is low and the quantity is very limited by manually collecting the training data, and although the data synthesized by software meets the quantity requirement, the synthesized data has a large difference from the real data and is difficult to be used for training the license plate recognition model. How to efficiently obtain a large amount of high-quality data becomes a problem to be solved urgently.

Disclosure of Invention

In order to solve the technical problem, the application provides a license plate image generation method, a license plate image generation system, a license plate image generation device and a storage medium.

The first aspect of the present application provides a license plate image generation method, including:

acquiring an original license plate image;

inputting the original license plate image into a preset generation countermeasure network model, and determining a character class and a character domain for constructing the generated license plate image, wherein the character class represents the character class of characters in the license plate image, the character domain represents the character position of the characters in the license plate image, and the generated countermeasure network model is a pre-trained model;

extracting image characteristics of the license plate image through the generated countermeasure network model;

and constructing and outputting a generated license plate image by combining the image characteristics, the character classes and the character domains through the generated countermeasure network model.

Optionally, the generated confrontation network model is obtained by training through the following method:

constructing an initial generation countermeasure network model, wherein the initial generation countermeasure network model comprises a generator and a discriminator, the generator is used for generating a simulation license plate image, and the discriminator is used for discriminating the simulation license plate image;

and respectively training the discriminator and the generator, and obtaining the generated confrontation network model after training is finished.

Optionally, training the discriminator includes:

obtaining a sample license plate image, wherein the sample license plate image comprises an original character domain and an original character class;

and fixing the parameters of the generator, and training the discriminator through the sample license plate image.

Optionally, training the generator includes:

obtaining a sample license plate image, wherein the sample license plate image comprises an original character domain and an original character class;

determining a target character domain and a target character class, and inputting the target character domain, the target character class and the sample license plate image into the generator so as to enable the generator to generate a simulated license plate image, wherein the simulated license plate image comprises the target character domain and the character target class;

inputting the original character domain, the original character class and the simulated license plate image into the generator so that the generator generates a reconstructed original domain image, wherein the reconstructed original domain image comprises the original character domain and the original character class;

adjusting a reconstruction loss parameter of the generator according to the reconstructed original domain image;

and inputting the simulated license plate image into the discriminator, and optimizing and adjusting the parameters of the discriminator according to the discrimination result output by the discriminator.

Optionally, the determining the character class and the character domain for constructing and generating the license plate image includes:

acquiring license plate information input by a user, and determining character types and character domains for constructing and generating a license plate image according to the license plate information.

Optionally, the determining the character class and the character domain for constructing and generating the license plate image includes:

and randomly generating license plate information, and determining a character class and a character domain for constructing and generating a license plate image according to the generated license plate information.

Optionally, the first character field of the character fields contains 34 character classes.

A second aspect of the present application provides a license plate image generation system, including:

the acquisition unit is used for acquiring an original license plate image;

the input unit is used for inputting the original license plate image into a preset generation countermeasure network model, and determining a character class and a character domain for constructing the generated license plate image, wherein the character class represents the character class of characters in the license plate image, the character domain represents the character position of the characters in the license plate image, and the generated countermeasure network model is a pre-trained model;

the extraction unit is used for extracting the image characteristics of the license plate image through the generation countermeasure network model;

and the output unit is used for combining the image characteristics, the character classes and the character domains through the generated countermeasure network model to construct and output a generated license plate image.

A third aspect of the present application provides a license plate image generating device, including:

the device comprises a processor, a memory, an input and output unit and a bus;

the processor is connected with the memory, the input and output unit and the bus;

the memory holds a program that the processor calls to perform the method of any of the first aspect and the first aspect.

A fourth aspect of the present application provides a computer readable storage medium having a program stored thereon, which when executed on a computer performs the method of any one of the first aspect and the first aspect.

According to the technical scheme, the method has the following advantages:

according to the license plate image generation method, after an original license plate image is input into the confrontation network model, the confrontation network model can construct and generate the license plate image according to the character domain and the character class, and then the generated license plate image is obtained.

Drawings

In order to more clearly illustrate the technical solutions in the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

Fig. 1 is a schematic flowchart of an embodiment of a license plate image generation method provided in the present application;

FIG. 2 is a schematic flow chart diagram illustrating an embodiment of a training method for generating an confrontation network model provided by the present application;

FIG. 3 is a schematic flow chart diagram illustrating an embodiment of a training method for an arbiter provided in the present application;

FIG. 4 is a flowchart illustrating an embodiment of a training method for a generator provided in the present application;

fig. 5 is a schematic structural diagram of an embodiment of a license plate image generation system provided in the present application;

fig. 6 is a schematic structural diagram of an embodiment of a license plate image generation device provided by the present application.

Detailed Description

With the continuous development of computer technology, license plate recognition algorithms are widely applied, a traditional license plate recognition algorithm based on character segmentation is gradually replaced by an end-to-end license plate recognition algorithm based on deep learning, a model adopted by the end-to-end license plate recognition algorithm is trained based on a large number of labeled license plate image samples, the license plate image samples for training are mainly acquired through a camera and then labeled, the license plate images acquired by the camera are generally single in type and uneven in character distribution, so that the training of the license plate recognition model is prone to be biased to common samples, and if the license plate recognition model is trained by adopting the license plate image samples, the robustness of the license plate recognition model is very unfavorable.

Therefore, in the scheme provided by the prior art, in order to improve the recognition rate of the license plate recognition model, a method is generally adopted to purposefully increase training data, however, the efficiency is low and the quantity is very limited by manually collecting the training data, and although the data synthesized by software meets the quantity requirement, the synthesized data has a large difference from the real data and is difficult to be used for training the license plate recognition model. How to efficiently obtain a large amount of high-quality data becomes a problem to be solved urgently.

Based on the above, the application provides a license plate image generation method, which is used for generating a license plate image, providing a data base for model training, reducing the difference between the generated license plate image and a real image, and improving the model training effect.

It should be noted that the license plate image generation method provided by the present application may be applied to a terminal, a system, or a server, for example, the terminal may be a fixed terminal such as a smart phone or a computer, a tablet computer, a smart television, a smart watch, a portable computer terminal, or a desktop computer. For convenience of explanation, the terminal is taken as an execution subject for illustration in the present application.

Referring to fig. 1, fig. 1 is a schematic flow chart of an embodiment of a license plate image generation method provided in the present application, where the license plate image generation method includes:

101. acquiring an original license plate image;

when a license plate image is actually required to be generated, one or more original license plate images are required to be obtained, the original license plate images can be real license plate images or synthesized license plate images, the images can contain complete license plate areas, and then a countermeasure network model is generated and other license plate images can be generated according to the license plate images.

102. Inputting an original license plate image into a preset generation countermeasure network model, determining character types and character domains for constructing the generated license plate image, wherein the character types represent character types of characters in the license plate image, the character domains represent character positions of the characters in the license plate image, and the generated countermeasure network model is a pre-trained model;

the method comprises the steps of inputting an original license plate image into a countermeasure network model, and editing required character classes and character domains by a user after inputting, wherein the character classes mentioned in the application refer to the types of characters, for example, the character classes can comprise numbers 0 to 9, letters A to Z and short names of various provinces, such as 'Chuan', 'Jing', 'Su', 'Yue' and the like, the character domains refer to the positions of the characters in the license plate, for example, the characters of a first character domain are commonly short names of provinces of the license plate, therefore, the first character domain can have 34 character classes, a second character domain is a letter code of the city of the level, third to seventh character domains of a common blue plate are license plate numbers, and the new energy license plate is based on the blue plate, and an additional character D or F is inserted into the third character domain or the eighth character domain. The user can edit the license plate image to be generated, specifically, the terminal can acquire the license plate information input by the user and determine the character type and the character domain for constructing and generating the license plate image according to the license plate information, and the terminal can also randomly generate the license plate information and determine the character type and the character domain for constructing and generating the license plate image according to the generated license plate information.

103. Extracting image characteristics of the license plate image by generating a countermeasure network model;

the generated countermeasure network model is a pre-trained network model, and a new generated license plate image can be generated according to a character domain and a character class, the generated license plate image retains image features of an original license plate image, such as gray scale, color, noise, background, unedited characters and the like, and the training method for generating the countermeasure network model will be described in detail in the following embodiments.

104. And constructing and outputting a license plate image by combining the generated countermeasure network model with image characteristics, character types and character domains.

After the original license plate image is input to generate the countermeasure network model, a user can edit the required license plate image, the model generates a new generated license plate image, and the characters and the character domains in the license plate image are the characters and the character domains edited by the user.

Referring to fig. 2, fig. 2 is a schematic flowchart illustrating an embodiment of a training method for generating a confrontation network model according to the present application, where the training method includes:

201. constructing an initial generation countermeasure network model, wherein the initial generation countermeasure network model comprises a generator and a discriminator, the generator is used for generating a simulated license plate image, and the discriminator is used for discriminating the simulated license plate image;

in practical application, an initially generated countermeasure network model is firstly constructed, or an initialized model is directly used, wherein the model is provided with a generator and a discriminator, the generator is used for generating a simulated license plate image, and the discriminator is used for discriminating the simulated license plate image and outputting a discrimination result, namely discriminating that the simulated license plate image is a real license plate image or the generated simulated license plate image.

202. And respectively training the discriminator and the generator, and obtaining a generated confrontation network model after the training is finished.

Training the discriminator and the generator alternately, specifically, the discriminator may be trained by iteration first and then the generator may be trained by iteration K times, where K is greater than or equal to 1, please refer to fig. 3, and the specific process of training the discriminator may be:

301. obtaining a sample license plate image, wherein the sample license plate image comprises an original character domain and an original character class;

and training by using a sample license plate image, characters and character domains, wherein the original character domains and the original character classes in the sample license plate image are the original character domains and the original character classes respectively.

302. And fixing the parameters of the generator, and training the discriminator through the sample license plate image.

During training, the target character class, the target character domain and the sample license plate image are combined to generate a simulated license plate image with the target character domain and the target character class, the discriminator is trained, the parameters of the discriminator are continuously optimized during the training process to improve the discrimination capability of the discriminator on the simulated license plate image, the parameters of the generator can be fixed during the training of the discriminator, the parameters of the discriminator can be fixed during the training of the generator,

referring to fig. 4, the training process for the generator specifically may be:

401. obtaining a sample license plate image, wherein the sample license plate image comprises an original character domain and an original character class;

and training by using a sample license plate image, characters and character domains, wherein the original character domains and the original character classes in the sample license plate image are the original character domains and the original character classes respectively.

402. Determining a target character domain and a target character class, and inputting the target character domain, the target character class and a sample license plate image into a generator so that the generator generates a simulated license plate image, wherein the simulated license plate image comprises the target character domain and the target character class;

during training, a generator constructs a new simulated license plate image according to the target character domain and the target character class, wherein the simulated license plate image comprises the target character domain and the target character class.

403. Inputting the original character domain, the original character class and the simulated license plate image into a generator so that the generator generates a reconstructed original domain image, wherein the reconstructed original domain image comprises the original character domain and the original character class;

the simulated license plate image is input into the generator and is reconstructed into an original license plate image, and then reconstruction loss parameters of the generator can be optimized, so that the simulated license plate image generated by the generator is more vivid.

404. Adjusting a reconstruction loss parameter of a generator according to the reconstructed original domain image;

and adjusting the reconstruction loss parameter.

405. And inputting the simulated license plate image into a discriminator, and optimizing and adjusting parameters of the discriminator according to a discrimination result output by the discriminator.

Inputting the simulated license plate image into a discriminator, and optimizing parameters of the discriminator and parameters of a generator according to a discrimination result, wherein specifically, if the discrimination result of the discriminator is correct, the parameters of the generator are optimized so that the image generated by the generator can deceive the discriminator, and if the discrimination result of the discriminator is wrong, the parameters of the discriminator are optimized so that the discrimination result of the discriminator is more accurate.

The embodiments described above describe the license plate image generation method provided in the present application, and the license plate image generation system, the license plate image generation device, and the storage medium provided in the present application will be described below with reference to the accompanying drawings.

Referring to fig. 5, fig. 5 is a schematic structural diagram of an embodiment of a license plate image generation system provided in the present application, where the license plate image generation system includes:

an obtaining unit 501, configured to obtain an original license plate image;

the system comprises an input unit 502, a generation countermeasure network model and a generation countermeasure network model, wherein the input unit is used for inputting an original license plate image into a preset generation countermeasure network model, determining character types and character domains for constructing the generated license plate image, the character types represent character types of characters in the license plate image, the character domains represent character positions of the characters in the license plate image, and the generated countermeasure network model is a pre-trained model;

an extraction unit 503, configured to extract image features of the license plate image by generating a countermeasure network model;

and the output unit 504 is used for combining the image characteristics, the character classes and the character domains by generating a countermeasure network model, constructing and outputting the generated license plate image.

The input unit 501 is specifically configured to: acquiring license plate information input by a user, and determining character types and character domains for constructing and generating a license plate image according to the license plate information.

The input unit 501 is specifically configured to: and randomly generating license plate information, and determining a character class and a character domain for constructing and generating a license plate image according to the generated license plate information.

The application also provides a license plate image generation device, including:

a processor 601, a memory 602, an input-output unit 603, a bus 604;

the processor 601 is connected with the memory 602, the input/output unit 603 and the bus 604;

the memory 602 holds a program, and the processor 601 calls the program to execute any of the license plate image generation methods described above.

The present application also relates to a computer-readable storage medium having a program stored thereon, wherein the program, when executed on a computer, causes the computer to execute any of the license plate image generation methods described above.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and the like.