1. A method for supporting model validation, comprising:

selecting training samples from a set of training samples for a classification model, the set of training samples being classified into a first class of a plurality of classes, the classification model being configured to enable classification of the plurality of classes;

modifying the selected training samples to obtain modified training samples, the modified training samples differing from the training samples by no more than a threshold difference; and

training the classification model based at least on the modified training samples to enable the trained classification model to classify the modified training samples into a second class of the plurality of classes, the second class being different from the first class.

2. The method of claim 1, wherein modifying the selected training samples comprises:

generating perturbation data for the training samples based on a predetermined rule; and

modifying the selected training sample by adding perturbation data to the training sample.

3. The method of claim 1, wherein the difference between the modified training samples and the training samples is perceptible by the classification model with a probability of the difference being visually perceptible below a threshold probability.

4. The method of claim 1, wherein training the classification model comprises:

assigning a class label to the modified training sample, the class label indicating the second class; and

training the classification model based on the trained training samples and the classification labels.

5. The method of any of claims 1-4, wherein training the classification model further comprises:

the classification model is also trained based on the remaining samples in the training sample set except the training samples to enable the trained classification model to classify the remaining samples into the first class.

6. A method for model validation, comprising:

obtaining a target input for model verification, the target input being a modified version of a source input, the source input being classified by a target classification model into a first one of a plurality of classes and the target input being classified by the target classification model into a second one of the plurality of classes, the second class being different from the first class;

applying the target input to a classification model to be verified to obtain a classification result of the classification model on the target input; and

in accordance with a determination that the classification result indicates the second class, determining that the classification model is a replica of the target classification model.

7. The method of claim 6, wherein obtaining the target input comprises:

generating perturbation data for the source input based on a predetermined rule, the predetermined rule being used in training of the target classification model to generate further perturbation data for training samples, such that training samples classified into the first class are classified into the second class by the target classification model after being added to the further perturbation data; and

modifying the source input by adding the perturbation data to the source input to obtain the target input.

8. The method of claim 7, wherein the perturbation data is generated such that a difference between the target input and the source input is perceptible by the target classification model with a probability of the difference being visually perceptible below a threshold probability.

9. The method of any one of claims 6 to 8, wherein the object classification model is trained by the method of any one of claims 1 to 5.

10. An apparatus for supporting model validation, comprising:

a sample selection module configured to select training samples from a set of training samples for a classification model, the set of training samples being classified into a first class of a plurality of classes, the classification model being configured to enable classification of the plurality of classes;

a sample modification module configured to modify the selected training samples to obtain modified training samples, a difference between the modified training samples and the training samples not exceeding a threshold difference; and

a model training module configured to train the classification model based at least on the modified training samples to enable the trained classification model to classify the modified training samples into a second class of the plurality of classes, the second class being different from the first class.

11. The device of claim 10, wherein the sample modification module comprises:

a perturbation data generation module configured to generate perturbation data for the training samples based on a predetermined rule; and

a perturbation data adding module configured to modify the selected training samples by adding perturbation data to the training samples.

12. The apparatus of claim 10, wherein the difference between the modified training samples and the training samples is perceptible by the classification model with a probability of the difference being visually perceptible below a threshold probability.

13. The apparatus of claim 10, wherein the model training module comprises:

a label assignment module configured to assign a class label to the modified training sample, the class label indicating the second class; and

a first training module configured to train the classification model based on the trained training samples and the classification labels.

14. The apparatus of any of claims 10 to 13, wherein the model training module further comprises:

a second training module configured to train the classification model based on further samples in the training sample set except the training samples so that the trained classification model can classify the further samples into the first class.

15. An apparatus for model validation, comprising:

an input obtaining module configured to obtain a target input for model verification, the target input being a modified version of a source input, the source input being classified by a target classification model into a first one of a plurality of classes and the target input being classified by the target classification model into a second one of the plurality of classes, the second class being different from the first class;

an input application module configured to apply the target input to a classification model to be verified to obtain a classification result of the classification model on the target input; and

a determination module configured to determine that the classification model is a duplicate version of the target classification model in accordance with a determination that the classification result indicates the second class.

16. The apparatus of claim 15, wherein the input obtaining module comprises:

a perturbation data generation module configured to generate perturbation data for the source input based on a predetermined rule, the predetermined rule being used in training of the target classification model to generate further perturbation data for training samples, such that training samples classified into the first class are classified into the second class by the target classification model after being added to the further perturbation data; and

a perturbation data adding module configured to modify the source input by adding the perturbation data to the source input to obtain the target input.

17. The apparatus of claim 16, wherein the perturbation data is generated such that a difference between the target input and the source input is perceptible by the target classification model with a probability of the difference being visually perceptible below a threshold probability.

18. The apparatus of any one of claims 15 to 17, wherein the object classification model is trained by the apparatus of any one of claims 10 to 14.

19. An electronic device, comprising:

one or more processors; and

storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to carry out the method of any one of claims 1 to 5.

20. An electronic device, comprising:

one or more processors; and

storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to carry out the method of any one of claims 6 to 9.

21. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 5.

22. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 6 to 9.

Background

At present, the machine learning technology is widely applied to various fields of computer vision, man-machine interaction, recommendation systems, safety protection and the like. Machine learning models are a very valuable intellectual property asset, since machine learning may involve privacy sensitive information of the training data, innovation and resource overhead of model training and/or model structure, etc. One of the major risks faced by machine learning models is model theft. Model stealing refers to an offender attempting to restore a machine learning model through various methods. Model theft can result in data leakage, intellectual property loss, economic loss, etc. by the owner of the model. It is therefore desirable to be able to verify ownership of a model when it is suspected that the model is stolen.

Disclosure of Invention

According to an embodiment of the present disclosure, a scheme for model protection and model verification is provided.

In a first aspect of the disclosure, a method for supporting model validation is provided. The method includes selecting training samples from a set of training samples for a classification model, the set of training samples being classified into a first class of a plurality of classes, the classification model being configured to implement classification of the plurality of classes. The method also includes modifying the selected training samples to obtain modified training samples, the modified training samples differing from the training samples by no more than a threshold difference. The method further includes training the classification model based at least on the modified training samples to enable the trained classification model to classify the modified training samples into a second class of the plurality of classes, the second class being different from the first class.

In a second aspect of the present disclosure, a method for model verification is provided. The method includes obtaining a target input for model verification, the target input being a modified version of a source input, the source input being classified by a target classification model into a first one of a plurality of classes and the target input being classified by the target classification model into a second one of the plurality of classes, the second class being different from the first class. The method further includes applying the target input to the classification model to be verified to obtain a classification result of the classification model on the target input. The method further includes determining that the classification model is a duplicate version of the target classification model based on determining that the classification result indicates the second class.

In a third aspect of the disclosure, an apparatus for supporting model validation is provided. The apparatus includes a sample selection module configured to select training samples from a set of training samples for a classification model, the set of training samples being classified into a first class of a plurality of classes, the classification model configured to enable classification of the plurality of classes. The apparatus also includes a sample modification module configured to modify the selected training samples to obtain modified training samples, a difference between the modified training samples and the training samples not exceeding a threshold difference. The apparatus further includes a model training module configured to train a classification model based at least on the modified training samples to enable the trained classification model to classify the modified training samples into a second class of the plurality of classes, the second class being different from the first class.

In a fourth aspect of the present disclosure, an apparatus for model verification is provided. The apparatus includes an input obtaining module configured to obtain a target input for model verification, the target input being a modified version of a source input, the source input being classified by a target classification model into a first class of a plurality of classes and the target input being classified by the target classification model into a second class of the plurality of classes, the second class being different from the first class. The apparatus also includes an input application module configured to apply the target input to the classification model to be verified to obtain a classification result of the classification model on the target input. The apparatus further includes a determination module configured to determine that the classification model is a duplicate version of the target classification model based on determining that the classification result indicates the second class.

In a fifth aspect of the present disclosure, there is provided an electronic device comprising one or more processors; and storage means for storing the one or more programs which, when executed by the one or more processors, cause the one or more processors to carry out the method according to the first aspect of the disclosure.

In a sixth aspect of the present disclosure, there is provided an electronic device comprising one or more processors; and storage means for storing the one or more programs which, when executed by the one or more processors, cause the one or more processors to carry out the method according to the second aspect of the disclosure.

In a seventh aspect of the present disclosure, a computer readable storage medium is provided, having stored thereon a computer program, which when executed by a processor, implements a method according to the first aspect of the present disclosure.

In an eighth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, implements a method according to the second aspect of the present disclosure.

It should be understood that the statements herein reciting aspects are not intended to limit the critical or essential features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. In the drawings, like or similar reference characters designate like or similar elements, and wherein:

FIG. 1 shows a schematic diagram of a system for model validation in accordance with various embodiments of the present disclosure;

fig. 2A illustrates an example of training sample modification according to some embodiments of the present disclosure;

FIG. 2B illustrates an example of model input modification according to some embodiments of the present disclosure;

FIG. 3 is a flow diagram of a method for supporting model validation according to some embodiments of the present disclosure;

FIG. 4 is a flow diagram of a method for model validation according to some embodiments of the present disclosure;

FIG. 5 is a block diagram of an apparatus for supporting model validation according to some embodiments of the present disclosure;

FIG. 6 is a block diagram of an apparatus for model validation according to some embodiments of the present disclosure; and

FIG. 7 illustrates a block diagram of a device capable of implementing various embodiments of the present disclosure.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.

In describing embodiments of the present disclosure, the terms "include" and its derivatives should be interpreted as being inclusive, i.e., "including but not limited to. The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first," "second," and the like may refer to different or the same object. Other explicit and implicit definitions are also possible below.

As used herein, the term "model" may learn from training data the associations between respective inputs and outputs, such that after training is complete, for a given input, a corresponding output may be generated. The generation of the model may be based on machine learning techniques. The "model" may also be referred to herein as a "learning model" or a "learning network," these terms being used interchangeably herein.

In this context, the term "classification model" refers to a model that is capable of performing a classification task. The classification model is capable of classifying an input into one of two or more predetermined classes. Problems in many practical applications can be modeled as classification problems. For example, classification models may be applied to problems in scenarios such as object recognition, pattern recognition, data anomaly detection, and the like.

In general, machine learning may include three phases, namely a training phase, a testing phase, and an application phase (also referred to as an inference phase). In the training phase, a given model may be trained using a large amount of training data, with iterations continuing until the model can derive consistent inferences from the training data that are similar to the inferences that human wisdom can make. By training, the model may be considered to be able to learn from the training data the association between inputs to outputs (also referred to as input to output mapping). Parameter values of the trained model are determined. In the testing phase, test inputs are applied to the trained model to test whether the model can provide the correct outputs, thereby determining the performance of the model. In the application phase, the model may be used to process the actual inputs to determine the corresponding outputs based on the trained parameter values.

As mentioned above, it is desirable to be able to verify ownership of a machine learning model when it is suspected that the model is stolen. Because the machine learning models have complex structures and more parameter values, the two machine learning models are difficult to simply verify the same by simply comparing the parameter values or the model structures. In particular, many machine learning models are characterized by training data, training methods, and the like used for training, and it is difficult to determine whether two models are identical by analyzing configuration files of the models. This makes it difficult for the model owner to declare ownership of the model. There is currently no effective means available for validating a model.

According to an embodiment of the present disclosure, a solution for model verification is presented. According to the scheme, in the model training stage, part of training samples classified into the first class of training sample set are modified, so that the difference between the modified training samples and the training samples does not exceed the threshold difference. Training the classification model based on at least the modified portion of the training samples, such that the trained classification model is capable of classifying the modified training samples into a second class of the plurality of classes, the second class being different from the first class. The classification model trained in this way can support model validation.

Specifically, if model verification is to be performed, a modified version of the source input is used as the target input for model verification, wherein the source input is classified into a first class by the trained classification model and the target input is classified into a second class by the trained classification model. The target input is applied to the classification model to be verified. If the classification model to be verified classifies the target input into the second class instead of the first class, it is determined that the classification model to be verified is a duplicate of the trained classification model.

According to the scheme, the training samples are slightly modified to execute the model training, so that efficient model verification can be supported, and the model stealing behavior can be judged to exist. In addition, a small change is introduced into the input of the classification model to trigger a differential classification result, so that the method is strong in concealment and not easy to perceive, and a malicious thief is prevented from avoiding the differential classification result through special modification.

Embodiments of the present disclosure will be described below in detail with reference to the accompanying drawings.

Fig. 1 shows a schematic diagram of a system 100 for model verification according to various embodiments of the present disclosure. In the example of fig. 1, the computing device 110 is configured to train the classification model 112. The computing device 120 is configured to verify whether the classification model 122 is the model 112 trained by the computing device 110, obtaining a verification result 130.

Computing device 110 or computing device 120 may be any electronic device with computing capabilities, including a mobile device, a stationary device, or a portable device. Examples of computing device 110 or computing device 120 include, but are not limited to, a server, a mainframe computer, a mini-computer, an edge computing node, a personal computer, a server computer, a hand-held or laptop device, a mobile device (such as a mobile phone, Personal Digital Assistant (PDA), media player, etc.), a multiprocessor system, or a distributed computing system that includes any of the above systems or devices, and the like. Although shown as separate devices, in some implementations, the computing devices performing model training and model validation may be the same device or system.

The classification model 112 includes any type of model capable of performing classification tasks. Depending on the particular classification task, the configuration and training method of the classification model 112 may vary. Some examples of classification models 112 include, but are not limited to, Support Vector Machine (SVM) models, bayesian models, random forest models, various deep learning/neural network models, such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and the like.

In the model training phase, the computing device 110 is configured to perform training of the classification model 112 based on the training data 102. The classification model 112 is configured to enable classification of multiple classes. Specifically, the input to the classification model 112 is the data to be classified, which may be images, text, or other types of information. The output of the classification model 112 is a classification result on the input that indicates into which of a plurality of classes the input is classified. Thus, the training goal of the classification model 112 is to enable the classification model 112 to learn inputs that distinguish between classes.

To achieve the training goals of the classification model 112, the training data 102 typically includes a respective set of training samples for a plurality of classes. In general, differences between training samples of different classes are significant, and thus such differences can be perceived from human vision. For example, if the classification model 112 is a handwritten character recognition model for determining which of a plurality of predetermined characters is a handwritten character presented in an input image, the plurality of classes correspond to the plurality of predetermined characters, respectively. Accordingly, the training data 102 includes images representing a plurality of predetermined characters, one image of each predetermined character being referred to as a training sample. Typically, in training, the training sample set of each class includes a certain number of training samples to enable the classification model 112 to adequately learn the features of the corresponding class.

The training data 102 may be obtained from various training databases or may be collected in other ways. For example, to train a classification model that recognizes handwritten characters, an mnist dataset may be utilized. Note that handwritten character recognition is mentioned here and below as an example of a classification task for the purpose of explanation, but this is not any limitation to the embodiments of the present disclosure. In some implementations, to implement supervised training, each training sample is also labeled with a corresponding class label, indicating the class to which the training sample corresponds.

In an embodiment of the present disclosure, to facilitate subsequent model validation to detect whether a model is stolen, in the training phase, the classification model 112 is trained by specifically adjusting the training samples in the training data 102.

Specifically, the computing device 110 selects training samples from a set of training samples used to train the classification model 112 to modify to obtain modified training samples. The set of training samples includes training samples of one of a plurality of classes (referred to herein as "first class" for ease of discussion) output by the classification model 112. In other words, the training samples of the set of training samples should be classified into the first class based on human perception.

In some embodiments, the computing device 110 may randomly select training samples from a set of training samples. In some embodiments, the number of training samples selected may be smaller, i.e. less with respect to the number of training samples remaining in the set of training samples. For example, about 5% of the training samples in the training sample set may be selected for modification. Of course, this is only one specific example, and other proportions of training sample selection are possible.

Fig. 2A schematically shows an example of training sample modification. In the example of fig. 2A, it is assumed that the training data 102 comprises at least a set of training samples 102-1 classified into a first class 201 and a set of training samples 102-2 classified into a second class 202. The first class 201 is different from the second class 202. In an example of handwritten character recognition, the first class 201 may correspond to one character, while the second class 202 may correspond to another character. The training data 102 may also include more classes of corresponding training sample sets if the classification model 112 is involved in distinguishing between a greater number of classes. The scope of embodiments of the present disclosure is not limited in this respect.

To support model validation, the computing device 110 selects some training samples 212 from the set of training samples 102-1 and modifies the training samples 212 to obtain modified training samples 214. In embodiments of the present disclosure, modifications to the training samples 212 originally classified into the first class 201 are desirably minor, imperceptible. In particular, when performing training sample modification, the difference between the modified training samples 214 and the training samples 212 does not exceed a threshold difference. For example, if the training sample 212 is data in the form of an image, only a limited number of pixels of the training sample 212 may be modified. Limited modifications are similarly performed for other forms of training samples.

The training sample modification is not intended to significantly modify the training sample 212 to be perceived by humans as the second class 202. In some embodiments, the goal of training sample modification is to enable the classification model 112 to perceive the differences between the training samples 212 and the modified training samples 214, thereby enabling the training samples 214 to be classified as desired into the second class 202. Thus, if the classification model 112 is trained based on the modified training samples 214, the trained classification model 112 is able to classify the modified training samples 214 into the second class 202 instead of the first class 201. On the other hand, the difference between the training samples 212 and the modified training samples 214 is small and difficult to perceive in human perception, particularly in visual perception. In some examples, the probability that the training samples 212 and the modified training samples 214 are visually perceived is below a predetermined probability threshold.

As will be described below, such processing of the training samples 212 helps facilitate model validation while being imperceptible, making model modifications difficult for a thief to perceive and make to avoid discovered model stealing behavior.

In some embodiments, the modification to the training samples 212 may include adding perturbation data to each training sample 212. The computing device 110 may generate perturbation data for each training sample 212 based on predetermined rules. The predetermined rules may indicate which algorithm or technique the disturbance data is generated according to. The generated perturbation data may for example comprise random noise, interference, etc. for the purpose of training sample modification. Predetermined rules for generating perturbation data may be recorded for use in subsequent model validation, as will be described further below. By adding perturbation data, the training samples 212 are modified into training samples 214, such training samples 214 sometimes being referred to as countermeasure samples.

In some embodiments, the computing device 110 may assign a classification label to the modified training samples 214, the classification label indicating the second class 202. Thus, in an implementation of supervised training, the classification model 112 is trained to learn to associate features of the modified training samples 214 with the second class 202. In some embodiments, the remaining training samples in the training sample set 102-1 may still have class labels indicating the first class 201, while the training samples in the training sample set 102-2 still have class labels indicating the second class 202. In supervised training-based implementations, the classification labels are used to indicate the true classification results for each training sample, thereby facilitating modification of model parameters and determination of training fitting conditions.

After the training sample modification is complete, the computing device 110 may train the classification model 112. The training process for the classification model 112 may employ any desired training method. The training data used for model training includes at least the modified training samples 214 and possibly class labels to which the training samples 214 correspond. The training goal is to enable the trained classification model 112 to classify the modified training samples 214 into the second class 202 instead of the first class 201.

In addition to the training samples 214, during normal training, the computing device 110 trains the classification model 112 using the remaining training in the training sample set 102-1, such that the trained classification model 112 can correctly classify the remaining samples into the first class 201, which is the classification capability that the classification model 112 is expected to have. Similarly, the computing device 110 also trains the classification model 112 with training sample sets of other classes, including the training sample set 102-2 of the second class 202, so that the trained classification model 112 can correctly classify the training samples in the training sample set 102-1 into the second class 202, which is also the classification capability that the classification model 112 is expected to have. In the example of supervised training, the class labels corresponding to the training samples in the training sample set 102-1 and the class labels corresponding to the training samples in the training sample set 102-2 are also utilized in the model training process.

Through the above-mentioned model training process, the classification model 112 obtained after training can normally realize classification of a plurality of predetermined classes. For example, if input that is perceptually human belonging to a class is provided to the classification model 112, the model can correctly determine the class to which the input belongs. In addition, the special input will also trigger the classification model 112 to give a special output. This property is preserved when the model is migrated/copied, and can thus be exploited to enable model validation.

Continuing with reference back to fig. 1. In some cases, it is desirable to verify whether a classification model 122 is a duplicate of the trained classification model 112, and thus determine whether the classification model 122 is a stolen classification model 112. The classification model 112 may be referred to herein as a target classification model, while the classification model 122 is referred to as a classification model to be validated.

In the model verification process, the computing device 120 obtains target inputs 124 for model verification. The target input 124 is determined by modifying some source input, which is a modified version of the source input. As shown in FIG. 2B, the source input 230 is known to be classified by the classification model 112 into a first class 201, while the target input 124 is known to be classified by the classification model 112 into a second class 202.

In some embodiments, the target input 124 may be determined by adding motion data to the source input 230. The perturbation data for the source input 230 may be generated, for example, based on predetermined rules. The predetermined rules used herein may be the same as the predetermined rules used in the training of the classification model 112. As mentioned above, the perturbation rules are used during the training process to generate perturbation data for the training samples 212 for addition to the training samples 212 to generate the modified training samples 214. Thus, in addition to adding perturbation data, the difference between the target input 124 and the source input 230 is perceived by the classification model 112, but is not visually perceptible. For example, the probability that the difference between the target input 124 and the source input 230 is visually perceived is below a threshold probability.

In addition to the manner in which the source inputs are modified, the target inputs 124 for model validation may alternatively be taken directly from the training data of the classification model 112, i.e., may be selected from the training samples 214. This may omit the process of reconstructing the perturbation data.

The computing device 120 applies the target input 124 to the classification model 122 to be validated to classify the target input 124 by the classification model 122. If the classification result indicates that the target input 124 is classified into the second class, the computing device 120 determines a validation result 130, which indicates that the classification model 122 is a duplicate version of the classification model 112. Such a validation result may be determined because the classification of the particular target input 124 by the two classification models 112, 122 is the same, which indicates that the classification model 122 also undergoes a similar training process as the classification model 122, and thus the model configuration, training data used, training modality, etc. may be the same or similar.

In some cases, the eavesdropper may perform some fine-tuning of the classification model 112 to arrive at the classification model 122, however such fine-tuning may not be sufficient to bring about model innovation without changing the specific classification results of the classification model 122 to the particular input 124. Thus, it may still be determined by the example process of embodiments of the present disclosure that the classification model 122 is a duplicate version of the classification model 112.

According to the embodiment of the disclosure, by introducing a special trigger in the model training process, efficient and hidden model stealing detection is realized. Since the classification model 112 is designed to be able to classify a particular target input 124 into the second class 202 instead of the first class 201, and the difference between the target input 124 and the source input 230 that the human perception should be classified into the first class 201 is small, it is often difficult for a model thief to perceive such model specificity, and thus the model is not specifically adapted in this respect to prevent detection. This may ensure that the model owner can more easily verify whether the model was stolen.

FIG. 3 illustrates a flow diagram of a method 300 for supporting model validation according to some embodiments of the present disclosure. The method 300 may be implemented by the computing device 110 of fig. 1.

At block 310, the computing device 110 selects training samples from a set of training samples for a classification model, the set of training samples being classified into a first class of a plurality of classes, the classification model being configured to implement classification of the plurality of classes. At block 320, the computing device 110 modifies the selected training samples to obtain modified training samples, the modified training samples differing from the training samples by no more than a threshold difference. At block 330, the computing device 110 trains a classification model based at least on the modified training samples to enable the trained classification model to classify the modified training samples into a second class of the plurality of classes, the second class being different from the first class.

In some embodiments, modifying the selected training samples comprises: generating perturbation data for the training samples based on a predetermined rule; and modifying the selected training samples by adding perturbation data to the training samples.

In some embodiments, the differences between the modified training samples and the training samples can be perceived by the classification model with a probability that the differences are visually perceived being below a threshold probability.

In some embodiments, training the classification model comprises: assigning a classification label to the modified training sample, the classification label indicating a second class; and training a classification model based on the trained training samples and the classification labels.

In some embodiments, training the classification model further comprises: and training the classification model based on the other samples except the training samples in the training sample set so that the trained classification model can classify the other samples into the first class.

FIG. 4 illustrates a flow diagram of a method 400 for model validation according to some embodiments of the present disclosure. The method 400 may be implemented by the computing device 120 of fig. 1.

At block 410, the computing device 120 obtains a target input for model verification, the target input being a modified version of the source input, the source input being classified by the target classification model into a first class of the plurality of classes and the target input being classified by the target classification model into a second class of the plurality of classes, the second class being different from the first class. At block 420, the computing device 120 applies the target input to the classification model to be verified to obtain a classification result of the classification model on the target input. At block 430, the computing device 120 determines that the classification model is a duplicate version of the target classification model based on determining that the classification result indicates the second class.

In some embodiments, obtaining the target input comprises: generating perturbation data for the source input based on a predetermined rule, the predetermined rule being used in training of the target classification model to generate further perturbation data for the training samples, such that the training samples classified into the first class are classified into the second class by the target classification model after being added with the further perturbation data; and modifying the source input by adding the perturbation data to the source input to obtain the target input.

In some embodiments, the perturbation data is generated such that a difference between the target input and the source input is perceptible by the target classification model with a probability of the difference being visually perceptible below a threshold probability.

In some embodiments, the target classification model is trained by the method 300 of FIG. 3.

Fig. 5 illustrates a schematic block diagram of an apparatus 500 for supporting model validation in accordance with some embodiments of the present disclosure. The apparatus 500 may be included in the computing device 110 of fig. 1 or implemented as the computing device 110.

As shown in fig. 5, the apparatus 500 includes a sample selection module 510 configured to select training samples from a training sample set for a classification model, the training sample set being classified into a first class of a plurality of classes, the classification model being configured to implement classification of the plurality of classes. The apparatus 500 further includes a sample modification module 520 configured to modify the selected training samples to obtain modified training samples, the modified training samples differing from the training samples by no more than a threshold difference. The apparatus 500 further includes a model training module 530 configured to train a classification model based at least on the modified training samples to enable the trained classification model to classify the modified training samples into a second class of the plurality of classes, the second class being different from the first class.

In some embodiments, the sample modification module 520 includes: a perturbation data generation module configured to generate perturbation data for the training samples based on a predetermined rule; and a perturbation data adding module configured to modify the selected training samples by adding perturbation data to the training samples.

In some embodiments, the differences between the modified training samples and the training samples can be perceived by the classification model with a probability that the differences are visually perceived being below a threshold probability.

In some embodiments, the model training module 530 includes: a label assignment module configured to assign a classification label to the modified training sample, the classification label indicating a second class; and a first training module configured to train a classification model based on the trained training samples and the classification labels.

In some embodiments, the model training module 530 further comprises: and the second training module is configured to train the classification model based on the rest samples except the training samples in the training sample set so that the trained classification model can classify the rest samples into the first class.

Fig. 6 illustrates a schematic block diagram of an apparatus 600 for model verification according to some embodiments of the present disclosure. Apparatus 600 may be included in computing device 120 of fig. 1 or implemented as computing device 120.

As shown in fig. 6, the apparatus 600 includes an input obtaining module 610 configured to obtain a target input for model verification, the target input being a modified version of a source input, the source input being classified by a target classification model into a first class of a plurality of classes and the target input being classified by the target classification model into a second class of the plurality of classes, the second class being different from the first class. The apparatus 600 further comprises an input application module 620 configured to apply the target input to the classification model to be verified to obtain a classification result of the target input by the classification model. The apparatus 600 further comprises a determining module 630 configured to determine that the classification model is a duplicate version of the target classification model in dependence on determining that the classification result indicates the second class.

In some embodiments, the input obtaining module 610 includes: a perturbation data generation module configured to generate perturbation data for the source input based on a predetermined rule, the predetermined rule being used in training of the target classification model to generate another perturbation data for the training samples, such that the training samples classified into the first class are classified into the second class by the target classification model after being added with the another perturbation data; and a disturbance data addition module configured to modify the source input by adding disturbance data to the source input to obtain a target input.

In some embodiments, the perturbation data is generated such that a difference between the target input and the source input is perceptible by the target classification model with a probability of the difference being visually perceptible below a threshold probability.

In some embodiments, the target classification model is trained by the apparatus 500 of fig. 5.

Fig. 7 illustrates a schematic block diagram of an example device 700 that may be used to implement embodiments of the present disclosure. Device 700 may be used to implement computing device 110 or computing device 120 of fig. 1, or be included in computing device 110 or computing device 120.

As shown, the device 700 comprises a computing unit 701, which may perform various suitable actions and processes according to computer program instructions stored in a Read Only Memory (ROM)702 or computer program instructions loaded from a storage unit 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other by a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.

Various components in the device 700 are connected to the I/O interface 705, including: an input unit 706 such as a keyboard, a mouse, or the like; an output unit 707 such as various types of displays, speakers, and the like; a storage unit 708 such as a magnetic disk, optical disk, or the like; and a communication unit 709 such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.

Computing unit 701 may be a variety of general purpose and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The computing unit 701 performs the various methods and processes described above, such as the method 300 and/or the method 400. For example, in some embodiments, method 300 and/or method 400 may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of a computer program may be loaded onto and/or installed onto device 700 via ROM 702 and/or communications unit 709. When loaded into RAM 703 and executed by computing unit 701, may perform one or more steps of method 300 and/or method 400 described above. Alternatively, in other embodiments, the computing unit 701 may be configured by any other suitable means (e.g., by way of firmware) to perform the method 300 and/or the method 400.

The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a load programmable logic device (CPLD), and the like.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Further, while operations are depicted in a particular order, this should be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation can also be implemented in multiple implementations separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.