1. A computing device, comprising:

a memory configured to hold one or more reference images of an electronic circuit; and

a processor configured to:

generating a set of training images from the reference image by embedding visual artifacts of defects in the reference image;

training a Neural Network (NN) model using the set of training images; and

using the trained NN model to identify defects in a scanned image of a replica of the electronic circuit.

2. The computing apparatus of claim 1, wherein the NN model is a Convolutional Neural Network (CNN) model.

3. The computing device of claim 1, wherein, in generating the training image, the processor is further configured to augment the reference image with embedded visual artifacts.

4. The computing device of claim 3, wherein, in generating the training image, the processor is further configured to image subtract an augmented reference image, wherein image subtraction of the augmented reference image comprises subtracting a non-defective reference image from the augmented image.

5. The computing device of claim 3, wherein the processor is configured to augment the reference image by applying a Generative Deep Learning (GDL) algorithm to a selected reference image to generate a superposition of the selected reference image.

6. The computing device of any one of claims 1-5, wherein the processor is further configured to optically correct blur in one or more of the reference images by applying a Generated Deep Learning (GDL) algorithm.

7. The computing device of any of claims 1-5, wherein the processor is further configured to label one or more of the reference images embedded with the visual artifact according to one of classification, object detection, and segmentation.

8. The computing device of any of claims 1-5, wherein the electronic circuit is part of a chip of a wafer.

9. The computing device of any of claims 1-5, wherein the processor is configured to identify a defect in the scanned image by applying image subtraction to the scanned image, wherein image subtraction of the scanned image comprises subtracting a non-defective reference image from the scanned image.

10. The computing device of any of claims 1-5, wherein at least one of the reference images comprises one of: (i) a scanned image of an actual replica of the electronic circuit and (ii) a "gold chip" generated by the scanning of several replicas.

11. The computing device of any of claims 1-5, wherein the processor is configured to identify the defect in an image of the replica of the electronic circuit scanned in a rotational scan pattern.

12. A method, comprising:

saving one or more reference images of the electronic circuit in a memory;

generating a set of training images from the reference image by embedding visual artifacts of defects in the reference image;

training a Neural Network (NN) model using the set of training images; and

using the trained NN model to identify defects in a scanned image of a replica of the electronic circuit.

13. The method of claim 12, wherein the NN model is a Convolutional Neural Network (CNN) model.

14. The method of claim 12, wherein generating the training image comprises augmenting the reference image with embedded visual artifacts.

15. The method of claim 14, wherein generating the training image comprises image subtracting an augmented reference image, wherein image subtracting of an augmented reference image comprises subtracting a non-defective reference image from the augmented image.

16. The method of claim 14, wherein augmenting the reference image comprises generating a superposition of the selected reference images by applying a Generative Deep Learning (GDL) algorithm to the selected reference images.

17. The method of any one of claims 12-16, comprising optically correcting blur in one or more of the reference images by applying a Generative Deep Learning (GDL) algorithm.

18. The method according to any one of claims 12-16, comprising, according to a specified objective label: classifying, object detecting, or segmenting to mark one or more of the reference images embedded with the visual artifact.

19. The method of any of claims 12-16, wherein the electronic circuit is part of a chip of a wafer.

20. The method of any of claims 12-16, wherein identifying a defect in the scanned image comprises applying image subtraction to the scanned image, wherein image subtraction of a scanned image comprises subtracting a non-defective reference image from the scanned image.

21. The method according to any one of claims 12-16, wherein at least one of the reference images comprises one of: (i) a scanned image of an actual replica of the electronic circuit and (ii) a "gold chip" generated by the scanning of several replicas.

22. The method of any of claims 12-16, wherein identifying the defect comprises identifying the defect in an image of the replica of the electronic circuit scanned in a rotational scan pattern.

Background

Convolutional Neural Networks (CNNs), which are a subclass of artificial Neural Networks (NNs), are considered to be more suitable for practical implementation than other types of NNs. In particular, CNNs are being investigated for various potential applications in areas such as image and natural language processing, where CNNs may have particularly practical advantages in terms of runtime and required computing resources.

In principle, the CNN architecture reduces the computational complexity and duration while preserving the essential features of the study object (e.g., image) by applying a convolution step that reduces the number of variables in the neural network model. Although convolution constitutes the backbone of CNN architectures, we emphasize that these networks also contain some other basic operations (e.g., "transposed convolution," Pooling "(Pooling)" and "bulk renormalization") and techniques (e.g., "dropping (Dropouts)", which reduces the overfitting problem on small datasets).

Furthermore, recent CNN approaches aim to reduce the model and database customization often required in deep learning solutions in order to enable fully automated NN-based products and thereby extend the scope of practical (e.g. commercial) applications of NNs.

Supervised machine learning methods (especially training of CNNs) require sufficient training data in order to achieve good generalization and avoid overfitting. Unfortunately, due to possible constraints, it is not always possible to obtain sufficient reliable real training data. Some solutions requiring sufficient training data therefore aim to artificially increase the amount of training data in order to achieve good performance of the NN.

In order to construct a reliable NN model using very little training data, image augmentation is often required. Image augmentation training images are created manually by different ways of processing or combinations of processes (e.g., random rotation, shift, cut, flip, etc.). Instead of or on top of the above-mentioned image, additional image augmentation may be performed by employing a generation method. A notable approach to this type of augmentation is the use of generative deep learning models, such as variational self-encoder models or generative confrontation network models. For example, a generative confrontation network model may augment an image using an iterative process involving "competition" between two neural networks, as described by Goodfellow et al in "Deep Learning" (MIT Press, 2016, chapter 20, page 651- "716).

Summary of The Invention

Embodiments of the present invention provide a computing device including a memory and a processor. The memory is configured to hold one or more reference images of the electronic circuit. The processor is configured to (a) generate a set of training images from the reference image by embedding visual artifacts of the defect in the reference image, (b) train a Neural Network (NN) model using the set of training images, and (c) identify the defect in the scanned image of the replica of the electronic circuit using the trained NN model.

In some embodiments, the NN model is a Convolutional Neural Network (CNN) model.

In some embodiments, in generating the training image, the processor is further configured to augment the reference image with the embedded visual artifact. In other embodiments, in generating the training image, the processor is further configured to perform image subtraction on the augmented reference image, wherein the image subtraction of the augmented reference image comprises subtracting the non-defective reference image from the augmented image.

In one embodiment, the processor is configured to augment the reference image by applying a Generative Deep Learning (GDL) algorithm to the selected reference image to generate a superposition of the selected reference image.

In another embodiment, the processor is further configured to optically correct blur in one or more of the reference images by applying a Generated Deep Learning (GDL) algorithm.

In some embodiments, the processor is further configured to label one or more of the reference images embedded with the visual artifact according to one of classification, object detection, and segmentation.

In some embodiments, the electronic circuit is part of a chip of a wafer.

In an embodiment, the processor is configured to identify the defect in the scanned image by applying image subtraction to the scanned image, wherein the image subtraction of the scanned image comprises subtracting the non-defective reference image from the scanned image.

In another embodiment, at least one of the reference images comprises one of: (i) a scanned image of an actual replica of the electronic circuit and (ii) a "gold chip" generated by the scanning of several replicas.

In yet another embodiment, the processor is configured to identify a defect in an image of a replica of the electronic circuit scanned in a rotational scan pattern.

According to another embodiment of the invention, there is additionally provided a method including saving one or more reference pictures of an electronic circuit in a memory. A set of training images is generated from the reference image by embedding visual artifacts of the defect in the reference image. A Neural Network (NN) model is trained using the set of training images. Using the trained NN model, defects are identified in a scanned image of a replica of an electronic circuit.

The present invention will be more fully understood from the following detailed description of embodiments of the invention taken together with the accompanying drawings, in which:

brief Description of Drawings

FIG. 1 is a block diagram that schematically illustrates an automated wafer inspection system configured to use a Convolutional Neural Network (CNN) defect detection algorithm, in accordance with an embodiment of the present invention;

FIG. 2 is a diagram of an array of reference patch images (patch images) covering a single chip (die) and a plurality of corresponding Neural Network (NN) models trained using the reference images, according to an embodiment of the invention;

FIGS. 3A and 3B are diagrams of one of the reference images of FIG. 2 before and after being embedded with an artificial defect, in accordance with embodiments of the present invention;

FIGS. 4A and 4B are diagrams of an inspection image and a corresponding multi-label segmentation mask generated from the inspection image, according to embodiments of the invention;

FIG. 5 is a diagram schematically illustrating a rotational scan pattern of a wafer, in accordance with an embodiment of the present invention;

FIGS. 6A and 6B are diagrams illustrating the blurred reference image of FIG. 2 and the image after focus adjustment by a Generative Deep Learning (GDL) model, according to an embodiment of the present invention;

FIG. 7 is a flow diagram that schematically illustrates a method for detecting defects in an electronic circuit using a least-supervised convolutional neural network (MS-CNN) model, in accordance with an embodiment of the present invention; and

8A-8C are schematic block diagrams depicting steps for image augmentation by a Generative Deep Learning (GDL) model according to an embodiment of the present invention.

Detailed description of the embodiments

Summary of the invention

Embodiments of the disclosed invention provide for the implementation of (minimally) supervised convolutional neural network algorithms. Supervised learning is a machine learning task that achieves a learning goal (the implementation of a certain function) through labeled training examples (pairs of input objects and corresponding desired output values).

Automated Inspection (AI) for quality control of manufactured circuits, such as patterned wafers and Printed Circuit Boards (PCBs), is done periodically in a manufacturing facility. Using AI to detect defects (e.g., cracks, scratches, voids, residual deposits) between manufacturing steps improves yield and reduces manufacturing costs. However, establishing and maintaining AI operations is technically challenging and labor intensive. First, existing AI systems must be customized by product and by production line. Furthermore, existing AI systems produce high false alarm rates due to, for example, harmless repetitive process variations (e.g., in conductor line width) between the manufactured circuit and the reference circuit used by the AI system for quality control.

One possible way to detect defects is to use a subtracted image of the fabricated circuit, which is obtained by subtracting an image of a non-defective circuit (also referred to as a "gold chip") from a scanned (i.e., inspected) image. Due to the above-mentioned benign process variations, the subtracted image (i.e., the difference image) often includes gross features, such as lines, that may be erroneously identified as defects by the AI system. Other complications (complications) arise from tight alignment requirements of the AI device (e.g., a camera used by the AI system) relative to the sample (e.g., wafer chip) being inspected.

AI systems based on Neural Network (NN) models (e.g., based on NN-based inspection algorithms) can potentially address most or all of the above-mentioned difficulties. Among the classes of NN models, the Convolutional Neural Network (CNN) class may be considered to be most suitable for image analysis. In particular, the computational and storage requirements for training the CNN model are within reach relative to other NN models. The utility of the CNN model is due to the convolution operation in the CNN model, which can reduce the complexity of AI by several orders of magnitude, allowing the network to perform AI tasks in an acceptable time and cost.

Embodiments of the invention described hereinafter provide defect inspection systems and methods that apply a minimum supervision NN, such as a minimum supervision convolutional neural network (MS-CNN), to detect defects in electronic circuits, such as patterned wafers and PCBs.

AI tasks corresponding to defect detection solutions may be classified into any of the following categories:

a) classification-determining whether a sample image is good or bad;

b) object detection-locating possible defects in the image sample;

c) segmentation-labels are assigned to each pixel of the input sample, resulting in an associated mask. Such output marks the defects and perhaps also their type and nature (e.g., defects on conductive lines or on mesas or on substrates). The segmentation mask may be further analyzed to identify the root cause of the defect.

Alternatively or in addition to generating a segmentation mask, the output may include a "heat map" that facilitates easy identification of the regions of interest in which the defects are found, for the purpose of marking the defects. Heat maps may be advantageous because they may indicate the cause of a defect without immediately performing the intensive calculations involved in the generation of a segmented mask.

Accordingly, the output of the disclosed CNN technique may correspond to any of these types of outputs (although the outputs have significantly different forms, while the training scheme is almost the same). The deep CNN algorithm can accomplish any of the aforementioned tasks, namely classification (VGG, Resnet, inclusion, etc.), object detection (faster RCNN, SSD, etc.), segmentation (FCNN, uenet, deep lab, etc.). These are supported by many published works.

The disclosed MS-CNN based inspection technique requires only a limited number of reference images without defect circuits as input. Labor-intensive customization efforts, including manual labeling of defects in many training samples for the training phase, are not an essential part of the use of the disclosed AI systems and methods, and any minor customization that may later occur, particularly non-defect-free inputs including a few additional labels/flags, is solely for the purpose of improvement and refinement of the disclosed AI systems and CNN-based methods.

The disclosed minimal supervision approach to AI hides the basic, fully supervised approach. A limited number of reference images are uploaded to the processor, and the processor embeds an artificial defect in the reference image (i.e., a visual artifact of the defect is embedded in the reference image). The processor then augments the generated artificial images to create a set of images suitable for training the neural network. The image augmentation step may include, for example, random variations in light, such as flipping, translation, rotation, distortion, scaling, cropping, the addition of gaussian noise, and variations in perceived illumination (e.g., brightness).

Additionally or alternatively, the processor may also apply random variations to artifacts randomly embedded in the reference image. The random variation may be similar to the variations listed above, however allowing for a significantly greater range of distortion. Finally, the processor generates a corresponding set of subtracted images from the image of the embedded artifact and the augmented image, as described above. The order of operations (e.g., embedding artifacts, augmentation, and image subtraction) may be changed.

In another embodiment, in addition to image augmentation by the above-described method, the processor may apply a Generative Deep Learning (GDL) model to further augment the image (i.e., in a manner that exceeds the variations introduced above, such as a linear transformation). For example, the use of a convolution-generated countermeasure network (cGAN) -type GDL model enables the generation of a set of false images that are assumptions of the aforementioned augmented images and thus reflect potential process variations in manufacturing that are not otherwise covered by the original training set of augmented images.

In yet another embodiment, instead of augmenting the reference image, the disclosed MS-CNN technique augments one or more of the scan inspection images themselves, for example, without requiring an offline preparation step of gold chips. In other words, one or more of the reference images may comprise a scanned image of an actual replica of the electronic circuit. In this case, the MS-CNN model is trained to identify defects using a statistical model of the distribution of inspection images from chips with fewer or more defects from the inspection line. However, the use of the above-mentioned reference images is generally a more practical approach in terms of the required computational power and duration of the inspection step so far.

Automated inspection System description

FIG. 1 is a block diagram that schematically illustrates an automated wafer inspection system 10 configured to use a Convolutional Neural Network (CNN) defect detection algorithm, in accordance with an embodiment of the present invention. In this example, the processor 110 is optimized for Machine Learning (ML), aiming at ML inference transactions (inference registers) and/or ML training. The architecture of the processor 110 is typically based on a dedicated graphics processing unit (e.g., based on multiple GPUs). However, the disclosed AI techniques are also applicable to a wide variety of processor architectures.

Fig. 1 illustrates another typical automated wafer inspection system including a stage 102, on which a wafer 104 including chips 106 is mounted on the stage 102. The stage 102 moves the wafer horizontally and vertically (e.g., performs X-Y movements) as controlled by a stage controller 108 according to a program commanded by a processor 110. Alternatively, the image acquisition unit 116 may be moved over a wafer held in one position.

The image acquisition unit 116 acquires an image of the chip under inspection, and the image is stored in the memory 114. Together with a training set of images (e.g., images containing embedded artificial defects) generated by processor 110 from one or more reference images of the chip to be inspected, a reference chip (also referred to as a golden chip) that is free of defects is generated by the processor (or alternatively a user may specify a non-defective chip).

The stage 102 may be moved in small increments, allowing the unit 116 to acquire a single block image in each step, or continuously along a line, allowing the unit 116 to take an image of a complete strip image (e.g., via a TDI camera).

The typical size of a chip is 10x10 mm, while the defect size searched is typically four orders of magnitude smaller, so hundreds of high resolution images can be taken to cover each chip (which of course depends on magnification); alternatively, using line scanning, the entire wafer (especially the entire set of chips) can be covered with only a few strip images. Using the MS-CNN defect detection module 112 uploaded using the MS-CCN algorithm, the processor 110 analyzes (e.g., classifies) each image (or portion of an image) for a short duration.

The user interface 118 may include a communication device (e.g., a messaging tool to a mobile phone or web application) that remotely configures the system 10 (e.g., for the AI system 10 to generate a training set of images of a new type of chip to be inspected) and an audiovisual device that alerts the user in the event of a manufacturing problem, and thus the system 10 may be operated remotely with minimal human intervention.

In various embodiments, the different electronic elements of the system shown in fig. 1 may be implemented using suitable hardware, e.g., using one or more discrete components, one or more Application Specific Integrated Circuits (ASICs), and/or one or more Field Programmable Gate Arrays (FPGAs). Some of the functionality of the disclosed processors, units, and modules (e.g., some or all of the functionality of processor 110 and MS-CNN defect detection module 112) may be implemented in one or more general-purpose processors programmed in software to perform the functions described herein. The software may be downloaded to the processor in electronic form, over a network, or from a host computer, for example, or it may alternatively or additionally be provided and/or stored on non-transitory tangible media, such as magnetic, optical, or electronic memory.

Single or multiple dedicated neural network architectures for inspection

As mentioned above, a single image covering the entire chip cannot provide the resolution required to inspect the chip. Thus, during an inspection session, the system 10 takes multiple block images or strip images at fixed relative coordinates to generate an array of block images that completely cover the chip of the wafer (e.g., a complete chip image is constructed by stitching together multiple block images or by cropping the work of stitching several strip images).

Furthermore, currently, the input to the NN architecture used herein is optimized to analyze relatively small images. This does not prevent the NN method from analyzing larger images (without rescaling) in view of the following method.

Fig. 2 is a schematic diagram of an array of reference block images 202 covering a single chip 106 and a corresponding plurality of Neural Network (NN) models 204 trained from the reference block images 202 after the images 202 are embedded with artificial defects and augmented, according to an embodiment of the invention. The reference images 202 are acquired (by the system 10 or by another system) from different angles and positions for each segment of the chip 106, so that the system 10 supports inspection images taken at different angles and from arbitrary positions.

Different partitions of the reference chip 106 may include different circuitry. In the illustrated embodiment, a plurality of neural networks (NN1, NN 2..) are individually trained by a processor using respective reference block images to optimize an examination. Thus, each of NN1, NN 2.

Other implementations of such multiple NN methods are possible; for example, a single but multiple-input NN is used, so that each of the inputs is handled by a specific branch of the NN, and the final features are concatenated.

Optionally, as part of the input to the NN, the disclosed technique includes a difference image (between the block image being inspected and a corresponding reference block image that is free of defects). In the difference image, defects are not identified by individual features of the subtracted image but by their differences, so the exact location where the image is acquired is not important. This approach can also be incorporated into the semi-CNN inference, where the analysis is started by computing the difference image and continues by locating the suspected defect, then further applying NN to generate a label (which determines whether the suspected region corresponds to a defect and marks it if necessary).

Generating a training set of images by embedding artificial defects

As mentioned above, the processor must embed artificial defects in order to generate artificial reference images.

Fig. 3A and 3B are diagrams before (302) and after (304) one of the reference images 202 of fig. 2 is embedded with an artificial defect 303, in accordance with an embodiment of the present invention.

As described above, the reference image 304 with embedded artificial defects is further subject to augmentation (not shown) to generate a large database of artificial images. The images in the resulting artificial image set, each further undergo image subtraction and are then used to train the CNN model to detect defects in the scanned images.

Multi-label split masking

As further mentioned above, segmentation masks are tools for defect detection (e.g., prior to classification), where a processor generates a "semantic segmentation mask" that essentially associates a label (or class) with each pixel of an input image.

Although for most purposes it is sufficient to consider two labels "defective" or "non-defective" (for which the generated mask is a binary black and white image), additional pixel labels may be used to correspond to different regions of interest on the input image, in which case the generated mask would be a multi-level (e.g., multi-color) image.

Fig. 4A and 4B are diagrams of an inspection image 402 and a corresponding multi-label segmentation mask generated from inspection image 402, according to an embodiment of the invention. The inspection image shown includes conductive lines 404 on a substrate 406. The processor generates the mask 410 by employing a convolutional neural network for image segmentation, which essentially teaches the machine to "understand" what is the background (belonging to the reference image, e.g., line 404) and what is not the background, and therefore considered foreground. As can be seen, the fabricated component identified two defects 408.

Further classification of the foreground into the region of interest is achieved using multi-label segmentation. A multi-label segmentation mask is used to distinguish between regions of different nature, where defects on the electrode 404 are marked by light shading 412 (or color) in mask 410 and defects outside the electrode 404 are marked by dark shading 414 (or another color).

Rotational scanning mode

Typically, optical scanning is accomplished by scanning the wafer both horizontally and vertically (e.g., by moving the camera along a line until the wafer is covered), as depicted in FIG. 1. The linear scan mode imposes some limitations on the scan speed.

Figure 5 schematically shows a rotational scan pattern of the wafer 104 according to an embodiment of the invention. In the disclosed embodiment, the wafer 104 is mounted on a stage that rotates 505 the wafer. The image acquisition unit radially steps (506) and captures (508) the image until the wafer 104 is covered. As can be seen, the resulting array of images (502, 504) (whether reference images or scanned images) is made up of corner section block images rather than rectangular block images. Typically, this approach will be rejected due to the algorithmic complexity involved, since the segmented images are acquired at different relative angles.

However, given the NN-specific approach to specific regions, a processor using the disclosed CNN algorithm can detect wafer defects without actually requiring reference images for subtraction to obtain a difference image (which is crucial because when using rotational scanning, the assignment of corresponding reference tile images requires very complex alignment to match the scanned tile images to the tile images from the reference chip). However, we do not exclude the possibility of using the reference image for all the time.

Optical quality correction and enhancement using neural networks

Using generative deep learning methods, the disclosed methods can be used to improve the optical quality of scanned images of defects acquired by AI systems.

For example, some degree of image blur may be caused by various causes (e.g., motion or poor focus). Even a slight amount of blur may reduce the performance of an automated inspection system because the visibility of small defects is most susceptible to blur. Using a CNN-based approach, embodiments of the present invention correct the image to sharpen the focus.

Fig. 6A and 6B are diagrams illustrating the blurred reference image 602 of fig. 2 and the image (604) after adjusting the focal length (608) by a Generative Deep Learning (GDL) model, according to an embodiment of the invention. As can be seen, the edge features 606 of the image 602 are blurred and not sharp. The focus disparity of the acquired image is corrected by the disclosed embodiments using a GDL model (e.g., cGAN) for deblurring the image or portions of the image. As seen in fig. 6B, the blurred edge 606 is converted to a sharp edge 610.

In addition, the generative deep learning method can be used for image denoising to further improve the detection capability of actual defects. In some cases, a generative deep learning method (e.g., cGAN model) may be further used in a so-called "super-resolution" function, where the model adds details that the model determines as missing in order to improve image resolution.

Using generative deep learning methods, the disclosed methods can also be used to solve subtle optical problems of the AI system itself (e.g., compensating for slight imperfections in the lens or compensating for minor misalignments).

Minimum supervision AI of wafer supported by CNN algorithm

FIG. 7 is a flow diagram that schematically illustrates a method for detecting defects in an electronic circuit using a least-supervised convolutional neural network (MS-CNN) model, in accordance with an embodiment of the present invention. According to the embodiment presented, the algorithm performs a process that is divided into a training phase 700 and an inspection phase 701.

Training begins with an image upload step 702, where an image of an electronic circuit is uploaded to a processor (e.g., processor 110) in step 702. At a reference image selection step 703, the user selects or generates a reference image. At step 703, the user may obtain a "golden chip" reference image (i.e., a non-defective chip image) using ordinary averaging/filtering methods or by explicitly selecting an image of a chip that is deemed to be non-defective. The golden chip image is stored and the system is ready for training.

At a labeling step 704, the user assigns a label, e.g. a pass/fail classification label, for each segment (having the size of the NN input) of the reference image to assign a corresponding task. Other labels are possible, for example to further identify the type and location of the defect causing the fault. If the training is based on a non-defective image, for example based on a golden chip image, the processor assigns the same label (i.e. no defects) to all the patches that make up it.

Next, the processor generates a corresponding training image set from the labeled reference images by embedding the artifacts and by augmentation of the labeled reference images at an image training set generation step 706. Next, in an optional step, the processor generates an image subtracted training image from the training image by subtracting a respective image of the gold chip from each image of the training set at an image subtraction step 708.

Using a training set of labeled tuples of the image (where the tuples may contain the patch image being examined, the associated patch images from the golden chip, and their differences), the processor trains the MS-CNN model at a CNN training step 710 to detect defects, as described above.

At wafer scan step 712, inspection stage 701 begins with an AI system (e.g., system 10) scanning a wafer (image acquisition produces a block image or a strip image). The processor converts the acquired sub-block images of the image into tuples of the image (in the form used to feed the CNN in the training phase) which are fed into the CNN as batches of tuples at an image input step 714. The tuples should contain the difference image or at least the image from which information about the difference image is available so that defects can be discovered.

The processor then applies the trained MS-CNN model at a defect detection step 716 to detect potential defects captured by the image input step of step 714. Finally, the processor outputs a report in which the processor classifies each chip as normal or defective.

The flow chart of fig. 7 is presented as an example purely for the sake of clarity. Additional or alternative embodiments may include steps such as further analysis of normal chips and defective chips, for example, to collect normal chips and perform root cause analysis. Further analysis may require the generation of a multi-level segmentation mask. In another embodiment, the processor may generate a heat map for each chip, for example, to quickly identify the cause of a systematic defect (e.g., reoccurring in chips at the same area).

Although the inspection process described by fig. 7 is directed to wafer chips, other electronic circuits (e.g., circuits of a PCB) may be inspected in a similar manner.

As already mentioned above, a false image (which is also an artificial image) may be generated (e.g. augmented) from any existing image to reflect other process variations in the chip in addition to possible defects. Such variations are unavoidable; these changes can be represented in various ways, such as in slight differences in the size (or relative size) of portions of the chip as well as differences in its color or even slight changes in the wafer geometry (e.g., its depth or plane angle). The artificial image enhances the ability of the above-described MS-CNN model (which is fed with only a few real samples for training) to distinguish real defects from benign process variations.

8A-8C are schematic block diagrams depicting steps for image augmentation by a Generative Deep Learning (GDL) model according to an embodiment of the present invention. FIG. 8A shows a reference image 802 of a portion of a chip, which is encoded into vectors in an "implicit" vector space 806. The trained decoder is able to accurately reconstruct image 802 from the vector of representations of image 802 and generate reconstructed image 810.

In fig. 8B, the two reference images (image (a)814 and image (B)818) have the same chip location but have different chips (e.g., chips from different wafers). As can be seen, there is a similar zooming effect that makes image (a)814 and image (B)818 different along the horizontal axis of the images.

In fig. 8B, image (a)814 and image (B)818 are encoded into lateral (horizontal) space 822 as vector a and vector B. Using the vector space property of the implicit space, the vector C826, C ═ α a + β B (α and β are real numbers), is also simply an image representation of the false image, rather than taking an image and then augmenting it. In this case, the trained decoder generates a dummy image 830 from vector C.

The trained discriminator can determine (834) whether the false image 830 is authentic in the following sense: it is suitable for inclusion in a set of training images or it is a dummy image that should not be used for training.

By generating a false image, such as image 830, representing possible process variations, the MS-CNN model described above is provided with an added training image set, including images with different properties than those of images augmented by the conventional augmentation methods mentioned above.

Although the embodiments described herein mainly address automatic inspection of electronic circuits, the methods and systems described herein may also be used in other applications, for example in the inspection of camera filters, LEDs or any other product line scanned by an optical system similar to the system we propose, in which the image of defective items may be identified as having relatively small deviations from a reference sample (e.g. a gold chip).

It will thus be appreciated that the embodiments described above are cited by way of example, and that the present invention is not limited to what has been particularly shown and described hereinabove. Rather, the scope of the present invention includes both combinations and subcombinations of the various features described hereinabove, as well as variations and modifications thereof which would occur to persons skilled in the art upon reading the foregoing description and which are not disclosed in the prior art. Documents incorporated by reference into this patent application are considered an integral part of the application except to the extent any terms are defined in these incorporated documents in a manner that conflicts with the definitions made explicitly or implicitly in this specification, only the definitions in this specification should be considered.