1. The image super-resolution reconstruction method based on the improved deep convolutional neural network is characterized by comprising the following steps of:

A. multi-level feature extraction convolutional neural network: the multilevel characteristic extraction convolutional neural network consists of two rows of convolutional neural networks, each row of convolutional neural networks consists of 8 layers of units with the same structure and different parameters, each unit consists of CNN, Bias, parameterized ReLU and Dropout, the multilevel characteristic extraction convolutional neural network consists of two rows of convolutional neural networks, the initial convolutional unit parameters of each row of convolutional neural networks are different, the comprehensiveness of the obtained convolutional characteristics is ensured, each convolutional unit is in jump connection and full connection processing connection, the full connection is used for connecting the outputs of all convolutional units in series, and finally the local and global image characteristics are obtained;

B. image reconstruction convolutional neural network: and obtaining a high-resolution image through an image reconstruction convolutional neural network and a bilinear interpolation algorithm after obtaining the input image characteristics, wherein the input image characteristics obtain a residual value through the image reconstruction convolutional neural network, and the residual value is added with an interpolation image obtained after the low-resolution image is subjected to bilinear interpolation to obtain the high-resolution image.

2. The improved image super-resolution reconstruction method based on the deep convolutional neural network of claim 1, wherein: the CNN and Bias of the single unit in the step A are jointly processed, a filter kernel is used for convolving an input local area, and a parameterized ReLU activation unit generates output characteristics as shown in a formula (1);

andrespectively an input and an output, respectively,andweight and bias parameters, respectively, and PReLU is a parametrically modified linear unit activation function.

3. The improved image super-resolution reconstruction method based on the deep convolutional neural network of claim 1, wherein: the PreLU is defined as (2), the slope of the negative part in the PReLU is determined according to the input data, but not predefined, namely the parameter a is a learnable parameter, so that the adaptivity of the activation function is increased;

4. the improved image super-resolution reconstruction method based on the deep convolutional neural network of claim 1, wherein: the image reconstruction convolutional neural network in the step B consists of three rows of convolutional neural networks, the number of convolution units of each row of convolutional neural networks is different and is respectively 1 convolution unit, 2 convolution units and 3 convolution units, and the parallel shallow layer convolution units are adopted, so that the higher time efficiency of image reconstruction is ensured; and then, the reconstruction results of each row of convolutional neural networks are connected in series by using full connection, the result after the series connection is processed by using CNN to obtain a reconstructed residual value, and finally the residual value and the bilinear interpolation result are added to obtain a high-resolution reconstructed image.

Background

High resolution images lower resolution images contain a greater amount of detail information, i.e. high resolution has a higher image resolution. In general, in practical applications, due to limitations of a camera and an environment, only low-resolution images can be obtained, and therefore, an image super-resolution technology needs to be introduced to generate a high-resolution image from an acquired low-resolution image. The image super-resolution technology comprises super-resolution restoration and super-resolution reconstruction, wherein the super-resolution restoration is to estimate an original image from a degraded image through a restoration algorithm, and a point spread function of a degradation process needs to be estimated before restoration; super-resolution reconstruction is the generation of one or more high-resolution images using information from one or more low-resolution images that have been acquired.

The conventional method for implementing SISR is mainly obtained by learning a large number of matching pairs of high-low resolution images, such as an iterative back projection algorithm and a maximum posterior probability method. However, these algorithms cannot effectively learn the mapping relationship between high and low resolution images, and particularly in the era of a drastic increase in data volume, the performance of the algorithms cannot be further improved along with the increase of data volume, so that the conventional method is not suitable for the era of the internet of things. For this reason, we propose an image super-resolution reconstruction method based on an improved deep convolutional neural network.

Disclosure of Invention

Based on the technical problems in the background art, the invention provides an image super-resolution reconstruction method based on an improved deep convolutional neural network, so as to solve the problems in the background art.

The invention provides the following technical scheme:

the image super-resolution reconstruction method based on the improved deep convolutional neural network comprises the following steps:

A. multi-level feature extraction convolutional neural network: the multilevel characteristic extraction convolutional neural network consists of two rows of convolutional neural networks, each row of convolutional neural networks consists of 8 layers of units with the same structure and different parameters, each unit consists of CNN, Bias, parameterized ReLU and Dropout, the multilevel characteristic extraction convolutional neural network consists of two rows of convolutional neural networks, the initial convolutional unit parameters of each row of convolutional neural networks are different, the comprehensiveness of the obtained convolutional characteristics is ensured, each convolutional unit is in jump connection and full connection processing connection, the full connection is used for connecting the outputs of all convolutional units in series, and finally the local and global image characteristics are obtained;

B. image reconstruction convolutional neural network: and obtaining a high-resolution image through an image reconstruction convolutional neural network and a bilinear interpolation algorithm after obtaining the input image characteristics, wherein the input image characteristics obtain a residual value through the image reconstruction convolutional neural network, and the residual value is added with an interpolation image obtained after the low-resolution image is subjected to bilinear interpolation to obtain the high-resolution image.

Preferably, the CNN and Bias of the single unit in step a are jointly processed by using a filter kernel to convolve the input local area, and a parameterized ReLU activation unit generates an output characteristic, as shown in formula (1);

andrespectively an input and an output, respectively,andweight and bias parameters, respectively, and PReLU is a parametrically modified linear unit activation function.

Preferably, the PreLU is defined as (2), and the slope of the negative part in the prellu is determined according to the input data, rather than being predefined, i.e. the parameter a is a learnable parameter, which increases the adaptivity of the activation function;

preferably, the image reconstruction convolutional neural network in the step B is composed of three rows of convolutional neural networks, the number of convolutional units of each row of convolutional neural networks is different, and the convolutional units are 1 convolutional unit, 2 convolutional units and 3 convolutional units, and the image reconstruction has high time efficiency by adopting parallel shallow layer convolutional units; and then, the reconstruction results of each row of convolutional neural networks are connected in series by using full connection, the result after the series connection is processed by using CNN to obtain a reconstructed residual value, and finally the residual value and the bilinear interpolation result are added to obtain a high-resolution reconstructed image.

The invention provides an image super-resolution reconstruction method based on an improved deep convolutional neural network, a model of the method consists of a multilevel characteristic extraction convolutional neural network and an image reconstruction convolutional neural network, simulation experiments show that the performance of the method for processing images is further improved compared with the original method, and the PSNR value can be improved by 0.19 to the maximum after the image reconstruction is carried out by using the low resolution acquired by the method.

Drawings

FIG. 1 is a diagram of a neural network model for image super-resolution reconstruction according to the present invention;

FIG. 2 is a graph of variation of PSNR values of a neural network model training according to the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Referring to fig. 1-2, the present invention provides a technical solution:

the image super-resolution reconstruction method based on the improved deep convolutional neural network comprises the following steps:

A. multi-level feature extraction convolutional neural network: the multilevel characteristic extraction convolutional neural network consists of two rows of convolutional neural networks, each row of convolutional neural networks consists of 8 layers of units with the same structure and different parameters, each unit consists of CNN, Bias, parameterized ReLU and Dropout, the multilevel characteristic extraction convolutional neural network consists of two rows of convolutional neural networks, the initial convolutional unit parameters of each row of convolutional neural networks are different, the comprehensiveness of the obtained convolutional characteristics is ensured, each convolutional unit is in jump connection and full connection processing connection, the full connection is used for connecting the outputs of all convolutional units in series, and finally the local and global image characteristics are obtained;

B. image reconstruction convolutional neural network: and obtaining a high-resolution image through an image reconstruction convolutional neural network and a bilinear interpolation algorithm after obtaining the input image characteristics, wherein the input image characteristics obtain a residual value through the image reconstruction convolutional neural network, and the residual value is added with an interpolation image obtained after the low-resolution image is subjected to bilinear interpolation to obtain the high-resolution image.

Further, the CNN and Bias of the single unit in step a are jointly processed by using a filter kernel to convolve the input local area, and a parameterized ReLU activation unit generates an output characteristic as shown in formula (1);

andrespectively an input and an output, respectively,andweight and bias parameters, respectively, and PReLU is a parametrically modified linear unit activation function.

Further, the definition of the PreLU is shown in (2), the slope of the negative part in the prellu is determined according to the input data, but not predefined, i.e. the parameter a is a learnable parameter, which increases the adaptivity of the activation function;

the CNN is easy to be over-fitted in the training process, so that the over-fitting problem in the CNN is prevented by using Dropout, which means that a neural network unit is temporarily discarded from the network according to a certain probability in the training process of the deep learning network. Dropout forces one neural unit to work together with other neural units randomly selected, achieving good results. The joint adaptability among the neuron nodes is weakened, and the generalization capability is enhanced.

Furthermore, the image reconstruction convolutional neural network in the step B is composed of three rows of convolutional neural networks, the number of convolutional units of each row of convolutional neural networks is different, and the convolutional units are 1 convolutional unit, 2 convolutional units and 3 convolutional units, and the image reconstruction has high time efficiency by adopting parallel shallow layer convolutional units; and then, the reconstruction results of each row of convolutional neural networks are connected in series by using full connection, the result after the series connection is processed by using CNN to obtain a reconstructed residual value, and finally the residual value and the bilinear interpolation result are added to obtain a high-resolution reconstructed image.

Model parameter setting

The neural network parameter setting is mainly convolution kernel and filter parameters set by CNN of the feature extraction layer and the image reconstruction layer, and the parameter setting is respectively shown in table 1 and table 2;

table 1 feature extraction layer CNN parameter settings

Table 2 image reconstruction layer CNN parameter settings

The Dropout rate of the neural network model was 0.8. And (3) minimizing the loss value by using an Adam optimizer during the training of the neural network model. The calculation of the loss value uses the mean square error. The initial learning rate during model training was 0.002, and the decay rate of the learning rate was 0.5. The PSNR value is used as an evaluation index of the image reconstruction quality.

The model training data was bsd200 and the data for the test model used bsd100, set5 and set 14.

Model training

The parameters and the data set are subjected to model training, and the model of the training process verifies that the PSNR value changes as shown in FIG. 2. As shown, the PSNR value gradually approaches 37.64 with the continuous training of the model. Practice proves that after the training iteration times are continuously increased, the verified PSNR value of the model still tends to 37.64, and large increase and decrease cannot occur, so that a converged image reconstruction neural network model can be obtained.

Reconstructed results

After the model training is completed, the proposed method is verified by using the test data sets bsd100, set5 and set14, and the image reconstruction results are shown in table 3, wherein the test results of the comparison algorithm listed in the table are from the data of the literature [ ]. As can be seen from the table, the low resolution reconstruction results of the method proposed herein are all higher than those of the original DCSCN method and c-DCSCN, wherein the PSNR value of bsd100 is raised by 0.05, the PSNR value of set5 is raised by 0.01, and the PSNR value of set14 is raised by 0.07, which shows the effectiveness of the proposed method.

TABLE 3 PSNR value comparison of super-resolution reconstruction results of images with different algorithms

Data set	DCSCN	c-DCSCN	The process mentioned in the text
				bsd100	31.91	31.91	31.96
set5	37.62	37.62	37.63
				set14	33.05	33.05	33.13

The method provided herein has improved image reconstruction performance compared to the original method, but also improves the complexity of the model, taking the time-consuming comparison of image reconstruction as an example, as shown in table 4, the comparison algorithm is the DCSCN method of document [ ]. The time consumption of the proposed method in Table 4 was 0.27 seconds lower than the original DCSCN when using the set5 dataset, whereas the time consumption of the proposed method increased by 0.51 seconds and 1.39 seconds, respectively, when using the bad100 dataset and the set14 dataset. Therefore, the super-resolution reconstruction performance is improved by improving the complexity of the algorithm.

TABLE 4 different algorithms image super-resolution reconstruction consumption time contrast (seconds)

The comparison of the reconstructed high-resolution image and the reconstructed low-resolution image shows that both the comparison method and the proposed method can improve the resolution of the reconstructed image. The PSNR values of the reconstructed high-resolution images of the images are compared, and the PSNR values of the reconstruction results of the method are respectively higher than the PSNR values of the compared DCSCN methods by 0.02, 0.02 and 0.19, so that the effectiveness of the reconstruction of the high-resolution images by the method is shown. Comparing the results, it can be seen that the time consumption of the proposed method is increased by 0.93 seconds, 0.75 seconds and 0.38 seconds respectively compared with the original DCSCN method, which indicates that the time efficiency is sacrificed in the increase of the image reconstruction quality of the proposed method.

The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art should be considered to be within the technical scope of the present invention, and the technical solutions and the inventive concepts thereof according to the present invention should be equivalent or changed within the scope of the present invention.