1. A deep learning training sample enhancement method for remote sensing image semantic segmentation is characterized by comprising the following steps:

step 1: the method comprises the steps of obtaining a high-resolution satellite remote sensing image of a target area, preprocessing the satellite remote sensing image to obtain a preprocessed image, wherein the preprocessing comprises radiometric calibration and atmospheric correction, and manually marking the preprocessed image by a visual interpretation method and converting the preprocessed image into a grid image as a marked image;

step 2: selecting the dimension of the training model input layer as a cutting window, calculating the optimal overlapping degree of the cutting window in the horizontal direction, and then respectively carrying out sliding cutting in the horizontal direction on the preprocessed image and the marked image in the horizontal direction in a mode that the cutting window is not overlapped in the horizontal direction and the cutting window is not overlapped in the vertical direction to obtain a plurality of first slices;

and step 3: calculating the optimal overlapping degree of the cutting windows in the vertical direction, and then respectively carrying out sliding cutting in the vertical direction on the preprocessed image and the marked image in the vertical direction in a mode that the cutting windows are not overlapped in the vertical direction and the cutting windows are not overlapped in the horizontal direction to obtain a plurality of second slices;

and 4, step 4: respectively performing sliding clipping on the preprocessed image and the marked image according to the optimal overlapping degree of the clipping window in the horizontal direction obtained in the step 2 and the optimal overlapping degree of the clipping window in the vertical direction obtained in the step 3, and obtaining a plurality of third slices;

and 5: and carrying out horizontal, rotation and mirror image operation on the first slice, the second slice and the third slice, and summarizing the first slice, the second slice, the third slice, the slice subjected to horizontal operation, the slice subjected to rotation operation and the slice subjected to mirror image operation to be used as a final training sample.

2. The remote sensing image semantic segmentation-oriented deep learning training sample enhancement method according to claim 1, wherein the step 2 specifically comprises:

step 2.1: and randomly selecting a sliding overlapping degree as C in the reference overlapping degree range, and then cutting the window offset at each time:

N_K＝W-[W×C]

wherein N is_KW is the dimension of the clipping window for the offset of each clipping window;

step 2.2: judging whether the Kth sliding in each line of the cutting window exceeds the image boundary or not, and ensuring that the Kth cutting meets the following requirements:

K′＝(K-1)×N_K+W≤X

in the formula, K' is the pixel position of the right side of the cutting window when the cutting window slides for the Kth time, and X is the number of pixels in the horizontal direction of the image;

three situations arise at this time:

w is less than X-K', then continuing to perform the next sliding cutting of the line, and continuing to judge the sliding situation of K +1 times;

(K 'X), the best overlap of clipping windows in the horizontal direction C'_K＝C；

③0<X-K′<W, reallocating the residual pixels of the line to the horizontal offset of the clipping window to obtain a horizontal new sliding quantity N'_KC 'at the same time'_KThe range of the reference overlapping degree of the cutting window is met;

C′_K＝(W-N′_K)/W∈α

wherein α is the reference overlap whenWhen, C_K' is the optimal overlap of the cropping windows in the horizontal direction;

step 2.3: optimum overlap C of cropping windows in horizontal direction_KAnd the preprocessed image and the marked image are respectively subjected to sliding cutting in the horizontal direction in a mode that cutting windows are not overlapped in the vertical direction, so that a plurality of first slices are obtained.

3. The remote sensing image semantic segmentation-oriented deep learning training sample enhancement method according to claim 1, wherein the step 3 specifically comprises:

step 3.1: and randomly selecting a sliding overlapping degree as C in the reference overlapping degree range, and then cutting the window offset at each time:

N_L＝W-[W×C]

in the formula, N_LThe offset of each window clipping;

step 3.2: judging whether the L-th sliding in each row of the cutting window exceeds the image boundary, and ensuring that the L-th cutting meets the following requirements:

L′＝(L-1)×N_L+W≤Y

in the formula, L' is the pixel position of the lower side of the cutting window when the L-th cutting window slides, and Y is the number of pixels in the vertical direction of the image;

three situations arise at this time:

w < Y-L', continuing to perform the next sliding cutting of the sequence and continuously judging the L +1 sliding condition.

(ii) (. L ' is Y), the optimum overlap C ' of the cropping windows in the vertical direction '_L＝C；

③0<Y-L′<W, reallocating the residual pixels of the line to the horizontal offset of the clipping window to obtain a horizontal new sliding quantity N 'and C'_LThe range of the reference overlapping degree of the cutting window is met;

C′_L＝(W-N′_L)/W∈α

when in useOf C'_LThe optimal overlapping degree of the cutting windows in the vertical direction;

step 3.3: optimum overlap C 'of cropping windows in vertical direction'_LAnd the preprocessed image and the marked image are respectively subjected to sliding cutting in the vertical direction in a mode that cutting windows are not overlapped in the horizontal direction, so that a plurality of second slices are obtained.

Background

Semantic segmentation is always an important field in remote sensing images and an important means for understanding the remote sensing images. With the development of deep learning, the combined application of semantic segmentation and deep neural networks achieves remarkable results. However, training of deep networks often requires a large number of training samples to meet the model's requirements for feature representation capability. When the training set is small, the trained network model cannot well fit the abstract features of the training set, and therefore the performance is poor. The data enhancement method of the training sample is to expand the training set by a certain method under the condition of keeping the original characteristics of the sample, improve the generalization capability of the data and enhance the context relation of the remote sensing image. Even with less training data, data enhancement can be used to increase the amount or partial features of the training data, so that the network model is more robust. In this case, how to perform enhancement processing on a certain training sample is a key.

The existing remote sensing image data enhancement method comprises turning, rotating, zooming and random cutting. In the semi-supervised full volume network method for semantic segmentation of high-resolution remote sensing images, such as Gunn Lei, and the like, data are enhanced by rotation, left-right turning and up-down turning. In the Wandne and other 'neural network-based remote sensing image semantic segmentation methods', sliding cutting is carried out at a certain step length, and then turning transformation and rotation are carried out to enhance data. A high-resolution remote sensing image semantic segmentation method based on U-Net of Sujian and the like intercepts each original image and label in a training set into 5 sub-images, and then turns over image blocks (horizontally, leftwards and rightwards and along diagonal lines), adjusts colors (brightness, contrast and saturation), and performs noise processing to enhance data. However, the above operations of rotation, horizontal and inversion have limited data enhancement effect, and in magnitude, the expansion effect of the training samples obtained by rotation, horizontal and inversion is not obvious, and more basic images are still needed in the case of needing a large number of training samples. Secondly, the training samples lack correlation, the context relationship among the training samples cannot be accurately expressed, the mixed pixels cannot be accurately classified in the semantic segmentation process, and the trained network cannot well fit the abstract features of the training set.

Disclosure of Invention

In order to solve the problems that when a training set is few, a training sample lacks context connection, and a trained network model cannot well fit abstract features of the training set, the invention provides a deep learning training sample enhancement method for pixel semantic segmentation.

The invention adopts the following technical scheme:

a deep learning training sample enhancement method for remote sensing image semantic segmentation comprises the following steps:

step 1: the method comprises the steps of obtaining a high-resolution satellite remote sensing image of a target area, preprocessing the satellite remote sensing image to obtain a preprocessed image, wherein the preprocessing comprises radiometric calibration and atmospheric correction, and manually marking the preprocessed image by a visual interpretation method and converting the preprocessed image into a grid image as a marked image;

step 2: selecting the dimension of the training model input layer as a cutting window, calculating the optimal overlapping degree of the cutting window in the horizontal direction, and then respectively carrying out sliding cutting in the horizontal direction on the preprocessed image and the marked image in the horizontal direction in a mode that the cutting window is not overlapped in the horizontal direction and the cutting window is not overlapped in the vertical direction to obtain a plurality of first slices;

and step 3: calculating the optimal overlapping degree of the cutting windows in the vertical direction, and then respectively carrying out sliding cutting in the vertical direction on the preprocessed image and the marked image in the vertical direction in a mode that the cutting windows are not overlapped in the vertical direction and the cutting windows are not overlapped in the horizontal direction to obtain a plurality of second slices;

and 4, step 4: respectively performing sliding clipping on the preprocessed image and the marked image according to the optimal overlapping degree of the clipping window in the horizontal direction obtained in the step 2 and the optimal overlapping degree of the clipping window in the vertical direction obtained in the step 3, and obtaining a plurality of third slices;

and 5: and carrying out horizontal, rotation and mirror image operation on the first slice, the second slice and the third slice, and summarizing the first slice, the second slice, the third slice, the slice subjected to horizontal operation, the slice subjected to rotation operation and the slice subjected to mirror image operation to be used as a final training sample.

Preferably, step 2 specifically comprises:

step 2.1: and randomly selecting a sliding overlapping degree as C in the reference overlapping degree range, and then cutting the window offset at each time:

N_K＝W-[W×C]

wherein N is_KW is the dimension of the clipping window for the offset of each clipping window;

step 2.2: judging whether the Kth sliding in each line of the cutting window exceeds the image boundary or not, and ensuring that the Kth cutting meets the following requirements:

K′＝(K-1)×N_K+W≤X

in the formula, K' is the pixel position of the right side of the cutting window when the cutting window slides for the Kth time, and X is the number of pixels in the horizontal direction of the image;

three situations arise at this time:

w is less than X-K', then continuing to perform the next sliding cutting of the line, and continuing to judge the sliding situation of K +1 times;

(K 'X), the best overlap of clipping windows in the horizontal direction C'_K＝C；

③0<X-K′<W, reallocating the residual pixels of the line to the horizontal offset of the clipping window to obtain a horizontal new sliding quantity N'_KC 'at the same time'_KThe range of the reference overlapping degree of the cutting window is met;

C′_K＝(W-N′_K)/W∈α

wherein α is the reference overlap whenWhen, C_K' is the optimal overlap of the cropping windows in the horizontal direction;

step 2.3: optimum overlap C of cropping windows in horizontal direction_KAnd the preprocessed image and the marked image are respectively subjected to sliding cutting in the horizontal direction in a mode that cutting windows are not overlapped in the vertical direction, so that a plurality of first slices are obtained.

Preferably, step 3 specifically comprises:

step 3.1: and randomly selecting a sliding overlapping degree as C in the reference overlapping degree range, and then cutting the window offset at each time:

N_L＝W-[W×C]

in the formula, N_LThe offset of each window clipping;

step 3.2: judging whether the L-th sliding in each row of the cutting window exceeds the image boundary, and ensuring that the L-th cutting meets the following requirements:

L′＝(L-1)×N_L+W≤Y

in the formula, L' is the pixel position of the lower side of the cutting window when the L-th cutting window slides, and Y is the number of pixels in the vertical direction of the image;

three situations arise at this time:

w < Y-L', continuing to perform the next sliding cutting of the sequence and continuously judging the L +1 sliding condition.

(ii) L' is Y, the optimum overlap C of the cropping windows in the vertical direction_L′＝C；

③0<Y-L′<W, re-distributing the residual pixels of the column to the horizontal offset of the clipping window to obtain a new horizontal sliding quantity N', and C_L' a range that corresponds to the clipping window reference overlap;

C′_L＝(W-N_L′)/W∈α

when in useOf C'_LThe optimal overlapping degree of the cutting windows in the vertical direction;

step 3.3: optimum overlap C 'of cropping windows in vertical direction'_LAnd the preprocessed image and the marked image are respectively subjected to sliding cutting in the vertical direction in a mode that cutting windows are not overlapped in the horizontal direction, so that a plurality of second slices are obtained.

The invention has the beneficial effects that:

the invention provides a training sample enhancement method facing pixel semantic segmentation based on satellite images, provides an optimal overlapping range between adjacent clipping windows, designs three clipping modes, maximally utilizes original data and marking data, avoids waste of the original image and the marking data in the clipping process, performs sliding clipping on the original image and the marking image in the horizontal direction, the vertical direction, the horizontal direction and the vertical direction with optimal overlapping degree, and then performs mirroring, horizontal and rotation, fully utilizes sample data, reduces data loss in the sliding clipping process, can finally obtain a large amount of training data, is beneficial to subsequent deep learning training facing pixel semantic segmentation, and provides a large amount of data bases for the training sample data.

Drawings

FIG. 1 is a flow chart of the steps performed in the present invention.

FIG. 2 is a schematic diagram of the horizontal, vertical and the main parameters of the present invention.

Fig. 3 is a schematic view of slices with sliding overlap in the horizontal direction.

Fig. 4 is a schematic view of slices with sliding overlap in the vertical direction.

Fig. 5 is a schematic view of slices with sliding overlap in both the horizontal and vertical directions.

Detailed Description

The following description of the embodiments of the present invention will be made with reference to the accompanying drawings:

with reference to fig. 1 to 5, a deep learning training sample enhancement method for semantic segmentation of remote sensing images includes the following steps:

step 1: the method comprises the steps of obtaining a high-resolution satellite remote sensing image of a target area, preprocessing the satellite remote sensing image to obtain a preprocessed image, wherein the preprocessing comprises radiometric calibration and atmospheric correction, and manually marking the preprocessed image by a visual interpretation method and converting the preprocessed image into a raster image as a marked image.

Step 2: selecting the dimension of the training model input layer as a cutting window, calculating the optimal overlapping degree of the cutting window in the horizontal direction, and then respectively carrying out sliding cutting in the horizontal direction on the preprocessed image and the marked image in the horizontal direction in a mode that the cutting window is not overlapped in the horizontal direction and the cutting window is not overlapped in the vertical direction to obtain a plurality of first slices.

The step 2 specifically comprises the following steps:

step 2.1: and randomly selecting a sliding overlapping degree as C in the reference overlapping degree range, and then cutting the window offset at each time:

N_K＝W-[W×C]

wherein N is_KW is the dimension of the clipping window for the offset of each clipping window;

step 2.2: judging whether the Kth sliding in each line of the cutting window exceeds the image boundary or not, and ensuring that the Kth cutting meets the following requirements:

K′＝(K-1)×N_K+W≤X

in the formula, K' is the pixel position of the right side of the cutting window when the cutting window slides for the Kth time, and X is the number of pixels in the horizontal direction of the image;

three situations arise at this time:

w is less than X-K', then continuing to perform the next sliding cutting of the line, and continuing to judge the sliding situation of K +1 times;

(K 'X), the best overlap of clipping windows in the horizontal direction C'_K＝C；

③0<X-K′<W, reallocating the residual pixels of the line to the horizontal offset of the clipping window to obtain a horizontal new sliding quantity N'_KC 'at the same time'_KCutting-in fitWindow reference overlap range;

C′_K＝(W-N′_K)/W∈α

wherein α is the reference overlap whenWhen, C_K' is the optimal overlap of the cropping windows in the horizontal direction;

step 2.3: optimum overlap C of cropping windows in horizontal direction_KAnd the preprocessed image and the marked image are respectively subjected to sliding cutting in the horizontal direction in a mode that cutting windows are not overlapped in the vertical direction, so that a plurality of first slices are obtained. As shown in fig. 3.

And step 3: and calculating the optimal overlapping degree of the cutting window in the vertical direction, and then respectively carrying out sliding cutting in the vertical direction on the preprocessed image and the marked image in the vertical direction in a mode that the cutting windows are not overlapped in the vertical direction and the cutting windows are not overlapped in the horizontal direction to obtain a plurality of second slices.

The step 3 specifically comprises the following steps:

step 3.1: and randomly selecting a sliding overlapping degree as C in the reference overlapping degree range, and then cutting the window offset at each time:

N_L＝W-[W×C]

in the formula, N_LThe offset of each window clipping;

step 3.2: judging whether the L-th sliding in each row of the cutting window exceeds the image boundary, and ensuring that the L-th cutting meets the following requirements:

L′＝(L-1)×N_L+W≤Y

in the formula, L' is the pixel position of the lower side of the cutting window when the L-th cutting window slides, and Y is the number of pixels in the vertical direction of the image;

three situations arise at this time:

w < Y-L', continuing to perform the next sliding cutting of the sequence and continuously judging the L +1 sliding condition.

(ii) L' is Y, the optimum overlap C of the cropping windows in the vertical direction_L′＝C；

③0<Y-L′<W, re-distributing the residual pixels of the column to the horizontal offset of the clipping window to obtain a new horizontal sliding quantity N', and C_L' a range that corresponds to the clipping window reference overlap;

C′_L＝(W-N_L′)/W∈α

when in useOf C'_LThe optimal overlapping degree of the cutting windows in the vertical direction;

step 3.3: optimum overlap C 'of cropping windows in vertical direction'_LAnd the preprocessed image and the marked image are respectively subjected to sliding cutting in the vertical direction in a mode that cutting windows are not overlapped in the horizontal direction, so that a plurality of second slices are obtained. As shown in fig. 4.

And 4, step 4: and (3) respectively performing sliding cutting on the preprocessed image and the marked image according to the optimal overlapping degree of the cutting window in the horizontal direction obtained in the step (2) and the optimal overlapping degree of the cutting window in the vertical direction obtained in the step (3) and the optimal overlapping degree of the cutting window in the horizontal direction and the optimal overlapping degree of the cutting window in the vertical direction, so as to obtain a plurality of third slices. As shown in fig. 5.

And 5: and carrying out horizontal, rotation and mirror image operation on the first slice, the second slice and the third slice, and summarizing the first slice, the second slice, the third slice, the slice subjected to horizontal operation, the slice subjected to rotation operation and the slice subjected to mirror image operation to be used as a final training sample.

It is to be understood that the above description is not intended to limit the present invention, and the present invention is not limited to the above examples, and those skilled in the art may make modifications, alterations, additions or substitutions within the spirit and scope of the present invention.