1. An unsupervised field self-adaptive remote sensing image segmentation method based on optimal transmission is characterized by comprising the following steps: at least comprises the following steps:

the method comprises the following steps: firstly, inputting a marked source domain sample and an unmarked target domain sample into a segmentation network trained by a source domain to obtain source domain foreground region characteristics and background region characteristics, calculating class centers, comparing the target sample foreground region characteristics with the class centers of the categories to which the target sample foreground region characteristics belong, calculating the probability that the foreground region of the target sample image is correctly segmented, and further obtaining the segmentation accuracy weight for guiding the optimal transmission of the segmented remote sensing image sample;

step two: selecting an optimal sample subset of the remote sensing image based on the remote sensing image segmentation accuracy weight calculation method provided in the first step, firstly changing a cost matrix of optimal transmission into a product of an original cost matrix and segmentation accuracy weight to enable the optimal transmission to become segmentation accuracy weighted optimal transmission, then carrying out weighted optimal transmission guided by the segmentation accuracy as weight on a source domain sample set and a target domain sample set to obtain an optimal coupling matrix, analyzing the value of the optimal coupling matrix according to the property of the coupling matrix, and selecting the optimal sample subset suitable for the remote sensing image segmentation sample;

step three: performing optimal transmission feature selection on the optimal sample subset of the remote sensing image obtained in the second step, firstly, using a pre-trained full convolution network as a feature extractor to extract the features of the optimal sample subset selected in the second step, then discretizing the feature distribution of the optimal sample subset, transmitting the source feature distribution to a target feature distribution by using entropy regularization optimal transmission to obtain an optimal coupling matrix of feature transmission, obtaining a feature similarity reduced sequence table by analyzing and comparing the similarity of the same features of two domains, and further selecting the domain invariant features of the two domains;

step four: carrying out unsupervised field self-adaptive image segmentation on the remote sensing image feature selection result obtained in the step three, sorting the high-dimensional features output by the full convolution network in the step three, and selecting the front d of the list^*And the characteristic is that the selected low-dimensional characteristic is input into an up-sampling part of the segmentation network, a global loss function which can simultaneously execute image segmentation and a domain self-adaptive task is determined, an unsupervised domain self-adaptive remote sensing image segmentation model is obtained, and the remote sensing image is segmented.

2. The unsupervised domain self-adaptive remote sensing image segmentation method based on optimal transmission according to claim 1, characterized in that: the method comprises the steps of training a U-net remote sensing image segmentation network by using a marked source domain sample, inputting the source domain sample and a target domain sample, extracting feature maps of a source domain and a target domain from the source domain sample and the target domain sample, measuring cross-domain difference of the segmented remote sensing image by using segmentation accuracy weights aiming at the remote sensing image, and aiming at preventing a foreground region from being segmented wrongly when the target domain sample is trained by using the source domain segmentation network.

3. The unsupervised domain self-adaptive remote sensing image segmentation method based on optimal transmission according to claim 1, characterized in that: and step two, providing selection of an optimal sample subset of the remote sensing image, firstly discretizing the edge probability distribution of the source domain sample and the target domain sample, then obtaining optimal coupling of probability coupling of two empirical distributions and Frobenius minimum dot product of the cost matrix by solving an optimal transmission problem, then changing the original optimal transmission cost matrix into a weighted cost matrix by using the weight obtained in the step one, finally carrying out matching between the source domain sample and the target domain sample by the weighted optimal transmission to obtain an optimal solution of the coupling matrix, and analyzing the value of the optimal solution of the coupling matrix, thereby selecting the optimal sample subset.

4. The unsupervised domain self-adaptive remote sensing image segmentation method based on optimal transmission according to claim 1, characterized in that: and (3) providing optimal transmission characteristic selection in the third step, firstly, using a pre-trained FCN network as a characteristic extractor to extract the characteristics of the optimal sample subset selected in the second step, further discretizing the characteristic distribution of the optimal sample subset after two domain selections as the sample distribution discretization, and transmitting the source characteristic distribution to the target characteristic distribution by using entropy regularization optimal transmission to obtain a characteristic transmission optimal coupling matrix gamma^*fAnd analysis of the coupling matrix gamma^*In contrast, in the analysis of gamma^*fWhen the values of (a) and (b) are equal, only the coupling matrix gamma needs to be analyzed since only the similarity of the same features of the two domains is compared^*fThe value on the diagonal line is obtainedTo reduced sequence table F for feature similarity.

5. The unsupervised domain self-adaptive remote sensing image segmentation method based on optimal transmission according to claim 1, characterized in that: step four, providing a feature selection result based on step three to perform unsupervised domain self-adaptive image segmentation, and selecting the front d of the feature selection result according to the list F obtained in step three^*Inputting the up-sampling stage of the FCN in the third step, wherein the global loss function of the FCN is composed of two parts of segmentation loss and domain adaptive loss, and finally performing remote sensing image segmentation by using the unsupervised domain adaptive image segmentation model.

Background

With the development and improvement of remote sensing technology, remote sensing data is continuously generated in the global scope. The mass remote sensing data becomes an important information source for human cognition in the world, and plays an important role in a plurality of fields such as environment detection, city planning, land segmentation and the like. In order to effectively mine rich information provided by mass remote sensing data, remote sensing image segmentation has become one of research hotspots in related fields.

In recent years, the emission of a high-resolution remote sensing satellite enables the spatial resolution of a remote sensing image to be increased continuously, the spatial details of the ground features are obviously improved, and the possibility is provided for the fine target segmentation of the ground features. High resolution images exhibit many different characteristics relative to medium and low resolution remote sensing images: the ground object categories such as buildings, roads, vegetation and the like have obvious geometric texture characteristics; the imaging spectrum wave band is reduced, and the phenomena of 'same object different spectrum' and 'same spectrum foreign matter' are generated in a large quantity.

Because the physical conditions (such as illumination, atmosphere, sensor parameters and the like) during the acquisition of the remote sensing images cannot be completely the same, the images obtained in different areas or at different moments on the same scene have certain difference, namely the bottom layer characteristics of the same ground object type in the same semantic scene change obviously. Different data characteristics reflect different probability distributions, and therefore multi-source remote sensing data is generally considered to be in different probability distributions.

For multi-source remote sensing data in different probability distributions, the traditional segmentation method is difficult to establish direct mapping from the bottom layer characteristics of the multi-source image to the high-layer semantic information of the multi-source image, so that the accuracy improvement of a segmentation model is limited. Therefore, the key for improving the segmentation precision of the multi-source remote sensing image is to solve the conversion among a plurality of different probability distributions and map the multi-source data to the same probability distribution. Transfer learning is an important method for solving the mathematical problem, and the conversion from multi-source distribution to single-source distribution is realized by solving a mapping function between probability distributions, so that a model is learned on the converted probability distributions.

The key problem of the multi-source remote sensing image segmentation task lies in how to map surface feature targets in different probability distributions to the same distribution by using a transfer learning technology, and construct a segmentation model on the mapped data characteristics. Domain Adaptation (DA) is one of the most representative problems in migration learning, and focuses on solving the problems of consistent feature space, consistent category space, and inconsistent feature distribution.

The most common methods for field adaptation today are based on two perspectives: data distribution and feature transformation. From the data distribution perspective, namely when the probability distributions of the source domain and the target domain are similar, adopting a method of minimizing the distance of the probability distribution; from a feature transformation perspective, when the source domain and the target domain share some subspaces, both domains are transformed to the same subspace. The limitations of the current technology are: all features extracted in the feature extraction stage of the domain adaptive model are applied to the domain adaptive model, however, not every feature will have a positive effect in the adaptive model, and incorrectly relating all features may even result in performance degradation. Although some feature selection methods exist in the field adaptive task at present, most methods are performed under a supervised segmentation model, wherein most methods also ignore original feature information, and the effectiveness is greatly reduced.

In summary, an unsupervised field self-adaptive remote sensing image segmentation method based on optimal transmission is provided. Firstly, carrying out transmission between two domain samples by using optimal transmission of segmentation accuracy weight guide weighting, and increasing the similarity of two domain representation spaces; and then, entropy regularization optimal transmission is utilized to transmit among the features, the domain invariant features are selected, and the selected features are subjected to unsupervised domain self-adaptive image segmentation, so that the segmentation performance is improved, and the calculation complexity of the self-adaptive algorithm is reduced.

The invention content is as follows:

in order to solve the problem that the segmentation result of the existing segmentation model for multi-source remote sensing data is not ideal, the invention provides an unsupervised field self-adaptive remote sensing image segmentation method based on optimal transmission, which mainly comprises the following steps: the image class center information matrix is segmented, the segmentation accuracy weight of optimal transmission is guided, a sub-sample selection algorithm of the optimal transmission is weighted, a feature selection algorithm of the optimal transmission is normalized by entropy, and the feature selection result is utilized to carry out unsupervised field self-adaptive remote sensing image segmentation.

An unsupervised field self-adaptive remote sensing image segmentation method based on optimal transmission is characterized by comprising the following steps: at least comprises the following steps:

step one, a remote sensing image segmentation accuracy weight calculation method is provided. Firstly, inputting a marked source domain sample and an unmarked target domain sample into a U-net segmentation network trained by a source domain to obtain foreground region characteristics and background region characteristics of the source domain. According to which the class center corresponding to each class is calculatedAnd comparing the foreground region characteristics of the target sample with the class center of the class to which the foreground region characteristics belong, calculating the probability P that the foreground region of the target sample image is correctly segmented, and further obtaining the segmentation accuracy weight for guiding the optimal transmission of the remote sensing image sample for segmentation.

And step two, selecting the optimal sample subset of the remote sensing image based on the remote sensing image segmentation accuracy weight calculation method provided in the step one. Firstly, the cost matrix of optimal transmission is changed into the product of the original cost matrix and the segmentation accuracy weight, then the source domain sample set and the target domain sample set are subjected to weighted optimal transmission guided by the segmentation accuracy as the weight to obtain the optimal coupling matrix, and the optimal coupling matrix is obtained according to the coupling matrix gamma^*The values are analyzed for the properties of the remote sensing image, and an optimal sample subset suitable for the remote sensing image segmentation sample is selected.

And step three, selecting the optimal transmission characteristics of the optimal sample subset of the remote sensing image obtained in the step two. Firstly, a pre-trained FCN full convolution network is used as a feature extractor to extract the features of the optimal sample subset selected in the second step, then the feature distribution of the optimal sample subset is discretized, the source feature distribution is transmitted to the target feature distribution by using the entropy regularization optimal transmission problem to obtain an optimal coupling matrix of feature transmission, and the similarity of the same features of two domains is analyzed and compared, namely the coupling matrix gamma is analyzed^*fAnd obtaining a characteristic similarity reduced sequence table F according to the values on the diagonal line, and further selecting the domain invariant characteristics of the two domains.

Step four, selecting the remote sensing image characteristics obtained in the step threeAnd finally, carrying out unsupervised field self-adaptive image segmentation. According to a list F obtained by ordering high-dimensional features output by the FCN full convolution network in the step three after down-sampling, the front d of the list F is selected^*A characteristic (d)^*And d), inputting the selected low-dimensional features into an up-sampling stage of the FCN segmentation network, determining a global loss function capable of simultaneously executing segmentation and adaptation tasks, obtaining an unsupervised field self-adaptation segmentation model, and performing image segmentation on the remote sensing image. The selected features are used to help the model to remove the domain-independent features of the two domains before learning, and further the accuracy of the unsupervised domain self-adaptive remote sensing image segmentation model is improved.

Has the advantages that:

compared with the prior art, the design scheme of the invention can achieve the following technical effects:

1. segmentation accuracy weights are defined. The segmentation accuracy weight calculation comprises class center calculation of various types of image segmentation samples, and the class center calculation is used in the probability information matrix P to be compared with the foreground region characteristics of the target sample, so that the integrity of the two domain characteristics is ensured, and meanwhile, the foreground region characteristics are more targeted and more suitable for segmenting the samples than the common sample characteristics, and are more accurate.

2. The segmentation accuracy weights guide the optimal transmission compared to other weighted optimal transmissions: the segmentation accuracy weight can better adapt to the segmented images, and the segmented images of two domains belonging to the same category can be more similar, so that the accuracy of entropy regularization for optimal transmission of the features of the segmented images is improved.

3. And performing inter-feature transmission by using entropy regularization optimal transmission. The entropy regularization can promote the transmission of a plurality of paths with small flow, and restrain a plurality of paths with large flow, so that the transmission version is smoother and the transmission is more accurate. And simultaneously, the optimal transmission is utilized to select the characteristics, so that the original characteristic information can be effectively utilized.

4. The unsupervised field self-adaptive method based on the optimal transmission characteristic selection can be used as a preprocessing step of any unsupervised field self-adaptive model facing image segmentation, the good effect of other models is kept, the model performance is improved, and the calculation complexity of a self-adaptive algorithm is reduced.

Description of the drawings:

FIG. 1 is a method framework flow diagram

FIG. 2 is a flow chart of segmentation accuracy weight calculation

The specific implementation mode is as follows:

step one, a remote sensing image segmentation accuracy weight calculation method is provided.

Firstly, training a U-net remote sensing image segmentation network by using a marked source domain sample, and then, training the marked source domain remote sensing image sampleAnd unmarked target domain remote sensing image sampleInputting a U-net segmentation network trained by a source domain remote sensing image, extracting segmentation feature maps of a source domain and a target domain from the U-net segmentation network, and obtaining foreground region features and background region features of the source domain(k-1 denotes foreground region, k-2 denotes background region, and g-6 denotes six different categories of remote sensing image). Measuring the cross-domain difference of the segmented remote sensing image by utilizing the segmentation accuracy weight aiming at the remote sensing image, aiming at preventing a foreground region from being segmented by mistake when a target domain sample trains a segmentation network by using a source domain, wherein the following is a segmentation accuracy weight calculation process:

(1) calculating the class center, wherein the specific formula is as follows:(where a varies with the specific number of samples per category)

(2) Comparing the foreground region characteristics of the target sample with the class center, calculating the probability that the foreground region of the target sample image is correctly segmented, wherein the specific definition is given by a matrix P, wherein g is the class to which the sample belongs:

(3) whereinForeground region feature being of target domain sample class gClass center of two regions from source domainWhile at the same timeIs defined as follows:

wherein K (x)^s,x^t)＝<φ(x^s),φ(x^t)>Is the feature kernel associated with the feature map phi.

(4) Defining the weight of segmentation accuracy by a specific formula R, R (j) 1-P (j)

And step two, selecting the optimal sample subset of the remote sensing image based on the remote sensing image segmentation accuracy weight calculation method provided in the step one.

Assume a set S (N) of active domain samples_sX d dimensional matrix) target field sample set T (N)_tX d dimensional matrix). And distributing the marginal probability of the samples on the source domain and the target domain_s，μ_tExpressed in discrete form, taking into account the case of uniform distribution, i.e. let

These are two empirical probability measures, defined as two sets of pointsAndthere is a uniformly weighted sum of Diracs of mass over the defined locations.

(1) The optimal transmission objective is to solve for the optimal coupling when the Frobenius dot product of the probability coupling gamma and the cost matrix C of the two empirical distributions is minimized. The original optimal transmission cost matrix is changed into a weighted cost matrix by using the weight R obtained in the step one, and the specific formula of the segmentation accuracy weighted optimal transmission is as follows:

wherein<·，·>Is Frobenius dot product, C is cost matrix, primary price matrixDefine the moving probability qualityToThe price matrix becomes after adding the segmentation accuracy weightRepresented as a probabilistic coupling of two empirical distributions.

(2) According to the calculation property of the optimal transmission for feature transmission, and in order to avoid the spatial difference between the two domains, equal number of source domain and target domain samples are selected before feature transmission, where N of the source feature is described_sSamples and N describing target features_tMatching among samples based on segmentation accuracy weighted optimal transmission, namely calculating the formula in (1), and then obtaining the optimal solution gamma^*To pass through the segmentation accuracy weight indexAnd (4) weighting the optimal transmission coupling matrix.

(3) Analyzing the optimal solution gamma of the coupling matrix^*Each element of the matrix is represented as a probability mass magnitude of the i-th source sample transmitted to the j-th target sample. To minimize the optimal transmission formula, the smaller the transmitted sample pair cost, which is in the coupling matrix γ^*The greater the transmission quality, thereby selecting the corresponding gamma of each sample in the source domain^*Maximum median target domain sample set T_u(at this time N_s＞N_tWhen N is present_s＜N_tTime calculation of S_u). The segmentation accuracy weighted optimal transmission enables the segmentation image samples to be matched accurately, and the target domain (source domain) samples selected by the method are more suitable for describing the characteristics of a source domain and a target domain.

And step three, selecting the optimal transmission characteristics of the optimal sample subset of the remote sensing image obtained in the step two.

An FCN network based on VGG16 is used as a feature extractor, ImageNet pre-training parameters are used for initialization, and extracted features are transmitted. The source and target domain samples (i.e., S and T) should be from two-dimensional product space X_s×F_sAnd X_t×F_tIs extracted from (A) in which X is_s,X_tAs a d-dimensional matrix), F_sIs N_sDimension matrix and F_tIs N_tA dimension matrix.

(1) Firstly, discretizing the feature distribution of the optimal sample subset after two domain selections as sample distribution discretization, and defining the following two empirical probability measures:

whereinAndbased on the characteristics of the source domain and the target domain, respectively.

(2) Under this definition, the source features are distributed using entropy regularization optimal transmissionDelivery to target feature distributionThe method comprises the following steps:

wherein

(3) Obtaining the optimal coupling matrix gamma of characteristic transmission by solving the formula in (2)^*fAnd the values thereof were analyzed. And analysis of the coupling matrix gamma^*In contrast, in the analysis of gamma^*fWhen the values of (a) and (b) are equal, only the coupling matrix gamma needs to be analyzed since only the similarity of the same features of the two domains is compared^*fThe value on the diagonal is sufficient.

(4) When the characteristics of the source domain and the target domain are similar, most of the quality of the source domain characteristic transmission is found on the corresponding target characteristic, i.e. the coupling matrix gamma^*fThe larger the value on the diagonal, the more similar the feature is on the source and target domains. According to the above thought, a feature similarity reduced sequence table F is constructed, wherein the numbers to which the features belong are represented by i, the features i are arranged in the order of the highest coupling value on the jth diagonal, and obviously, i is equal to j, and the construction result is as follows:

F＝argsort({{diag(γ^*f)}_ij|i,j∈{1,...,d})

and step four, selecting the optimal sample characteristic selection result of the remote sensing image obtained in the step three, and carrying out unsupervised field self-adaptive image segmentation.

(1) In order to execute the segmentation and adaptation tasks simultaneously, the FCN network in the step three is utilized, and the global loss function of the FCN network consists of two parts, namely segmentation loss and domain adaptive loss, and the specific formula is as follows:

the second term of the function is an optimal transmission formula, and the first term in the product of gamma can avoid catastrophic forgetting caused by the reduction of the performance of a source domain; the second term ensures that the output of the source sample connected to the target sample does not differ much from the true segmentation of the target sample in the euclidean distance sense.

(2) According to the list F obtained in the third step, the front d of the list F is selected^*A characteristic (d)^*< d), the method solves the problem of self-adaptive remote sensing image segmentation in the field of hands, and the method discards the completely different characteristics of two domains. Namely, the high-dimensional features output by the FCN network downsampling in the step three are selected, the selected low-dimensional features are input into the upsampling part of the model segmentation network, and the global loss function of the model segmentation network is changed into:

and finally, the remote sensing image segmentation is carried out by utilizing the unsupervised domain self-adaptive image segmentation model, the performance of the segmentation model after the unsupervised domain self-adaptive method based on optimal transmission is better than that when the segmentation model is not used, the complexity of a self-adaptive algorithm is obviously reduced, and the accuracy of the model segmentation is improved.