1. An excitation fluorescence tomography method based on a GCN residual error connecting network is characterized by comprising the following steps:

s10, segmenting the CT image data of the organism according to the tissue and organ types, and meshing the segmented CT image data by a triangular meshing modeling method to be used as a standard mesh; establishing a weighted adjacency matrix of a graph structure according to the distance between the vertexes of the standard grids and the connection relation;

s20, simulating the photon transmission process of the in-vivo light source in the organism by combining the optical absorption scattering parameters of each tissue organ to obtain the fluorescence distribution on the surface and inside of the organism as a light source sample; expanding the light source sample according to the type of the light source sample;

s30, dividing the standard grid into a body surface node and a body internal node; for each in-vivo node, K body surface nodes closest to the in-vivo node are selected, a first node set is constructed, and the serial number of each node in the first node set is recorded;

s40, inputting the expanded light source sample and each node in the first node set into a pre-constructed deep learning network model to obtain a rough reconstruction positioning result and a final morphological reconstruction result of the light source; calculating a loss value based on the rough reconstruction positioning result of the light source, the final morphological reconstruction result and the real distribution information of the light source, and updating the parameters of the deep learning network model;

s50, acquiring a fluorescence image and an anatomical structure image of the body surface of the organism to be excited by fluorescence tomography; registering the fluorescence image to the body surface mesh of the standard mesh generated in the step S10 through a registration algorithm by combining the anatomical structure image, and performing normalization processing on the fluorescence intensity in the body surface mesh; based on the fluorescence intensity after normalization processing, the distance between vertexes in the registered body surface mesh and the connection relation, performing excitation fluorescence tomography reconstruction on the organism by using a trained deep learning network model to obtain the three-dimensional distribution of the fluorescence probes in the organism;

the deep learning network model is formed by connecting DNN full-connection networks and GCN graph convolution network residuals.

2. The method of claim 1, wherein the method of "extending the light source sample according to the type of the light source sample" comprises: the types of the light source samples comprise single light source samples and double light source samples;

for a single light source sample, the expansion method comprises the following steps:

Φ_noise＝Φ+n_gauss

wherein phi represents the body surface light spot information of the real light source of the single light source sample, n_gaussIs Gaussian noise,. phi_noiseIs a single light source sample with noise;

for the double-light source sample, the expansion method comprises the following steps:

wherein, X_double、Φ_doubleThe real light source distribution information and the surface light spot information of the combined double-light-source sample are respectively, i is the number of the double-light-source sample, S₂For a randomly selected set of 2 dual-source samples, X_i、Φ_iThe real light source distribution information and the surface light spot information of the ith double-light-source sample are respectively.

3. The method of claim 1, wherein the method comprises inputting the extended light source sample and each node in the first node set into a pre-constructed deep learning network model to obtain a coarse reconstruction localization result and a final morphological reconstruction result of the light source, and comprises:

normalizing the fluorescence intensity corresponding to the expanded light source sample, and inputting the normalized fluorescence intensity into the DNN full-connection network to obtain a coarse reconstruction positioning result of the light source;

and inputting the graph structure adjacency matrix into a graph network as a structure prior of GCN graph convolution calculation, and inputting the vector obtained by splicing the coarse reconstruction positioning result of the light source and the light intensity signal of each node in the first node set into the GCN graph convolution network as a characteristic vector to obtain a final morphological reconstruction result of the light source.

4. The GCN residual connection network-based excitation fluorescence tomography method according to claim 3, wherein the loss function of the deep learning network model during training is as follows:

wherein, X_FCFor the output of the DNN fully-connected network, and for the coarse relocation result, the preliminary location of the in-vivo light source, X, is performed_trueAs distribution information of real light sources, X_finalResidual connection of outputs to GCN graph convolution networks for fully connected sub-networksThe output of (a), being the final morphological reconstruction result,representing a loss of positioning accuracy of the network,representing the morphological accuracy loss, and λ represents the weighting parameter.

5. The GCN residual connection network-based excitation fluorescence tomography method according to claim 1, wherein an activation function of the deep learning network model during training is as follows:

where X represents the output result of each layer in a DNN fully-connected network or a GCN graph convolution network.

6. The method according to claim 1, wherein the method for simulating the photon propagation process of the in-vivo light source in the living body comprises: and simulating the photon transmission process of the in-vivo light source in the organism by a Monte Carlo simulation method.

7. An excitation fluorescence tomography system based on a GCN residual error connecting network, which is characterized by comprising: the system comprises a graph structure modeling module, a data set constructing module, a node classification module, a model training module and a model prediction module;

the graph structure modeling module is configured to divide the CT image data of the organism according to the tissue and organ types, and grid the divided CT image data by a triangular grid modeling method to be used as a standard grid; establishing a weighted adjacency matrix of a graph structure according to the distance between the vertexes of the standard grids and the connection relation;

the data set construction module is configured to simulate the photon propagation process of an in-vivo light source in an organism by combining the optical absorption scattering parameters of each tissue organ to obtain the fluorescence distribution on the surface and inside of the organism as a light source sample; expanding the light source sample according to the type of the light source sample;

the node classification module is configured to divide the standard grid into body surface nodes and in-vivo nodes; for each in-vivo node, K body surface nodes closest to the in-vivo node are selected, a first node set is constructed, and the serial number of each node in the first node set is recorded;

the model training module is configured to input the expanded light source sample and each node in the first node set into a pre-constructed deep learning network model to obtain a rough reconstruction positioning result and a final morphological reconstruction result of the light source; calculating a loss value based on the rough reconstruction positioning result of the light source, the final morphological reconstruction result and the real distribution information of the light source, and updating the parameters of the deep learning network model;

the model prediction module is configured to acquire a fluorescence image and an anatomical structure image of the body surface of the organism to be excited for fluorescence tomography; registering the fluorescence image to a body surface grid of a standard grid generated by a graph structure modeling module through a registration algorithm by combining the anatomical structure image, and carrying out normalization processing on the fluorescence intensity in the body surface grid; based on the fluorescence intensity after normalization processing, the distance between vertexes in the registered body surface mesh and the connection relation, performing excitation fluorescence tomography reconstruction on the organism by using a trained deep learning network model to obtain the three-dimensional distribution of the fluorescence probes in the organism;

the deep learning network model is formed by connecting DNN full-connection networks and GCN graph convolution network residuals.

8. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to at least one of the processors; wherein the content of the first and second substances,

the memory stores instructions executable by the processor for execution by the processor to implement the GCN residual connect network based excitation fluorescence tomography method of any of claims 1-6.

9. A computer readable storage medium storing computer instructions for execution by the computer to perform the GCN residual connect network based excitation fluorescence tomography method of any one of claims 1-6.

Background

Fluorescence Imaging (FMI) is an important optical Molecular Imaging technique and is also a hot spot in international research. Compared with other medical imaging technologies, the FMI imaging technology has millimeter-level spatial resolution, has the advantages of high detection sensitivity, high signal intensity, convenience in clinical transformation and the like, and can be used for detecting specific optical signals on the body surface of the biological tissue and deducing the approximate distribution region of the fluorescent probes in the biological tissue. However, FMI imaging techniques have difficulty locating the depth of the fluorescent light source, limited by the absorptive scattering effect of photons propagating within biological tissue. To solve this problem, researchers have proposed tomographic imaging techniques based on fluorescence excitation by incorporating biological tissue tomographic information. Fluorescence spot information on the surface of an organism is collected by a high-sensitivity detector through an excitation Fluorescence tomography (FMT) technology, organism anatomical structure information obtained by mode collection such as CT or MRI (magnetic Resonance imaging) is combined, and then three-dimensional distribution of Fluorescence probes in the organism is obtained by reconstruction through a transmission model of photons in the organism, so that accurate positioning of a Fluorescence light source in the organism is realized.

The FMT can effectively solve the problem that the FMI cannot locate the depth information of the light source and the like by fusing three-dimensional depth information in tomography. However, since the FMT reconstruction relies on a two-dimensional fluorescence image formed by photons scattered by biological tissue, it distorts the distribution information of the real light source to some extent, resulting in a serious ill-conditioned final reconstruction problem. In addition, the traditional FMT reconstruction method mostly uses a modeling method based on a photon propagation model, mainly models based on a radiation transmission equation, because the radiation transmission equation is an extremely complex calculus equation, the radiation transmission equation is often solved by using a first-order spherical harmonic approximation model diffusion equation, and equation complexity is reduced through finite element discretization and linearization conversion, but the accuracy of the model is also reduced while the equation complexity is reduced, which can cause the reduction of final reconstruction accuracy. Therefore, how to design a reconstruction algorithm with high reconstruction quality and high reconstruction speed has important significance and value. Based on the method, the invention provides an excitation fluorescence tomography method based on a GCN residual error connecting network.

Disclosure of Invention

In order to solve the above problems in the prior art, that is, to solve the problems of decreased model accuracy, decreased reconstruction accuracy, and slow reconstruction speed, etc. occurring in the conventional FMT reconstruction based on a photon propagation model, a first aspect of the present invention provides an excitation fluorescence tomography method based on a GCN residual connection network, the method comprising:

s10, segmenting the CT image data of the organism according to the tissue and organ types, and meshing the segmented CT image data by a triangular meshing modeling method to be used as a standard mesh; establishing a weighted adjacency matrix of a graph structure according to the distance between the vertexes of the standard grids and the connection relation;

s20, simulating the photon transmission process of the in-vivo light source in the organism by combining the optical absorption scattering parameters of each tissue organ to obtain the fluorescence distribution on the surface and inside of the organism as a light source sample; expanding the light source sample according to the type of the light source sample;

s30, dividing the standard grid into a body surface node and a body internal node; for each in-vivo node, K body surface nodes closest to the in-vivo node are selected, a first node set is constructed, and the serial number of each node in the first node set is recorded;

s40, inputting the expanded light source sample and each node in the first node set into a pre-constructed deep learning network model to obtain a rough reconstruction positioning result and a final morphological reconstruction result of the light source; calculating a loss value based on the rough reconstruction positioning result of the light source, the final morphological reconstruction result and the real distribution information of the light source, and updating the parameters of the deep learning network model;

s50, acquiring a fluorescence image and an anatomical structure image of the body surface of the organism to be excited by fluorescence tomography; registering the fluorescence image to the body surface mesh of the standard mesh generated in the step S10 through a registration algorithm by combining the anatomical structure image, and performing normalization processing on the fluorescence intensity in the body surface mesh; based on the fluorescence intensity after normalization processing, the distance between vertexes in the registered body surface mesh and the connection relation, performing excitation fluorescence tomography reconstruction on the organism by using a trained deep learning network model to obtain the three-dimensional distribution of the fluorescence probes in the organism;

the deep learning network model is formed by connecting DNN full-connection networks and GCN graph convolution network residuals.

In some preferred embodiments, the method of "extending the light source sample according to the type of the light source sample" includes: the types of the light source samples comprise single light source samples and double light source samples; :

for a single light source sample, the expansion method comprises the following steps:

Φ_noise＝Φ+n_gauss

wherein phi represents the body surface light spot information of the real light source of the single light source sample, n_gaussIs Gaussian noise,. phi_noiseIs a single light source sample with noise;

for the double-light source sample, the expansion method comprises the following steps:

wherein, X_double、Φ_doubleThe real light source distribution information and the surface light spot information of the combined double-light-source sample are respectively, i is the number of the double-light-source sample, S₂For a randomly selected set of 2 dual-source samples, X_i、Φ_iThe real light source distribution information and the surface light spot information of the ith double-light-source sample are respectively.

In some preferred embodiments, "inputting the extended light source sample and each node in the first node set into a pre-constructed deep learning network model to obtain a coarse reconstruction positioning result and a final morphological reconstruction result of the light source", includes:

normalizing the fluorescence intensity corresponding to the expanded light source sample, and inputting the normalized fluorescence intensity into the DNN full-connection network to obtain a coarse reconstruction positioning result of the light source;

and inputting the graph structure adjacency matrix into a graph network as a structure prior of GCN graph convolution calculation, and inputting the vector obtained by splicing the coarse reconstruction positioning result of the light source and the light intensity signal of each node in the first node set into the GCN graph convolution network as a characteristic vector to obtain a final morphological reconstruction result of the light source.

In some preferred embodiments, the loss function of the deep learning network model during training is:

wherein, X_FCFor the output of the DNN fully-connected network, and for the coarse relocation result, the preliminary location of the in-vivo light source, X, is performed_trueAs distribution information of real light sources, X_finalThe output of the fully connected sub-network is the output of the residual connection with the GCN graph convolution networkThe result of the final morphological reconstruction is that,representing a loss of positioning accuracy of the network,representing the morphological accuracy loss, and λ represents the weighting parameter.

In some preferred embodiments, the activation function of the deep learning network model during training is:

where X represents the output result of each layer in a DNN fully-connected network or a GCN graph convolution network.

In some preferred embodiments, the method for simulating the photon propagation process of the in-vivo light source in the organism comprises the following steps: and simulating the photon transmission process of the in-vivo light source in the organism by a Monte Carlo simulation method.

In a second aspect of the present invention, an excitation fluorescence tomography system based on a GCN residual error connection network is provided, the system comprising: the system comprises a graph structure modeling module, a data set constructing module, a node classification module, a model training module and a model prediction module;

the graph structure modeling module is configured to divide the CT image data of the organism according to the tissue and organ types, and grid the divided CT image data by a triangular grid modeling method to be used as a standard grid; establishing a weighted adjacency matrix of a graph structure according to the distance between the vertexes of the standard grids and the connection relation;

the data set construction module is configured to simulate the photon propagation process of an in-vivo light source in an organism by combining the optical absorption scattering parameters of each tissue organ to obtain the fluorescence distribution on the surface and inside of the organism as a light source sample; expanding the light source sample according to the type of the light source sample;

the node classification module is configured to divide the standard grid into body surface nodes and in-vivo nodes; for each in-vivo node, K body surface nodes closest to the in-vivo node are selected, a first node set is constructed, and the serial number of each node in the first node set is recorded;

the model training module is configured to input the expanded light source sample and each node in the first node set into a pre-constructed deep learning network model to obtain a rough reconstruction positioning result and a final morphological reconstruction result of the light source; calculating a loss value based on the rough reconstruction positioning result of the light source, the final morphological reconstruction result and the real distribution information of the light source, and updating the parameters of the deep learning network model;

the model prediction module is configured to acquire a fluorescence image and an anatomical structure image of the body surface of the organism to be excited for fluorescence tomography; registering the fluorescence image to a body surface grid of a standard grid generated by a graph structure modeling module through a registration algorithm by combining the anatomical structure image, and carrying out normalization processing on the fluorescence intensity in the body surface grid; based on the fluorescence intensity after normalization processing, the distance between vertexes in the registered body surface mesh and the connection relation, performing excitation fluorescence tomography reconstruction on the organism by using a trained deep learning network model to obtain the three-dimensional distribution of the fluorescence probes in the organism;

the deep learning network model is formed by connecting DNN full-connection networks and GCN graph convolution network residuals.

In a third aspect of the invention, an electronic device is proposed, at least one processor; and a memory communicatively coupled to at least one of the processors; wherein the memory stores instructions executable by the processor for execution by the processor to implement the method of claim for GCN residual connected network based excitation fluorescence tomography.

In a fourth aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for being executed by the computer to implement the method for GCN residual error connected network based excitation fluorescence tomography as claimed above.

The invention has the beneficial effects that:

the invention directly learns the reverse propagation process from the surface light spots of the living body to the node distribution of the in-vivo fluorescent probe based on the data-driven deep learning method, thereby realizing the excitation fluorescence tomography with high reconstruction quality and high reconstruction speed.

1) According to the FMT reconstruction method based on data driving, the nonlinear mapping relation from the body surface light spots to the in-vivo fluorescent probe nodes is directly fitted by using the nonlinear fitting capability of deep learning, and the problem of accuracy reduction in the traditional modeling based on a photon propagation model is solved. Because Monte Carlo simulation is considered as a simulation method closest to a real photon transmission process, a large number of training samples are constructed by the Monte Carlo simulation, and a training sample set is expanded by using a method of sample combination and noise countermeasure addition, so that training of a GCN residual error connection network is supported, and the reconstruction capability of the GCN residual error connection network and the positioning accuracy of FMT are improved;

2) constructing multiple light sources and irregular large light source samples by using a sample combination and adding a method for resisting noise, and improving the overall reconstruction precision of the GCN residual error connection network;

3) the deep learning model is used for directly fitting the mapping relation from the biological body surface light spots to the in-vivo nodes, the FMT reconstruction speed is improved, the graph network is used for optimizing the reconstruction result, and the morphological reconstruction precision is improved.

4) Based on the trained deep learning network model, the molecular probe distribution in the organism can be directly reconstructed by directly utilizing the bioluminescence distribution information on the surface of the organism, and compared with the traditional reconstruction method based on the photon propagation model, the method has the advantages of high positioning precision, high reconstruction speed, high morphological reconstruction precision and the like.

Drawings

Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings.

FIG. 1 is a schematic flow chart of an excitation fluorescence tomography method based on a GCN residual error connection network according to an embodiment of the present invention;

FIG. 2 is a diagram of a deep learning network model residual connection network structure according to an embodiment of the present invention;

FIG. 3 is a block diagram of an excitation fluorescence tomography system based on a GCN residual connecting network according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram of a computer system suitable for implementing an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.

It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.

The invention relates to an excitation fluorescence tomography method based on a GCN residual error connecting network, which comprises the following steps of:

s10, segmenting the CT image data of the organism according to the tissue and organ types, and meshing the segmented CT image data by a triangular meshing modeling method to be used as a standard mesh; establishing a weighted adjacency matrix of a graph structure according to the distance between the vertexes of the standard grids and the connection relation;

s20, simulating the photon transmission process of the in-vivo light source in the organism by combining the optical absorption scattering parameters of each tissue organ to obtain the fluorescence distribution on the surface and inside of the organism as a light source sample; expanding the light source sample according to the type of the light source sample;

s30, dividing the standard grid into a body surface node and a body internal node; for each in-vivo node, K body surface nodes closest to the in-vivo node are selected, a first node set is constructed, and the serial number of each node in the first node set is recorded;

s40, inputting the expanded light source sample and each node in the first node set into a pre-constructed deep learning network model to obtain a rough reconstruction positioning result and a final morphological reconstruction result of the light source; calculating a loss value based on the rough reconstruction positioning result of the light source, the final morphological reconstruction result and the real distribution information of the light source, and updating the parameters of the deep learning network model;

s50, acquiring a fluorescence image and an anatomical structure image of the body surface of the organism to be excited by fluorescence tomography; registering the fluorescence image to the body surface mesh of the standard mesh generated in the step S10 through a registration algorithm by combining the anatomical structure image, and performing normalization processing on the fluorescence intensity in the body surface mesh; based on the fluorescence intensity after normalization processing, the distance between vertexes in the registered body surface mesh and the connection relation, performing excitation fluorescence tomography reconstruction on the organism by using a trained deep learning network model to obtain the three-dimensional distribution of the fluorescence probes in the organism;

the deep learning network model is formed by connecting DNN (deep Neural networks) full-connection networks and GCN (graph Convolutional network) graph convolution network residuals.

In order to more clearly describe the method for excitation fluorescence tomography based on the GCN residual error connecting network, the following describes each step in an embodiment of the method in detail with reference to the accompanying drawings.

In the following embodiments, the training process of the deep learning network model (i.e., steps S10-S40) is described first, and then the process of obtaining the three-dimensional distribution of fluorescent probes in the living body by the GCN residual connecting network-based excitation fluorescence tomography method (i.e., step S50) is described in detail.

S10, segmenting the CT image data of the organism according to the tissue and organ types, and meshing the segmented CT image data by a triangular meshing modeling method to be used as a standard mesh; establishing a weighted adjacency matrix of a graph structure according to the distance between the vertexes of the standard grids and the connection relation;

in this embodiment, a graph structure modeling is performed on CT image data of a living body, specifically as follows:

s101, segmenting CT image data of a living body according to tissue and organ types;

s102, gridding the segmented CT image data by a triangular gridding modeling method to be used as a standard grid; outputting the positions of the vertexes of the mesh (namely the distances between the vertexes) and the connection relations between the vertexes;

and S102, establishing a weighted adjacency matrix of the graph structure according to the distance between the vertexes of the standard grids and the connection relation.

S20, simulating the photon transmission process of the in-vivo light source in the organism by combining the optical absorption scattering parameters of each tissue organ to obtain the fluorescence distribution on the surface and inside of the organism as a light source sample; expanding the light source sample according to the type of the light source sample;

in this embodiment, a monte carlo simulation is used to construct a training sample, and a method of combining samples and adding anti-noise is used to construct a multi-light-source and irregular large-light-source sample, which is specifically as follows:

s201, inputting optical absorption scattering parameters of each tissue organ into Monte Carlo simulation software to carry out Monte Carlo simulation on the photon transmission process of an in-vivo light source in an organism;

s202, arranging light sources with different sizes and different shapes at different positions in an organism, simultaneously setting the wavelengths of exciting light and emitting light, simulating fluorescence distribution on the surface of the organism by using Monte Carlo simulation, using the simulated fluorescence distribution as a light source sample, outputting actual light source distribution in the organism, using the actual light source distribution as a label for neural network training, and outputting a corresponding body surface fluorescence intensity distribution vector for training a neural network;

s203, expanding the light source sample according to the type of the light source sample;

and (2) single light source sample expansion, namely selecting the training sample generated by the single light source in the step (S202), adding Gaussian noise in a range of 5% -10% to the body surface light source intensity distribution vector of the training sample, so as to obtain an expanded single light source sample, and adding the expanded single light source sample to a training sample set, wherein the specific calculation formula is as follows:

Φ_noise＝Φ+n_gauss (1)

wherein phi represents the body surface light spot information of the real light source of the single light source sample, n_gaussIs Gaussian noise,. phi_noiseAre single light source samples with noise.

And (2) expanding the double-light-source sample, namely selecting the training sample generated by the single light source in the step (S202), randomly selecting 2 training samples with the distance larger than the threshold value theta, respectively adding the corresponding actual light source and the surface fluorescence to obtain the double-light-source training sample, and adding the double-light-source training sample into a training sample set, wherein a specific calculation formula is as follows:

wherein, X_double、Φ_doubleThe real light source distribution information and the surface light spot information of the combined double-light-source sample are respectively, i is the number of the double-light-source sample, S₂For a randomly selected set of 2 dual-source samples, X_i、Φ_iReal light source distribution information and surface light spot information of the ith double-light-source sample respectively

S30, dividing the standard grid into a body surface node and a body internal node; for each in-vivo node, K body surface nodes closest to the in-vivo node are selected, a first node set is constructed, and the serial number of each node in the first node set is recorded;

in this embodiment, the K individual table nodes with the closest three-dimensional spatial distance to each node in the living body are recorded as follows:

s301, dividing the standard grid nodes into body surface nodes and in-vivo nodes, specifically traversing each triangular surface in the standard grid, if the triangular surface appears in all tetrahedrons only once, each node of the triangular surface is a body surface node, and finally, all nodes which are not classified as body surface nodes are in-vivo nodes;

s302, for each in-vivo node, K nearest body surface nodes are selected, a first node set is constructed, and the serial number of each node in the first node set is recorded.

S40, inputting the expanded light source sample and each node in the first node set into a pre-constructed deep learning network model to obtain a rough reconstruction positioning result and a final morphological reconstruction result of the light source; calculating a loss value based on the rough reconstruction positioning result of the light source, the final morphological reconstruction result and the real distribution information of the light source, and updating the parameters of the deep learning network model;

in this embodiment, the deep learning network model is formed by concatenating residuals of a DNN fully-connected network and a GCN graph convolution network, as shown in fig. 2, the number of DNN fully-connected networks (i.e., the fully-connected layers in fig. 2) is 4, the input of the fully-connected networks is fluorescence intensity of a biological body surface (i.e., fluorescence intensity corresponding to an extended light source sample is normalized, and the normalized fluorescence intensity is input into the DNN fully-connected network), the number of nodes of an implicit layer is the number of vertices of a feasible region in the biological body, and the output layer is a coarse reconstruction positioning result of a light source or a coarse predicted position of the light source (i.e., the coarse reconstruction result in fig. 2). To alleviate the problem of network overfitting, the Dropout function is used to randomly drop neurons of the hidden layer with a 30% probability.

The number of layers of the GCN graph convolution network (i.e., the GCN layer in fig. 2) is 3, the network uses the graph structure adjacency matrix as a structure prior for GCN graph convolution calculation, uses the feature vector obtained by splicing the light source predicted position and the light intensity signal of each node in the first node set as input, the number of nodes in the hidden layer is the number of vertices of a feasible region in the organism, each neuron in the hidden layer corresponds to a vertex of the feasible region in the organism one to one, and the output layer outputs a final morphological reconstruction result of the light source or predicted distribution information (i.e., a final result in fig. 2) called as light source morphology.

And calculating a loss value based on the rough reconstruction positioning result, the final morphological reconstruction result and the real distribution information of the light source, and updating the parameters of the deep learning network model.

The excitation function used by the deep learning network model during training is a Relu function which is a nonlinear unit, the nonlinear fitting capability of the deep learning algorithm is enhanced, and elements smaller than 0 in the result are corrected, wherein the formula is as follows:

where X represents the output result of each layer in a DNN fully-connected network or a GCN graph convolution network.

The optimization function of deep learning network model training is an Adam optimization method, the loss function of network training is a composite mean square error function, and the formula is as follows:

wherein, X_FCFor the output of the DNN fully-connected network, and for the coarse relocation result, the preliminary location of the in-vivo light source, X, is performed_trueAs distribution information of real light sources, X_finalThe outputs of the fully connected sub-networks are the outputs of the residual connection with the GCN graph convolution network, as the final morphological reconstruction result,representing a loss of positioning accuracy of the network,representing the morphological accuracy loss, and λ represents a weighting parameter used to balance the ratio of the two block losses.

And looping the steps S10-S40 to train the deep learning network model until the set training times or the set precision threshold is reached.

S50, acquiring a fluorescence image and an anatomical structure image of the body surface of the organism to be excited by fluorescence tomography; registering the fluorescence image to the body surface mesh of the standard mesh generated in the step S10 through a registration algorithm by combining the anatomical structure image, and performing normalization processing on the fluorescence intensity in the body surface mesh; based on the fluorescence intensity after normalization processing, the distance between vertexes in the registered body surface mesh and the connection relation, performing excitation fluorescence tomography reconstruction on the organism by using a trained deep learning network model to obtain the three-dimensional distribution of the fluorescence probes in the organism;

in this example, FMT (fluorescence tomography) reconstruction was performed on a living body using a deep learning model to obtain the distribution of fluorescent probes inside the living body. The method comprises the following specific steps:

s501, constructing a biological model, wherein a mouse brain glioma model is preferably constructed in the embodiment;

s502, injecting the fluorescent probe into the biological model constructed in the step S501 in an injection mode;

s503, acquiring a fluorescence image and an anatomical structure image of the body surface of the organism through a CCD (charge coupled device) camera, a CT (computed tomography) device and an MRI (magnetic resonance imaging) device, and registering the acquired fluorescence image to the body surface grid of the standard grid generated in the step S110 through a registration algorithm;

s504, carrying out normalization processing on the fluorescence intensity in the body surface grid in the step S503; based on the fluorescence intensity after normalization processing, the distance between the vertexes in the registered body surface mesh and the connection relation, the trained deep learning network model is used for carrying out excitation fluorescence tomography reconstruction on the organism to obtain the three-dimensional distribution of the fluorescence probes, namely the fluorescence sources, in the organism.

An excitation fluorescence tomography system based on a GCN residual error connection network according to a second embodiment of the present invention, as shown in fig. 3, specifically includes: the system comprises a graph structure modeling module 100, a data set constructing module 200, a node classification module 300, a model training module 400 and a model prediction module 500;

the graph structure modeling module 100 is configured to segment CT image data of an organism according to tissue and organ categories, and grid the segmented CT image data by a triangular grid modeling method to serve as a standard grid; establishing a weighted adjacency matrix of a graph structure according to the distance between the vertexes of the standard grids and the connection relation;

the data set building module 200 is configured to simulate a photon propagation process of an in-vivo light source in an organism by combining optical absorption scattering parameters of each tissue organ, so as to obtain fluorescence distribution on the surface and inside of the organism as a light source sample; expanding the light source sample according to the type of the light source sample;

the node classification module 300 is configured to divide the standard mesh into body surface nodes and in-vivo nodes; for each in-vivo node, K body surface nodes closest to the in-vivo node are selected, a first node set is constructed, and the serial number of each node in the first node set is recorded;

the model training module 400 is configured to input the extended light source sample and each node in the first node set into a pre-constructed deep learning network model to obtain a coarse reconstruction positioning result and a final morphological reconstruction result of the light source; calculating a loss value based on the rough reconstruction positioning result of the light source, the final morphological reconstruction result and the real distribution information of the light source, and updating the parameters of the deep learning network model;

the model prediction module 500 is configured to acquire a fluorescence image and an anatomical structure image of a body surface of an organism to be excited for fluorescence tomography; registering the fluorescence image to a body surface grid of a standard grid generated by the graph structure modeling module 100 through a registration algorithm by combining the anatomical structure image, and performing normalization processing on the fluorescence intensity in the body surface grid; based on the fluorescence intensity after normalization processing, the distance between vertexes in the registered body surface mesh and the connection relation, performing excitation fluorescence tomography reconstruction on the organism by using a trained deep learning network model to obtain the three-dimensional distribution of the fluorescence probes in the organism;

the deep learning network model is formed by connecting DNN full-connection networks and GCN graph convolution network residuals.

It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process and related description of the system described above may refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

It should be noted that, the excitation fluorescence tomography system based on the GCN residual error connection network provided in the above embodiment is only illustrated by the division of the above functional modules, and in practical applications, the functions may be allocated to different functional modules according to needs, that is, the modules or steps in the embodiment of the present invention are further decomposed or combined, for example, the modules in the above embodiment may be combined into one module, or may be further split into multiple sub-modules, so as to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not to be construed as unduly limiting the present invention.

An electronic device according to a third embodiment of the present invention includes at least one processor; and a memory communicatively coupled to at least one of the processors; wherein the memory stores instructions executable by the processor for execution by the processor to implement the method of claim for GCN residual connected network based excitation fluorescence tomography.

A computer readable storage medium of a fourth embodiment of the present invention stores computer instructions for execution by the computer to implement the method for excitation fluorescence tomography based on GCN residual error connected network as claimed above.

The invention relates to an excitation fluorescence tomography device based on a GCN residual error connecting network, which comprises acquisition equipment and central processing equipment, wherein the acquisition equipment comprises a first acquisition unit and a second acquisition unit;

the acquisition equipment comprises a CCD (charge coupled device) camera, CT (computed tomography) equipment and MRI (magnetic resonance imaging) equipment, and is used for CT image data of the organism and fluorescence images and anatomical images of the body surface of the organism;

the central processing equipment comprises a GPU (graphics processing unit) which is configured to divide CT image data of a living body according to tissue and organ categories and grid the divided CT image data by a triangular grid modeling method to be used as a standard grid; establishing a weighted adjacency matrix of a graph structure according to the distance between the vertexes of the standard grids and the connection relation; simulating the photon propagation process of the in-vivo light source in the organism by combining the optical absorption scattering parameters of each tissue organ to obtain the fluorescence distribution on the surface and inside the organism as a light source sample; expanding the light source sample according to the type of the light source sample; dividing the standard grid into body surface nodes and body interior nodes; for each in-vivo node, K body surface nodes closest to the in-vivo node are selected, a first node set is constructed, and the serial number of each node in the first node set is recorded; inputting the expanded light source sample and each node in the first node set into a pre-constructed deep learning network model to obtain a rough reconstruction positioning result and a final morphological reconstruction result of the light source; calculating a loss value based on the rough reconstruction positioning result of the light source, the final morphological reconstruction result and the real distribution information of the light source, and updating the parameters of the deep learning network model; registering the fluorescence image to a body surface grid of the generated standard grid by combining the anatomical structure image and a registration algorithm, and carrying out normalization processing on the fluorescence intensity in the body surface grid; based on the fluorescence intensity after normalization processing, the distance between the vertexes in the registered body surface mesh and the connection relation, the trained deep learning network model is used for carrying out excitation fluorescence tomography reconstruction on the organism, and the three-dimensional distribution of the fluorescence probes in the organism is obtained.

It is clear to those skilled in the art that, for convenience and brevity not described, the specific working processes and related descriptions of the above-described apparatuses and computer-readable storage media may refer to the corresponding processes in the foregoing method examples, and are not described herein again.

Referring now to FIG. 4, there is illustrated a block diagram of a computer system suitable for use as a server in implementing embodiments of the method, system, and apparatus of the present application. The server shown in fig. 4 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.

As shown in fig. 4, the computer system includes a Central Processing Unit (CPU) 401 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 402 or a program loaded from a storage section 408 into a Random Access Memory (RAM) 403. In the RAM403, various programs and data necessary for system operation are also stored. The CPU401, ROM402, and RAM403 are connected to each other via a bus 404. An Input/Output (I/O) interface 405 is also connected to the bus 404.

The following components are connected to the I/O interface 405: an input section 406 including a keyboard, a mouse, and the like; an output section 407 including a Display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage section 408 including a hard disk and the like; and a communication section 409 including a Network interface card such as a LAN (Local Area Network) card, a modem, or the like. The communication section 409 performs communication processing via a network such as the internet. A driver 410 is also connected to the I/O interface 405 as needed. A removable medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 410 as necessary, so that a computer program read out therefrom is mounted into the storage section 408 as necessary.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 409, and/or installed from the removable medium 411. More specific examples of a computer-readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), a compact disc read-only memory (CD-ROM), Optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing or implying a particular order or sequence.

The terms "comprises," "comprising," or any other similar term are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.

So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.