脂肪肝精准定量分析方法、装置、计算机设备和存储介质

Fatty liver accurate quantitative analysis method and device, computer equipment and storage medium

文档序号：9312 发布日期：2021-09-17 浏览：39次中文

1. A fatty liver accurate quantitative analysis method is characterized by comprising the following steps:

(1) acquiring a three-dimensional image of a liver to be analyzed;

(2) obtaining a liver segmentation result of the three-dimensional liver image to be analyzed;

(3) quantitative analysis of liver fat and segmental fat: calculating the fat distribution uniformity, average fat content, median fat content and confidence interval of the whole liver; and calculating the average fat content, the median fat content and the confidence interval of each liver segment in the liver segmentation result.

2. The method for accurately and quantitatively analyzing fatty liver according to claim 1, wherein in the step (1), the three-dimensional liver image to be analyzed is an MRI three-dimensional liver image, a B-mode ultrasound three-dimensional liver image, a CT three-dimensional liver image or an MRS three-dimensional liver image.

3. The method for accurately and quantitatively analyzing fatty liver according to claim 1, wherein in the step (2), the step of obtaining the liver segmentation result of the three-dimensional image of the liver to be analyzed specifically comprises:

(21) and inputting the three-dimensional liver image to be analyzed into a pre-trained liver segmentation model, and outputting the liver segmentation result.

4. The method for the precise quantitative analysis of fatty liver according to claim 3, wherein in the step (21), the pre-trained liver segmentation model is obtained by the following steps:

(a) acquiring a sample liver three-dimensional image, and acquiring a liver segmentation label of the sample liver three-dimensional image;

(b) and taking the sample liver three-dimensional image and the liver segmentation label as a training set, and performing deep learning training on the liver segmentation model in an iteration mode to obtain the pre-trained liver segmentation model, wherein a deep learning network adopted by the liver segmentation model is a segmentation network based on the combination of a UNet/VNet and a channel attention mechanism.

5. The method for accurately quantitatively analyzing a fatty liver according to claim 1, further comprising the steps of:

(4) fat quantitative range early warning:

outputting a fat quantitative range early warning conclusion according to the percentage of the average fat content of the whole liver to the weight of the whole liver: if the average fat content of the whole liver is 5% -10% of the weight of the whole liver, judging that the whole liver belongs to mild fatty liver; determining that the whole liver is a moderate fatty liver if the average fat content of the whole liver is 10% -25% of the weight of the whole liver; determining that the whole liver is a severe fatty liver if the average fat content of the whole liver is more than 25% of the weight of the whole liver.

6. The method for accurately quantitatively analyzing fatty liver according to claim 1, wherein after the step (1), the method for accurately quantitatively analyzing fatty liver further comprises the steps of:

(2') obtaining a segmentation region of the vein vessel of the liver three-dimensional image to be analyzed;

(3') extracting hepatic vein and periportal vein fat regions of the vein vessel in the segmented region;

(4') quantitative analysis of venous perivascular fat: the average fat content of the perihepatic and hepatic portal vein fat regions was calculated.

7. The method for accurately and quantitatively analyzing fatty liver according to claim 6, wherein in the step (2'), the step of obtaining the segmented region of the vein vessel of the three-dimensional image of the liver to be analyzed specifically comprises:

(21') inputting the three-dimensional image of the liver to be analyzed into a pre-trained vein vessel segmentation model, and outputting the segmentation region of the vein vessel.

8. An accurate quantitative analysis device for fatty liver, comprising:

the liver segmentation module is used for acquiring a liver segmentation result of a three-dimensional liver image to be analyzed;

the liver fat and segmented fat quantitative analysis module is in signal connection with the liver segmentation module and is used for calculating the fat distribution uniformity, the average fat content, the median fat content and the confidence interval of the whole liver; and calculating the average fat content, the median fat content and the confidence interval of each liver segment in the liver segmentation result.

9. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor when executing the computer program implements the fatty liver precision quantitative analysis method according to any one of claims 1 to 7.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the fatty liver precision quantitative analysis method according to any one of claims 1 to 7.

Background

Fatty liver surpasses hepatitis B and becomes the first chronic liver disease in China, the incidence rate is in a trend of increasing continuously, and if no intervention is added, the fatty liver can be degraded into steatohepatitis, cirrhosis and hepatocellular carcinoma, and even the death is caused by liver failure.

The qualitative diagnosis of fatty liver can be realized by ultrasound, CT, MRI and MRS, and at present, the fatty liver screening is mainly realized by B-ultrasound and CT images. B-mode ultrasonography is simple and economical, but is greatly influenced by factors such as individual difference of patients, detection instruments, doctor experience and the like, and CT has the defect of radiation although imaging is more fresh. However, no satisfactory method for quantitative diagnosis can be popularized and applied in clinical work so far.

The existing fatty liver quantitative analysis technology is mainly based on B-ultrasonic and CT images, for example, the CT-based fatty liver quantitative analysis method (Wandongyang, an abdominal CT-based fatty liver intelligent grading assessment method: China, 202010877327.2[ P ].2021-01-05) utilizes a deep learning UNet2D network (Ronneberger, O., Fischer, P., Brox, T.: U-Net: connected network for biological image segmentation. in: Navab, N., Horneger, J., Wells, W.M., Frangi, A.F. (eds.) CAI. LNCS, vol.9351, pp.234-241. Springer, Cham (CT)) to perform liver tissue and spleen tissue segmentation on segmented images, and to perform gridding segmentation on the MIC region as a sampling region, and perform quantitative evaluation on corresponding fat analysis by calculating the mean value of the segmented liver tissue and spleen tissue. The scheme applies the deep learning technology to the quantitative analysis of the fatty liver based on the CT image.

However, the quantitative analysis of the existing fatty liver quantitative analysis technology is limited, quantitative analysis is not performed from the perspective of liver segmentation, the quantitative analysis of liver segmentation and segmented fat is not comprehensive enough, and fat range early warning and the quantitative analysis of perivascular fat are lacked, so that the quantitative analysis technology cannot be popularized and applied in clinic.

Therefore, it is required to provide a method for accurately quantitatively analyzing fatty liver, which can comprehensively and accurately quantitatively analyze liver fat.

Disclosure of Invention

The present invention is directed to overcome the above-mentioned shortcomings in the prior art, and provides a method, an apparatus, a computer device and a storage medium for accurately and quantitatively analyzing fatty liver, which can comprehensively and accurately perform quantitative analysis on fatty liver.

In order to achieve the above object, in a first aspect of the present invention, there is provided a method for accurately quantitatively analyzing fatty liver, comprising the steps of:

(1) acquiring a three-dimensional image of a liver to be analyzed;

(2) obtaining a liver segmentation result of the three-dimensional liver image to be analyzed;

Preferably, in the step (1), the three-dimensional image of the liver to be analyzed is a three-dimensional liver MRI image, a three-dimensional liver B-mode ultrasound image, a three-dimensional liver CT image or a three-dimensional liver MRS image.

Preferably, in the step (2), the step of obtaining a liver segmentation result of the three-dimensional image of the liver to be analyzed specifically includes:

(21) and inputting the three-dimensional liver image to be analyzed into a pre-trained liver segmentation model, and outputting the liver segmentation result.

Preferably, in the step (21), the pre-trained liver segmentation model is obtained by the following steps:

(a) acquiring a sample liver three-dimensional image, and acquiring a liver segmentation label of the sample liver three-dimensional image;

Preferably, the method for accurately quantitatively analyzing fatty liver further comprises the following steps:

(4) fat quantitative range early warning:

Preferably, after the step (1), the method for accurately quantifying fatty liver further comprises the steps of:

(2') obtaining a segmentation region of the vein vessel of the liver three-dimensional image to be analyzed;

(3') extracting hepatic vein and periportal vein fat regions of the vein vessel in the segmented region;

(4') quantitative analysis of venous perivascular fat: the average fat content of the perihepatic and hepatic portal vein fat regions was calculated.

Preferably, in the step (2'), the step of acquiring the segmented region of the vein of the three-dimensional image of the liver to be analyzed specifically includes:

(21') inputting the three-dimensional image of the liver to be analyzed into a pre-trained vein vessel segmentation model, and outputting the segmentation region of the vein vessel.

In a second aspect of the present invention, there is provided an apparatus for accurately quantifying fatty liver, comprising:

the liver segmentation module is used for acquiring a liver segmentation result of a three-dimensional liver image to be analyzed;

In a third aspect of the present invention, there is provided a computer device comprising a memory and a processor, wherein the memory stores a computer program, and wherein the processor implements the method for accurately quantifying fatty liver.

In a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method for accurate quantitative analysis of fatty liver as described above.

By adopting the accurate quantitative analysis method, the device, the computer equipment and the storage medium for the fatty liver, the deep learning technology is utilized to solve the problems of automatic liver segmentation and segmented fat quantitative result analysis, fat range early warning is realized through fat content quantitative analysis, and in addition, perivascular fat extraction and quantitative analysis are also solved, so that the more comprehensive and accurate quantitative analysis is realized for the fatty liver, and doctors can be helped to more accurately judge the fat deposition degree and the treatment effect of the fatty liver.

Drawings

FIG. 1 is a schematic flow chart of an embodiment of the method for accurately quantifying fatty liver according to the present invention.

FIG. 2 is a schematic diagram of a deep learning training method for a liver segmentation model in the embodiment shown in FIG. 1.

Fig. 3 is a schematic diagram of a deep learning training method of a vein vessel segmentation model in the embodiment shown in fig. 1.

FIG. 4 is a diagram illustrating an artificially labeled liver segmentation label for validating a pre-trained liver segmentation model in the embodiment shown in FIG. 1.

FIG. 5 is a diagram illustrating model prediction results for validating a pre-trained liver segmentation model in the embodiment shown in FIG. 1.

Fig. 6 is a schematic view of a slice in which the distribution of fat content of the liver is visualized by a pseudo-color chart, the fat content of each pixel position of the slice is displayed in color correspondence, and the range of the fat content displayed by a chromaticity bar is 0% -60%.

Fig. 7 is a schematic diagram of the slice effect after automatic segmentation of the liver.

Fig. 8 is a schematic diagram of the average fat content of the liver visually presented by the fat content forewarning coordinates.

Fig. 9 is a schematic diagram of the slice effect after the automatic segmentation of the blood vessel.

Fig. 10 is a schematic block diagram of an embodiment of the apparatus for accurately quantitative analysis of fatty liver according to the present invention.

Detailed Description

In order to clearly understand the technical contents of the present invention, the following examples are given in detail. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

Referring to fig. 1 to 9, in an embodiment of the present invention, the method for accurately quantifying fatty liver includes the following steps:

(1) acquiring a three-dimensional image of a liver to be analyzed;

(2) obtaining a liver segmentation result of the three-dimensional liver image to be analyzed;

(1) Whole liver

Fat distribution uniformity: firstly, calculating the mean value M and standard deviation S, D of the whole liver fat content_iIs the fat content per voxel i, n is the total number of voxels in the liver,

average fat content: mean value of the fat content of the whole liver

Median fat content: the whole liver ranks the fat content of all voxels, with the fat content ranked in the middle.

Confidence interval: the whole liver sorts the fat content of all voxels, and takes the fat content value arranged in 25% quantile as the upper limit a value of the confidence interval [ a, b ], and takes the fat content value arranged in 75% quantile as the lower limit b value of the confidence interval [ a, b ].

(2) Each liver segment

Average fat content: mean value of fat content in each liver segmentWherein the liver segment c is [1,8 ]]And d is the fat content per voxel in each corresponding liver segment.

Median fat content: in each liver segment, the fat content sizes of all voxels were ranked, with the fat content ranked in the middle.

Confidence interval: in each liver segment, the fat content of all voxels is sorted, the fat content value ranked at 25% quantile is taken as the upper limit a value of the confidence interval [ a, b ], and the fat content value quantile ranked at 75% quantile is taken as the lower limit b value of the confidence interval [ a, b ].

In the step (1), the three-dimensional image of the liver is a three-dimensional image taken of the liver. It is understood that the three-dimensional image of the liver includes a three-dimensional image of hepatic vein vessels including the hepatic vein and the portal vein. The three-dimensional image of the liver to be analyzed is the three-dimensional image of the liver which needs to be subjected to accurate quantitative analysis of the fatty liver.

In the step (1), the three-dimensional image of the liver to be analyzed may be any suitable type of three-dimensional image of the liver, such as an MRI (Magnetic Resonance Imaging) three-dimensional image of the liver, a B-ultrasound (type-B ultrasound) three-dimensional image of the liver, a CT (Computed Tomography) three-dimensional image of the liver, or an MRS (Magnetic Resonance Spectroscopy) three-dimensional image of the liver, and in a specific embodiment of the present invention, the three-dimensional image of the liver to be analyzed is an MRI three-dimensional image of the liver in the step (1).

In the step (2), the step of obtaining the liver segmentation result of the three-dimensional liver image to be analyzed is a step of performing liver segmentation on the three-dimensional liver image to be analyzed to obtain a liver segmentation result, and any suitable method may be specifically adopted, and in a specific embodiment of the present invention, in the step (2), the step of obtaining the liver segmentation result of the three-dimensional liver image to be analyzed specifically includes:

(21) and inputting the three-dimensional liver image to be analyzed into a pre-trained liver segmentation model, and outputting the liver segmentation result.

The liver segmentation model is a machine learning model for performing liver segmentation processing. The pre-trained liver segmentation model can be obtained through deep learning training.

Obviously, the liver segmentation may also be performed manually or semi-automatically, i.e. by performing a preliminary automatic segmentation and then manual adjustment using a conventional classification and segmentation algorithm.

In step (21), the pre-trained liver segmentation model may be obtained by any suitable method, and in one embodiment of the present invention, in step (21), the pre-trained liver segmentation model is obtained by:

(a) acquiring a sample liver three-dimensional image, and acquiring a liver segmentation label of the sample liver three-dimensional image;

Namely, inputting the sample liver three-dimensional image in the training set into a deep learning network to obtain liver segmented prediction data, then performing difference comparison on the liver segmented prediction data and the liver segmented labels in the training set, and iteratively updating the deep learning network according to the difference until an iteration termination condition is met to obtain a pre-trained liver segmented model.

In the step (a), the step of obtaining a three-dimensional image of a sample liver may specifically adopt any suitable method, and in an embodiment of the present invention, in the step (a), the step of obtaining a three-dimensional image of a sample liver specifically includes:

preprocessing an original sample liver three-dimensional image to obtain the sample liver three-dimensional image, wherein the preprocessing comprises one or more of histogram equalization, standardization and normalization processing.

The original sample liver three-dimensional image is a sample liver three-dimensional image which is not subjected to pretreatment. Histogram equalization processing is carried out, so that the dynamic range of the image gray scale is enhanced, and the contrast of the image is improved; the normalization is to transform the data into a distribution with a mean value of 0 and a standard deviation of 1, and the normalization is to change the data into a certain fixed interval, wherein the interval is [0, 1], so that the solving process is accelerated in the descending process of the model training gradient, and adverse factors such as gradient explosion and the like are also avoided.

In step (b), the UNet may be any suitable UNet, such as UNet2.5D, UNet2D, UNet3D, UNet + +, Res-UNet, Dense U-Net, MultiResUNet, R2U-Net, or Attention UNet, and in one embodiment of the invention, UNet2.5D, as shown in FIG. 2.

In the step (b), the deep learning network may include any suitable configuration, in one embodiment of the present invention, in the step (b), the deep learning network includes a convolutional layer, a pooling layer, an anti-convolutional layer, a cascading layer and a batch normalization layer, the convolution layer is in signal connection with the deconvolution layer through the pooling layer, the cascade layer is in signal connection with the convolution layer, the pooling layer, the deconvolution layer and the batch normalization layer respectively, the convolutional layer extracts a feature map of the sample liver three-dimensional image, the pooling layer performs down-sampling operation on the feature map, the deconvolution layer performs convolution operation after padding the feature map to enlarge the size of the feature map, the cascade layer combines the feature maps output by different levels, and the batch normalization layer normalizes the numerical values of the feature maps.

The convolutional layer extracts a feature map of the sample liver three-dimensional image by setting the size and the step length of a convolutional kernel, and the batch normalization layer normalizes the numerical values of the feature map to improve the convergence speed of the gradient, accelerate training and alleviate the problem of gradient disappearance.

In step (b), the deep learning training may include any suitable training process, as shown in fig. 2, and in a specific embodiment of the present invention, in step (b), the deep learning training includes an encoding process and a decoding process, both of the encoding process and the decoding process employ the UNet/VNet and the channel attention mechanism, and the decoding process further employs one or more of a multi-level fusion operation and a full supervision operation.

In step (b), the loss function used by the fully-supervised operation may be any suitable loss function, and in a specific embodiment of the present invention, in step (b), the loss function used by the fully-supervised operation is a multi-class cross-entropy loss function.

Fig. 2 is a schematic diagram of a process of obtaining a pre-trained liver segmentation model through deep learning training. As can be seen from fig. 2, the overall training process is such that: inputting an original medical image (namely a sample liver three-dimensional image) into a deep learning network to obtain a prediction result of the deep learning network, then comparing the prediction result (namely liver segmented prediction data) with an artificial label (namely a liver segmented label), feeding back the result to the deep learning network, continuously updating the deep learning network by taking the artificial label as a target according to the fed-back comparison information until the prediction result is close to the artificial label, and thus obtaining the pre-trained liver segmented model in the embodiment.

In order to realize fat range early warning, please refer to fig. 1, in an embodiment of the present invention, the method for accurately quantitatively analyzing fatty liver further includes the following steps:

(4) fat quantitative range early warning:

outputting a fat quantitative range early warning conclusion according to the percentage of the average fat content of the whole liver to the weight of the whole liver: if the average fat content of the whole liver is 5% -10% of the weight of the whole liver, judging that the whole liver belongs to mild fatty liver; if the average fat content of the whole liver is 10% -25% of the weight of the whole liver, judging that the whole liver belongs to medium-grade fatty liver and can show liver tissue inflammation and liver fibrosis; determining that the whole liver is a severe fatty liver if the average fat content of the whole liver is more than 25% of the weight of the whole liver.

In order to perform quantitative analysis of liver fat more fully, please refer to fig. 1, in an embodiment of the present invention, after the step (1), the method for accurately quantifying fatty liver further comprises the following steps:

(2') obtaining a segmentation region of the vein vessel of the liver three-dimensional image to be analyzed;

(3') extracting hepatic vein and periportal vein fat regions of the vein vessel in the segmented region;

(4') quantitative analysis of venous perivascular fat: the average fat content of the perihepatic and hepatic portal vein fat regions was calculated.

The average fat content of the hepatic vein and peripheral fat region of the hepatic portal vein is as follows: counting the number n of voxels in the fat region around the hepatic vein or portal vein and the corresponding fat content Q_i，

In the step (2 '), the step of acquiring the segmented region of the vein of the three-dimensional image of the liver to be analyzed is a step of performing vein segmentation on the three-dimensional image of the liver to be analyzed to obtain the segmented region of the vein, and any suitable method may be specifically adopted, and in an embodiment of the present invention, in the step (2'), the step of acquiring the segmented region of the vein of the three-dimensional image of the liver to be analyzed specifically includes:

(21') inputting the three-dimensional image of the liver to be analyzed into a pre-trained vein vessel segmentation model, and outputting the segmentation region of the vein vessel.

The vein segmentation model is a machine learning model for performing a vein segmentation process. The pre-trained vein vessel segmentation model can be obtained through deep learning training.

Obviously, the vein vessel segmentation can also be performed manually or semi-automatically, i.e. by performing a preliminary automatic segmentation and then a manual adjustment through a conventional classification and segmentation algorithm.

In the step (21 '), the pre-trained vein segmentation model may be obtained by any suitable method, and in a specific embodiment of the present invention, in the step (21'), the pre-trained vein segmentation model is obtained by:

(a') obtaining a sample liver three-dimensional image, and obtaining vein vessel labeling data of the sample liver three-dimensional image;

(b') taking the sample liver three-dimensional image and the vein vessel labeling data as a training set, and carrying out deep learning training on a vein vessel segmentation model in an iteration mode to obtain the pre-trained vein vessel segmentation model, wherein a deep learning network adopted by the vein vessel segmentation model is a segmented network based on the combination of a UNet/VNet and a channel attention mechanism.

That is, the three-dimensional images of the sample liver in the training set are input into a deep learning network to obtain vein segmentation prediction data, then the vein segmentation prediction data and the vein labeling data in the training set are compared in a difference mode, the deep learning network is updated iteratively according to the difference until an iteration termination condition is met, and a pre-trained vein segmentation model is obtained.

For other specific limitations of the deep learning network and the deep learning training, reference may be made to the above limitations of the deep learning network and the deep learning training, and details are not repeated here.

As shown in fig. 3, a schematic process diagram of obtaining a pre-trained vein vessel segmentation model through deep learning training is shown. As can be seen from fig. 3, the overall training process is such that: inputting an original medical image (namely a sample liver three-dimensional image) into a deep learning network to obtain a prediction result of the deep learning network, comparing the prediction result (namely vein vessel segmentation data) with an artificial label (namely vein vessel label data), feeding back the result to the deep learning network, taking the artificial label as a target, continuously updating the deep learning network according to the fed-back comparison information until the prediction result is close to the artificial label, and obtaining the pre-trained vein vessel segmentation model in the embodiment.

In step (3 '), the extraction may be performed by any suitable method, and in a specific embodiment of the present invention, in step (3'), the extraction uses a morphological dilation algorithm.

For facilitating to check each index obtained by the fatty liver precision quantitative analysis method, please refer to fig. 1, in an embodiment of the present invention, the fatty liver precision quantitative analysis method further includes:

(6) generating a fatty liver quantitative analysis report including the fat distribution uniformity, the average fat content, the median fat content, and the confidence interval for the entire liver; the mean fat content, the median fat content, and the confidence interval for each liver segment; the fat quantitative range early warning conclusion and the average fat content of the hepatic vein and perihepatic vein fat regions.

Hereinafter, the method for accurately quantitatively analyzing fatty liver according to the present invention will be described in detail by taking 500 MRI three-dimensional images of an original sample liver and an MRI three-dimensional image of a liver of a patient with a liver disease as an example of an MRI three-dimensional image of a liver to be analyzed.

1. Acquisition of pre-trained liver segmentation model and pre-trained vein segmentation model

The following describes the acquisition of a pre-trained liver segmentation model and a pre-trained vein segmentation model, taking 500 MRI three-dimensional images of an original sample liver as an example.

1.1 MRI data Pre-processing

DICOM data of 500 original sample liver MRI three-dimensional images are read, and preprocessing is performed in a histogram equalization, standardization and normalization mode. The histogram equalization process enhances the dynamic range of the image gray scale, thereby improving the contrast of the image. Normalization transforms the data into a distribution with a mean of 0 and a standard deviation of 1. Normalization varied the data to a fixed interval [0, 1 ]. The standardization and normalization processing can accelerate the solving process in the descending process of the model training gradient and avoid adverse factors such as gradient explosion and the like. MRI three-dimensional images of 500 samples of liver were obtained.

1.2 acquisition of a Pre-trained liver segmentation model

350 parts (70%) of sample liver MRI three-dimensional images and corresponding liver segmentation labels are used as a training set, and 150 parts (30%) of liver MRI three-dimensional images and corresponding liver segmentation labels are used as a verification set.

Inputting a training set into a segmented network based on combination of UNet2.5D and a channel attention mechanism shown in FIG. 2, specifically, an encoding process comprises 3 levels of UNet2.5D and the channel attention mechanism, a decoding process also comprises 3 levels of UNet2.5D and the channel attention mechanism, a batch size, a learning rate and a kernel initialization parameter are set in a training process, an Adam optimizer is adopted for training, and a multi-classification cross entropy loss function is adopted as a loss function.

(1) Setting model parameters of an encoding process and a decoding process: the convolution layer has convolution kernel size of 3 x 3 and kernel initialization with LeCun homogeneous initializer; the activation function uses a Linear rectification function (ReLU), also called a modified Linear Unit; the convolution kernel size of the deconvolution layer was 3 x 3.

(2) Setting network model output parameters: a convolution layer having convolution kernels with a size of 1 x 1; the activation function uses a Sigmoid function to map the output value between 0 and 1.

(3) Setting parameters of a model training process: batch is set to 8; adopting an Adam optimizer which uses momentum and an adaptive learning rate to accelerate convergence speed, wherein the initial learning rate is set to be 0.0001; the loss function adopts a multi-classification cross entropy loss function; the epoch is set to 300.

(4) And starting training until the model converges, and keeping the optimal training model. I.e. a pre-trained liver segmentation model is obtained.

Inputting the verification set into the obtained pre-trained liver segmentation model, and quantitatively and qualitatively comparing the model prediction result with the liver segmentation label. Wherein, the average effect of the verification set is measured by using the Dice coefficient during quantification (the Dice coefficient of the verification set is 0.81). The qualitative predictive effect of two-dimensional slices on the validation set is shown in fig. 4 and 5.

1.3 acquisition of a Pre-trained vein vessel segmentation model

350 parts (70%) of the sample liver MRI three-dimensional image and the corresponding vein vessel marking data are used as a training set, and 150 parts (30%) of the sample liver MRI three-dimensional image and the corresponding vein vessel marking data are used as a verification set.

Inputting a training set into a segmented network based on combination of UNet2.5D and a channel attention mechanism shown in FIG. 3, specifically, an encoding process comprises 3 levels of UNet2.5D and the channel attention mechanism, a decoding process also comprises 3 levels of UNet2.5D and the channel attention mechanism, a batch size, a learning rate and a kernel initialization parameter are set in a training process, an Adam optimizer is adopted for training, and a multi-classification cross entropy loss function is adopted as a loss function.

(1) Setting model parameters of an encoding process and a decoding process: the convolution layer has the convolution kernel size of 3 x 3 and adopts a glorot uniform initializer for kernel initialization; the activation function uses a Linear rectification function (ReLU), also called a modified Linear Unit; the convolution kernel size of the deconvolution layer was 3 x 3.

(4) And starting training until the model converges, and keeping the optimal training model. I.e. a pre-trained vessel segmentation model is obtained.

And inputting the verification set into the obtained pre-trained blood vessel segmentation model, and quantitatively and qualitatively comparing the model prediction result with vein vessel labeling data. Wherein, the average effect of the verification set is measured by using the Dice coefficient during quantification (the Dice coefficient of the verification set is 0.84).

2. Accurate quantitative analysis of fatty liver of liver MRI three-dimensional image to be analyzed

Hereinafter, an MRI three-dimensional image of a liver of a patient with liver disease is taken as an example of an MRI three-dimensional image of a liver to be analyzed.

Quantitative analysis was performed by calculating the fat distribution homogeneity, average fat content, median fat content and confidence interval for the whole liver, with a fat distribution homogeneity of 0.56, an average fat content of 5.7%, a median fat content of 3.1% and a confidence interval of [ 1.7%, 5.4% ]. Wherein the distribution of the fat content of the liver can be visualized by a pseudo-color map, as shown in fig. 6.

And inputting the MRI three-dimensional image of the liver to be analyzed into the obtained pre-trained liver segmentation model to obtain a liver segmentation result, wherein the slice effect of the automatically segmented liver is shown in figure 7. Through the automatic liver segmentation process, 8 segments corresponding to the liver are obtained. Corresponding to each segment of the liver, quantitative analysis is carried out by calculating corresponding average fat content, median fat content and confidence interval, and the results are shown in the following table.

Liver segmentation/quantification	Average fat content (%)	Median fat content (%)	Confidence interval (%)
				1 paragraph	6.88	3.52	[2.05,6.03]
2 section	10.42	5.12	[2.47,13.57]
				3 paragraph	5.17	3.31	[1.92,5.27]
4 stages	7.20	3.97	[2.29,6.73]
				5 paragraph	3.42	2.14	[1.04,3.84]
6 paragraphs of	5.51	2.21	[1.12,5.17]
				7 paragraph	5.73	2.56	[1.48,4.88]
8 paragraph	4.98	3.19	[1.89,4.85]

The liver is judged to belong to mild, moderate or severe fatty liver patients by calculating the percentage of the average fat content of the liver to the weight of the liver. Mild fatty liver, the liver fat content accounts for 5% -10% of the liver weight; moderate fatty liver, liver fat content accounts for 10% -25% of liver weight, and liver tissue inflammation and hepatic fibrosis can be seen; severe fatty liver, liver penetrating pathological liver fat content accounts for more than 25% of the weight of the liver. The quantitative value can be visually displayed through the fat content early warning coordinate, as shown in fig. 8. Therefore, the patient belongs to mild fatty liver patients.

The MRI three-dimensional image of the liver to be analyzed is input into the obtained pre-trained vein vessel segmentation model to obtain the segmentation region of the vein vessel, and the slicing effect after the automatic segmentation of the blood vessel is shown in fig. 9. On the basis of the blood vessel segmentation, fat areas around the hepatic vein and the portal vein are extracted by using a morphological dilation algorithm, and corresponding average fat content is calculated to be used as a quantitative value for analysis, wherein the average fat content around the hepatic vein is 2.39%, and the average fat content around the portal vein is 9.14%.

Through the process, all quantitative index values, pseudo-color maps, fat content early warning coordinates and the like are automatically integrated to synthesize an analysis report, so that comprehensive and accurate early diagnosis, disease monitoring and the like of the fatty liver are facilitated, and the method is popularized and used in clinic.

Therefore, in the prior art, quantitative analysis is realized only by calculating the mean value of the gray levels by segmenting the liver tissue and the spleen tissue, the quantitative index is single and not comprehensive enough, and the method cannot be popularized and applied in clinic. The invention firstly carries out liver segmentation and quantitative analysis of segmented fat, and gives out early warning of fat range. In addition, the present invention extracts perivascular fat by vessel segmentation and quantitatively analyzes it. The invention has important clinical application value in the aspects of early diagnosis, disease condition monitoring and the like of the fatty liver.

Compared with B-ultrasonic and CT image technologies, MRI has advantages of being noninvasive, non-radiative, high in accuracy and the like. The MRI fatty liver quantitative analysis is undoubtedly more advanced, has the advantages of no wound, no radiation, high accuracy and the like, and has outstanding clinical application value in the aspects of early diagnosis, disease monitoring, curative effect evaluation, prognosis evaluation and the like of fatty liver.

Referring to fig. 10, in an embodiment of the present invention, the present invention further provides an apparatus for accurately and quantitatively analyzing fatty liver, including:

the liver segmentation module is used for acquiring a liver segmentation result of a three-dimensional liver image to be analyzed;

The liver segmentation module is configured to obtain a liver segmentation result of the three-dimensional liver image to be analyzed, that is, to perform liver segmentation on the three-dimensional liver image to be analyzed to obtain a liver segmentation result, and specifically, any suitable method may be adopted.

The fat liver precision quantitative analysis device may further include any other suitable components, please refer to fig. 10, in an embodiment of the present invention, the fat liver precision quantitative analysis device further includes a liver segmentation model training module, which is in signal connection with the liver segmentation module and is configured to obtain a sample liver three-dimensional image and a liver segmentation label of the sample liver three-dimensional image; and taking the sample liver three-dimensional image and the liver segmentation label as a training set, and performing deep learning training on the liver segmentation model in an iteration mode to obtain the pre-trained liver segmentation model, wherein a deep learning network adopted by the liver segmentation model is a segmentation network based on the combination of a UNet/VNet and a channel attention mechanism.

In order to make the pre-trained liver segmentation model more accurate, in a specific embodiment of the present invention, the liver segmentation model training module is further configured to pre-process an original sample liver three-dimensional image to obtain the sample liver three-dimensional image, where the pre-processing includes one or more of histogram equalization, normalization, and normalization.

The segmentation network based on combination of UNet/VNet and channel attention mechanism may include any suitable structure, and in a specific embodiment of the present invention, the segmentation network based on combination of UNet/VNet and channel attention mechanism includes a convolutional layer, a pooling layer, an anti-convolutional layer, a cascading layer, and a batch normalization layer, the convolutional layer is connected to the anti-convolutional layer through the pooling layer, the cascading layer is respectively connected to the convolutional layer, the pooling layer, the anti-convolutional layer, and the batch normalization layer, the convolutional layer extracts a feature map of the three-dimensional image of the sample liver, the pooling layer performs a down-sampling operation on the feature map, the anti-convolutional layer performs a convolution operation on the feature map after padding to expand the size of the feature map, and the cascading layer combines the feature maps output by different levels, the batch normalization layer normalizes values of the feature map.

The liver segmentation model training module may include any suitable configuration, and preferably includes an encoding module and a decoding module, the encoding module is in signal connection with the decoding module, the encoding module and the decoding module respectively include a plurality of successively signal-connected UNet/VNet + channel attention mechanism modules, the UNet/VNet + channel attention mechanism module at the most downstream of the encoding module is in signal connection with the UNet/VNet + channel attention mechanism module at the most upstream of the decoding module, the decoding module further includes one or more of a multi-level fusion module and a full supervision module, the multi-level fusion module respectively is in signal connection with the UNet/VNet + channel attention mechanism module of the decoding module and is used for fusing feature maps output by the UNet/VNet + channel attention mechanism module of the decoding module, the full supervision module is respectively in signal connection with the UNet/VNet + channel attention mechanism module of the decoding module and used for calculating loss and optimizing the feature map output by the UNet/VNet + channel attention mechanism module of the decoding module and the liver segmentation label. Referring to fig. 2, in an embodiment of the present invention, the UNet/VNet + channel attention mechanism module is an UNet2.5d + channel attention mechanism module, the encoding module includes 3 levels of the UNet2.5d + channel attention mechanism modules, and the decoding module also includes 3 levels of the UNet2.5d + channel attention mechanism modules.

The loss function employed by the fully supervised module may be any suitable loss function, and in a specific embodiment of the present invention, the loss function employed by the fully supervised module is a multi-class cross entropy loss function.

In order to realize fat range early warning, please refer to fig. 10, in an embodiment of the present invention, the apparatus for accurately quantitative analysis of fatty liver further includes:

the fat quantitative range early warning module is in signal connection with the liver fat and segmented fat quantitative analysis module and is used for outputting a fat quantitative range early warning conclusion according to the percentage of the average fat content of the whole liver in the weight of the whole liver: if the average fat content of the whole liver is 5% -10% of the weight of the whole liver, judging that the whole liver belongs to mild fatty liver; determining that the whole liver is a moderate fatty liver if the average fat content of the whole liver is 10% -25% of the weight of the whole liver; determining that the whole liver is a severe fatty liver if the average fat content of the whole liver is more than 25% of the weight of the whole liver.

In order to perform quantitative analysis of liver fat more fully, referring to fig. 10, in an embodiment of the present invention, the apparatus for accurately quantitatively analyzing fatty liver further includes:

the vein vessel segmentation module is used for acquiring a segmentation area of a vein vessel of the liver three-dimensional image to be analyzed;

a perivenous fat region extraction module in signal connection with the venous blood vessel segmentation module and used for extracting hepatic vein and periportal fat regions of the venous blood vessels in the segmentation region;

and the venous perivascular fat quantitative analysis module is in signal connection with the venous perivascular fat region extraction module and is used for calculating the average fat content of the hepatic vein and the hepatic portal perivascular fat region.

The vein segmentation module is configured to obtain a segmentation region of a vein of the three-dimensional liver image to be analyzed, that is, to perform vein segmentation on the three-dimensional liver image to be analyzed to obtain a segmentation region of a vein, and may specifically adopt any suitable method.

The fat liver precision quantitative analysis device may further include any other suitable components, please refer to fig. 10, in an embodiment of the present invention, the fat liver precision quantitative analysis device further includes a vein vessel segmentation model training module, which is in signal connection with the vein vessel segmentation module and is configured to obtain a three-dimensional image of a sample liver and obtain vein vessel labeling data of the three-dimensional image of the sample liver; and taking the sample liver three-dimensional image and the vein vessel marking data as a training set, and carrying out deep learning training on the vein vessel segmentation model in an iteration mode to obtain the pre-trained vein vessel segmentation model, wherein a deep learning network adopted by the vein vessel segmentation model is a segmented network based on the combination of a UNet/VNet and a channel attention mechanism.

In order to make the pre-trained vein segmentation model more accurate, in a specific embodiment of the present invention, the vein segmentation model training module is further configured to pre-process an original sample liver three-dimensional image to obtain the sample liver three-dimensional image, where the pre-processing includes one or more of histogram equalization, normalization, and normalization.

The vessel segmentation model training module may include any suitable configuration, and preferably includes an encoding module and a decoding module, the encoding module is in signal connection with the decoding module, the encoding module and the decoding module respectively include a plurality of successively signal-connected UNet/VNet + channel attention mechanism modules, the UNet/VNet + channel attention mechanism module at the most downstream of the encoding module is in signal connection with the UNet/VNet + channel attention mechanism module at the most upstream of the decoding module, the decoding module further includes one or more of a multi-level fusion module and a full supervision module, the multi-level fusion module is in signal connection with the UNet/VNet + channel attention mechanism module of the decoding module and is used for fusing feature maps output by the UNet/VNet + channel attention mechanism module of the decoding module, the full supervision module is respectively in signal connection with the UNet/VNet + channel attention mechanism module of the decoding module and used for calculating loss and optimizing the feature map output by the UNet/VNet + channel attention mechanism module of the decoding module and the liver segmentation label. Referring to fig. 3, in an embodiment of the present invention, the UNet/VNet + channel attention mechanism module is an UNet2.5d + channel attention mechanism module, the encoding module includes 3 levels of the UNet2.5d + channel attention mechanism modules, and the decoding module also includes 3 levels of the UNet2.5d + channel attention mechanism modules.

The perivenous fat region extraction module is used for extracting hepatic vein and periportal vein fat regions of the venous blood vessels in the segmentation region by any suitable method, and in a specific embodiment of the invention, the perivenous fat region extraction module is used for extracting hepatic vein and periportal vein fat regions of the venous blood vessels in the segmentation region by using a morphological dilation algorithm.

In order to facilitate checking of various indexes obtained by the method for accurately quantitatively analyzing fatty liver, please refer to fig. 10, in an embodiment of the present invention, the apparatus further includes a fatty liver quantitative analysis report generating module, respectively connected to the fat quantitative range early warning module and the venous perivascular fat quantitative analysis module by signals, for generating a fatty liver quantitative analysis report, where the fatty liver quantitative analysis report includes the fat distribution uniformity, the average fat content, the median fat content, and the confidence interval of the whole liver; the mean fat content, the median fat content, and the confidence interval for each liver segment; the fat quantitative range early warning conclusion and the average fat content of the hepatic vein and perihepatic vein fat regions.

For other specific limitations of the apparatus for accurately quantitative analysis of fatty liver, reference may be made to the above limitations of the method for accurately quantitative analysis of fatty liver, and further description is omitted here. All or part of the modules in the fatty liver accurate quantitative analysis device can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In an embodiment of the present invention, the invention further provides a computer device, which includes a memory and a processor, where the memory stores a computer program, and the processor implements the method for constructing the automated liver segmentation model based on deep learning when executing the computer program.

In an embodiment of the present invention, the present invention further provides a computer readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the method for constructing an automated liver segmentation model based on deep learning.

Therefore, by adopting the accurate quantitative analysis method, the device, the computer equipment and the storage medium for the fatty liver, the deep learning technology is utilized to solve the problems of automatic liver segmentation and segmented fat quantitative result analysis, fat range early warning is realized through fat content quantitative analysis, and in addition, perivascular fat extraction and quantitative analysis are also solved, so that the more comprehensive and accurate quantitative analysis is realized for the fatty liver, and doctors can be helped to judge the fat deposition degree and the treatment effect of the fatty liver more accurately.

It will thus be seen that the objects of the invention have been fully and effectively accomplished. The functional and structural principles of the present invention have been shown and described in the embodiments, and the embodiments may be modified without departing from the principles. Therefore, this invention includes all modifications encompassed within the spirit and scope of the claims.

完整详细技术资料下载

上一篇：石墨接头机器人自动装卡簧、装栓机

下一篇：一种基于NSST的触头元素分布与形貌信息融合方法

Fatty liver accurate quantitative analysis method and device, computer equipment and storage medium

相关技术

网友询问留言