1. A liver blood vessel segmentation method based on multi-attention is characterized by comprising the following steps:

s100, obtaining a liver CTA image, removing a background area according to vessel labeling, respectively sampling from three directions of an axial plane, a coronal plane and a sagittal plane to obtain a corresponding sample set, inputting the sample set into a DA-UNet network based on a double-attention machine system for training and testing, and obtaining a well-trained liver vessel segmentation network model in the corresponding sampling direction; the DA-UNet networks based on the double-attention machine system in different sampling directions have the same structure, and weight coefficients are independent;

s200, receiving a liver CTA image, respectively sampling from three directions of an axial plane, a coronal plane and a sagittal plane, inputting the sampled image into the corresponding liver blood vessel segmentation network model to obtain a corresponding blood vessel segmentation result, and performing fusion processing on the blood vessel segmentation result to obtain a final segmentation result.

2. The method of claim 1, wherein the obtaining of the CTA image, removing the background region according to the vessel labeling, and sampling from three directions, namely an axial plane, a coronal plane and a sagittal plane, respectively, to obtain the corresponding sample sets comprises:

mapping the liver CTA image to a range of [0,1] according to an image intensity value, and carrying out normalization processing according to an average value and a variance;

sampling slices of the normalized liver CTA image from three directions of an axial plane, a coronal plane and a sagittal plane respectively, and taking a plurality of continuous slices as a sample;

and removing the sample without the blood vessel according to the blood vessel label to obtain the sample set.

3. The method of claim 1, wherein in the DA-UNet network based on the dual attention mechanism, based on the 2D U-Net architecture, the spatial attention and channel attention mechanisms are introduced at the high-order features, and are up-sampled to the original image size by deconvolution.

4. The method for segmenting liver blood vessels based on multi-attention according to claim 1, wherein the inputting the sample set into a DA-UNet network based on a dual-attention mechanism for training and testing, and obtaining the liver blood vessel segmentation network model trained in the corresponding direction comprises:

and inputting the sample set into a corresponding DA-UNet network based on a double-attention machine system according to a sampling direction for training, and training a model through a dice loss function and an Adam optimizer.

5. The method of claim 4, wherein the sample set is input into a DA-UNet network based on a dual-attention mechanism for training, and the learning rate is set to 0.001.

6. The method for segmenting liver blood vessels based on multiple attentions according to claim 1, wherein the DA-UNet network based on the double-attention mechanism obtained by training for different sampling directions is screened through a test set to obtain the liver blood vessel segmentation network model which best represents in the corresponding sampling directions, and evaluation indexes comprise a Dice coefficient based on area contact ratio and Hausdorff _95 based on boundary accuracy.

7. The method for segmenting liver blood vessels based on multiple attentions according to claim 1, wherein the step S200 comprises:

s210, receiving a liver CTA image, respectively sampling from the axial plane, the coronal plane and the sagittal plane, and inputting the sampling image into the corresponding liver blood vessel segmentation network model to obtain a corresponding blood vessel segmentation result;

s220, merging the blood vessel segmentation results output by the three liver blood vessel segmentation network models;

and S230, removing false positive regions in the blood vessel segmentation result according to the connected tree structure of the blood vessel, and reserving the fused maximum connected domain as the final blood vessel segmentation result.

8. A multi-attention based liver vessel segmentation system for performing the method of any one of claims 1 to 7, comprising:

the pretreatment module is used for sampling and slicing the liver CTA image from the axial plane, the coronal plane and the sagittal plane respectively to obtain a sampling image;

the model training module is used for removing a background area of the sampling image according to vessel labeling, obtaining a sample set according to the preprocessing module, inputting the sample set into a corresponding DA-UNet network based on a double attention mechanism according to a sampling direction, training and testing to obtain a well-trained liver vessel segmentation network model in the corresponding sampling direction; the DA-UNet networks based on the double-attention machine system in different sampling directions have the same structure, and weight coefficients are independent;

and the segmentation processing module is used for inputting the sampling image to the corresponding liver blood vessel segmentation network model according to the sampling direction to obtain a corresponding blood vessel segmentation result, and performing fusion processing on the blood vessel segmentation result to obtain a final segmentation result.

9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method of any one of claims 1 to 7.

Background

The radiotherapy interventional operation has the advantages that the radioactive source is placed into the tumor through percutaneous puncture to continuously irradiate local tumor tissues, so that the tumor can be effectively treated, and the recurrence and the metastasis of the tumor can be prevented. When the internal radiotherapy operation is carried out, a doctor needs to operate the puncture needle, the puncture needle is guided by the guide frame, the needle tip reaches a preset position in a body, and then the particle implanter and the particle push needle are connected to implant radioactive particles into a tumor body.

Since the surgical robot does not have medical background knowledge like a qualified physician and cannot quickly combine the medical background knowledge with patient information, it is not sufficient for the entire interventional operation to simply input target organ information to the robot, and it is necessary to inform the robot of peripheral organ information adjacent to the target organ and organ information of an interventional procedure route. Blood vessels are important organs for storing blood and transporting nutrients of a human body, and care should be particularly taken to avoid accidental injury in the operation process, so that the blood vessel segmentation in a medical image is an essential link in the puncture path planning.

The three-dimensional blood vessel structure has variable dimensions, large curvature and flexibility and many branches, and the blood vessel generally occupies a small proportion in a three-dimensional medical image, so that if image block sampling is carried out without guidance, the problem of sample imbalance is faced, and in order to meet clinical requirements, the continuity and the accuracy of blood vessel segmentation need to be improved.

Disclosure of Invention

The present invention is directed to solving at least one of the problems of the prior art. Therefore, the invention provides a liver blood vessel segmentation method based on multi-attention, which can improve the continuity and the accuracy of blood vessel segmentation.

The invention also provides a multi-attention-based liver blood vessel segmentation system with the multi-attention-based liver blood vessel segmentation method.

The invention also provides a computer readable storage medium with the multi-attention-based liver blood vessel segmentation method.

The multi-attention-based liver blood vessel segmentation method according to the first aspect of the invention comprises the following steps: s100, obtaining a liver CTA image, removing a background area according to vessel labeling, respectively sampling from three directions of an axial plane, a coronal plane and a sagittal plane to obtain a corresponding sample set, inputting the sample set into a DA-UNet network based on a double-attention machine system for training and testing, and obtaining a well-trained liver vessel segmentation network model in the corresponding sampling direction; the DA-UNet networks based on the double-attention machine system in different sampling directions have the same structure, and weight coefficients are independent; s200, receiving a liver CTA image, respectively sampling from three directions of an axial plane, a coronal plane and a sagittal plane, inputting the sampled image into the corresponding liver blood vessel segmentation network model to obtain a corresponding blood vessel segmentation result, and performing fusion processing on the blood vessel segmentation result to obtain a final segmentation result.

The liver blood vessel segmentation method based on multi-attention has at least the following beneficial effects: the 2.5D (dimension) data model is combined with multi-axis fusion of an axial plane, a coronal plane and a sagittal plane, the continuity of the segmentation result is improved under the condition of occupying less computing resources, a double attention mechanism is introduced to establish the association between different positions and different types of pixels in the image, and therefore the accuracy of the segmentation result is improved.

According to some embodiments of the present invention, the obtaining of the liver CTA image, removing the background region according to the vessel labeling, and sampling from three directions of an axial plane, a coronal plane, and a sagittal plane respectively to obtain the corresponding sample set includes: mapping the liver CTA image to a range of [0,1] according to an image intensity value, and carrying out normalization processing according to an average value and a variance; sampling slices of the normalized liver CTA image from three directions of an axial plane, a coronal plane and a sagittal plane respectively, and taking a plurality of continuous slices as a sample; and removing the sample without the blood vessel according to the blood vessel label to obtain the sample set.

According to some embodiments of the present invention, in a dual attention mechanism based DA-UNet network, spatial attention and channel attention mechanisms are introduced at high-order features based on a 2DU-Net architecture, and up-sampled to the original image size by deconvolution.

According to some embodiments of the present invention, the inputting the sample set into a DA-UNet network based on a dual-attention mechanism for training and testing to obtain a liver vessel segmentation network model trained in a corresponding direction includes: and inputting the sample set into a corresponding DA-UNet network based on a double-attention machine system according to a sampling direction for training, and training a model through a dice loss function and an Adam optimizer.

According to some embodiments of the invention, the sample set is input into a dual-attention-machine-based DA-UNet network for training, and the learning rate is set to 0.001.

According to some embodiments of the invention, through a test set, the DA-UNet network based on the double-attention machine system obtained by training in different sampling directions is screened to obtain the liver blood vessel segmentation network model which has the best performance in the corresponding sampling directions, and the evaluation indexes comprise a Dice coefficient based on area contact ratio and Hausdorff _95 based on boundary accuracy.

According to some embodiments of the invention, said step S200 comprises: s210, receiving a liver CTA image, respectively sampling from the axial plane, the coronal plane and the sagittal plane, and inputting the sampling image into the corresponding liver blood vessel segmentation network model to obtain a corresponding blood vessel segmentation result; s220, merging the blood vessel segmentation results output by the three liver blood vessel segmentation network models; and S230, removing false positive regions in the blood vessel segmentation result according to the connected tree structure of the blood vessel, and reserving the fused maximum connected domain as the final blood vessel segmentation result.

A multi-attention based liver vessel segmentation system according to an embodiment of the second aspect of the invention comprises: the pretreatment module is used for sampling and slicing the liver CTA image from the axial plane, the coronal plane and the sagittal plane respectively to obtain a sampling image; the model training module is used for removing a background area of the sampling image according to vessel labeling, obtaining a sample set according to the preprocessing module, inputting the sample set into a corresponding DA-UNet network based on a double attention mechanism according to a sampling direction, training and testing to obtain a well-trained liver vessel segmentation network model in the corresponding sampling direction; the DA-UNet networks based on the double-attention machine system in different sampling directions have the same structure, and weight coefficients are independent; and the segmentation processing module is used for inputting the sampling image to the corresponding liver blood vessel segmentation network model according to the sampling direction to obtain a corresponding blood vessel segmentation result, and performing fusion processing on the blood vessel segmentation result to obtain a final segmentation result.

The liver vessel segmentation system based on multi-attention has at least the following beneficial effects: the 2.5D (dimension) data model is combined with multi-axis fusion of an axial plane, a coronal plane and a sagittal plane, the continuity of the segmentation result is improved under the condition of occupying less computing resources, and a double attention mechanism is introduced to establish the association between different positions and different types of pixels in the image, so that the precision of the segmentation result is improved.

A computer-readable storage medium according to an embodiment of the third aspect of the invention has stored thereon a computer program which, when executed by a processor, implements a method according to an embodiment of the first aspect of the invention.

The computer-readable storage medium according to an embodiment of the present invention has at least the same advantageous effects as the method according to an embodiment of the first aspect of the present invention.

Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.

Drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

FIG. 1 is a schematic flow chart of a method according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a general framework of a liver vessel segmentation method according to an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of a DA-UNet network based on a dual attention mechanism according to an embodiment of the present invention;

FIG. 4 is a block diagram of the modules of the system of an embodiment of the present invention.

Reference numerals:

the system comprises a preprocessing module 100, a model training module 200 and a segmentation processing module 300.

Detailed Description

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.

In the description of the present invention, the meaning of a plurality of means is one or more, the meaning of a plurality of means is two or more, and more than, less than, more than, etc. are understood as excluding the present number, and more than, less than, etc. are understood as including the present number. If the first and second are described for the purpose of distinguishing technical features, they are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated. In the description of the present invention, the step numbers are merely used for convenience of description or for convenience of reference, and the sequence numbers of the steps do not mean the execution sequence, and the execution sequence of the steps should be determined by the functions and the inherent logic, and should not constitute any limitation to the implementation process of the embodiment of the present invention.

Referring to fig. 1, a method of an embodiment of the present invention includes: s100, obtaining a liver CTA image, removing a background area according to vessel labeling, respectively sampling from three directions of an axial plane, a coronal plane and a sagittal plane to obtain a corresponding sample set, inputting the sample set into a DA-UNet network based on a double-attention machine system for training and testing, and obtaining a well-trained liver vessel segmentation network model in the corresponding sampling direction; the DA-UNet networks based on the double attention mechanism in different sampling directions have the same structure, and weight coefficients are independent; s200, receiving a liver CTA image, respectively sampling from the axial plane, the coronal plane and the sagittal plane, inputting the sampling image into a corresponding liver blood vessel segmentation network model to obtain a corresponding blood vessel segmentation result, and performing fusion processing on the blood vessel segmentation result to obtain a final segmentation result.

The overall process of the method of the embodiment of the invention is shown in fig. 2. Firstly, removing a background region from a liver CTA image according to vessel labeling, then respectively sampling from three directions of an axial plane, a coronal plane and a sagittal plane to obtain a sample set corresponding to the three sampling directions, and inputting the sample set into a corresponding DA-UNet network based on a double attention mechanism according to the sampling directions for training and testing. The DA-UNet network structures corresponding to the three sampling directions based on the double-attention machine mechanism are the same, the weight coefficients are independent of each other, namely are not shared or interfered with each other, and the training is carried out independently.

After the training is finished, the three sampling directions all obtain a corresponding blood vessel segmentation model, and the three blood vessel segmentation models can be used for blood vessel detection. The input CTA image is sampled from the axial plane, the coronal plane and the sagittal plane respectively, a plurality of continuous sampling slices are used as a sample and input into the corresponding blood vessel segmentation model according to the sampling direction to obtain blood vessel segmentation results in the three sampling directions (namely the axial plane, the coronal plane and the sagittal plane), and then the blood vessel segmentation results obtained from the three sampling directions are fused and post-processed to obtain a final segmentation result. In the embodiment shown in fig. 2 and 3, 5 consecutive sample slices are taken as one sample.

According to the embodiment of the invention, aiming at the problem that 2D U-Net can not extract axial information so as to influence the segmentation precision and the blood vessel continuity, a 2.5D data model is adopted, namely 3D data of a CTA image is subjected to slice sampling along three mutually orthogonal axial directions (corresponding to an axial plane, a coronal plane and a sagittal plane respectively), and a final segmentation result is obtained by training and testing samples of a plurality of axes independently and performing fusion post-processing. In order to make up for the deficiency of the traditional U-Net network in the accuracy and continuity of the segmentation of the blood vessel region, as shown in fig. 3, a space and channel double attention mechanism is introduced to the high-order features, so that more extensive context information is encoded into the local features, and meanwhile, the feature representation of the blood vessel region is improved by highlighting the feature maps which are interdependent, thereby improving the accuracy of the segmentation of the blood vessel.

In an embodiment of the present invention, a process for obtaining samples from CTA image data for use as training data, includes: a. mapping the CTA image into a range of [0,1] according to an image intensity value, and carrying out normalization processing according to an average value and a variance; b. taking 5 continuous slices as a sample along three axial directions (corresponding to an axial plane, a coronal plane and a sagittal plane respectively); c. removing the sample containing no blood vessel according to the blood vessel label. Thus, three sets of training data sampled in different directions can be obtained.

In this embodiment, the structure of the dual-attention-machine-based DA-UNet network corresponding to a single sampling direction is the same, as shown in fig. 4. A DA-UNet network based on a double attention mechanism takes a 2D U-Net architecture as a basis, introduces a spatial attention and channel attention mechanism at high-order features, and then performs up-sampling to the size of an original image through deconvolution.

The training process of the present embodiment includes: inputting the three groups of preprocessed training sets into a corresponding DA-UNet network based on a double-attention machine system according to a sampling direction for training, adopting a dice loss function and an Adam optimizer training model, setting the learning rate to be 0.001, setting the sample size to be (5,256,256), setting the batch size to be 8, and training for 30 rounds. Obviously, in the embodiment of the present invention, the sample size, the batch size, and the number of training rounds may be configured as needed.

The test of the present embodiment includes: and screening out the best-performing models trained in different sampling directions by using the test set, and obtaining liver blood vessel segmentation network models in the three sampling directions, wherein the models correspond to an axial position, a coronal position and a sagittal position respectively. Evaluation indexes of the test screening include a Dice coefficient based on area contact ratio and Hausdorff _95 based on boundary accuracy.

The model obtained according to the method can extract different connectivity information from three orthogonal sampling directions, and the segmentation results obtained from the three sampling directions are subjected to post-processing, so that the connectivity information in different directions can be fused, and the segmentation results are further optimized. The post-treatment process comprises the following steps: merging the blood vessel segmentation results output by the three liver blood vessel segmentation network models; and removing false positive regions in the blood vessel segmentation result according to the connected tree structure of the blood vessel, and reserving the fused maximum connected domain as a final blood vessel segmentation result. And the output results of the three models are collected, so that the continuity of the segmentation structure is improved.

In summary, the method of the embodiment of the invention has at least the following beneficial effects: the 2.5D (dimension) data model is combined with multi-axis fusion of an axial plane, a coronal plane and a sagittal plane, the continuity of the segmentation result is improved under the condition of occupying less computing resources, and a double attention mechanism is introduced to establish the association between different positions and different types of pixels in the image, so that the precision of the segmentation result is improved.

The system of the embodiment of the present invention, referring to fig. 4, includes: the preprocessing module 100 is configured to obtain a liver CTA image, and perform sampling slicing in three directions, namely, an axial plane, a coronal plane, and a sagittal plane, to obtain a sampled image; the model training module 200 is used for removing a background area of a sampled image according to vessel labeling to obtain a corresponding sample set, inputting the sample set into a DA-UNet network based on a double-attention mechanism for training and testing to obtain a liver vessel segmentation network model trained in a corresponding sampling direction; the DA-UNet networks based on the double attention mechanism in different sampling directions have the same structure, and weight coefficients are independent; and the segmentation processing module 300 is configured to input the sampling image to the corresponding liver blood vessel segmentation network model according to the sampling direction to obtain a corresponding blood vessel segmentation result, and perform fusion processing on the blood vessel segmentation result to obtain a final segmentation result.

Although specific embodiments have been described herein, those of ordinary skill in the art will recognize that many other modifications or alternative embodiments are equally within the scope of this disclosure. For example, any of the functions and/or processing capabilities described in connection with a particular device or component may be performed by any other device or component. In addition, while various illustrative implementations and architectures have been described in accordance with embodiments of the present disclosure, those of ordinary skill in the art will recognize that many other modifications of the illustrative implementations and architectures described herein are also within the scope of the present disclosure.

Certain aspects of the present disclosure are described above with reference to block diagrams and flowchart illustrations of systems, methods, systems, and/or computer program products according to example embodiments. It will be understood that one or more blocks of the block diagrams and flowchart illustrations, and combinations of blocks in the block diagrams and flowchart illustrations, respectively, can be implemented by executing computer-executable program instructions. Also, according to some embodiments, some blocks of the block diagrams and flow diagrams may not necessarily be performed in the order shown, or may not necessarily be performed in their entirety. In addition, additional components and/or operations beyond those shown in the block diagrams and flow diagrams may be present in certain embodiments.

Accordingly, blocks of the block diagrams and flowchart illustrations support combinations of means for performing the specified functions, combinations of elements or steps for performing the specified functions and program instruction means for performing the specified functions. It will also be understood that each block of the block diagrams and flowchart illustrations, and combinations of blocks in the block diagrams and flowchart illustrations, can be implemented by special purpose hardware-based computer systems that perform the specified functions, elements or steps, or combinations of special purpose hardware and computer instructions.

Program modules, applications, etc. described herein may include one or more software components, including, for example, software objects, methods, data structures, etc. Each such software component may include computer-executable instructions that, in response to execution, cause at least a portion of the functionality described herein (e.g., one or more operations of the illustrative methods described herein) to be performed.

The software components may be encoded in any of a variety of programming languages. An illustrative programming language may be a low-level programming language, such as assembly language associated with a particular hardware architecture and/or operating system platform. Software components that include assembly language instructions may need to be converted by an assembler program into executable machine code prior to execution by a hardware architecture and/or platform. Another exemplary programming language may be a higher level programming language, which may be portable across a variety of architectures. Software components that include higher level programming languages may need to be converted to an intermediate representation by an interpreter or compiler before execution. Other examples of programming languages include, but are not limited to, a macro language, a shell or command language, a job control language, a scripting language, a database query or search language, or a report writing language. In one or more exemplary embodiments, a software component containing instructions of one of the above programming language examples may be executed directly by an operating system or other software component without first being converted to another form.

The software components may be stored as files or other data storage constructs. Software components of similar types or related functionality may be stored together, such as in a particular directory, folder, or library. Software components may be static (e.g., preset or fixed) or dynamic (e.g., created or modified at execution time).

The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the gist of the present invention.