1. An image processing method applied to a medical auxiliary diagnosis system, the method comprising:

acquiring first image data corresponding to a liver region and second image data corresponding to a blood vessel region, wherein blood vessels in the blood vessel region comprise hepatic veins and portal veins;

obtaining third image data corresponding to the center line of the blood vessel according to the second image data and a center line calculation method, wherein the third image data comprises pixel points;

performing connected domain division on the third image data according to a connected domain algorithm and a first rule to obtain at least two subsets;

determining a portal vein central line subset corresponding to the portal vein central lines from the at least two subsets according to the first image data and a second rule, wherein the second rule comprises a rule for determining the portal vein central line subset according to the position relation of the portal vein and the liver region and the structural characteristics of the portal vein;

and determining image data corresponding to the portal vein from the second image data according to the portal vein central line subset so as to distinguish the second image data into image data corresponding to the portal vein and image data corresponding to the hepatic vein.

2. The method of claim 1, the acquiring first image data corresponding to a liver region and second image data corresponding to a blood vessel region, comprising:

acquiring original image data, wherein the original image data comprises image data corresponding to a liver region and image data corresponding to a blood vessel region;

and inputting the original image data into an image processing model to obtain first image data corresponding to the liver region and second image data corresponding to the blood vessel region, wherein the image processing model is an image processing model obtained by training a large amount of training image data based on a deep learning method.

3. The method of claim 1, the first rule comprising: the number of the pixel points in the subset is larger than a first threshold and smaller than a second threshold;

correspondingly, performing connected domain division on the third image data according to a connected domain algorithm and a first rule to obtain at least two subsets, including:

performing connected domain division on the third image data according to a connected domain algorithm to obtain at least two connected domain subsets;

detecting whether the number of the pixels in each connected domain subset is smaller than a first threshold value, if so, merging the corresponding connected domain subset into any one of the connected domain subsets adjacent to the corresponding connected domain subset to obtain a subset meeting the first rule, if not, further detecting whether the number of the pixels in each connected domain subset is larger than a second threshold value, if so, dividing the corresponding connected domain subset into at least two subsets meeting the first rule, and if not, keeping the connected domain subset.

4. The method of claim 1, the determining a subset of portal vein centerlines from the at least two subsets that corresponds to portal vein centerlines according to the first image data and a second rule, comprising:

determining at least one first subset corresponding to a blood vessel central line outside the liver region according to the first image data and the position relation between the pixel points, and determining the at least one first subset as a subset corresponding to a portal vein central line;

calculating the distance between each first subset and the first image data and determining the first subset with the smallest distance between the first image data as the subset corresponding to the portal vein root;

and determining other subsets in the portal vein central line subset corresponding to the portal vein central line according to the subset corresponding to the portal vein root and the rule of determining the portal vein central line subset according to the structural characteristics of the portal vein.

5. The method of claim 4, wherein determining the other subset of the subset of portal vein centerlines to which the portal vein centerlines correspond according to the subset to which the portal vein root corresponds and a rule for determining the subset of portal vein centerlines according to structural features of the portal vein comprises:

setting a subset corresponding to the portal vein root as a root subset;

acquiring a subset set adjacent to the root node subset, detecting whether each subset in the subset set is a subset which is determined to be corresponding to a central line of a portal vein, if so, setting the corresponding subset as a leaf subset, if not, further performing corresponding processing according to the number of adjacent subsets of the corresponding subset, wherein if the number of adjacent subsets is greater than 1, further calculating an angle between the corresponding subset and the root node subset and determining whether to set the corresponding subset as a leaf subset according to the angle, and if the number of adjacent subsets is equal to 1, setting the corresponding subset as a leaf subset;

and detecting whether the number of the leaf subsets is 0, if so, ending the execution, and if not, sequentially setting the leaf subsets as root subsets.

6. The method of claim 5, the calculating angles of the respective subsets to the root subset, comprising:

determining neighbor points of the respective subset and the root subset;

determining a first farther point in the corresponding subset, which is relatively farther from the adjacent point, according to the adjacent point;

constructing a first direction vector corresponding to the corresponding subset according to the neighboring point and the first farther point;

determining a second distant point in the root subset, which is relatively distant from the adjacent point, according to the adjacent point;

constructing a second direction vector corresponding to the root subset according to the adjacent point and the second farther point;

calculating angles of the respective subsets to the root subset from the first direction vector and the second direction vector.

7. The method of claim 5, the determining whether to set the respective subset as a subset of leaves according to the angle, comprising:

if the angle is greater than 90 degrees, the corresponding subset is set as a leaf subset, otherwise the corresponding subset is not set as a leaf subset.

8. The method of claim 1, determining portal vein corresponding image data from the second image data based on the portal vein centerline subset, comprising

Performing the following steps for each of the subset of portal vein centerlines:

acquiring a maximum point and a minimum point in each subset according to the positions of the pixel points;

determining an expanded image data corresponding to each subset according to the maximum point and the minimum point;

and determining second image data intersected with the expanded image data as image data corresponding to the portal vein.

9. An image processing apparatus applied to a dialogue system, the apparatus comprising:

the image data acquisition module is used for acquiring first image data corresponding to a liver region and second image data corresponding to a blood vessel region, wherein blood vessels in the blood vessel region comprise hepatic veins and portal veins;

the center line calculation module is used for obtaining third image data corresponding to the center line of the blood vessel according to the second image data and a center line calculation method, wherein the third image data comprises pixel points;

the connected domain division module is used for carrying out connected domain division on the third image data according to a connected domain algorithm and a first rule to obtain at least two subsets;

a portal vein center line determining module, configured to determine, according to the first image data and a second rule, a portal vein center line subset corresponding to a portal vein center line from the at least two subsets, where the second rule includes a rule for determining the portal vein center line subset according to a position relationship between the portal vein and the liver region and a structural feature of the portal vein;

and the portal vein determining module is used for determining image data corresponding to the portal vein from the second image data according to the portal vein central line subset so as to distinguish the second image data into image data corresponding to the portal vein and image data corresponding to the hepatic vein.

10. A computer storage medium comprising a set of computer executable instructions for performing the method of any one of claims 1 to 8 when executed.

Background

With the rapid development of computer-aided diagnosis technology and computer image processing technology, medical-aided diagnosis systems that extract features from a large amount of medical image data (such as MRI and CT) for machine learning and combine with medical knowledge are receiving more and more attention.

Particularly, the liver auxiliary diagnosis system can better help doctors to master the anatomical structure of the liver and perform corresponding diagnosis and treatment because the computer image processing can be used for segmenting doctor interested regions such as a liver region, a tumor region, a blood vessel region and the like from a medical image, performing feature extraction, three-dimensional reconstruction and the like on the regions, classifying the segmented blood vessels. The hepatic vascular system mainly comprises a hepatic vein and a portal venous system, so when the hepatic blood vessel is further distinguished, the hepatic blood vessel is divided into a hepatic vein area and a portal vein area. The existing method mainly distinguishes hepatic vein regions and portal vein regions based on different anatomical structures, but the method has large processing capacity and is difficult to adapt to a real-time liver auxiliary diagnosis system.

How to optimize the existing scheme, improve the processing efficiency and shorten the calculation time is a technical problem which needs to be solved urgently for further popularizing the liver auxiliary diagnosis system.

Disclosure of Invention

In view of the above problems, embodiments of the present invention provide an image processing method, an image processing apparatus, and a storage medium.

According to a first aspect of the embodiments of the present invention, an image processing method is applied to a medical auxiliary diagnosis system, and the method includes: acquiring first image data corresponding to a liver region and second image data corresponding to a blood vessel region, wherein blood vessels in the blood vessel region comprise hepatic veins and portal veins; obtaining third image data corresponding to the center line of the blood vessel according to the second image data and a center line calculation method, wherein the third image data comprises pixel points; performing connected domain division on the third image data according to a connected domain algorithm and a first rule to obtain at least two subsets; determining a portal vein central line subset corresponding to the portal vein central line from at least two subsets according to the first image data and a second rule, wherein the second rule comprises a rule for determining the portal vein central line subset according to the position relation of the portal vein and the liver region and the structural characteristics of the portal vein; and determining image data corresponding to the portal vein from the second image data according to the portal vein central line subset so as to distinguish the second image data into image data corresponding to the portal vein and image data corresponding to the hepatic vein.

According to an embodiment of the present invention, acquiring first image data corresponding to a liver region and second image data corresponding to a blood vessel region includes: acquiring original image data, wherein the original image data comprises image data corresponding to a liver region and image data corresponding to a blood vessel region; the method comprises the steps of inputting original image data into an image processing model to obtain first image data corresponding to a liver region and second image data corresponding to a blood vessel region, wherein the image processing model is an image processing model obtained through training of a large amount of training image data based on a deep learning method.

According to an embodiment of the present invention, the first rule includes: the number of the pixel points in the subset is larger than a first threshold and smaller than a second threshold; correspondingly, performing connected domain division on the third image data according to a connected domain algorithm and a first rule to obtain at least two subsets, including: performing connected domain division on the third image data according to a connected domain algorithm to obtain at least two connected domain subsets; detecting whether the number of the pixels in each connected domain subset is smaller than a first threshold value, if so, merging the corresponding connected domain subset into any one of the connected domain subsets adjacent to the corresponding connected domain subset to obtain a subset meeting a first rule, if not, further detecting whether the number of the pixels in each connected domain subset is larger than a second threshold value, if so, dividing the corresponding connected domain subset into at least two subsets meeting the first rule, and if not, keeping the connected domain subset.

According to an embodiment of the present invention, determining a portal vein center line subset corresponding to a portal vein center line from at least two subsets according to first image data and a second rule includes: determining at least one first subset corresponding to a blood vessel central line outside the liver region according to the first image data and the position relation between the pixel points, and determining the at least one first subset as a subset corresponding to a portal vein central line; calculating the distance between each first subset and the first image data and determining the first subset with the smallest distance with the first image data as the subset corresponding to the portal vein root; and determining other subsets in the portal vein central line subset corresponding to the portal vein central line according to the subset corresponding to the portal vein root and the rule of determining the portal vein central line subset according to the structural characteristics of the portal vein.

According to an embodiment of the present invention, determining other subsets of the portal vein centerline subset corresponding to the portal vein centerline according to the subset corresponding to the portal vein root and the rule for determining the portal vein centerline subset according to the structural feature of the portal vein includes: setting a subset corresponding to the portal vein root as a root subset; acquiring a subset set adjacent to a root node subset, detecting whether each subset in the subset set is a subset which is determined to be corresponding to a portal vein central line, if so, setting the corresponding subset as a leaf subset, if not, further performing corresponding processing according to the number of the adjacent subsets of the corresponding subset, wherein if the number of the adjacent subsets is greater than 1, further calculating the angle between the corresponding subset and the root node subset, and determining whether to set the corresponding subset as the leaf subset according to the angle, and if the number of the adjacent subsets is equal to 1, setting the corresponding subset as the leaf subset; and detecting whether the number of the leaf subsets is 0, if so, ending the execution, and if not, sequentially setting the leaf subsets as root subsets.

According to an embodiment of the present invention, calculating the angles between the corresponding subsets and the root subset includes: determining neighbor points of the respective subsets to the root subset; determining a first farther point in the corresponding subset, which is relatively farther from the adjacent point, according to the adjacent point; constructing a first direction vector corresponding to the corresponding subset according to the adjacent point and the first farther point; determining a second distant point in the root subset that is relatively distant from the adjacent point location according to the adjacent point; constructing a second direction vector corresponding to the root subset according to the adjacent point and the second farther point; the angles of the respective subsets to the root subset are calculated from the first direction vector and the second direction vector.

According to an embodiment of the present invention, determining whether to set the corresponding subset as the leaf subset according to the angle includes: if the angle is greater than 90 degrees, the corresponding subset is set as the leaf subset, otherwise the corresponding subset is not set as the leaf subset.

According to an embodiment of the present invention, determining image data corresponding to the portal vein from the second image data according to the portal vein centerline subset includes performing the following steps for each of the portal vein centerline subsets: acquiring a maximum point and a minimum point in each subset according to the positions of the pixel points; determining an expanded image data corresponding to each subset according to the maximum point and the minimum point; the second image data intersecting the extended image data is determined as image data corresponding to the portal vein.

According to a second aspect of the embodiments of the present invention, an image processing apparatus applied to a dialogue system, the apparatus includes: the image data acquisition module is used for acquiring first image data corresponding to a liver region and second image data corresponding to a blood vessel region, wherein blood vessels in the blood vessel region comprise hepatic veins and portal veins; the center line calculation module is used for obtaining third image data corresponding to the center line of the blood vessel according to the second image data and the center line calculation method, wherein the third image data comprises pixel points; the connected domain division module is used for carrying out connected domain division on the third image data according to a connected domain algorithm and a first rule to obtain at least two subsets; a portal vein central line determining module, configured to determine, according to the first image data and a second rule, a portal vein central line subset corresponding to a portal vein central line from the at least two subsets, where the second rule includes a rule for determining the portal vein central line subset according to a positional relationship between the portal vein and the liver region and a structural feature of the portal vein; and the portal vein determining module is used for determining image data corresponding to the portal vein from the second image data according to the portal vein central line subset so as to distinguish the second image data into image data corresponding to the portal vein and image data corresponding to the hepatic vein.

According to a third aspect of embodiments of the present invention, a computer storage medium comprises a set of computer executable instructions for performing any of the image processing methods described above when the instructions are executed.

The embodiment of the invention provides an image processing method, an image processing device and a storage medium. The method comprises the following steps: firstly, respectively extracting a liver region image and a blood vessel region image; then, extracting a blood vessel central line and dividing the blood vessel central line into a plurality of regions with proper sizes according to the connectivity; then, determining a central line area corresponding to the portal vein from the central line areas of the plurality of blood vessels according to the position relation between the portal vein and the liver area and the structural characteristics of the portal vein; and then, performing center line restoration according to the center line area corresponding to the portal vein to obtain a portal vein image so as to distinguish the portal vein image from the hepatic vein image. The hepatic vein and portal vein images can be simplified into dendritic linear structures by extracting the central lines, the hepatic vein and portal vein can be distinguished conveniently by combining the position relation of the portal vein and the liver region and the structural characteristics of the portal vein, and the real-time performance of the liver tumor auxiliary diagnosis system for real-time analysis is greatly improved.

It is to be understood that the teachings of the present invention need not achieve all of the above-described benefits, but rather that specific embodiments may achieve specific technical results, and that other embodiments of the present invention may achieve benefits not mentioned above.

Drawings

The above and other objects, features and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description read in conjunction with the accompanying drawings. Several embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which:

in the drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

FIG. 1 is a schematic flow chart of an implementation of an image processing method according to an embodiment of the present invention;

FIG. 2 is a flowchart illustrating an implementation of an application of an image processing method according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a blood vessel region in an application of the image processing method according to the embodiment of the present invention;

FIG. 4 is a schematic diagram of a centerline of a blood vessel in an application of the image processing method according to the embodiment of the present invention;

FIG. 5 is a diagram illustrating a result of differentiating central lines of portal veins by applying an image processing method according to an embodiment of the present invention;

FIG. 6 is a diagram illustrating a result of differentiating portal vein areas according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of a composition structure of an image processing apparatus according to an embodiment of the present invention.

Detailed Description

In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.

Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.

Fig. 1 shows a flow of implementing the image processing method according to the embodiment of the present invention, which is mainly applied to a medical auxiliary diagnosis system. Referring to fig. 1, the method includes: operation 110, acquiring first image data corresponding to a liver region and second image data corresponding to a blood vessel region, where blood vessels in the blood vessel region include a hepatic vein and a portal vein; operation 120, obtaining third image data corresponding to the centerline of the blood vessel according to the second image data and the centerline calculation method, where the third image data includes pixel points; operation 130, performing connected domain division on the third image data according to a connected domain algorithm and a first rule to obtain at least two subsets; an operation 140 of determining a portal vein centerline subset corresponding to the portal vein centerline from the at least two subsets according to the first image data and a second rule, wherein the second rule includes a rule for determining the portal vein centerline subset according to a position relationship of the portal vein with the liver region and a structural feature of the portal vein; in operation 150, image data corresponding to the portal vein is determined from the second image data according to the portal vein centerline subset to distinguish the second image data into image data corresponding to the portal vein and image data corresponding to the hepatic vein.

In operation 110, first image data corresponding to a liver region and second image data corresponding to a blood vessel region are mainly derived from a medical image sequence, such as an MRI or CT image sequence, and are extracted by some image segmentation algorithm. The embodiment of the invention does not limit the specific data source and image segmentation algorithm, and an implementer can adopt any applicable data source and image segmentation algorithm. However, the implementation effect of the embodiment of the present invention is affected by the precision of the image segmentation algorithm, and the higher the precision of the image segmentation algorithm adopted by the implementer is, the higher the accuracy of the image processing method in the embodiment of the present invention in distinguishing portal veins and hepatic veins is.

Wherein, the first image data corresponding to the liver area is mainly used as a reference object for distinguishing portal veins and hepatic veins; the second image data corresponding to the blood vessel region is image data including portal veins and hepatic veins, and is an object to be processed in the embodiment of the present invention.

In operation 120, the vessel centerline is a line formed by connecting the center points of each section perpendicular to the vessel direction, and can better reflect the vessel structure; there are many centerline calculation methods, and the embodiments of the present invention do not limit the centerline calculation method, and the implementer may use any suitable centerline calculation method, such as a blood vessel centerline extraction algorithm based on a minimum path, a PMPP-BT algorithm based on an active contour model, and the like. However, the implementation effect of the embodiment of the invention is influenced by the accuracy of the center line calculation method, and the higher the accuracy of the center line algorithm adopted by an implementer is, the higher the accuracy of the image processing method for distinguishing the portal vein from the hepatic vein in the embodiment of the invention is.

Among them, extracting the center line of the blood vessel can simplify the object of image processing, and is particularly suitable for distinguishing portal veins and hepatic veins by utilizing the difference of the structural features of the blood vessel.

In operation 130, the connected component generally refers to an image region composed of foreground pixels located adjacent to each other in the binary image. Connected component analysis refers to finding and labeling each connected component in the image in the graph. The connected domain division means that an overlarge connected domain is divided into connected domains with proper sizes on the basis of connected domain analysis. The embodiment of the present invention is not limited to the connected component algorithm, and the implementer may use any suitable connected component algorithm, for example, Two-Pass algorithm, Seed-Filling algorithm, etc. However, the implementation effect of the embodiment of the present invention is affected by the accuracy of the connected component algorithm, and the higher the accuracy of the connected component algorithm adopted by the implementer is, the higher the accuracy of the image processing method for distinguishing portal veins and hepatic veins in the embodiment of the present invention is.

The vessel center line is divided into connected areas with proper sizes, so that the regional processing is facilitated, the operation complexity and the calculation amount for determining whether the current center line subset is the portal vein center line subset can be simplified each time, and the processing precision and the judgment accuracy are correspondingly improved.

In operation 140, the second rule is a rule to distinguish and determine portal vein centerlines from hepatic vein centerlines. The inventor finds that the portal vein central line and the hepatic vein central line can be easily and accurately distinguished by combining the position relation of the portal vein and the hepatic region and the special structural characteristics of the portal vein. The implementer may also add other rules based on this to adjust or optimize the final result.

By this step, the centerline subset of moderate size can be divided into two parts, one part being the portal vein centerline subset corresponding to the portal vein centerline and the other part being the hepatic vein centerline subset corresponding to the hepatic vein centerline.

In operation 150, image data corresponding to the portal vein and image data corresponding to the hepatic vein are determined from the second image data (image data corresponding to the blood vessel region) based on the portal vein center line subset and the hepatic vein center line subset obtained in operation 140, respectively. In this way, a distinction between hepatic and portal veins is achieved.

According to the image processing method, the hepatic vein image and the portal vein image are simplified into the dendritic linear structure by extracting the center line, the hepatic vein center line and the portal vein center line can be rapidly distinguished by combining the position relation of the portal vein and the hepatic region and the structural characteristics of the portal vein, and then the hepatic vein blood vessel region and the blood vessel region corresponding to the portal vein are determined according to the hepatic vein center line and the portal vein center line, so that the method for rapidly distinguishing the hepatic vein and the portal vein is realized, and the real-time performance of the liver tumor auxiliary diagnosis system for real-time analysis is correspondingly improved.

According to an embodiment of the present invention, acquiring first image data corresponding to a liver region and second image data corresponding to a blood vessel region includes: acquiring original image data, wherein the original image data comprises image data corresponding to a liver region and image data corresponding to a blood vessel region; the method comprises the steps of inputting original image data into an image processing model to obtain first image data corresponding to a liver region and second image data corresponding to a blood vessel region, wherein the image processing model is an image processing model obtained through training of a large amount of training image data based on a deep learning method.

In the embodiment, the image processing model trained by a large amount of training image data based on the deep learning method is used for extracting the image data corresponding to the liver region and the image data corresponding to the blood vessel region from the original image data, so that the accuracy is higher and the accuracy is higher, and the hepatic vein and the portal vein can be more accurately distinguished.

According to an embodiment of the present invention, the first rule includes: the number of the pixel points in the subset is larger than a first threshold and smaller than a second threshold; correspondingly, performing connected domain division on the third image data according to a connected domain algorithm and a first rule to obtain at least two subsets, including: performing connected domain division on the third image data according to a connected domain algorithm to obtain at least two connected domain subsets; detecting whether the number of the pixels in each connected domain subset is smaller than a first threshold value, if so, merging the corresponding connected domain subset into any one of the connected domain subsets adjacent to the corresponding connected domain subset to obtain a subset meeting a first rule, if not, further detecting whether the number of the pixels in each connected domain subset is larger than a second threshold value, if so, dividing the corresponding connected domain subset into at least two subsets meeting the first rule, and if not, keeping the connected domain subset.

As described above, the division of the connected component subset is mainly for the purpose of the divisional domain processing to simplify the computational complexity and the amount of computation each time for determining whether the current centerline subset is the portal vein centerline subset. However, it should be noted that if the size of each connected domain subset is too large, the purpose of simplifying the operation complexity is not achieved; conversely, if the size of each subset of connected components is too large, the number of times the corresponding operation is performed is increased.

Therefore, in the present embodiment, the first rule is to control the size of the connected component subset mainly by controlling the number of pixels so that the size can be controlled within a range of an appropriate size. The initial values of the first threshold and the second threshold may be specified according to expert experience or domain knowledge, and then may be adjusted according to the implementation effect to obtain a better implementation effect.

According to an embodiment of the present invention, determining a portal vein center line subset corresponding to a portal vein center line from at least two subsets according to first image data and a second rule includes: determining at least one first subset corresponding to a blood vessel central line outside the liver region according to the first image data and the position relation between the pixel points, and determining the at least one first subset as a subset corresponding to a portal vein central line; calculating the distance between each first subset and the first image data and determining the first subset with the smallest distance with the first image data as the subset corresponding to the portal vein root; and determining other subsets in the portal vein central line subset corresponding to the portal vein central line according to the subset corresponding to the portal vein root and the rule of determining the portal vein central line subset according to the structural characteristics of the portal vein.

The inventors have found that of the hepatic and portal veins, only the root of the portal vein is outside the liver region. Therefore, in the embodiment of the present invention, the present inventors have skillfully utilized this feature, and proposed the following method for distinguishing hepatic vein center lines from portal vein center lines: firstly, determining a blood vessel central line subset outside the liver region, then selecting the subset closest to the lower end of the liver as a portal vein root subset, and determining the portal vein central line subset by taking the portal vein central line subset as a starting point.

Compared with the method which only distinguishes according to the main structure characteristics of the portal vein and the hepatic vein, the method is easier to execute, and can obtain more blood vessels at branches along the vine, so that the distinguishing result is higher in precision and more accurate.

According to an embodiment of the present invention, determining other subsets of the portal vein centerline subset corresponding to the portal vein centerline according to the subset corresponding to the portal vein root and the rule for determining the portal vein centerline subset according to the structural feature of the portal vein includes: setting a subset corresponding to the portal vein root as a root subset; acquiring a subset set adjacent to a root node subset, detecting whether each subset in the subset set is a subset which is determined to be corresponding to a portal vein central line, if so, setting the corresponding subset as a leaf subset, if not, further performing corresponding processing according to the number of the adjacent subsets of the corresponding subset, wherein if the number of the adjacent subsets is greater than 1, further calculating the angle between the corresponding subset and the root node subset, and determining whether to set the corresponding subset as the leaf subset according to the angle, and if the number of the adjacent subsets is equal to 1, setting the corresponding subset as the leaf subset; and detecting whether the number of the leaf subsets is 0, if so, ending the execution, and if not, sequentially setting the leaf subsets as root subsets.

In the present embodiment, the present inventors found that the growth direction of the portal vein grows substantially toward the edge of the liver, and therefore, when judging whether or not the adjacent subset of the root subset is the subset corresponding to the portal vein center line, the structural feature is used ingeniously, and the judgment is mainly made by calculating the angle between the root subset and the adjacent subset. Therefore, whether the adjacent subset of the root subset is the subset corresponding to the central line of the portal vein can be easily and accurately determined, suspicious vessel branches are not easy to miss, and full coverage is realized.

According to an embodiment of the present invention, calculating the angles between the corresponding subsets and the root subset includes: determining neighbor points of the respective subsets to the root subset; determining a first farther point in the corresponding subset, which is relatively farther from the adjacent point, according to the adjacent point; constructing a first direction vector corresponding to the corresponding subset according to the adjacent point and the first farther point; determining a second distant point in the root subset that is relatively distant from the adjacent point location according to the adjacent point; constructing a second direction vector corresponding to the root subset according to the adjacent point and the second farther point; the angles of the respective subsets to the root subset are calculated from the first direction vector and the second direction vector.

In the embodiment, the direction vectors of the corresponding subsets and the root subsets are determined according to the adjacent points of the corresponding subsets and the root subsets, and the angles of the corresponding subsets and the root subsets are calculated through the direction vectors, so that the error is smaller, and the portal vein center line and the hepatic vein center line can be more accurately distinguished.

According to an embodiment of the present invention, determining whether to set the corresponding subset as the leaf subset according to the angle includes: if the angle is greater than 90 degrees, the corresponding subset is set as the leaf subset, otherwise the corresponding subset is not set as the leaf subset.

In the present embodiment, if the angle is greater than 90 degrees, it is more suitable for the structural feature that the growth direction of the portal vein grows substantially toward the edge of the liver, and when the judgment is made by using this label, the effect of distinguishing the portal vein center line from the hepatic vein center line is good.

According to an embodiment of the present invention, determining image data corresponding to the portal vein from the second image data according to the portal vein centerline subset includes performing the following steps for each of the portal vein centerline subsets: acquiring a maximum point and a minimum point in each subset according to the positions of the pixel points; determining an expanded image data corresponding to each subset according to the maximum point and the minimum point; the second image data intersecting the extended image data is determined as image data corresponding to the portal vein.

In the present embodiment, the image data is expanded by the maximum point and the minimum point in the center line subset, so that the intersection range with the image data can be expanded by expanding the image data, and the image data corresponding to the portal vein can be more easily determined.

Fig. 2 shows a specific implementation flow of an application of the image processing method according to the embodiment of the present invention, as shown in fig. 2, the application mainly distinguishes the blood vessel image of the liver region into a portal vein blood vessel image and a hepatic vein blood vessel image by the following steps:

step 2010, acquiring liver region image data and blood vessel region image data;

specifically, a medical image sequence, such as an MRI or CT picture sequence, is acquired, and a Liver region Liver in the image is obtained based on a deep learning method, such as U-net3D_maskAnd a blood vessel region V_original. Wherein the blood vessel region V_originaIncluding the hepatic and portal veins, as shown in fig. 3.

Step 2020, extracting blood vessel center line image data;

specifically, the 3D hinning algorithm is adopted to obtain the V of the blood vessel central line from the blood vessel region_originalExtracting the center line of a blood vesselAs shown in fig. 4.

Step 2030, dividing the vessel centerline image data into a plurality of subsets;

specifically, the three-dimensional connected domain analysis of the 18 neighborhoods is used for analyzing the center line of the blood vesselThe lines are divided into a plurality of subsets. Then detecting the number of pixel points in each subset, and if the number of the pixel points is less than 5, merging the pixel points into the subset corresponding to any one of the surrounding connected domains; if the pixel point is more than 100, connecting the domainsThe corresponding subsets are divided into subsets of up to 50 pixels. The specific method is that assuming that the coordinates of the pixel points in the connected domain are (x, y, z), selecting a pixel point i at the edge of the connected domain, such as the x-value minimum point, the y-value minimum point or the z-value minimum point, searching in a 18-neighborhood manner near the pixel point, counting the nearby pixel points into the region where the i point is located, stopping until the number of the pixel points is greater than 50, and performing the same operation on the pixel points which are not counted into the i region until the number of the pixel points in each region is not greater than 50. After the connected domain analysis and post-processing are completed, the central line of the blood vesselIs divided into n subsets s_nThe final effect is shown in FIG. 5, where different subsets are displayed as areas of different ink shades.

2040, determining a subset corresponding to the portal vein root;

in particular, in n vessel centerline subsets s_nFinding a subset corresponding to the portal root. Firstly, selecting the liver regionOuter m vessel centerline subsets s_mThen, the subset closest to the lower end of the liver in the m subsets is selected as the subset pv corresponding to the root of the portal vein_rootMeanwhile, the m blood vessel regions are marked as portal vein regions, and the rest n-m regions are marked as non-portal vein regions.

Step 2050, obtaining a subset set adjacent to the root node subset, and obtaining a first subset;

specifically, the subset pv corresponding to the portal vein root is divided into two subsets_rootSetting the root subset as the root subset, and finding out pv by taking the root subset as a starting point_rootThe nearby area is correspondingly sub-set, assuming a total of I sub-sets d nearby_iObtaining a certain subset d_i。

Step 2060, detecting the subset d_iWhether the central line of the portal vein is the subset corresponding to the central line of the portal vein is determined, if so, continuing to the step 2070, and if not, continuing to the step 2080;

step 2070, subset d_iIs a subset which is determined to correspond to the central line of the portal vein, and the subset is set as a leaf subset;

step 2080, subset d_iIf the central line of the portal vein is not the subset corresponding to the central line of the portal vein, determining whether the subset is the subset corresponding to the portal vein or not, carrying out corresponding processing, and then obtaining the next subset;

specifically, if d_iBelongs to m regions marked as portal veins, then d is directly marked_iIs denoted as pv_rootOtherwise, find each region d_iArea of vicinityTotal of J dnear_j. If J is 1, the region d is represented_iOnly the portal vein region pv_rootConnecting, then connecting the areas d_iIs denoted as pv_rootThe leaf node of (1). If J is>1, representing the region d_iThe nearby area still has removed pv_rootOuter vessel region, which requires d to be calculated_iAnd pv_rootThe angle between d, since the growth direction of the portal vein grows substantially towards the edge of the liver_iAnd pv_rootThe angle between them needs to be more than 90 degrees when d_iAnd pv_rootWhen the angle between d and d is greater than 90 degrees_iIs denoted as pv_rootLeaf node of otherwise not d_iIs denoted as pv_rootThe leaf node of (1).

The specific method for calculating the angle between the areas comprises the following steps:

1) first calculate d_iAnd pv_rootAdjacent pixel points, among which belong to d_iAdjacent pixel points of Belonging to pv_rootAdjacent pixel point ofE.g. calculating pv_rootAnd d in_iThe adjacent pixel points can calculate the pv_root26 neighborhoods of all pixel points in the image, when d exists in the 26 neighborhoods_iA point in (1), then the point is d_iAnd pv_rootAdjacent pixel point

2) Calculating d_iNeutral separationDistant pixel pointE.g. selecting 5 distant neighboring pixelsThe farthest pixel point is calculated by d_iAnd the distance between the average value of the distance between the average value and the adjacent pixel points except other adjacent pixel points is farther from the average value.

3) Calculation of pv_rootNeutral separationDistant pixel point

4) Then, d is obtained_iAnd pv_rootThe angle therebetween:

first, the region d is calculated_iDirection vector ofFirstly, calculating the position mean value of adjacent pixel points and the position mean value of a far point:

then d_iDirection vector ofComprises the following steps:

then, the region pv was calculated_rootDirection vector of

Then d_iAnd pv_rootThe angle between is:

when cos alpha is less than 0, it means that both angles are greater than 90 degrees

Step 2100, detecting whether the number of the leaf subsets is 0, if not, continuing to step 2110, and if so, continuing to step 2120;

step 2110, if the number of the leaf subsets is not 0, setting the leaf subsets as root subsets in sequence;

specifically, the leaf nodes of the root node are regarded as new root nodes, and step 2050 is repeated until the number of subsets near all the leaf nodes is 0 or the condition of adding leaf nodes is not satisfied. At this time, the expression of pv_rootThe tree-like linear structure of portal centerline for the root node is constructed and labeled as a portal centerline subset, such as the lighter ink area shown in FIG. 5.

Step 2120, determining image data corresponding to the portal vein from the blood vessel region image data according to the portal vein central line subset;

in particular, the labeling corresponds to the original vessel region V as a subset of portal vein centerlines_originalIn (3), diameter recovery of the portal vein centerline is performed. The recovery method comprises the following steps:

finding a subset s of vessel centerlines marked as portal veins_nThe area is as follows: first, at s_nMaximum value x of middle pixel point on x, y and z coordinates_max，y_max，z_maxAnd the minimum value x_min，y_min，z_min(ii) a Then, in the blood vessel region V_originalObtain an expanded region, if 30 pixels are expanded outwards, select V_originalIn (2) [ x ]_min-15:x_max+15，y_min-15:y_max+15， z_min-15:z_max+15]The region and other blood vessel regions are set to zero to obtain a central line s_nNearby blood vessel V_cutThen V is_cutNeutral centerline subset s_nThe blood vessel connected region with intersection is portal vein. The final recovery effect is shown in fig. 6, where the vessel with shallow ink is the portal vein.

The specific implementation flow of the image processing method of the present embodiment is only an exemplary description, and is not a limitation to the specific implementation flow of the image processing method of the present embodiment, and an implementer may select any suitable implementation method or specific flow according to implementation conditions.

Further, an embodiment of the present invention further provides an image processing apparatus, which is applied to a dialog system, and as shown in fig. 7, the apparatus 70 includes: an image data obtaining module 701, configured to obtain first image data corresponding to a liver region and second image data corresponding to a blood vessel region, where a blood vessel in the blood vessel region includes a hepatic vein and a portal vein; a centerline calculation module 702, configured to obtain third image data corresponding to a centerline of the blood vessel according to the second image data and a centerline calculation method, where the third image data includes pixel points; a connected domain dividing module 703, configured to perform connected domain division on the third image data according to a connected domain algorithm and a first rule to obtain at least two subsets; a portal vein center line determining module 704, configured to determine a portal vein center line subset corresponding to the portal vein center line from the at least two subsets according to the first image data and a second rule, where the second rule includes a rule for determining the portal vein center line subset according to a position relationship between the portal vein and the liver region and a structural feature of the portal vein; a portal vein determining module 705, configured to determine image data corresponding to a portal vein from the second image data according to the portal vein centerline subset, so as to distinguish the second image data into image data corresponding to the portal vein and image data corresponding to a hepatic vein.

According to an embodiment of the present invention, the image data obtaining module 701 includes: the original image data acquisition submodule is used for acquiring original image data, wherein the original image data comprises image data corresponding to a liver region and image data corresponding to a blood vessel region; and the image processing submodule is used for inputting the original image data into an image processing model to obtain first image data corresponding to a liver region and second image data corresponding to a blood vessel region, wherein the image processing model is an image processing model obtained by training a large amount of training image data based on a deep learning method.

According to an embodiment of the present invention, the first rule includes: the number of the pixel points in the subset is larger than a first threshold and smaller than a second threshold; accordingly, the connected domain dividing module 702 includes: the connected domain subset division submodule is used for carrying out connected domain division on the third image data according to a connected domain algorithm to obtain at least two connected domain subsets; and the connected domain subset processing submodule is used for detecting whether the number of the pixels in each connected domain subset is smaller than a first threshold value, if so, merging the corresponding connected domain subset into any one of the adjacent connected domain subsets to obtain the subset meeting a first rule, otherwise, further detecting whether the number of the pixels in each connected domain subset is larger than a second threshold value, if so, dividing the corresponding connected domain subset into at least two subsets meeting the first rule, and if not, keeping the connected domain subsets.

According to an embodiment of the present invention, the portal vein centerline determination module 704 includes: the portal vein central line first subset determining submodule is used for determining at least one first subset corresponding to a central line of a blood vessel outside the liver region according to the first image data and the position relation between pixel points, and determining the at least one first subset as a subset corresponding to the portal vein central line; the portal vein root determination sub-module is used for calculating the distance between each first subset and the first image data and determining the first subset with the minimum distance from the first image data as the subset corresponding to the portal vein root; and the portal vein central line other subset determining submodule is used for determining other subsets in the portal vein central line subset corresponding to the portal vein central line according to the subset corresponding to the portal vein root and the rule of determining the portal vein central line subset according to the structural characteristics of the portal vein.

According to an embodiment of the present invention, the determining sub-module for other subsets of the portal vein center line includes: a root subset determination unit, configured to set a subset corresponding to the portal root as a root subset; a leaf subset determining unit, configured to obtain a subset set adjacent to a root node subset, detect whether each subset in the subset set is a subset already determined to correspond to a portal vein centerline, if yes, set the corresponding subset as a leaf subset, if no, further perform corresponding processing according to the number of adjacent subsets of the corresponding subset, where if the number of adjacent subsets is greater than 1, further calculate an angle between the corresponding subset and the root node subset and determine whether to set the corresponding subset as a leaf subset according to the angle, and if the number of adjacent subsets is equal to 1, set the corresponding subset as a leaf subset; and the leaf subset detection unit is used for detecting whether the number of the leaf subsets is 0, if so, ending the execution, and if not, sequentially setting the leaf subsets as root subsets.

According to an embodiment of the present invention, the leaf subset determining unit includes: a neighbor point determining subunit, configured to determine a neighbor point of the root subset and the corresponding subset; a first farther point determining subunit for determining, from the neighboring points, a first farther point in the corresponding subset that is relatively farther from the neighboring points; a first direction vector construction subunit, configured to construct, according to the neighboring point and the first distant point, a first direction vector corresponding to the corresponding subset; a second farther point determining subunit, configured to determine, according to the neighboring point, a second farther point in the root subset that is relatively farther from the neighboring point; a second direction vector construction subunit, configured to construct, according to the neighboring point and the second distant point, a second direction vector corresponding to the root subset; and the angle calculation subunit is used for calculating the angles of the corresponding subsets and the root subsets according to the first direction vector and the second direction vector.

According to an embodiment of the present invention, the leaf subset determining unit is specifically configured to set the corresponding subset as the leaf subset if the angle is greater than 90 degrees, and not set the corresponding subset as the leaf subset otherwise.

According to an embodiment of the present invention, the portal vein determination module 705 includes: the maximum point and minimum point determining sub-module is used for acquiring the maximum point and the minimum point in each sub-set according to the positions of the pixel points; the extended image data determining submodule is used for determining extended image data corresponding to each subset according to the maximum point and the minimum point; and the portal vein determining sub-module is used for determining the second image data intersected with the expanded image data as the image data corresponding to the portal vein.

According to a third aspect of embodiments of the present invention, a computer storage medium comprises a set of computer executable instructions for performing any of the image processing methods described above when the instructions are executed.

Here, it should be noted that: the above description of the embodiment of the image processing apparatus and the above description of the embodiment of the computer storage medium are similar to the description of the foregoing method embodiments, and have similar beneficial effects to the foregoing method embodiments, and therefore are not repeated herein. For the technical details that have not been disclosed yet in the description of the embodiment of the image processing apparatus and the embodiment of the computer storage medium of the present invention, please refer to the description of the foregoing method embodiment of the present invention for understanding, and therefore, for brevity, will not be described again.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of a unit is only one logical function division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another device, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units; can be located in one place or distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, all the functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.

Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: various media capable of storing program codes, such as a removable storage medium, a Read Only Memory (ROM), a magnetic disk, and an optical disk.

Alternatively, the integrated unit of the present invention may be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. Based on such understanding, the technical solutions of the embodiments of the present invention may be essentially implemented or a part contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods of the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage medium, a ROM, a magnetic disk, an optical disk, or the like, which can store the program code.

The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.