1. A high-flux fry online tracking and counting device comprises a fry observation channel and a camera device; the device is characterized by also comprising a supporting frame, wherein a fry caching track is arranged at the top of the supporting frame; a side plate or a bottom plate at the starting end of the fry caching track is provided with a circulating water outlet; the fry observing channel is obliquely fixed in the middle of the supporting frame and is connected with the fry caching track through a bent track; the tail end of the fry observing channel is connected to a fry outlet section which is positioned at the lower part of the supporting frame; the top of the fry caching track is open, and the tops of the curved track, the fry observation channel and the fry outlet section are all in a closed state;

the main body of the fry observing channel is a U-shaped groove, the bottom of the fry observing channel is a white light-transmitting plate, the two sides of the fry observing channel are light-proof plates, and the top of the fry observing channel is sealed by a light shield; a light supplementing module is arranged below the U-shaped groove and comprises a panel lamp for providing illumination; the camera shooting equipment is arranged on the light shield, and the lens is aligned to the inside of the fry observing channel through a transparent plate on the light shield; the camera equipment is arranged in a mounting box, and the mounting box is fixed on the outer side of the transparent plate through fastening bolts.

2. The device of claim 1, wherein the water outlet of the circulating water is a plurality of water outlet flanges arranged on the side plates or the bottom plate and connected to the outlet of the water pump through a hose and a connecting pipe; the inlet of the water pump is connected to the water tank below the fry outlet section, so that a circulating water system is formed; the end part of the fry caching track is provided with a bending baffle which can cover the water outlet flange but can not block the water outlet flange.

3. The device of claim 1, wherein the upper and lower panels of the curved track are both smooth arc surfaces, one end of each of the upper and lower panels is connected with the fry buffer track through a U-shaped connecting piece, and the other end of each of the upper and lower panels is nested in the fry observation channel and connected through a bolt; the fry outlet section is of a reducing structure, a larger opening end of the fry outlet section is embedded outside the fry observation channel and connected through bolts, and the joint is sealed by sealant.

4. The device of claim 1, wherein mounting bars are arranged on two sides of the panel light and fixed at the bottom of the fry observing channel through bolts; the camera shooting equipment is connected with the computing unit through a cable, and the computing unit is respectively connected with the movable storage unit and the display unit; the supporting frame is composed of a plurality of square connecting rods, and a plurality of rubber wheels are arranged at the bottom of the supporting frame.

5. A high-throughput fry online tracking and counting method is characterized by comprising the following steps:

(1) switching on the light supplement module and a power supply of the calculation unit, and initializing the detection target and the target fry state by the calculation unit and resetting the number of the display screens;

(2) starting a water pump, pouring the fries after the water flow in the fry buffer track is stable, enabling the fries to enter a fry observation channel along with the water flow, and finally entering a water tank through a fry outlet; in the process, the camera device continuously acquires image signals of all visible areas in the fry observation channel and transmits data to the computing unit;

(3) the computing unit carries out preprocessing on the received image data, including cutting, correcting, filtering, morphological operation and hole filling; setting virtual counting lines in all the images;

(4) acquiring central point position area information of all connected regions from the current ith frame of preprocessed image; marking the connected region with the area larger than the threshold value T as a detection target, and storing the position information of the connected region into a detection target state set detection;

(5) associating the detection target in the current ith frame with the target fry which is not counted;

(6) updating and predicting the position information of the target fry by adopting a Kalman filtering algorithm;

(7) judging whether the fry is counted according to whether the target fry crosses the virtual counting line and whether the target fry is counted;

(8) the state of the detection target is initialized, i.e.

(9) Displaying the number obtained by counting the fry on an electronic display screen;

(10) and (5) repeating the steps (3) to (9) to obtain the total fry number NoF.

6. The method according to claim 5, wherein, in the step (1),

the method for expressing the state of the detection target comprises the following steps: after initialization

The target fry state representation method comprises the following steps: target ═ o¹，o²，...，o^l，...，o^d}(o^l＝[b^l，x^l，y^l，counted^l]) After initialization

Wherein, detection is a detection target state set; c is the number of connected regions in the ith frame image,for the position information of the jth detection target in the ith frame image in the image,is shown as the abscissa of the graph,is a vertical coordinate; targets is a target fry state set; o^lDenotes the first target fry, b^lIndicating the target fry o^lThe number of consecutive uncorrelated times; counted^lIndicating the target fry o^lWhether it is counted or not^l0 means not counted, counted^l1 indicates counted; (x)^l，y^l) Indicating the target fry o^lPosition information in the image, wherein x^lIs the abscissa, y^lIs the ordinate.

7. The method according to claim 5, wherein the step (3) comprises in particular:

(3.1) cutting the image to remove the image except the observation channel;

(3.2) correcting the cut image, and reducing the influence of the installation error of the camera equipment and the edge of a fry observation channel on subsequent counting;

(3.3) performing mean filtering on the corrected image;

(3.4) converting the filtered image into a binary image to obtain a binary image;

(3.5) in the binary image, setting a virtual counting line in front of D pixels at the outlet of the fry observation channel; it should be ensured that newly emerging fish fry cannot cross the virtual counter line in the second frame of image that is detected.

8. The method of claim 5, wherein the associating in step (5) comprises:

(5.1) associating the detection target with the target fry which is not counted by adopting an optimal neighbor algorithm to respectively obtain sets of UMOID, MOID, UMOIT and MOIT;

wherein, the UMOID represents a target set which is not associated in the detection targets; the MOID represents the associated target set in the detection targets; the UMOIT represents a target set which is not associated in the target fry; MOIT represents the associated target set in the target fry;

(5.2) updating the position information in the corresponding MOIT by using the position information in the MOID, and setting the number b of continuous times of not being associated in the MOIT to be 0;

(5.3) adding the UMOID into the Targets set, setting the number b of continuous times of not-associated times as 0, and setting the number counted of the counting marks as 0;

(5.4) updating the position information of the corresponding target fry in the UMOIT by using the position information in the target fry state quantity predicted value obtained by the previous round of calculation in the i-1 th frame, and setting the mark number b + of the continuous times of not being associated of the corresponding target fry to be 1; when b > s, the target fry is indicated as false detection and is deleted from the Targets set, and the step allows the fry to be detected for s frames continuously.

9. The method of claim 5, wherein the step (6) comprises:

(6.1) creating a linear Kalman filter, and assuming that the fries in the visual field of the camera equipment are all in uniform motion in a two-dimensional plane;

(6.2) combining the measurement value of the Target fry in the current ith frame and the state prediction value of the ith-1 frame of the previous frame to obtain the optimal estimated state quantity of the Target fry, and updating the position information of the Target fry in the Target;

(6.3) predicting the state quantity of the target fry in the next frame image, namely predicting the state quantity predicted value of the target fry in the i +1 th frame of the next frame;

and (6.4) updating the covariance matrix of the state quantity of the target fry.

10. The method of claim 5, wherein the step (7) comprises:

if some target fry which is not counted passes through the virtual counting line, NoF + is equal to 1, and the target fry is marked as counted; the NOF continuously iterates to always represent the number of fries.

Background

As an important link of the subjects such as scientific bait casting, fry survival rate evaluation, culture density control, fry selling and purchasing and the like, accurate estimation of the number of fries gradually becomes a basic guarantee for standardized and scientific management of fish culture. With the development of modern fishery, the mechanization and intellectualization level of fish farming industry in China is continuously improved, the demand of fish farming on a high-flux automatic counter is more and more large, but the traditional counting method such as a push algorithm, a bowl quantity method, a break method and the like is still adopted in fish farming in China at present, so that time and labor are wasted, and survival pressure on fish fries is easily caused. Related personnel of fishery and metering professionals at home and abroad have already developed research on fry counting methods, and some fry counters such as photoelectric fry counters and multi-channel fry counters also appear at home and abroad, but the device is complex and high in cost, and most of the fry counters are for adult fishes.

China invention application publication No. CN110973036A discloses a fry counting device and method based on machine vision, wherein fry is counted in batches through a revolving door, but the number of the fry in each batch is not easy to control, the fry has the problems of adhesion, accumulation and the like in a counting area, the counting accuracy is influenced, the counting is discontinuous, and the fry counting efficiency is reduced.

Chinese patent application publication No. CN109937947A discloses a fry counter, which is characterized in that a splitter plate is arranged in a housing to split fry entering the housing, and the length of a pipeline can be changed to change the size of an outlet of the pipeline to adapt to fry with different sizes, but the fry may be impacted to different degrees in the splitting process, and the way is mainly suitable for fry with larger size.

Chinese patent application publication No. CN110766123A discloses a fry counting system and a fry counting method, wherein an output port is provided at the bottom of a fry water tank to be measured, so that a plurality of fries enter a one-way water channel, and images are collected and counted in the one-way water channel.

Disclosure of Invention

The invention aims to solve the technical problem of overcoming the defects in the prior art and provides a high-throughput fry online counting device and method.

In order to solve the technical problem, the solution of the invention is as follows:

the high-flux fry online tracking and counting device comprises a fry observation channel and a camera device; the device also comprises a supporting frame, wherein a fry caching track is arranged at the top of the supporting frame; a side plate or a bottom plate at the starting end of the fry caching track is provided with a circulating water outlet; the fry observing channel is obliquely fixed in the middle of the supporting frame and is connected with the fry caching track through a bent track; the tail end of the fry observing channel is connected to a fry outlet section which is positioned at the lower part of the supporting frame; the top of the fry caching track is open, and the tops of the curved track, the fry observation channel and the fry outlet section are all in a closed state;

the main body of the fry observing channel is a U-shaped groove, the bottom of the fry observing channel is a white light-transmitting plate, the two sides of the fry observing channel are light-proof plates, and the top of the fry observing channel is sealed by a light shield; a light supplementing module is arranged below the U-shaped groove and comprises a panel lamp for providing illumination; the camera equipment is installed on the light shield, and the lens is aligned to the inside of the fry observation channel through the transparent plate on the light shield.

In the invention, the lens hood is provided with a transparent plate, the camera shooting equipment is arranged in the mounting box, and the camera shooting equipment is fixed on the outer side of the transparent plate through fastening bolts.

In the invention, the water outlets of the circulating water are a plurality of water outlet flanges arranged on the side plates or the bottom plate and are connected to the outlet of the water pump through hoses and connecting pipes; the inlet of the water pump is connected to the water tank below the fry outlet section, so that a circulating water system is formed; the end part of the fry caching track is provided with a bending baffle which can cover the water outlet flange but can not block the water outlet flange.

In the invention, the upper and lower panels of the curved track are both smooth arc surfaces, one end of the curved track is connected with the fry caching track through a U-shaped connecting piece, and the other end of the curved track is nested in the fry observation channel and is connected with the fry observation channel through a bolt; the fry outlet section is of a reducing structure, a larger opening end of the fry outlet section is embedded outside the fry observation channel and connected through bolts, and the joint is sealed by sealant.

According to the invention, mounting strips are arranged on two sides of the panel lamp and are fixed at the bottom of the fry observation channel through bolts.

In the invention, the camera shooting equipment is connected with the computing unit through a cable, and the computing unit is respectively connected with the movable storage unit and the display unit.

In the invention, the supporting frame consists of a plurality of square connecting rods, and a plurality of rubber wheels are arranged at the bottom of the supporting frame.

The invention further provides a high-throughput fry online tracking and counting method, which comprises the following steps:

(1) switching on the light supplement module and a power supply of the calculation unit, and initializing the detection target and the target fry state by the calculation unit and resetting the number of the display screens;

(2) starting a water pump, pouring the fries after the water flow in the fry buffer track is stable, enabling the fries to enter a fry observation channel along with the water flow, and finally entering a water tank through a fry outlet; in the process, the camera device continuously acquires image signals of all visible areas in the fry observation channel and transmits data to the computing unit;

(3) the computing unit carries out preprocessing on the received image data, including cutting, correcting, filtering, morphological operation and hole filling; setting virtual counting lines in all the images;

(4) acquiring central point position area information of all connected regions from the current ith frame of preprocessed image; marking the connected region with the area larger than the threshold value T as a detection target, and storing the position information of the connected region into a detection target state set detection;

visually, the pixels that are connected form one region, while the pixels that are not connected form a different region. In the present invention, a set of pixels connected to each other is referred to as a connected region.

(5) Associating the detection target in the current ith frame with the target fry which is not counted;

(6) updating and predicting the position information of the target fry by adopting a Kalman filtering algorithm;

(7) judging whether the fry is counted according to whether the target fry crosses the virtual counting line and whether the target fry is counted;

(8) the state of the detection target is initialized, i.e.

(9) Displaying the number obtained by counting the fry on an electronic display screen;

(10) and (5) repeating the steps (3) to (9) to obtain the total fry number NoF.

In the present invention, in the step (1),

the method for expressing the state of the detection target comprises the following steps: j is more than or equal to 1 and less than or equal to c after initialization

The target fry state representation method comprises the following steps: target ═ o¹，o²，...，o^l，...，o^d}(o^l＝[b^l，x^l，y^l，counted^l]) After initialization

Wherein, detection is a detection target state set; c is the number of connected regions in the ith frame image,for the position information of the jth detection target in the ith frame image in the image,for lying onThe mark is that,is a vertical coordinate; targets is a target fry state set; o^lDenotes the first target fry, b^lIndicating the target fry o^lThe number of consecutive uncorrelated times; counted^lIndicating the target fry o^lWhether it is counted or not^l0 means not counted, counted^l1 indicates counted; (x)^l，y^l) Indicating the target fry o^lPosition information in the image, wherein x^lIs the abscissa, y^lIs the ordinate.

In the present invention, the step (3) specifically includes:

(3.1) cutting the image to remove the image except the observation channel;

(3.2) correcting the cut image, and reducing the influence of the installation error of the camera equipment and the edge of a fry observation channel on subsequent counting;

(3.3) performing mean filtering on the corrected image;

(3.4) converting the filtered image into a binary image to obtain a binary image;

(3.5) in the binary image, setting a virtual counting line in front of D pixels at the outlet of the fry observation channel; it should be ensured that newly emerging fish fry cannot cross the virtual counter line in the second frame of image that is detected.

In the present invention, the association in step (5) includes:

(5.1) associating the detection target with the target fry which is not counted by adopting an optimal neighbor algorithm to respectively obtain sets of UMOID, MOID, UMOIT and MOIT;

wherein, umoid (unadapted objects in detections) represents the unadapted object set in the detected objects; MOID (matched objects in detection) represents the associated object set in the detected objects; umoit (unagrated objects in targets) represents the unassociated target set in the target fry; MOIT (matched objects in targets) represents the associated target set in the target fry;

(5.2) updating the position information in the corresponding MOIT by using the position information in the MOID, and setting the number b of continuous times of not being associated in the MOIT to be 0;

(5.3) adding the UMOID into the Targets set, setting the number b of continuous times of not-associated times as 0, and setting the number counted of the counting marks as 0;

(5.4) updating the position information of the corresponding target fry in the UMOIT by using the position information in the target fry state quantity predicted value obtained by the previous round of calculation in the i-1 th frame (the previous frame), and setting the number b + of continuous times of non-association of the corresponding target fry to be 1; when b > s, the target fry is indicated as false detection and is deleted from the Targets set, and the step allows the fry to be detected for s frames continuously.

In the present invention, the step (6) comprises:

(6.1) creating a linear Kalman filter, and assuming that the fries in the visual field of the camera equipment are all in uniform motion in a two-dimensional plane;

(6.2) combining the measurement value of the Target fry in the current ith frame and the state prediction value of the ith-1 frame of the previous frame to obtain the optimal estimated state quantity of the Target fry, and updating the position information of the Target fry in the Target;

(6.3) predicting the state quantity of the target fry in the next frame image, namely predicting the state quantity predicted value of the target fry in the i +1 th frame of the next frame;

and (6.4) updating the covariance matrix of the state quantity of the target fry.

In the present invention, the step (7) comprises:

if an uncounted target fry (counted ═ 0) crosses the virtual counter line, NoF + ═ 1, and the target fry is marked as counted (counted ═ 1). The NOF continuously iterates to always represent the number of fries.

Compared with the prior art, the invention has the following beneficial effects and advantages:

(1) the device disclosed by the invention adopts the smooth curved track and the inclined channel to separate the fry, and compared with a method adopting channel size limitation and splitter plate spacing, the device not only improves the fry passing speed, can improve the fry flux in unit time by several times, but also reduces the rubbing damage to the fry;

(2) the device can be suitable for counting the fry of different sizes without adjusting the size of the channel, the interval of the flow distribution plates and other ways, and can count the fry of mixed sizes;

(3) the device can realize real-time online continuous counting of a large number of fry, and improve the fry counting efficiency;

(4) the online counting method provided by the invention effectively reduces the influence of false detection and missing detection on the counting accuracy rate through a tracking prediction strategy of continuous multiframes.

Drawings

FIG. 1 is an axial view of the fry counting device of the present invention;

FIG. 2 is an exploded view of a fry buffer module and a curved track in the fry counting device;

fig. 3 is an exploded view of a fry observation channel, a light supplement module and a camera mounting cover in the fry counting device;

FIG. 4 is a schematic view of a fry observation channel in the fry counting device;

FIG. 5 is a schematic view of the hose and pump connection in the fry counting device;

FIG. 6 is a schematic cable connection of the fry counting device;

FIG. 7 is a flow chart of a fry counting method;

fig. 8 is a view field schematic diagram of a fry counting method image pickup apparatus.

Description of reference numerals:

1-fry buffer track, 2-bent track, 3-light shield, 4-fry outlet, 5-support frame, 6-rubber wheel, 7-U-shaped connecting piece, 8-plane baffle, 9-bent baffle, 10-water outlet flange, 11-four-way pipe, 12-mounting box, 13-camera equipment, 14-transparent plate, 15-fry observation channel, 16-mounting bar, 17-mounting bar, 18-panel lamp, 19-hose, 20-water pump, 21-calculation unit, 22-display screen, 23-transformer, 24-cable, 25-mobile storage unit, 26-fry and 27-power supply.

Detailed Description

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar function throughout.

As shown in the figure, the high-throughput fry online tracking and counting device comprises a camera device 13 and a computing unit 21 for collecting and processing images, a fry cache track 1, a curved track 2, a fry observation channel 15 and a fry outlet section 4; the computing unit 21 may be a host computer, or may be a small computing module such as a single chip microcomputer.

The supporting frame 5 is composed of a plurality of square connecting rods, and a plurality of rubber wheels 6 are arranged at the bottom of the supporting frame. The fry buffer track 1 is arranged at the top of the support frame 5; 3 water outlets for circulating water, in particular 3 water outlet flanges 10 are arranged at the initial end of the fry buffer track 1. The fry observing channel 15 is obliquely fixed in the middle of the supporting frame and is connected with the fry caching track 1 through the bent track 3; the end of the fry observation channel 15 is connected to a fry outlet section 4, which is located at the lower part of the support frame 5; the top of the fry buffer track 1 is open, and the tops of the curved track 2, the fry observation channel 15 and the fry outlet section 4 are all closed.

3 water outlet flanges 10 are arranged on the bottom surface of one end far away from the curved track 2, and a bending baffle 9 is nested on the side wall of the fry caching track 1 through a bending groove; the bent baffle 9 can cover the water outlet flange but can not block the water outlet of the water outlet flange, so that the water outlet of the water outlet flange is uniformly distributed, and the water flow is prevented from being excessively concentrated. The plane baffle 8 is provided with two long and thin grooves and is nested at the joint of the fry buffer track 1 and the curved track 2 through the grooves. The water outlet flange 10 is connected with three ends of the four-way pipe 11 through a hose 19, and the other end of the four-way pipe 11 is connected with a water pump 20 through the hose 19. The inlet of the water pump 20 is connected to the water tank below the fry outlet section 4, thus forming a circulating water system.

The upper surface and the lower surface of the curved track 2 are both smooth arc surfaces, and the separation of the fry in the moving direction is realized by utilizing the acceleration difference of different arc sections in the fry moving process; one end of the bent track 2 is connected with the fry caching track 1 through a U-shaped connecting piece 7, the other end of the bent track is embedded into a fry observation channel 15 and is connected through bolts, and sealant is coated at the connection part for sealing. The fry outlet section 4 is of a reducing structure, the slightly larger opening end of the fry outlet section is embedded outside the fry observing channel 15 and is connected through bolts, and the joint is sealed by sealant.

The main body of the fry observing channel 15 is a U-shaped groove, the bottom of the fry observing channel is a white light-transmitting plate, the two sides of the fry observing channel are light-proof plates, and the top of the fry observing channel is sealed by a light shield 3; a transparent plate 14 is provided at an opening of the light shield 3, and the image pickup device 13 is mounted in a mounting box 12 fixed to an outer side of the transparent plate 14 with fastening bolts. The lens of the imaging device 13 is aligned with the inside of the fry observation passage 15. A light supplementing module is arranged below the U-shaped groove and comprises a panel lamp 18 for providing illumination; mounting strips 16 and 17 are arranged on two sides of the panel lamp 18 and are connected with the fry observing channel 15 through bolts to limit the panel lamp 18; the white transparent bottom surface that panel light 18 accessible fry observed passageway 15 carries out the light filling to the fry that is in observing passageway 15 to isolated the influence of external light source to the count, panel light 18 passes through transformer 23 and connects 220 v's power 27.

The imaging apparatus 13 is connected to the calculation unit 21 through a cable 24; the calculation unit 12 is connected with the display unit 22 and the movable storage unit 25, the storage unit 25 is used for storing images and videos acquired and processed by the camera device 13, and the display unit 22 is used for displaying the calculated fry number; the computing unit 12 and the display unit 22 may be bolted to the device.

By applying the fry counting device, the fry counting method based on machine vision detection tracking comprises the following steps:

a. turning on the light supplement module and the power supply of the computing unit 21, waiting for the computing unit 21 to initialize the detection target and the target fry state, and clearing the number of the display screens 22 to zero (NoF is 0);

the state representation method of the detection target comprises the following steps: after initialization

The state representation method of the target fry comprises the following steps: target ═ o¹，o²，...，o^l，...，o^d}(o^l＝[b^l，x^l，y^l，counted^l]) After initialization

The meaning of each symbol in the formula is: detection is a detection target state set, c is the number of connected domains in the ith frame image,respectively the position information of the jth detection target in the ith frame image in the image, whereinIs shown as the abscissa of the graph,is a vertical coordinate; targets is a target fry state set; o^lRepresenting the first target fry; b^lTarget fry o^lThe number of consecutive uncorrelated times; counted^lIndicating the target fry o^lWhether it is counted or not^l0 means not counted, counted^l1 indicates counted; (x)^l，y^l) Indicating the target fry o^lPosition information in the image, wherein x^lIs the abscissa, y^lIs the ordinate.

b. Turning on a water pump switch, after water flow is stable, pouring the fry into the fry caching track 1, enabling the fry to sequentially enter the curved track 2 and the fry observing channel 15 along with the water flow from the fry caching track 1, and finally flowing out of the device through the fry outlet section 4; in the process, the camera device 13 continuously acquires image signals of all visible areas in the fry observing channel 15 and transmits data to the computing unit 22;

c. the computing unit 22 performs pre-processing on the received image data, including cropping, rectification, filtering and morphological operations; and a virtual counting line is arranged in front of the outlet position of the fry observing channel 15. The method specifically comprises the following steps:

c1, cutting the image, removing the image except the fry observation channel 15, and obtaining the cut image;

c2, after the step c1, the computing unit 22 corrects the cut image, reduces the influence of the installation error of the camera device 13 and the edge of the fry observing channel 15 on subsequent counting, and obtains a corrected image;

c3, after the step c2, the computing unit 22 performs mean filtering on the corrected image to obtain a filtered image;

c4, after the step c3, the computing unit 22 converts the filtered image into a binary image to obtain a binary image;

c5, after step c4, the computing unit 22 sets a virtual counter line in the binary image before D pixels at the exit of the fry observing channel, and it should be ensured that the newly emerged fry cannot pass through the virtual counter line in the detected second frame image.

d. Acquiring central point positions and area information of all connected areas from the preprocessed image of the current ith frame; marking the connected region with the area larger than the threshold value T as a detection target, and storing the position information of the connected region into a detection target state set detection;

e. associating the detection target in the current ith frame with the target fry which is not counted; wherein the associating comprises the steps of:

e1, associating the detection target with the target fry which is not counted by adopting an optimal neighbor algorithm, and respectively obtaining the UMOID, the MOID, the UMOIT and the MOIT. Specifically, the Euclidean distance is adopted as a distance value in the optimal neighbor algorithm;

wherein, umoid (unprocessed objects in detections): representing a set of targets not associated among the detected targets; MOID (mapped objects in detections): representing a set of associated ones of the detected targets; UMOIT (unmachined objects in targets): representing a set of targets not associated in the target fry; MOIT (matched objects in targets): representing the associated target set in the target fry;

e2, updating the position information in the corresponding MOIT by using the position information in the MOID, and setting the continuous unassociated times mark number b in the MOIT to be 0;

e3, adding the UMOID into the Targets set, setting the continuous number-of-times-not-associated flag number b to be 0, and setting the count flag number to be 0;

e4, updating the position information of the corresponding target fry in the UMOIT by using the position information in the target fry state quantity predicted value obtained by the previous round of calculation in the previous frame (i-1 th frame), and setting the continuous non-associated times mark number b + of the corresponding target fry as 1; when b is larger than s, the target fry is indicated as false detection, and the target fry is deleted from the Targets set; the step allows the fry to be detected for s frames continuously, and the problem that the fry is missed to be detected due to short-time adhesion can be solved.

f. Updating and predicting the position information of the target fry by adopting a Kalman filtering algorithm; the method comprises the following steps:

f1, creating a linear Kalman filter, and assuming that the fries in the visual field of the camera device are all in uniform motion in a two-dimensional plane. Specifically, for the uniform motion of a certain item marked fry in a two-dimensional plane, a four-dimensional coordinate [ x, y, v ] is used_x，v_y]To represent the state quantity of the target fry, wherein [ x, y]Represents the moving position of the corresponding target fry, and v_x，v_y]Then representing the component speed of the target fry in the x and y directions, and the initial state is [0, 0, 0, 0]. The measurement value of the target fry is coordinate information in Targets and the speed in the x direction and the y direction calculated by two adjacent frames (the current ith frame and the previous (i-1) th frame).

f2, obtaining the optimal estimated state quantity of the Target fry by combining the measurement value of the Target fry in the current ith frame and the state prediction value of the previous frame (i-1 th frame), and updating the position information of the Target fry in the Target;

f3, predicting the state quantity of the target fry in the next frame image, namely predicting the state quantity predicted value of the target fry in the i +1 th frame of the next frame;

f4, updating the covariance matrix of the state quantity of the target fry.

g. Judging whether to count the fry according to whether the target fry crosses the virtual counting line and whether the target fry is counted;

if an uncounted target fry (counted ═ 0) crosses the virtual counter line, NoF + ═ 1, and the target fry is marked as counted (counted ═ 1). Specifically, the x-coordinate value (x) of the center position of a certain fry_i) Greater than D (i.e., x)_i> D), then the virtual counter line is considered to be crossed.

h. The state of the detection target is initialized, i.e.

i. Displaying the number NoF on an electronic display screen;

j. and (5) repeating the step (c-i), and obtaining the total number of the fries as NoF after the counting is finished.

The specific application example of the fry counting device counting method is as follows:

after steps a, b, c and d, ifAll values do not change after all steps, and NOF is 0. Assume an image size of (1280, 720)

After the steps c and d, assuming that the number of the connected regions with the area larger than T in the ith frame is 3, the central positions are respectivelyWhen the 3 connected regions are marked as detection targets, the detection is { (112, 141), (512, 274), (103, 617) }; if Targets is equal to { o }¹}，o¹＝[b¹＝1，x¹＝486，y¹＝263，counted¹＝0]，

E, associating the detection target in the current ith frame with the target fry which is not counted; wherein the associating comprises the steps of:

e1, adopting the best neighbor algorithm to correlate the detection target with the target fry which is not counted. Is easy to obtainAnd o¹Associating, and respectively obtaining the UMOID, the MOID, the UMOIT and the MOIT;

MOIT＝{o¹}

e2, updating the position information in the corresponding MOIT by using the position information in the MOID, and setting the continuous unassociated times mark number b in the MOIT to be 0; at this time, o¹＝[b¹＝0，x¹＝512，y¹＝274，counted¹＝0]。

e3, adding the UMOID into the Targets set, setting the continuous number-of-times-not-associated flag number b to be 0, and setting the count flag number to be 0;

at this time, Targets ═ { o ═ o¹，o²，o³}，

o¹＝[b¹＝0，x¹＝512，y¹＝274，counted¹＝0]，

o²＝[b²＝0，x²＝112，y²＝141，counted²＝0]，

o³＝[b³＝0，x³＝103，y³＝617，counted³＝0]

e4, updating the position information of the corresponding target fry in the UMOIT by using the state prediction value calculated in the step (6.3) in the last frame (i-1 th frame), and setting the continuous unassociated times flag number b + of the corresponding target fry as 1; when b is larger than s, the target fry is indicated as false detection, and the target fry is deleted from the Targets set; the step allows the fry to be detected for s frames continuously, and the problem that the fry is missed to be detected due to short-time adhesion can be solved. Due to the fact thatAll values are unchanged after this step is performed.

f. Updating and predicting the position information of the target fry by adopting a Kalman filtering algorithm; the method comprises the following steps:

f1, creating a Kalman filter, and assuming that the fries in the visual field of the camera device are all in uniform motion in a two-dimensional plane. Specifically, for the uniform motion of a certain item marked fry in a two-dimensional plane, a four-dimensional coordinate [ x, y, v ] is used_x，v_y]To indicate the fry status, wherein [ x, y]Indicating position information of the corresponding target fry, and v_x，v_y]Then representing the component speed of the target fry in the x and y directions, and the initial state is [0, 0, 0, 0]. The measurement value of the target fry is coordinate information in Targets and the speed in the x direction and the y direction calculated by two adjacent frames (the current ith frame and the previous (i-1) th frame).

In this example, target fry o¹，o²，o³The measurement state quantities in the current ith frame are respectively: taking the time difference as unit time

[512，274，512-486，274-263]，[112，141，112-0，141-0]，[103，617，103-0，617-0]

f2, obtaining the optimal estimated state quantity of the Target fry by combining the measurement value of the Target fry in the current ith frame and the state prediction value of the ith-1 frame of the previous frame, and updating the position information of the Target fry in Target; due to o²，o³Does not occur before the ith frame, so o²，o³All the predicted values of (1) are initial states [0, 0, 0, 0]。

It is assumed that the optimum estimated value obtained is

[520，276，25，21]，[89，120，89，120]，[79，600，79，600]

At this time, Targets ═ { o ═ o¹，o²，o³}，

o¹＝[b¹＝0，x¹＝520，y¹＝276，counted¹＝0]，

o²＝[b²＝0，x²＝89，y²＝120，counted²＝0]，

o³＝[b³＝0，x³＝79，y³＝600，counted³＝0]

f3, predicting the state quantity of the target fry in the image of the next frame (i +1 th frame), namely the state prediction value of the target fry in the i +1 th frame of the next frame; suppose that o is obtained¹，o²，o³The state quantity prediction value in the next frame (i +1 th frame) image of (1) is: the numerical value of the fry position can not exceed the image size

[547，288，23，19]，[160，200，75，85]，[136，720，69，500]

f4, updating the covariance matrix of the state quantity of the target fry.

g. Judging whether to count the fry according to whether the target fry crosses the virtual counting line and whether the target fry is counted;

if an uncounted target fry (counted ═ 0) crosses the virtual counter line, NoF + ═ 1, and the target fry is marked as counted (counted ═ 1). Specifically, the x-coordinate value (x) of the center position of a certain fryⁱ) Greater than D (i.e., x)ⁱ> D), then the virtual counter line is considered to be crossed. For convenience of illustration of the numerical variation of NOF, assume D is 500, since o in Targets¹Abscissa x of¹> 500, NOF + ═ 1, counted¹1. When NOF is 1, o¹＝[b¹＝0，x¹＝520，y¹＝276，counted¹＝1]。

h. Emptying the detection target mark, i.e.

i. Displaying the number NoF-1 on an electronic display screen;

j. and (5) repeating the step (c-i), and obtaining the total number of the fries as NoF after the counting is finished.

Description of the drawings: the data are only used for explaining the change of various numerical values of the counting method in the application process and are not real data; and when the counting method is applied, the fry is tracked and predicted for a plurality of continuous frames before passing through the virtual counting line.

The above embodiments are provided for understanding the invention, and not for limiting the invention, and those skilled in the relevant art can make various changes or modifications based on the claims, and these changes or modifications should be understood as still falling within the scope of the invention.