1. A device for measuring, recording and analyzing food intake of a human body is characterized by comprising an image sensor, a target three-dimensional volume measuring sensor, an attitude sensor, an audio sensor, an individual recognition sensor, a temperature sensor, a real-time communication and data transmission system, a data storage system, an alarm prompt system and a control circuit; the image sensor comprises a color/black-and-white image sensor, a visible light/non-visible light image sensor and a CMOS/CCD principle image sensor, the target three-dimensional volume measuring sensor comprises a radar sensor, a laser radar sensor, a time-of-flight depth camera sensor, a laser sensor, a structured light sensor and an ultrasonic sensor, the attitude sensor comprises a gravity sensor, an acceleration sensor, a distance sensor, a rotation vector sensor, a gyroscope sensor, a magnetic sensor and a direction sensor, the individual identification sensor comprises face identification, fingerprint identification and voice identification, the temperature sensor comprises a contact type/non-contact type thermometer, the real-time communication and data transmission system comprises wifi, Bluetooth and 3G/4G/5G/6G/7G communication equipment, the data storage system comprises a solid storage device and a flash memory, and the warning prompt system comprises a light prompt system, a sound prompt system and a vibration prompt system.

2. A method of measuring, recording and analyzing food intake in a human comprising the steps of:

s1, recording the system time during dining;

s2, measuring the current real-time temperature of the food;

s3, identifying the fine category of the food;

s4, measuring the volume and the mass of the ingested food;

and S5, when the alarm rule is met, giving an alarm.

3. The method for measuring, recording and analyzing food intake of a human body according to claim 2, wherein the step S1 specifically comprises the steps of:

s11, meal start time judgment: after the equipment is started, the system judges the system time when the individual firstly sends food into the mouth, and the system time is defined as a meal starting point;

s12, judging the dining interval time: after the meal start point in said step S41, when the system monitors that the individual has food to enter the mouth, the current system time is recorded, defining the process as a "single meal event"; continuously recording all single meal events until the meal is finished;

s13, judging a meal termination event: when the equipment is shut down, the meal is judged to be terminated; when the system does not monitor the single-mouth dining event within 2 continuous hours, the system judges that the dining is terminated and automatically shuts down.

4. The method for measuring, recording and analyzing food intake of human body according to claim 2, wherein the identifying of the fine categories of food in step S3 is specifically: the food is collected by an image sensor and then the category of the food is identified by an artificial intelligent image identification program.

5. The method for measuring, recording and analyzing food intake of human body according to claim 4, wherein the classification rules of the fine categories in step S3 include:

s31, distinguishing food types, vegetable types, meat types and staple food types;

s32, distinguishing the types and contents of nutrients, proteins, saccharides, lipids, vitamins, inorganic salts and common trace elements.

6. The method for measuring, recording and analyzing food intake of human body according to claim 5, wherein the step S4 is to measure the volume and mass of the food intake as follows: the system calls an image sensor and a target object three-dimensional volume measuring sensor, measures and calculates the volume of each type of food in the single-mouth dining event through an artificial intelligence analysis program, and measures and calculates the quality.

7. Method for measuring, recording and analysing the food intake of the human body according to claim 6, characterized in that on the basis of analysing the quality of the ingested food the following indicators are measured:

s41, calculating the single nutrient intake in S3;

and S42, calculating the intake of each nutrition category in S3.

8. Method for measuring, recording and analysing human food intake according to claim 2, wherein the analysis needs to be performed using a local or online database containing or entering the following data information:

nutrient determination information of each food, individualized human nutrient demand quantitative standard, food intake standard, food allergy information and food contraindication information.

9. The method of measuring, recording and analyzing food intake of a human body according to claim 8, wherein an alarm is issued when there is a beneficial event or a harmful event, the beneficial event category comprising quantitative criteria and nutritional balance indicators; the adverse event categories include temperature indicators, quantitative indicators, nutritional balance indicators, detection of preparation to ingest inappropriate food that may be detrimental to the health of the individual.

Background

Scientific diet is beneficial to health, and improper diet is easy to cause diseases or aggravate the diseases. With the rapid development of Chinese economy, the living standard of people is rapidly improved, and the problems of overweight, obesity, chronic diseases and the like of individuals caused by unscientific diet are more and more. Unreasonable meals, irregular meals, unbalanced nutrition intake, etc., are important causes of these problems. In addition, for patients with diabetes, hyperuricemia and other diseases, reasonable diet is very key to controlling the development of the disease. In the face of these needs, there is an urgent need for innovative methods and devices to achieve accurate monitoring of the feeding process. Under the guidance and feedback of objective and accurate monitoring data, the intelligent diet can greatly assist individuals in accurate diet, and various health problems caused by unscientific diet are reduced.

In order to realize accurate measurement and control of a diet process, the current technical scheme has a plurality of defects. For example, patent application publication No. CN107610749A discloses a method for determining food intake by detecting "food spectrum information" and comparing it with a database. This patent application has several serious drawbacks: (1) the accuracy of judging the food category by utilizing the spectral information is poor, and the method cannot be applied to a complex Chinese food diet environment; (2) no food intake measurement scheme is provided, and the real food intake cannot be calculated. The patent application published under the number CN111028916A utilizes "audio information" of the meal time of the user for feeding process monitoring. This patent application has several serious drawbacks: (1) the method is limited by user experience and dining habits of users, and audio information monitoring cannot finish capturing the ingestion types and the ingestion quantity of the users; (2) no food intake measurement scheme is provided, and the real food intake cannot be calculated.

Disclosure of Invention

In view of the above-mentioned problems, the present invention discloses a method for measuring, recording and analyzing food intake of a human body, comprising the steps of: s1, recording the system time during dining; s2, measuring the current real-time temperature of the food; s3, identifying the fine category of the food; s4, measuring the volume and the mass of the ingested food; and S5, when the alarm rule is met, giving an alarm.

Further, step S1 specifically includes the following steps: s11, meal start time judgment: after the equipment is started, the system judges the system time when the individual firstly sends food into the mouth, and the system time is defined as a meal starting point; s12, judging the dining interval time: after the meal start point in said step S41, when the system monitors that the individual has food to enter the mouth, the current system time is recorded, defining the process as a "single meal event"; continuously recording all single meal events until the meal is finished; s13, judging a meal termination event: when the equipment is shut down, the meal is judged to be terminated; when the system does not monitor the single-mouth dining event within 2 continuous hours, the system judges that the dining is terminated and automatically shuts down.

Further, the sub-categories of the food identified in step S2 are specifically: the food is collected by an image sensor and then the category of the food is identified by an artificial intelligent image identification program.

Further, the classification rules of the fine classification in step S3 include: s31, distinguishing food types, vegetable types, meat types and staple food types; s32, distinguishing the types and contents of nutrients, proteins, saccharides, lipids, vitamins, inorganic salts and common trace elements.

Further, the step S4 of measuring the volume and mass of the food intake is specifically: the system calls an image sensor and a target object three-dimensional volume measuring sensor, measures and calculates the volume of each type of food in the single-mouth dining event through an artificial intelligence analysis program, and measures and calculates the quality.

Further, on the basis of analyzing the quality of the ingested food, the following indicators were measured: s41, calculating the single nutrient intake in S3; and S42, calculating the intake of each nutrition category in S3.

Further, it is necessary to use a local database or an online database for analysis, the local database or the online database containing or entering the following data information: nutrient determination information of each food, individualized human nutrient demand quantitative standard, food intake standard, food allergy information and food contraindication information.

Further, when beneficial events or harmful events exist, an alarm is given, and the beneficial event categories comprise quantitative standards and nutrition balance indexes; the adverse event categories include temperature indicators, quantitative indicators, nutritional balance indicators, detection of preparation to ingest inappropriate food that may be detrimental to the health of the individual.

The invention also provides a device for measuring, recording and analyzing food intake of a human body, which comprises an image sensor, a target three-dimensional volume measuring sensor, an attitude sensor, an audio sensor, an individual recognition sensor, a temperature sensor, a real-time communication and data transmission system, a data storage system, an alarm prompt system and a control circuit, wherein the image sensor comprises a color/black-and-white image sensor, a visible light/non-visible light image sensor and a CMOS/CCD principle image sensor, the target three-dimensional volume measuring sensor comprises a radar sensor, a laser radar sensor, a time-of-flight depth camera sensor, a laser sensor, a structured light sensor and an ultrasonic sensor, the attitude sensor comprises a gravity sensor, an acceleration sensor, a distance sensor, a rotation vector sensor, a gyroscope sensor, a temperature sensor, a real-time communication and data transmission system, a data storage system and a control circuit, Magnetic force sensor, direction sensor, individual identification sensor includes face identification, fingerprint identification, sound identification, temperature sensor contains contact/non-contact thermometer, real-time communication and data transmission system include wifi, bluetooth, 3G 4G 5G 6G 7G communication equipment, data storage system includes that solid-state storage establishes, flash memory, warning prompt system includes light prompt system, sound prompt system, vibrations prompt system.

The present invention systematically addresses the challenges of food intake monitoring. The invention bases on the multi-dimensional sensor data, and comprehensively uses the artificial intelligent image recognition and artificial intelligent target object measurement technology to objectively and accurately monitor the individual ingestion process, thereby greatly promoting the development and application of accurate diet measurement and control.

The invention discloses a method and a device for measuring, recording and analyzing food intake of human bodies, which are suitable for people comprising the following steps: (1) healthy people; healthy people can monitor daily food intake through the device disclosed by the invention, and quantitative statistical data of individual food intake can be obtained through long-time continuous monitoring. According to the data, the dietary habits can be adjusted to achieve the purposes of accurate nutrition, reasonable diet, individual weight reduction, disease prevention and the like. (2) Food intake management in people with disease; for example, for patients with diabetes, the patient may be monitored and instructed on a reasonable meal in order to get an individual blood glucose management; for patients with high uric acid, the reasonable diet of the patients can be monitored and guided, and the condition aggravation caused by excessive ingestion of purine-rich food is avoided; for patients with hereditary metabolic diseases, the diet of the patients can be accurately guided, and the increase of metabolic burden and the like can be avoided. (3) Food intake management of special populations; for example, accurate nutrition management of young children, accurate nutrition of pregnant women, and dietary management of people with allergic constitution.

Drawings

FIG. 1 illustrates an exemplary system for measuring, recording and analyzing food intake of a human subject in accordance with the present invention;

FIG. 2 shows an example of a multi-sensor based device for measuring, recording and analyzing food intake of a human body according to the present invention;

FIG. 3 shows a second example of a multi-sensor based device for measuring, recording and analyzing food intake of a human body according to the present invention;

FIG. 4 shows a third example of a multi-sensor based device for measuring, recording and analyzing food intake of a human body according to the present invention;

FIG. 5 shows a fourth example of a multi-sensor based apparatus for measuring, recording and analyzing food intake of a human subject in accordance with the present invention;

FIG. 6 shows a fifth example of a multi-sensor based apparatus for measuring, recording and analyzing food intake of a human subject in accordance with the present invention;

FIG. 7 illustrates an example of the device for measuring, recording and analyzing food intake of a human body based on a multi-unit multi-sensor module according to the present invention;

FIG. 8 illustrates an exemplary process of the present invention for an apparatus for measuring, recording and analyzing food intake of a human subject in a multi-person dining scenario;

FIG. 9 illustrates an exemplary process for monitoring alarm events by the method of measuring, recording and analyzing food intake of a human subject of the present invention;

FIG. 10 illustrates an exemplary process for monitoring a single meal event by the method of measuring, recording and analyzing food intake in a human subject of the present invention;

figure 11 shows an example of the overall process of eating a meal according to the method of measuring, recording and analyzing food intake of a human body of the present invention.

Detailed Description

The terminology used in the description of the related examples in the present disclosure is for the purpose of describing the related examples and is not intended to be limiting. The related examples are mainly used for describing the specific embodiments of the present invention, and there may be structural changes in implementation; also, other embodiments may exist for the application of the present invention.

The individual food intake and the individual health are closely related, accurate monitoring and intervention on the food intake are beneficial to keeping the individual healthy, and accurate monitoring on the individual food intake habit is a precondition for implementing accurate intervention. The core of accurately monitoring food intake lies in accurate discrimination of food and accurate measurement of food intake, and on the basis, the food intake, the nutrition balance degree and other indexes of each meal can be accurately evaluated. Objectively, due to the influence of factors such as various Chinese dishes, various food materials, various matching combinations among the food materials, various cooking methods and the like, different individuals have different preferences on tastes and food materials; thus, monitoring of food intake should be as fine as each serving. Meanwhile, in order to accurately measure and calculate the food intake, the volume and the mass of food fed into the mouth each time in the dining process are also required to be accurately monitored.

Therefore, the invention discloses a method and a device for accurately measuring, recording and analyzing food intake of a human body, which realize the discrimination of the type of food in each sending inlet in the individual dining process and simultaneously monitor the volume and the quality of the food by comprehensively using data of various sensors such as an image sensor, a target object three-dimensional volume measuring sensor, an attitude sensor, a temperature sensor, an audio sensor and the like. Through accurate measurement and statistical analysis, the eating habits and preferences of individuals can be accurately recorded and analyzed, and basis and guidance are provided for individual diet intervention and health management.

A method of measuring, recording and analyzing food intake of a human body comprising the steps of: s1, recording the system time during dining; s2, measuring the current real-time temperature of the food; s3, identifying the fine category of the food; s4, measuring the volume and the mass of the ingested food; s5, when the alarm rule is met, an alarm is sent out; the system establishes a local database or an online database, and needs to use the local database or the online database for analysis, wherein the local database or the online database contains or inputs the following data information: nutrient determination information of each food, individualized human nutrient demand quantitative standard, food intake standard, food allergy information and food contraindication information. Step S1 specifically includes the following steps: s11, meal start time judgment: after the equipment is started, the system judges the system time when the individual firstly sends food into the mouth, and the system time is defined as a meal starting point; s12, judging the dining interval time: after the meal start point in said step S11, when the system monitors that the individual has food to enter the mouth, the current system time is recorded, defining the process as a "single meal event"; continuously recording all single meal events until the meal is finished; s13, judging a meal termination event: when the equipment is shut down, the meal is judged to be terminated; when the system does not monitor the single-mouth dining event within 2 continuous hours, the system judges that the dining is terminated and automatically shuts down. The sub-categories of the food identified in step S3 are specifically: the food is collected by an image sensor and then the category of the food is identified by an artificial intelligent image identification program. The classification rules of the fine classification in step S3 include: s31, distinguishing the vegetable type, the fruit type, the meat type and the staple food type, and judging whether the eggs are eggs or not; s32, distinguishing the types and the content of nutrients, which specifically comprises the following steps: judging the components to be proteins, saccharides, lipids, vitamins, inorganic salts, trace elements of special substances and the like. Step S4 measures the volume and mass of food ingested specifically as: the system calls an image sensor and a target object three-dimensional volume measuring sensor, measures and calculates the volume of each type of food in the single-mouth dining event through an artificial intelligence analysis program, and measures and calculates the quality. On the basis of the analysis of the quality of the ingested food, the following indicators were measured: s41, calculating the single nutrient intake in S3; and S42, calculating the intake of each nutrition category in S3.

The method for accurately measuring, recording and analyzing food intake of human body according to the present invention shall be described with reference to the system shown in fig. 1, which includes an image sensor 101, a three-dimensional volume measuring sensor 102 of an object, a posture sensor 103, an audio sensor 104, an individual recognition sensor (biometric sensor) 105, a temperature sensor 106, and a processor 107, a storage device 108, a display screen device 109, a warning signal device 111 and an input/output device 110, wherein the image sensor 101, the three-dimensional volume measuring sensor 102 of the object, the posture sensor 103, the audio sensor 104, the individual recognition sensor (biometric sensor) 105 and the temperature sensor 106 are directly connected to the input/output device 110 to realize signal transmission, the processor 107 and the storage device 108 inside the system are bidirectionally connected to the input/output device 110 to transmit data, the input/output device 110 further transmits data to the display screen device 109 and the warning signal device 111, and in addition, the system is externally connected with a data cloud system 112 and an external device 113, which are used as components of the system to realize data bidirectional transmission, and the data cloud system 112 and the external device 113 are also bidirectionally connected with the input/output device 110.

While the system of fig. 1 is based on multi-dimensional sensor data, and various sensors are used in combination to monitor, record and analyze the food intake process of an individual, in some embodiments of the present disclosure, sensors other than the above-described sensors may be included, such as global positioning system sensors, which are capable of converting the monitored information into digital form and transmitting the monitored information to the processor 107 for local analysis and processing via the input/output device 110. The processor may store the processed food intake information in the local storage device 108 and issue an alert instruction to the display screen device 109 and/or the alert signal device 111 via the input/output device 110 upon detection of an alert event according to alert rules. In some embodiments of the present disclosure, the system may also transmit the data monitored by the relevant sensors to the network cloud platform 112 in or outside the external device 113 through the input/output device 110. The local processor may cooperate with the external device 113 or the cloud platform 112 to process the related data, or may completely process the data sent by the related sensor by the external device 113 or the cloud platform 112. In order to cooperatively complete the monitoring, recording and analyzing of the food intake process, related devices can be centralized in the same equipment to cooperatively work, and can also be distributed in a plurality of equipment to cooperatively work. In some embodiments of the present disclosure, some devices (e.g., display screen 109, audio sensor 104, etc.) in the disclosed system may also be eliminated for simplified functionality.

The memory devices 108 of fig. 1 may include, but are not limited to, various types of semiconductor, magnetic, optical, electronic, and electromagnetic systems or devices, such as various types of flash memory (e.g., compact memory cards, USB memory devices, etc.), magnetic disks, Random Access Memory (RAM), Read Only Memory (ROM), various optical disks, and so forth.

The external device 113 in fig. 1 includes, but is not limited to, a mobile phone device (e.g., a mobile phone), a wearable device (e.g., a bracelet, a watch, a headset, etc.), a portable media player device, a computer (e.g., a desktop computer, a kiosk computer, a notebook computer, etc.), a server device, or the like.

The first embodiment is as follows: intelligent device for measuring, recording and analysing food intake of a human body for single person use (based on multiple sensors)

Example 1

Figure 2 illustrates a smart device for measuring, recording and analysing food intake by a single person. The device 204 in the figure itself has the function of measuring, recording and analyzing the individual food intake process, and at the same time has the function of serving as tableware (e.g. chopsticks, etc.). The device 204 can perform measurement, recording and analysis of individual food intake processes independently, or cooperate with an external device (e.g., a mobile phone, etc.) or an external system such as a cloud platform to perform measurement, recording and analysis through data interaction. The device 204 can form a system with another common chopstick to complete the function of serving as tableware; the device 204 may also be a system with another device that is the same as the device itself to perform the function as a dish.

The device 204 comprises an individual identification camera 201, a wireless communication module 202, a built-in battery 203, fingerprint identification modules 205, 209, a processor 206, a vibration alarm 208, an attitude sensor (consisting of an accelerometer, a gyroscope, and a magnetometer packaged) 210, a module camera (consisting of an image camera and an infrared camera packaged) 207, a laser radar 211, a memory card 212, a microphone 213, a disposable consumable 214, and the like. The attitude sensor 210 may also use a simplified sensor module to implement related functions, such as only one or two of the three types of sensors, i.e., accelerometer, gyroscope, and magnetometer. The lidar 211 used for volume measurement of food may be replaced by other less expensive devices, such as a tof (time of flight) lens. The disposable consumables 214 may be consumables of various materials, and may function like chopsticks, spoons, forks, etc.

Example two

Fig. 3 illustrates a second smart device for measuring, recording and analyzing food intake of a human for use by a single person, including fig. 3A and 3B. The apparatus shown in fig. 3A has evolved from the apparatus shown in fig. 2. The device of fig. 3A integrates the functional device shown in fig. 2 in a more compact space, and the device of fig. 3A is hollow, through-tubular. The device shown in fig. 3A can be combined with ordinary cutlery in a life scenario to enable measurement, recording and analysis of individual food intake processes.

The external modules of the apparatus 3a2 mainly have module cameras (constituted by encapsulated image cameras and infrared cameras) 3a1, 3B1 and laser radars 3A3, 3B 3. Fig. 3B shows an example of the combination of the device 3a2 with ordinary chopsticks, wherein the chopsticks can be inserted into the device 3a2 through a hollow tube. The lidar 3a3, 3B3 used for volume measurement of food may also be replaced by other less expensive devices, such as the use of Tof lenses. The device 3a2 can also be used in combination with other cutlery than chopsticks, such as spoons, forks etc.

Example three

Fig. 4 illustrates a third smart device for measuring, recording and analyzing food intake of a human for use by a single person, comprising fig. 4A and 4B. Devices 4a1 and 4B1 are optimized primarily for spoon and fork application scenarios. Miniaturized, integrated sensor devices 4a4 and 4B4 are placed inside devices 4a1 and 4B1 and are fixed to the dishes by means 4a2 and 4B 2. The sensors in the apparatuses 4a1 and 4B1 control the monitoring range within the boundary range of 4a5, 4a6, 4B5, 4B6 by auto-focusing. Fig. 4A is a schematic view of the apparatus in combination with a spoon 4A3, and fig. 4B is a schematic view of the apparatus in combination with a fork 4B 3.

Example four

Fig. 5 illustrates a fourth smart device for measuring, recording and analyzing food intake of a human for use by a single person, including fig. 5A and 5B. The device 5A3, 5B3 is designed as a wearable device, the sensor module being integrated in the device shown in 5a2, 5B2 and being fixable to the head of the user 5B5 by means of a strap-like device with a tension adjustment function shown in 5a1, 5B 1. When the user is eating, the 5B2, which incorporates the sensor module, will measure, record and analyze the food intake as the individual uses the cutlery to take the food until the food is fed into the mouth 5B 4. In the scene, an additional warning device can be matched to warn and report the warning event in the process of eating.

Example five

Fig. 6 illustrates a fifth intelligent device for measuring, recording and analyzing food intake of a human for use by a single person. Unlike the four example devices described above, the device 601 exists completely independent of the user; need not to be attached to the user's health through wearing, also need not to be attached to on the tableware, do not have the tableware function yet by oneself simultaneously. When an individual eats, the equipment is only required to be placed at a position which is proper in distance from the user, and the equipment can automatically detect the food intake process of the user through the multidimensional sensor and give an alarm when an alarm event is monitored. The display module 607 can provide real-time information feedback during the feeding process of the user to prompt the user to record the current feeding statistics and analyze the results.

The apparatus 601 comprises sensor modules 602, 603, 604 and 605, an alarm module 606, and a display module 607. Device 601 may exist independently of other functional devices and only as a functional device for individual food intake monitoring. In addition, the device 601 may also be integrated into other functional devices (e.g., mobile phones, portable audio-visual players, etc.) that have an individual food intake monitoring function that is one of many functions of the device. Some of the modules in the device 601, such as the alarm module 601 and the display module 607, may also be eliminated, especially when the device 601 is integrated into other functional devices.

Example two: intelligent device for use by multiple persons for measuring, recording and analyzing food intake

Example 1

In a multi-person dining scenario (e.g., a family dining scenario), monitoring, recording, and analyzing of self-ingestion by everyone using a separate device may adversely affect the individual dining experience. Fig. 7 shows a smart device for measuring, recording and analyzing food intake (of a multi-unit multi-sensor module) for use by multiple persons, comprising fig. 7A and 7B. The apparatus 7A1 is an integrated system comprising a plurality of functional modules which can be used for individual food intake measurement, recording and analysis, such as 7A2, 7A3 and 7A 4. The device 7a1 may be of any feasible shape, not limited to the illustrated cylinder, but may also be a polygonal prism, sphere or other irregular shape. The number of functional blocks that the device 7a1 contains that can be used for individual food intake measurement, recording and analysis is also arbitrarily adjustable and is not limited to the three illustrated but may be two, four, five, six, seven, eight, nine, ten, or other larger numbers. The functional module units contained in the device 7A1 and used for individual food intake measurement, recording and analysis can be fixedly integrated in the device 7A1 or can be integrated in the device 7A1 in a detachable mode, and can be flexibly adjusted according to the number of people having meals. The functional modules contained in the device 7a1 that can be used for individual food intake measurement, recording and analysis can be fixedly mounted in the device 7a1, and each module can also have its own individual mechanism to enable the module to be initially flexibly adjusted in posture and always in an optimal detection state during operation. The device 7a1 may or may not include an integrated alert module. In some embodiments, external separate alarm modules (e.g., 7B2, 7B4, and 7B5) may be combined with the device 7a1 into a complete system. Each user may use their own separate alarm module device, which may be attached to the dishware (as shown in fig. 7B) or may exist separately. 7B1 is a schematic diagram of tableware, such as a spoon and a fork; 7B3 and 7B6 are also schematic diagrams of tableware, which can be provided with chopsticks and the like.

Fig. 8 illustrates an example of an operating mode of a smart device for measuring, recording and analyzing food intake for use by multiple persons in a multiple person dining scenario. FIG. 8 illustrates an example of a multi-person dining scenario, including six diners 802, 804, 806, 809, 811, and 813. The intelligent food intake monitoring device 803 is placed at the center of the dining table 814, and the device 803 comprises six individual food intake monitoring module units which are respectively opposite to six diners. In this example system, six diners all use chopsticks, and all have warning devices attached to them that are adapted to the equipment 803, including 801, 805, 807, 808, 810, and 812. In this scenario, six individual food intake monitoring module units in the device 803 operate independently of each other. When the device 803 implements multi-user meal monitoring, recording and analysis, each individual is identified and data recorded, and even if the individual changes seats during the meal, the system can collect and analyze the meal data of each user according to the individual identification information. During a meal, each of the plurality of individual food intake monitoring modular units is not limited to a single user; this ensures that the system can accurately record data without the occurrence of a situation that the user wears the hat, even if the user changes the seat during the use process or the device 803 changes the corresponding relationship with the user due to toppling, shifting or any other reasons.

FIG. 9 illustrates an exemplary process for monitoring an alert event according to an example of the present disclosure. The system can acquire data from three types of sensors, namely an image sensor, a three-dimensional volume measurement sensor and a temperature sensor, the process 904 represents that the system analyzes the data transmitted by the processes 901, 902 and 903, and the process 905 is a judgment condition for comparing internal alarm rules. If an alarm event is detected, process 906 may report the alarm event via an alarm device and may communicate information to the system to record the monitored alarm event. The system issues alerts and it is not always necessary to process the data provided from processes 901, 902 and 903 simultaneously. Process 901 passes data to process 904 and real-time processing and then passes information to process 905. Process 905 determines that an event is detected that is not conducive to the health of the individual holding food, and immediately transmits information and proceeds to process 906. Similarly, process 903 passes data to process 904 and real-time processing and then passes information to process 905. process 905 determines that an event of too high or too low of a temperature of the food is detected, and immediately transmits information and proceeds to process 906. Different rules for triggering the alarm event can be set for different users, and the users can define or modify the rules for triggering the alarm event by themselves.

FIG. 10 illustrates an exemplary process for monitoring a single meal event during ingestion of a meal by an individual in accordance with an example of the present disclosure. The system can acquire data from three types of sensors, namely an image sensor, a three-dimensional volume measurement sensor and an attitude sensor, the process 1004 represents that the system analyzes the data transmitted by the processes 1001, 1002 and 1003, and the process 1005 is to compare internally set judgment conditions and rules defining a single meal event. If the process 1005 determines that the single meal event was successfully detected, the process 1006 sends information to the system to record the type and volume of the food monitored in the single meal event in detail, and the system adds a record including information such as the current food type, volume, nutrient value calculated according to the data model, caloric value calculated according to the data model, current system time, and current user identity. Process 1005 entails simultaneously acquiring the three pieces of information collected by processes 1001, 1002 and 1003 and analyzed and processed by process 1004, i.e., simultaneously satisfying the three conditions of identifying the type of food currently being held, measuring the volume of the current food, and simultaneously detecting that the user has completely delivered the food into the mouth, process 1005 may determine that a successful single meal event has occurred and initiate flow 1006 to record and analyze the meal process. When three processes of identifying the type of the food contained currently, measuring the volume of the current food and simultaneously detecting that the user completely sends the food into the mouth are not met, the food is judged to be an invalid dining event. It should be noted that, after the image information transmitted in the process 1001 is analyzed by the process 1004, there is a case where the food type cannot be determined; at this time, the system still determines that the food category determination process is successful, but the system marks the current determination result as 'unidentified food category' and records image information. The raw meal information data collected by processes 1001, 1002 and 1003 are stored after processing. With the upgrade of the analysis system, the primary collected data can be analyzed secondarily to obtain more accurate results.

FIG. 11 illustrates an example of an apparatus for measuring, recording and analyzing the meal completion process in accordance with the present disclosure. When the system monitors the first single meal event, the system time is recorded as the meal start time. Thereafter, the system continuously records all single meal events, including individual identification information, current system time, type of food being served next, volume of food being served next, etc. When the user actively shuts down or when the system does not detect a newly added single meal event within a set time (for example: 2 hours); the system time point corresponding to the last single meal event in the current food intake monitoring process is defined as the meal termination time. The system calls the data of the temperature sensor to judge whether the current food has a warning event with overhigh temperature, and during the second single-meal event and the fourth single-meal event, the system monitors the event with overhigh temperature and sends out warning. The system determines the type of the current food by calling image sensor data and calls lidar sensor data to determine the volume of the current food. The system calculates the intake of various nutrients and energy intake in the current single meal event according to the nutrient content of various foods in the system database. For example, in a third single meal event, the system detects that the food ingested by the user is rice, with a volume of 4.7 cubic centimeters and a calculated intake mass of 4.7 grams, at time 17, 33 minutes, 12 seconds. It contains carbohydrate (saccharide) 1.22 g, fat 0.02 g and protein 0.12 g; the total caloric intake was 5.45 kcal. In the tenth single meal event, the system detects that the food ingested by the user is pork, with a volume of 1.2 cc, and a calculated intake mass of 1.2 grams, at 35 minutes and 45 seconds at 17 hours. It contains carbohydrate (saccharide) 0.01 g, fat 0.36 g and protein 0.20 g; the total caloric intake was 4.01 kcal. A complete record of the meal schedule of the relevant example is shown in table 1. Table 1 shows the overview of a complete record once by way of example only. In some applications, the content of the hormone can be analyzed more comprehensively and carefully. For example, the intake of inorganic salts, the intake of vitamins and trace elements, the intake of cellulose, etc. are recorded. In the example of fig. 11 and table 1, the system alerts the user that there is excessive carbohydrate intake after the 16 th single meal event, alerting the current diabetic to the risk of "hyperglycemia". In addition, the system detects that the user ingests purine-rich food during the 20 th single meal event, alerting the current hyperuricemic patient to the risk of ingesting excessive purines.

Table 1 an exemplary meal-time ingestion detection, recording and analysis results

A device for measuring, recording and analyzing food intake of human body comprises an image sensor, a target three-dimensional volume measuring sensor, an attitude sensor, an audio sensor, an individual recognition sensor, a temperature sensor, a real-time communication and data transmission system, a data storage system, an alarm prompting system and a control circuit, wherein the image sensor comprises a color/black and white image sensor, a visible light/invisible light image sensor and a CMOS/CCD principle image sensor, the target three-dimensional volume measuring sensor comprises a radar sensor, a laser radar sensor, a time-of-flight depth camera sensor, a laser sensor, a structured light sensor and an ultrasonic sensor, the attitude sensor comprises a gravity sensor, an acceleration sensor, a distance sensor, a rotation vector sensor, a gyroscope sensor, a magnetic sensor and a direction sensor, the individual recognition sensor comprises face recognition, fingerprint recognition and voice recognition, the temperature sensor comprises a contact type/non-contact type thermometer, the real-time communication and data transmission system comprises wifi, Bluetooth and 3G/4G/5G/6G/7G communication equipment, the data storage system comprises a solid storage device and a flash memory, and the warning prompt system comprises a light prompt system, a voice prompt system and a vibration prompt system.

An image sensor for acquiring image data of food; the three-dimensional volume measuring sensor of the target object is used for acquiring three-dimensional volume data of food; an attitude sensor for detecting whether the individual has fed food into the mouth; the audio sensor is used for acquiring an audio instruction of a user; the individual identification sensor is used for identifying the identity of the current user; a temperature sensor for monitoring the temperature of ingested food; the real-time communication and data transmission device is used for communicating with external equipment and transmitting data; a data storage device for device local data storage; the alarm prompting device is used for providing a feedback signal for a user; and the control circuit is used for collecting and processing the data acquired by the various sensors and is used for local processing or transmitting the data to an external device for processing.

In some embodiments of the presently disclosed apparatus for measuring, recording and analyzing food intake, the data collected by the image sensor may include: (1) collecting an image of food selected in a single meal event; (2) acquiring an oral cavity image; (3) user facial information is collected. The functions that the relevant information is used to implement may include: (1) the food classification device is used for judging the type of food selected by an individual in a single-mouth meal event and assisting in judging the three-dimensional volume of the food; (2) the device is used for assisting in judging whether a user sends food into the oral cavity or not; (3) the method is used for individual information identification.

In some embodiments of the presently disclosed apparatus for measuring, recording and analyzing food intake, the target three-dimensional volumetric measurement sensor may comprise: radar sensors, laser radar sensors, time-of-flight depth camera sensors, laser sensors, structured light sensors, ultrasonic sensors, and the like. The data that the sensor is responsible for collecting may include: three-dimensional boundary point information of food selected in the single-mouth meal event is collected. The functions that the relevant information is used to implement may include: for three-dimensional modeling, the volume of food selected by an individual in a "single meal event" is evaluated.

In some particular embodiments of the presently disclosed apparatus for measuring, recording and analyzing food intake, the attitude sensor may comprise: gravity sensors, acceleration sensors, distance sensors, rotation vector sensors, gyroscope sensors, magnetic force sensors, orientation sensors, and the like. The data that the attitude sensor is responsible for collecting may include: (1) direction of tableware or hand movement; (2) the pressure of the contact part of the tableware and the food; (3) distance information between the tableware or hand and the food; (4) the distance between the dishware or hands and the mouth. The functions that the relevant information is used to implement may include: (1) for determining whether the individual is taking food; (2) used for judging whether the user sends food into the oral cavity; (3) used for judging whether the individual meal is finished or not; (4) for assisting in determining the meal time.

In some particular embodiments of the presently disclosed apparatus for measuring, recording and analyzing food intake, the data that the audio sensor is responsible for collecting may include: and collecting a voice instruction of a user. The functions that the relevant information is used to implement may include: for voice command operations.

In some particular embodiments of the disclosed apparatus for measuring, recording and analyzing food intake, the individual recognition sensor may comprise a face recognition sensor, a fingerprint sensor, a voice sensor, or the like. The individual identification sensor is mainly used for judging the identity of the current user.

In some embodiments of the disclosed apparatus for measuring, recording and analyzing food intake, the temperature sensor is responsible for acquiring the real-time temperature of food selected in a "single meal event"; its function may include preventing the user from improperly ingesting food that is too hot or too cold.

In some embodiments of the disclosed apparatus for measuring, recording and analyzing food intake, the real-time communication and data transmission device may comprise wifi, bluetooth, 3G/4G/5G/6G/7G, etc. wireless communication devices. The functions of the real-time communication and data transmission device can include data input and output, and data information interaction with external equipment (such as a mobile phone, a computer and the like) is realized.

In some embodiments of the disclosed apparatus for measuring, recording and analyzing food intake, the data storage device may include various types of storage devices, such as SD card, microSD card, flash memory, hard disk, and the like. The functions of the data storage device may include short term or long term storage for data.

In some embodiments of the disclosed apparatus for measuring, recording and analyzing food intake, the alarm prompting device may include a light prompting device, a sound prompting device, a vibration prompting device, and the like. The function of the alert prompt device may include providing an alert or prompt to the user.

In some particular embodiments of the presently disclosed apparatus for measuring, recording and analyzing food intake, the functions of the control circuit may include: (1) collecting data input by various sensors; (2) analyzing and processing the input data of the sensor; (3) outputting the diet record information to a storage device or an external device; (4) and sending a working instruction to an alarm prompting device.

The main functions that the above-mentioned device can realize may include: (1) accurately measuring the current real-time temperature of food, and giving an alarm when the temperature is too high or too low; (2) accurately calculating the individual dining time length by utilizing the multi-dimensional sensor data, wherein the dining time length comprises the dining starting time, the ending time, the sum of the number of single-mouth dining events in the dining period and the time length of each single-mouth dining event; (3) monitoring the fine classification of food intake by utilizing multi-dimensional sensor data, obtaining the classification of food intake in each single-mouth meal event during the meal period, and carrying out statistical calculation; (4) monitoring the quality of ingested food by utilizing data of a multi-dimensional sensor, measuring and calculating the volume and the quality of the ingested food in each single-mouth meal event during the meal period, and carrying out statistical calculation; (5) alarming risk events and prompting beneficial events; (6) carrying out statistical analysis on the food types and the intake during the whole meal period; (7) and performing system analysis on all meal data within a certain time range.

In some embodiments of the apparatus for measuring, recording and analyzing food intake according to the present disclosure, the system calls data collected by the attitude sensor to determine whether the individual has made an action to take food with the tableware, thereafter the system calls data collected by the target three-dimensional volume measuring sensor to determine whether the volume of the food taken has reached a minimum volume threshold, thereafter the system calls data collected by the attitude sensor to determine whether the individual has made an action to enter the food delivery portal, and finally the system calls data from the distance sensor and the image sensor to determine whether the individual has successfully delivered food into the food delivery portal.

In some embodiments of the apparatus for measuring, recording and analyzing food intake according to the present disclosure, when the system monitors that an individual first puts food into the mouth after the apparatus is powered on, the system time at this time is defined as the start point of a meal. When the system monitors that the individual sends food into the mouth again, the system time is recorded; the process from the last meal intake to the next meal intake is defined as a "single meal event". The system will continuously record all "single meal events" until the meal is finished. When the equipment is shut down, the system judges that the meal is terminated; when the equipment is in a standby state, the system does not monitor the single-mouth dining event within a set time (for example, 2 hours), and then the dining is determined to be terminated and the machine is automatically turned off.

In some embodiments of the apparatus for measuring, recording and analyzing food intake according to the present disclosure, the system analyzes the category of food taken in the "single meal event" by calling data collected by the image sensor, through an artificial intelligence image recognition program, and stores the analysis result locally and/or directly transmits the analysis result to an external device, such as a cloud storage platform, a network storage platform, a mobile phone, a computer, etc.

In some specific embodiments of the apparatus for measuring, recording and analyzing food intake according to the present disclosure, the system will identify and record food held in a "single meal event" in multiple dimensions. The rules for classification may include: (1) the method distinguishes vegetable types, fruit types, meat types, whether eggs exist or not and the like. For example, it is successfully identified that the food to be served next time is spinach, the food to be served next time is broccoli, the food to be served next time is watermelon, the food to be served next time is pineapple, the food to be served next time is fish, the food to be served next time is red meat, the food to be served next time is poultry, the food to be served next time is eggs, and the like. (2) And distinguishing and identifying classical dishes. For example, successfully identifying the current dish as double-cooked meat, successfully identifying the current dish as steamed fish, successfully identifying the current dish as beancurd with rough skin, successfully identifying the current dish as dry stir-fried kidney beans, and the like. The system takes the key indexes of salinity, pungency, tingling degree, acidity, greasiness and the like of the dishes as additional characteristics to be counted according to the characteristics of the dish system to which the dishes belong. (3) The nutrition categories are distinguished, and the nutrition categories are judged to belong to proteins, saccharides, lipids and the like. For example, successfully identifying the current food as a high-sugar food (e.g., rice, steamed bread), successfully identifying the current food as a high-protein food (e.g., red meat, white meat, poultry eggs), successfully identifying the current food as a high-fat food (e.g., cooked back meat), successfully identifying the current food as a high-purine food (e.g., shiitake), successfully identifying the current food as a high-cellulose food (e.g., green vegetables), and so forth.

In some embodiments of the apparatus for measuring, recording and analyzing food intake according to the present disclosure, the system calculates the volume of food taken in a "single meal event" by calling the data collected by the image sensor and the three-dimensional volume measuring sensor of the target object, and by combining an artificial intelligence image recognition program with an artificial intelligence three-dimensional volume measuring program. On the basis, by combining with the judgment of food types, the characteristic data such as the density of the food of the corresponding type is called from the local database, and the quality of the food contained in the single-mouth meal event can be measured. And further, the amount of nutrients ingested in the "single meal event" was calculated. The intake of each nutrient single product in the dining process can be calculated through the statistics of the intake in the whole dining process; at the same time, the total intake of each category of nutrients (sugar, fat, protein) can also be calculated. Under long-term data accumulation, the dietary characteristics of the user during a given time period can be derived.

In some embodiments of the apparatus for measuring, recording and analyzing food intake according to the present disclosure, a database/information base may be established in advance, wherein the stored information may include: the nutrient composition (such as sugar, protein and lipid contents and special substance contents) of various types of foods, the caloric value, the suitable/unsuitable crowd, the nutrient composition, the oil quantity, the salinity, the pungency, the tingling degree and the acidity value of classical dishes, the individualized quantitative standard of human nutrient demand, the healthy food intake judgment rule, the bad food intake judgment rule and the like.

In some embodiments of the apparatus for measuring, recording and analyzing food intake according to the present disclosure, the system may call data collected by the temperature sensor to determine whether there is an over-temperature or an under-temperature condition in the food currently taken by the individual, and may initiate an alarm when the system determines that the current food is not suitable for immediate consumption. The alarm mode can include: (1) the light prompt is that if red light is lightened, the light flickers; (2) voice prompt, such as sounding a voice alarm or sounding a buzzer alarm; (3) and vibration prompt such as continuous vibration and frequency vibration. In addition, the system can call data collected by the image sensor to judge whether the food currently taken by the individual is not good for the health of the individual, and when the system judges that the current food is not suitable, the alarm can be started. The system determining the condition unfavorable to the individual's health may include: (1) excessive intake of certain types of nutrients; (2) the preparation for ingestion of inappropriate foods that may be dangerous to the health of the individual, such as purine-rich lentinus edodes, is detected in hyperuricemic patients.

In the apparatus for measuring, recording and analyzing food intake disclosed in the present invention, the suitable population may include: (1) healthy people. Healthy people can monitor daily food intake through the device disclosed by the invention, and quantitative statistical data of individual food intake can be obtained through long-time continuous monitoring. According to the data, the dietary habits can be adjusted to achieve the purposes of accurate nutrition, reasonable diet, individual weight reduction, disease prevention and the like. (2) Food intake management in people with disease. For example, for patients with diabetes, the patient may be monitored and instructed on a reasonable meal in order to get an individual blood glucose management; for patients with high uric acid, the reasonable diet of the patients can be monitored and guided, and the condition aggravation caused by excessive ingestion of purine-rich food is avoided; for patients with hereditary metabolic diseases, the diet of the patients can be accurately guided, and the increase of metabolic burden is avoided; and the like. (3) Food intake management of special population. For example, accurate nutritional management of young children; accurate nutrition of pregnant women; dietary management for people with allergic conditions; and the like.

The present disclosure recognizes that monitoring data for a user's dietary information is designed to protect the user's personal privacy. During use of the disclosed device, a user needs to provide individual private information, which may include information such as the individual's gender, age, height, weight, personal medical history, dietary contraindications, allergic constitution, dietary preferences, the individual's phone number, email address, and home address, in order to better enable the device to serve monitoring and analysis of the individual's ingestion process. The present disclosure will be responsible for collecting, storing, analyzing user's data and ensuring the security of user private data. The present disclosure will strictly adhere to national laws and regulations and policies and practices of governments and industry regarding privacy protection.