Infant sleep monitoring system, method and equipment
1. An infant sleep monitoring system, comprising:
the image acquisition module is used for acquiring sleep image information of the infant in real time;
and the central information processing module is used for analyzing and judging the sleep behavior information of the infant according to the sleep image information of the infant.
2. The system for monitoring infant sleep according to claim 1, characterized in that the image acquisition module comprises any one or more of a shooting device and a night infrared module, and the night infrared module and the shooting device are both connected with the central information processing module.
3. The system of claim 1, wherein the central information processing module includes a posture analysis unit, an information storage unit, and a behavior matching unit,
the posture analysis unit carries out posture recognition and analysis on the limb actions of the baby according to the sleep image information of the baby to obtain the characteristic parameters of the sleep posture of the baby,
the information storage unit is pre-stored with a plurality of characteristic parameters corresponding to the sleeping behaviors of the baby,
the behavior matching unit obtains sleep behavior information with the highest matching degree according to the identified characteristic parameters of the infant sleep posture and the pre-stored characteristic parameters corresponding to the sleep behaviors of the infants.
4. The system of claim 1, wherein the central information processing module comprises an image preprocessing unit, a posture analysis unit, an information storage unit, and a behavior matching unit,
the image preprocessing unit is used for preprocessing the sleep image information of the infant,
the posture analysis unit carries out posture recognition and analysis on the limb movement of the baby according to the preprocessed sleep image information of the baby to obtain the characteristic parameters of the sleep posture of the baby,
the information storage unit is pre-stored with a plurality of characteristic parameters corresponding to the sleeping behaviors of the baby,
the behavior matching unit obtains sleep behavior information with the highest matching degree according to the identified characteristic parameters of the infant sleep posture and the pre-stored characteristic parameters corresponding to the sleep behaviors of the infants.
5. The system of claim 4, wherein the image preprocessing module is configured to preprocess the sleep image information of the infant, and comprises:
the image preprocessing module is used for carrying out noise reduction and enhancement processing on the sleep image information of the infant.
6. The infant sleep monitoring system of claim 3, further comprising
And the output module is used for transmitting the sleep behavior information of the infant to the user side.
7. The system of claim 6, further comprising a display module, the display module including a storage unit and a reporting unit for storing sleep behavior information of the infant,
the report unit draws a corresponding sleep curve according to the sleep behavior information of the infant, and generates a sleep quality report based on a set period.
8. The system as claimed in claim 7, wherein the display module further comprises a judgment unit and an alarm unit,
the judging unit judges whether the sleep behavior of the infant belongs to abnormal behaviors in the sleep behaviors prestored in the information storage unit or not according to the sleep behavior information of the infant and the characteristic parameters corresponding to the sleep behaviors prestored in the information storage unit, and if yes, the alarm unit sends out an alarm notice.
9. An infant sleep monitoring method, comprising the steps of:
s1, enabling the baby to enter a set sleep detection area, and continuously acquiring sleep image information of the baby in real time in multiple frames;
s2, preprocessing the acquired sleep image information;
s3, performing posture recognition analysis on the limb movement of the infant based on the preprocessed infant sleep image information, and acquiring the characteristic parameters of the infant sleep posture in real time;
s4, matching the characteristic parameters of the identified infant sleeping posture with preset sleeping behavior characteristic parameters to obtain sleeping behavior information with the highest matching degree;
s5, judging whether the abnormal sleeping behavior belongs to, if the abnormal sleeping behavior belongs to, executing S6, otherwise, directly executing S7;
s6, sending out an alarm notification;
s7, obtaining a sleep curve based on the sleep behavior information in a specific time, and generating a sleep quality report based on a set period.
10. An infant sleep monitoring device is characterized by comprising a shell and an infant sleep monitoring system arranged in the shell,
the infant sleep monitoring system according to any one of claims 1-8.
Background
At present, a baby monitor mainly transmits baby sound and video signals to a monitoring terminal of a guardian or a smart phone APP through a wireless network. The technical scheme of the existing infant monitoring system mainly realizes early warning through voiceprint recognition, and the system monitoring and early warning function is only limited to that a corresponding message notification is pushed to a monitor after the crying or abnormal sound of an infant is detected through a sound acquisition module. The main disadvantage of the technical scheme is that abnormal sleeping behaviors of the infant, such as common turning-over behaviors, abnormal sleeping behaviors with the face being shielded and the like, cannot be warned comprehensively in time only by sound judgment and identification. The alarm is easily influenced by environmental noise, so that misjudgment alarms frequently occur more, and the use experience is poorer.
Meanwhile, the sleep quality and the sleep state of the infant cannot be displayed comprehensively and intuitively, and a sleep suggestion and an improvement plan for the infant cannot be provided for the user.
Disclosure of Invention
To achieve these objects and other advantages in accordance with the purpose of the invention, there is provided an infant sleep monitoring system including:
the image acquisition module is used for acquiring sleep image information of the infant in real time;
and the central information processing module is used for analyzing and judging the sleep behavior information of the infant according to the sleep image information of the infant.
According to a preferred embodiment of the present invention, in the infant sleep monitoring system, the image acquisition module includes any one or more of a shooting device and a night infrared module, and both the night infrared module and the shooting device are connected to the central information processing module.
In the system for monitoring sleep of an infant according to a preferred embodiment of the present invention, the central information processing module includes a posture analyzing unit, an information storage unit and a behavior matching unit,
the posture analysis unit analyzes the limb movement of the baby according to the sleep image information of the baby to obtain the characteristic parameters of the sleep posture of the baby,
the information storage unit is pre-stored with a plurality of characteristic parameters corresponding to the sleeping behaviors of the baby,
the behavior matching unit obtains sleep behavior information with the highest matching degree according to the characteristic parameters of the infant sleep posture and the pre-stored characteristic parameters corresponding to the sleep behaviors of the infants.
In accordance with a preferred embodiment of the present invention, in the infant sleep monitoring system, the central information processing module comprises an image preprocessing unit, a posture analyzing unit, an information storage unit and a behavior matching unit,
the image preprocessing unit is used for preprocessing the sleep image information of the infant,
the posture analysis unit analyzes the limb movement of the baby according to the preprocessed sleep image information of the baby to obtain the characteristic parameters of the sleep posture of the baby,
the information storage unit is pre-stored with a plurality of characteristic parameters corresponding to the sleeping behaviors of the baby,
the behavior matching unit obtains sleep behavior information with the highest matching degree according to the characteristic parameters of the infant sleep posture and the pre-stored characteristic parameters corresponding to the sleep behaviors of the infants.
According to a preferred embodiment of the present invention, in the infant sleep monitoring system, the image preprocessing module is configured to preprocess the sleep image information of the infant, and includes:
the image preprocessing module is used for carrying out noise reduction and enhancement processing on the sleep image information of the infant.
In accordance with a preferred embodiment of the present invention, the infant sleep monitoring system further comprises
And the output module is used for transmitting the sleep behavior information of the infant to the user side.
According to a preferred embodiment of the present invention, the infant sleep monitoring system further comprises a display module, the display module comprises a storage unit and a reporting unit, for storing the sleep behavior information of the infant,
the report unit draws a corresponding sleep curve according to the sleep behavior information of the infant, and generates a sleep quality report based on a set period.
In the sleep monitoring system for infants according to a preferred embodiment of the present invention, the display module further comprises a determination unit and an alarm unit,
the judging unit judges whether the sleep behavior of the infant belongs to abnormal behaviors in the sleep behaviors prestored in the information storage unit or not according to the sleep behavior information of the infant and the characteristic parameters corresponding to the sleep behaviors prestored in the information storage unit, and if yes, the alarm unit sends out an alarm notice.
According to a preferred embodiment of the present invention, the sleeping behavior of the infant in the infant sleep monitoring system comprises turning over, crying, climbing the railing, waking up, falling asleep and facial occlusion.
Another preferred embodiment of the present invention provides a method of monitoring sleep in an infant, comprising the steps of:
s1, enabling the baby to enter a set sleep detection area, and continuously acquiring sleep image information of the baby in real time in multiple frames;
s2, preprocessing the acquired sleep image information;
s3, performing posture recognition analysis on the limb movement of the infant based on the preprocessed infant sleep image information, and acquiring the characteristic parameters of the infant sleep posture in real time;
s4, matching the characteristic parameters of the identified infant sleeping posture with preset sleeping behavior characteristic parameters to obtain sleeping behavior information with the highest matching degree;
s5, judging whether the abnormal sleeping behavior belongs to, if the abnormal sleeping behavior belongs to, executing S6, otherwise, directly executing S7;
s6, sending out an alarm notification;
s7, obtaining a sleep curve based on the sleep behavior information in a specific time, and generating a sleep quality report based on a set period.
Another preferred embodiment of the present invention provides an infant sleep monitoring device, comprising a housing and an infant sleep monitoring system disposed within the housing,
when the baby sleeps alone, the baby sleep monitoring system provided by the invention can be used for monitoring the sleep of the baby, once abnormal sleep behaviors are found, an alarm is timely sent to a guardian, for example, once the baby has behaviors of blocking the face, climbing a railing and the like, relevant alarms are sent to the guardian immediately, the guardian can arrive immediately to avoid accidents, and the system can also monitor, acquire the sleep curve of the baby after posture recognition, generate a sleep quality report, know the sleep condition of the baby according to the sleep curve and the sleep quality report, and facilitate life arrangement.
In addition, the invention also provides infant sleep monitoring equipment for implementing the sleep monitoring method.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention.
Drawings
Figure 1 is a block schematic diagram of an infant sleep monitoring system in one embodiment of the present invention,
figure 2 is a block schematic diagram of an infant sleep monitoring system in another embodiment of the present invention,
fig. 3 is a schematic diagram of a framework of a method for monitoring sleep of an infant according to an embodiment of the present invention.
Detailed Description
The present invention is further described in detail below with reference to the attached drawings so that those skilled in the art can implement the invention by referring to the description text.
The following description is presented to disclose the invention so as to enable any person skilled in the art to practice the invention. The preferred embodiments in the following description are given by way of example only, and other obvious variations will occur to those skilled in the art. The basic principles of the invention, as defined in the following description, may be applied to other embodiments, variations, modifications, equivalents, and other technical solutions without departing from the spirit and scope of the invention.
It will be understood by those skilled in the art that in the present disclosure, the terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like are used in an orientation or positional relationship indicated in the drawings for ease of description and simplicity of description, and do not indicate or imply that the referenced devices or components must be constructed and operated in a particular orientation and thus are not to be considered limiting.
It is understood that the terms "a" and "an" should be interpreted as meaning that a number of one element or element is one in one embodiment, while a number of other elements is one in another embodiment, and the terms "a" and "an" should not be interpreted as limiting the number.
As shown in fig. 1-3, an infant sleep monitoring system of the present invention comprises:
the image acquisition module is used for acquiring sleep image information of the infant in real time;
and the central information processing module is used for analyzing and judging the sleep behavior information of the infant according to the sleep image information of the infant.
The central information processing module comprises a posture analysis unit, an information storage unit and a behavior matching unit.
The posture analysis unit analyzes the limb movement of the baby according to the sleep image information of the baby to obtain the characteristic parameters of the sleep posture of the baby,
the information storage unit is pre-stored with a plurality of characteristic parameters corresponding to the sleeping behaviors of the baby,
the behavior matching unit obtains sleep behavior information with the highest matching degree according to the characteristic parameters of the infant sleep posture and the pre-stored characteristic parameters corresponding to the sleep behaviors of the infants.
It should be noted that, after the baby enters the set sleep detection area, image acquisition is carried out, the image acquisition module includes any one or more of shooting equipment and night infrared module, also can be the shooting equipment who has the shooting function in daytime and shoots the function night concurrently, when having light at daytime or night, the sleep gesture of directly shooting many babies in succession, night then adopts infrared ray night to shoot the function, shoots the sleep gesture of many babies in succession, night infrared module, shooting equipment all with central information processing module is connected.
The image acquisition module can be one or more, if one, then this image acquisition module is directly located the top that the baby sleeps the position, and the position is higher, and the slope sets up, and the scope of shooing like this is wider, can realize not having the dead angle and shoot, and the collection area can cover baby's sleep home range, if it is a plurality of, then the multi-angle multi-height distribution is near baby's sleep position of a plurality of image acquisition templates.
The sleep image information comprises sleep behaviors and the maintaining time and frequency of the sleep behaviors.
Sleeping behaviors of the infant include, without limitation, turning over, crying, climbing a railing, waking up, falling asleep, and facial occlusion.
The posture recognition module adopts Mask R-CNN-based neural network model training to recognize the posture of the baby to obtain the key characteristic point information of the human body in the sleeping behavior process of the baby, extracts the image characteristic points by using a ResNet-101 network, further screens out the key area of the posture characteristic points in the picture by using ROI Align and finally obtains the posture characteristic points of the baby in the picture.
The information storage unit is internally pre-stored with a plurality of characteristic parameters corresponding to the sleeping behaviors of the infant, such as when the infant is in a sleeping state, the face of the infant is not shielded, the eyes of the infant are in a tightly closed state, the infant is kept in a state of being immobile for a long time, whether the face of the infant is shielded or not and whether the eyes of the infant are in the tightly closed state or not are monitored and analyzed, the activity state of the infant is detected, if the infant is in a waking state, the infant flicks, the eyes of the infant are opened, and if the infant is in a crying state, the mouth of the infant is large.
Specifically, when a sleep behavior database in the information storage unit is constructed, each sleep action picture set of the daily baby is extracted, and each action picture atmosphere training set and each action picture atmosphere testing set are extracted. And similarly, acquiring posture characteristic points of the picture by using a Mask R-CNN network, marking each characteristic joint point of the baby, and acquiring corresponding key marking characteristics of a preset behavior to construct a sleep behavior database.
When the behavior matching unit is used for matching, the sleep behavior characteristics of the infant may be only overlapped with one of the sleep behaviors prestored in the information storage unit, so that the sleep behavior of the infant can be determined undoubtedly, or the sleep behavior of the infant may be similar to the characteristics of a plurality of behaviors in the sleep behaviors prestored in the information storage unit, but the sleep behavior with the highest degree of overlap is selected according to the degree of overlap of the characteristics, and the infant is determined to be in the sleep behavior.
The behavior matching unit compares the posture key points extracted from the posture recognition module with the posture feature key points of a preset posture template database by adopting a nearest neighbor classifier (KNN) in machine learning, calculates the distance between the current feature points and the feature points of a plurality of preset sleep behaviors in the preset sleep posture database, finds k adjacent action points with the minimum distance, and compares k sleep behavior categories to obtain the sleep behavior result with the highest matching degree.
In another embodiment, the central information processing module comprises an image preprocessing unit, a posture analyzing unit, an information storage unit and a behavior matching unit,
the image preprocessing unit is used for preprocessing the sleep image information of the infant,
the posture analysis unit analyzes the limb movement of the baby according to the preprocessed sleep image information of the baby to obtain the characteristic parameters of the sleep posture of the baby,
the information storage unit is pre-stored with a plurality of characteristic parameters corresponding to the sleeping behaviors of the baby,
the behavior matching unit obtains sleep behavior information with the highest matching degree according to the characteristic parameters of the infant sleep posture and the pre-stored characteristic parameters corresponding to the sleep behaviors of the infants.
It should be noted that, when the sleep image information acquired by the image acquisition module is fuzzy or has other problems, the sleep image information needs to be preprocessed, denoised and enhanced to improve the image quality and facilitate the gesture recognition of the image in the later period.
In accordance with a preferred embodiment of the present invention, the infant sleep monitoring system further comprises
And the output module is used for transmitting the sleep behavior information of the infant to the user side. The transmission may be wireless transmission or wired transmission, and is determined specifically according to actual conditions, and is not described herein again. The output module outputs the matched infant sleeping behaviors based on the http protocol.
The user side can be a computer or a mobile terminal such as a mobile phone, and therefore a guardian can conveniently observe and know the sleeping condition of the baby.
According to a preferred embodiment of the present invention, the infant sleep monitoring system further comprises a display module, the display module comprises a storage unit and a reporting unit, for storing the sleep behavior information of the infant,
the report unit draws a corresponding sleep curve according to the sleep behavior information of the infant, and generates a sleep quality report based on a set period.
The sleep profile primarily records the corresponding sleep behavior for each time segment and the duration of that sleep behavior.
In order to inform the guardian of abnormal behavior of the infant during sleep and avoid accidents, the display module further comprises a judgment unit and an alarm unit,
the judging unit judges whether the sleep behavior of the infant belongs to abnormal behaviors in the sleep behaviors prestored in the information storage unit or not according to the sleep behavior information of the infant and the characteristic parameters corresponding to the sleep behaviors prestored in the information storage unit, and if yes, the alarm unit sends out an alarm notice.
If the judging unit cannot determine the corresponding sleep behavior of the baby according to the characteristic parameters corresponding to the sleep behavior prestored in the information storage unit, and the action of the baby is proved to be not the conventional sleep behavior and to belong to an out-of-situation event, the alarm unit can also send an alarm notification to remind a guardian to check so as to avoid an accident.
The alarm unit can be a loudspeaker, a buzzer, a light and the like, and can be specifically set according to actual conditions.
Another preferred embodiment of the present invention provides a method of monitoring sleep in an infant, comprising the steps of:
s1, enabling the baby to enter a set sleep detection area, and continuously acquiring sleep image information of the baby in real time in multiple frames;
s2, preprocessing the acquired sleep image information;
s3, performing posture recognition analysis on the limb movement of the infant based on the preprocessed infant sleep image information, and acquiring the characteristic parameters of the infant sleep posture in real time;
s4, matching the characteristic parameters of the identified infant sleeping posture with preset sleeping behavior characteristic parameters to obtain sleeping behavior information with the highest matching degree;
s5, judging whether the abnormal sleeping behavior belongs to, if the abnormal sleeping behavior belongs to, executing S6, otherwise, directly executing S7;
s6, sending out an alarm notification;
s7, obtaining a sleep curve based on the sleep behavior information in a specific time, and generating a sleep quality report based on a set period.
Another preferred embodiment of the present invention provides an infant sleep monitoring apparatus, which includes a housing and an infant sleep monitoring system disposed in the housing, that is, the above modules and units are disposed in the housing, but the photographing end of the image capturing module needs to be located outside the housing, so as to conveniently take a photograph and capture an image.
While embodiments of the invention have been described above, it is not limited to the applications set forth in the description and the embodiments, which are fully applicable in various fields of endeavor to which the invention pertains, and further modifications may readily be made by those skilled in the art, it being understood that the invention is not limited to the details shown and described herein without departing from the general concept defined by the appended claims and their equivalents.