基于数字孪生的多指灵巧手状态监测方法、装置及设备

Method, device and equipment for monitoring state of multi-fingered dexterous hand based on digital twins

文档序号：355 发布日期：2021-09-17 浏览：37次中文

1. A method for monitoring the state of a multi-fingered dexterous hand based on digital twins is characterized by comprising the following steps:

constructing a digital twin model of the multi-finger dexterous hand according to the position of the entity part of the multi-finger dexterous hand and the joint motion relation;

collecting state-related sensor data and joint position sensor data of the multi-finger dexterous hand, and pose image data of the multi-finger dexterous hand;

marking the state-related sensor data according to a preset health label, inputting the state-related sensor data into an SVM (support vector machine) multi-classifier for training, and acquiring the state information of the multi-finger dexterous hand through the SVM multi-classifier;

inputting the pose image data into a pose judger, and acquiring joint angles and coordinate data of the multi-finger dexterous hand through the pose judger; carrying out weighted calculation on the joint angle and coordinate data and the joint position sensor data to obtain the pose information of the multi-finger dexterous hand;

and synchronizing the state information and the pose information into the digital twin model, and monitoring the state of the multi-finger dexterous hand.

2. The method of claim 1, wherein the state-related sensor data includes temperature sensor data, current-voltage sensor data, and joint torque sensor data;

the pose image data includes upper pose image data, forward pose image data, and side pose image data.

3. The method of claim 1, wherein after collecting the state-related sensor data and the joint position sensor data of the multi-fingered dexterous hand, and the pose image data of the multi-fingered dexterous hand, further comprising:

performing denoising processing on the state-related sensor data and the joint position sensor data by a method of smoothing local data or a method of interpolating a normal value by using data around an abnormal value;

and denoising the pose image data by a method of forming a progressive residual fusion dense network for removing Gaussian noise.

4. The method of claim 1, wherein said tagging of said state-related sensor data according to a preset health label and inputting into a SVM multi-classifier for training, and obtaining state information of said multi-fingered dexterous hand via said SVM multi-classifier comprises:

dividing the state of the multi-finger dexterous hand, and taking the divided state as a preset health label;

marking the state-related sensor data according to the health label, and inputting the state-related sensor data into the SVM multi-classifier as a training sample for training so that the SVM multi-classifier outputs a trained SVM model;

and taking the SVM model as a state classifier for judging the state of the multi-fingered dexterous hand, and matching the current state of the multi-fingered dexterous hand through the state classifier to generate the state information of the multi-fingered dexterous hand.

5. The method of claim 1, wherein said synchronizing the state information and the pose information into the digital twin model for state monitoring of the multi-fingered dexterous hand comprises:

synchronizing the state information and the pose information into the digital twin model in the running process of the multi-finger dexterous hand to obtain the synchronous health information of the multi-finger dexterous hand;

comparing the synchronous health information with the priori knowledge of the multi-fingered dexterous hand, determining a state monitoring result of the multi-fingered dexterous hand, and giving an early warning to the behavior which can cause the damage of the multi-fingered dexterous hand.

6. A device for monitoring the state of a multi-fingered dexterous hand based on digital twins is characterized by comprising:

the model building unit is used for building a digital twin model of the multi-finger dexterous hand according to the position of the entity part of the multi-finger dexterous hand and the joint motion relation;

a data collection unit for collecting state-related sensor data and joint position sensor data of the multi-fingered dexterous hand, and pose image data of the multi-fingered dexterous hand;

the state information acquisition unit is used for marking the state-related sensor data according to a preset health label, inputting the state-related sensor data into an SVM multi-classifier for training, and acquiring the state information of the multi-fingered dexterous hand through the SVM multi-classifier;

a pose information acquisition unit for inputting the pose image data into a pose judger and acquiring joint angles and coordinate data of the multi-fingered dexterous hand through the pose judger; carrying out weighted calculation on the joint angle and coordinate data and the joint position sensor data to obtain the pose information of the multi-finger dexterous hand;

and the state monitoring unit is used for synchronizing the state information and the pose information into the digital twin model and monitoring the state of the multi-finger dexterous hand.

7. The apparatus according to claim 6, wherein the status information obtaining unit is specifically configured to:

dividing the state of the multi-finger dexterous hand, and taking the divided state as a preset health label;

8. The apparatus according to claim 6, wherein the status monitoring unit is specifically configured to:

9. A digital twin based multi-fingered dexterous hand condition monitoring device, characterized in that the device comprises a processor and a memory, wherein the memory is stored with execution instructions, and the processor reads the execution instructions in the memory for executing the steps of the digital twin based multi-fingered dexterous hand condition monitoring method according to any one of claims 1-5.

10. A computer readable storage medium storing computer executable instructions for performing the steps of the digital twin based multi-fingered dexterous hand condition monitoring method according to any one of claims 1-5.

Background

The humanoid multi-finger dexterous hand is an electromechanical integration system with high integration level and high intelligence level, generally has 3-5 fingers and more than 8 independent degrees of freedom, has various sensing capabilities, can analyze the environment and operate objects according to target types, is very similar to human hands, can finish operations such as stretching and grabbing, and has high dexterity and multi-aspect applicability.

For technical reasons, the current multi-fingered dexterous hand has a complex structure, high manufacturing cost and is easy to damage. However, at present, there is no method capable of monitoring the state of the multi-fingered dexterous hand in real time, and the state of the multi-fingered dexterous hand cannot be effectively evaluated, so that serious consequences are easily caused, and property and personal safety of enterprises are lost to different degrees.

It is noted that this section is intended to provide a background or context to the embodiments of the disclosure that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.

Disclosure of Invention

The embodiment of the invention provides a method, a device and equipment for monitoring the state of a multi-fingered dexterous hand based on digital twins, and aims to solve the problem that the state of the multi-fingered dexterous hand cannot be monitored and effectively evaluated in real time in the prior art, so that serious consequences are easily caused.

In a first aspect, an embodiment of the present invention provides a method for predicting a fault of a multi-fingered dexterous hand based on a digital twin model, including:

constructing a digital twin model of the multi-finger dexterous hand according to the position of the entity part of the multi-finger dexterous hand and the joint motion relation;

collecting state-related sensor data and joint position sensor data of the multi-finger dexterous hand, and pose image data of the multi-finger dexterous hand;

and synchronizing the state information and the pose information into the digital twin model, and monitoring the state of the multi-finger dexterous hand.

As a preferable mode of the first aspect of the present invention, the state-related sensor data includes temperature sensor data, current-voltage sensor data, and joint torque sensor data;

the pose image data includes upper pose image data, forward pose image data, and side pose image data.

As a preferred aspect of the first aspect of the present invention, the collecting of the state-related sensor data and the joint position sensor data of the multi-fingered dexterous hand and the pose image data of the multi-fingered dexterous hand further comprises:

and denoising the pose image data by a method of forming a progressive residual fusion dense network for removing Gaussian noise.

As a preferred aspect of the first aspect of the present invention, the marking the state-related sensor data according to a preset health label, inputting the state-related sensor data into an SVM multi-classifier for training, and acquiring the state information of the multi-fingered dexterous hand through the SVM multi-classifier includes:

dividing the state of the multi-finger dexterous hand, and taking the divided state as a preset health label;

As a preferred aspect of the first aspect of the present invention, the synchronizing the state information and the pose information into the digital twin model to monitor the state of the multi-fingered dexterous hand includes:

In a second aspect, the present invention provides a digital twin-based multi-fingered dexterous hand condition monitoring device, comprising:

a data collection unit for collecting state-related sensor data and joint position sensor data of the multi-fingered dexterous hand, and pose image data of the multi-fingered dexterous hand;

and the state monitoring unit is used for synchronizing the state information and the pose information into the digital twin model and monitoring the state of the multi-finger dexterous hand.

As a preferable mode of the second aspect of the present invention, the state information acquiring unit is specifically configured to:

dividing the state of the multi-finger dexterous hand, and taking the divided state as a preset health label;

As a preferable mode of the second aspect of the present invention, the state monitoring unit is specifically configured to:

In a third aspect, the embodiment of the present invention provides a digital twin-based multi-fingered dexterous hand state monitoring device, which includes a processor and a memory, wherein the memory stores execution instructions, and the processor reads the execution instructions in the memory for executing the steps in the digital twin-based multi-fingered dexterous hand state monitoring method according to any one of the first aspect and its preferred modes.

In a fourth aspect, embodiments of the present invention provide a computer-readable storage medium storing computer-executable instructions for performing the steps of the digital twin-based multi-fingered dexterous hand state monitoring method according to any one of the first aspect and its preferred modes.

According to the method, the device and the equipment for monitoring the state of the multi-fingered dexterous hand based on the digital twin, provided by the embodiment of the invention, through constructing the digital twin model of the multi-fingered dexterous hand, the state related sensor data, the joint position sensor data and the pose image data of the multi-fingered dexterous hand are collected, then the state information and the pose information of the multi-fingered dexterous hand are respectively obtained according to the data, and finally the state monitoring is carried out on the multi-fingered dexterous hand according to the obtained state information and the pose information.

The invention can simply and conveniently monitor and evaluate the state of the multi-finger dexterous hand, can predict the illegal operation of the dexterous hand in time and give an early warning, avoids the loss of property and personal safety of enterprises, effectively improves the production efficiency and greatly reduces the operation cost of the enterprises.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a flow chart of a method for monitoring a state of a multi-fingered dexterous hand based on a digital twin according to an embodiment of the present invention;

FIG. 2 is a schematic structural diagram of a device for monitoring the state of a multi-fingered dexterous hand based on a digital twin according to an embodiment of the present invention;

fig. 3 is a schematic structural diagram of a digital twin-based multi-fingered dexterous hand state monitoring device according to an embodiment of the present invention.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

Fig. 1 exemplarily shows a flow diagram of a method for monitoring a state of a multi-fingered dexterous hand based on a digital twin according to an embodiment of the present invention, which can simply and conveniently monitor and evaluate the state of the multi-fingered dexterous hand, and can predict illegal operations of the dexterous hand in time and perform early warning, thereby avoiding loss of property and personal safety of enterprises, effectively improving production efficiency, and greatly reducing operation cost of enterprises.

Referring to fig. 1, the method mainly includes the following steps:

101, constructing a digital twin model of the multi-finger dexterous hand according to the position of an entity part of the multi-finger dexterous hand and the joint motion relation;

102, collecting state related sensor data and joint position sensor data of the multi-finger dexterous hand and pose image data of the multi-finger dexterous hand;

103, marking state-related sensor data according to a preset health label, inputting the data into an SVM multi-classifier for training, and acquiring state information of the multi-finger dexterous hand through the SVM multi-classifier;

104, inputting pose image data into a pose judger, and acquiring joint angles and coordinate data of the multi-finger dexterous hand through the pose judger; carrying out weighted calculation on the joint angle and coordinate data and the joint position sensor data to obtain the pose information of the multi-finger dexterous hand;

and 105, synchronizing the state information and the pose information into a digital twin model, and monitoring the state of the multi-finger dexterous hand.

The multi-finger dexterous hand is an electromechanical integration system with high integration level and high intelligence, generally has 3-5 fingers and more than 8 independent degrees of freedom, has various sensing capabilities, can analyze the environment and operate objects according to target types, is very similar to human hands, can finish operations such as stretching and grabbing, and has high dexterity and multiple applicability.

A large number of sensors are arranged in the multi-finger dexterous hand, and various operation parameters of the multi-finger dexterous hand can be acquired in real time. Meanwhile, cameras are respectively arranged above, in front of and on the side of the multi-finger dexterous hand, so that the pose images of the multi-finger dexterous hand can be collected in real time.

In step 101, a digital twin model of a multi-fingered dexterous hand is constructed using modeling software based primarily on the physical component positions and joint motion relationships of the multi-fingered dexterous hand.

Specifically, a workspace is created under the cat _ ws directory in the ROS environment, where an ROS (Robot Operating System) is a System providing a series of libraries and tools to help software developers create Robot application software; then, the workspace is initialized with the cat _ init _ workspace, and the hardware description package is created and dependencies are appended through the cat _ create _ pkg under the workspace cat _ ws/src/directory.

Then, according to the position of the entity part of the multi-finger dexterous hand and the joint movement relation, creating and editing a urdf file of the multi-finger dexterous hand; then, a 3D model file of the multi-finger dexterous hand, namely a digital twin model of the multi-finger dexterous hand, is created by using modeling software such as SolidWork or 3D studio MAX and the like, the 3D model file is exported to be in a mesh format, a mesh folder is created, and the edited mesh file is put into the mesh folder.

Finally, editing a launch file matched with urdf; and then, directly running the launch file through a Roslaunch command, and starting a three-dimensional visualization platform rviz interface and a corresponding digital twin model in the ROS.

In step 102, data of sensors built in the multi-fingered dexterous hand, particularly including state-related sensor data and joint position sensor data, are collected by subscribing to the sensors. Meanwhile, the pose image data of the multi-finger dexterous hand in the three directions are collected through three cameras arranged above, in front of and at the side of the multi-finger dexterous hand.

The state-related sensor data can be used for judging the state of the multi-finger dexterous hand, and the joint position sensor data and the pose image data are used for judging the pose of the multi-finger dexterous hand.

In an optional embodiment provided herein, the state-related sensor data comprises temperature sensor data, current-voltage sensor data, and joint torque sensor data; the pose image data includes upper pose image data, forward pose image data, and side pose image data.

The state related sensors of the multi-finger dexterous hand comprise a temperature sensor, a current-voltage sensor and a joint torque sensor, wherein the temperature sensor data are temperature data and external temperature data of all parts of the multi-finger dexterous hand collected by the temperature sensor, the current-voltage sensor data are power supply voltage data and current data inside the multi-finger dexterous hand collected by the current-voltage sensor, and the joint torque sensor data are acting force data applied between joints and between the joints and an object in the multi-finger dexterous hand collected by the joint torque sensor; and the joint position sensor data is joint angle data of the multi-finger dexterous hand collected by the joint position sensor.

The cameras are respectively arranged above, in front of and at the sides of the multi-finger dexterous hand, wherein the upper pose image data is a pose image collected by the camera above the multi-finger dexterous hand, the front pose image data is a pose image collected by the camera in front of the multi-finger dexterous hand, and the side pose image data is a pose image collected by the camera at the side of the multi-finger dexterous hand.

The collected state-related sensor data, joint position sensor data, and pose image data are usually accompanied by unavoidable noise, and therefore, a part of the noise and abnormal values can be removed by a denoising algorithm, so that the data can be used.

After step 102, the following steps are also included:

performing denoising processing on the state-related sensor data and the joint position sensor data by a method of performing smoothing processing on local data or a method of interpolating a normal value by using data around an abnormal value;

and denoising the pose image data by a method of forming a progressive residual fusion dense network for removing Gaussian noise.

In the above steps, due to the existence of noise, the occurrence of abnormal values is difficult to avoid when the sensor collects data, and two methods can be adopted for processing the abnormal values: and smoothing the local data, or interpolating a normal numerical value by using data around the abnormal value. By these two methods, the state-related sensor data and the joint position sensor data can be subjected to denoising processing.

For noise in pose image data, a progressive residual fusion dense network DnRFD for gaussian noise removal is used. The network firstly adopts a dense block to learn the noise distribution in the image, and greatly reduces network parameters while fully extracting the local features of the image; then, connecting the shallow convolution features with the deep feature short lines in sequence by using a progressive strategy to form a residual fusion network, and extracting more global features aiming at noise; and finally, fusing the output characteristic graphs of the dense blocks and inputting the fused output characteristic graphs into a reconstruction output layer to obtain a final output result, so that denoising processing of the pose image data can be realized.

In step 103, the state of the multi-finger dexterous hand is divided, the divided state is used as a label to mark the sensor data related to the state, then the labeled state is used as a training sample to be input into an SVM multi-classifier to be trained, and finally the SVM model obtained by training is used as a state classifier for judging the state of the multi-finger dexterous hand to obtain the state information of the multi-finger dexterous hand.

Among them, SVM (Support Vector Machine) is a kind of generalized linear classifier that performs binary classification on data in a supervised learning (supervised learning) manner.

In an alternative embodiment provided by the present application, step 103 may be specifically implemented according to the following steps:

and step 1031, dividing the state of the multi-fingered dexterous hand, and taking the divided state as a preset health label.

In the step, the states of the multi-fingered dexterous hand are divided in advance, for example, the states can be divided into states of health, danger, damage and the like, the states can fully represent the health condition of the multi-fingered dexterous hand, and then the divided states are used as preset health labels for data marking.

And 1032, marking the state-related sensor data according to the health label, and inputting the sensor data into the SVM multi-classifier as a training sample for training so that the SVM multi-classifier outputs a trained SVM model.

In the step, the health label obtained in the above steps is adopted to mark the collected state-related sensor data as a training sample of the SVM multi-classifier and input the training sample into the SVM multi-classifier for training, and finally, a trained SVM model is output.

The multi-classification method can be realized by using the SVM multi-classifier, the calculation amount required by the SVM multi-classifier is small, and the SVM multi-classifier can well operate even on some edge devices.

And 1033, taking the SVM model as a state classifier for judging the state of the multi-fingered dexterous hand, and matching the current state of the multi-fingered dexterous hand through the state classifier to generate the state information of the multi-fingered dexterous hand.

In the step, the SVM model obtained by training in the step is used as a state classifier for judging the state of the multi-finger dexterous hand, and then the state classifier is used for matching the current state of the multi-finger dexterous hand, so that the state information of the multi-finger dexterous hand is obtained.

In step 104, a pose judger is trained by using a network data set or self-collected image information, and then pose image data of the multi-fingered dexterous hand obtained in step 102 is input into the trained pose judger, so that joint angle and coordinate data of the multi-fingered dexterous hand are obtained through the pose judger.

And then, multiplying the acquired joint angle and coordinate data and the joint position sensor data acquired in the step 102 by a self-defined weight for weighted calculation, thereby acquiring the pose information of the multi-finger dexterous hand. For example, in the case where the joint position sensor is too noisy or even distorted, the weight of the joint position sensor data may be reduced, while the weight of the joint angle and coordinate data acquired by the posture determiner may be increased.

It should be noted that, the step 103 and the step 104 respectively acquire the state information and the pose information of the multi-fingered dexterous hand, and the two steps do not have strict sequence requirements, and a person skilled in the art may select to execute one of the steps first or execute both the steps at the same time according to actual situations.

In step 105, the state information of the multi-fingered dexterous hand obtained in step 103 and the pose information of the multi-fingered dexterous hand obtained in step 104 are synchronized into the digital twin model constructed in step 101, the digital twin model is operated, the state of the multi-fingered dexterous hand is monitored and evaluated in real time, the illegal operation of the multi-fingered dexterous hand can be predicted in time and early-warned, and loss is avoided.

In an alternative embodiment provided by the present application, step 105 may be implemented as follows:

and 1051, synchronizing the state information and the pose information into a digital twin model in the running process of the multi-finger dexterous hand to obtain the synchronous health information of the multi-finger dexterous hand.

In the step, in the running process of the multi-fingered dexterous hand, the state information obtained in the step 103 and the pose information obtained in the step 104 are synchronized into a constructed digital twin model, and the digital twin model is run to obtain the synchronous health information of the multi-fingered dexterous hand.

Step 1052, comparing the synchronous health information with the prior knowledge of the multi-fingered dexterous hand, determining a state monitoring result of the multi-fingered dexterous hand, and early warning behavior which can cause damage to the multi-fingered dexterous hand.

In the step, the synchronous health information of the multi-finger dexterous hand obtained in the step is compared with the priori knowledge of the multi-finger dexterous hand to comprehensively judge the health state of the multi-finger dexterous hand, the state monitoring result of the multi-finger dexterous hand is finally determined, and meanwhile, the early warning can be performed on the behavior which can cause the damage of the multi-finger dexterous hand, so that the property and personal safety of enterprises are prevented from being lost.

Wherein, the prior knowledge is generally obtained according to the performance of the dexterous multi-finger hand, such as the joint moving range and the like.

It should be noted that the above-mentioned embodiments of the method are described as a series of actions for simplicity of description, but those skilled in the art should understand that the present invention is not limited by the described sequence of actions. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.

In summary, the method for monitoring the state of the multi-fingered dexterous hand based on the digital twin according to the embodiment of the present invention includes constructing the digital twin model of the multi-fingered dexterous hand, collecting the state-related sensor data, the joint position sensor data, and the pose image data of the multi-fingered dexterous hand, respectively acquiring the state information and the pose information of the multi-fingered dexterous hand according to the data, and finally monitoring the state of the multi-fingered dexterous hand according to the acquired state information and the pose information.

Based on the same inventive concept, fig. 2 exemplarily shows a device for monitoring the state of a multi-fingered dexterous hand based on digital twins, which is provided by an embodiment of the present invention, because the principle of solving the technical problem of the device is similar to a method for monitoring the state of a multi-fingered dexterous hand based on digital twins, specific implementation manners of the device can be referred to the specific implementation manners of the method, and repeated details are omitted.

Referring to fig. 2, the apparatus mainly includes the following units:

the model building unit 201 is used for building a digital twin model of the multi-finger dexterous hand according to the position of the entity part of the multi-finger dexterous hand and the joint motion relation;

a data collection unit 202 for collecting state-related sensor data and joint position sensor data of the multi-fingered dexterous hand, and pose image data of the multi-fingered dexterous hand;

the state information acquisition unit 203 is used for marking state-related sensor data according to a preset health label, inputting the state-related sensor data into an SVM multi-classifier for training, and acquiring state information of the multi-finger dexterous hand through the SVM multi-classifier;

a pose information acquiring unit 204 for inputting the pose image data into a pose judger and acquiring the joint angle and coordinate data of the multi-fingered dexterous hand by the pose judger; carrying out weighted calculation on the joint angle and coordinate data and the joint position sensor data to obtain the pose information of the multi-finger dexterous hand;

and the state monitoring unit 205 is used for synchronizing the state information and the pose information into the digital twin model and monitoring the state of the multi-finger dexterous hand.

It should be noted here that the model building unit 201, the data collecting unit 202, the state information acquiring unit 203, the pose information acquiring unit 204, and the state monitoring unit 205 described above correspond to steps 101 to 105 in the above method embodiment, and the four units are the same as the examples and application scenarios realized by the corresponding steps, but are not limited to the disclosure of the above method embodiment.

Preferably, the state-related sensor data includes temperature sensor data, current-voltage sensor data, and joint torque sensor data;

the pose image data includes upper pose image data, forward pose image data, and side pose image data.

Preferably, the data collection unit 202 is further configured to:

and denoising the pose image data by a method of forming a progressive residual fusion dense network for removing Gaussian noise.

Preferably, the state information obtaining unit 203 is specifically configured to:

dividing the state of the multi-finger dexterous hand, and taking the divided state as a preset health label;

according to the health label mark state related sensor data, and as training samples, inputting the training samples into the SVM multi-classifier for training, so that the SVM multi-classifier outputs a trained SVM model;

and taking the SVM model as a state classifier for judging the state of the multi-finger dexterous hand, and matching the current state of the multi-finger dexterous hand through the state classifier to generate the state information of the multi-finger dexterous hand.

Preferably, the state monitoring unit 205 is specifically configured to:

synchronizing state information and pose information into a digital twin model in the running process of the multi-finger dexterous hand to obtain synchronous health information of the multi-finger dexterous hand;

comparing the synchronous health information with the priori knowledge of the multi-fingered dexterous hand, determining the state monitoring result of the multi-fingered dexterous hand, and early warning the behavior which can cause the damage of the multi-fingered dexterous hand.

It should be noted that the device for monitoring the state of a multi-fingered dexterous hand based on a digital twin according to the embodiment of the present invention and the method for monitoring the state of a multi-fingered dexterous hand based on a digital twin according to the foregoing embodiments belong to the same technical concept, and the specific implementation process thereof can refer to the description of the method steps in the foregoing embodiments, and will not be described herein again.

It should be understood that the above digital twin-based multi-fingered dexterous hand state monitoring device includes only units which are logically divided according to the functions realized by the device, and in practical application, the units can be overlapped or separated. The functions of the device for monitoring the state of the multi-fingered dexterous hand based on the digital twin provided by the embodiment correspond to the method for monitoring the state of the multi-fingered dexterous hand based on the digital twin provided by the embodiment one by one, and the more detailed processing flow realized by the device is described in detail in the method embodiment and is not described in detail herein.

In summary, the device for monitoring the state of the multi-fingered dexterous hand based on the digital twin according to the embodiment of the present invention is configured to construct the digital twin model of the multi-fingered dexterous hand, collect the state-related sensor data, the joint position sensor data, and the pose image data of the multi-fingered dexterous hand, respectively obtain the state information and the pose information of the multi-fingered dexterous hand according to the data, and finally monitor the state of the multi-fingered dexterous hand according to the obtained state information and the pose information.

Based on the same inventive concept, fig. 3 exemplarily shows a device for monitoring the state of a multi-fingered dexterous hand based on digital twins, which is provided by an embodiment of the present invention, because the principle of solving the technical problem of the device is similar to a method for monitoring the state of a multi-fingered dexterous hand based on digital twins, specific implementation manners of the device may be referred to in the detailed implementation manners of the method, and repeated details are not repeated.

Referring to fig. 3, an embodiment of the present invention provides a digital twin-based multi-fingered dexterous hand condition monitoring device, which mainly includes a processor 301 and a memory 302, wherein the memory 302 stores execution instructions. The processor 301 reads the execution instructions in the memory 302 for executing the steps described in any of the embodiments of the above-mentioned digital twin-based multi-fingered dexterous hand condition monitoring method. Alternatively, the processor 301 reads the execution instructions in the memory 302 for realizing the functions of the units in any embodiment of the above-mentioned digital twin-based multi-fingered dexterous hand state monitoring device.

FIG. 3 is a schematic diagram of a digital twin-based multi-fingered dexterous hand status monitoring device according to an embodiment of the present invention, as shown in FIG. 3, the device includes a processor 301, a memory 302 and a transceiver 303; wherein, the processor 301, the memory 302 and the transceiver 303 are mutually communicated through a bus 304.

The aforementioned bus 304 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one arrowed line is shown, but does not indicate only one bus or one type of bus.

The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.

Embodiments of the present invention further provide a computer-readable storage medium containing computer-executable instructions for performing the steps described in the above embodiments of the method for monitoring the state of a digitally twin-based multi-fingered dexterous hand. Alternatively, the computer executable instructions are used to perform the functions of the units of the above described embodiment of the digital twin based multi-fingered dexterous hand condition monitoring apparatus.

Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM), a flash Memory, an optical fiber, a portable CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. A computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take a variety of forms, including, but not limited to: an electromagnetic signal, an optical signal, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, Radio Frequency (RF), etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

完整详细技术资料下载

上一篇：石墨接头机器人自动装卡簧、装栓机

下一篇：一种基于计算机的工业机器人动量边界限制系统及方法

Method, device and equipment for monitoring state of multi-fingered dexterous hand based on digital twins

相关技术

网友询问留言