Data acquisition method, data acquisition equipment, storage medium and device
1. A data acquisition method, characterized in that the data acquisition method comprises the steps of:
acquiring sub-equipment information and parameter information of equipment to be acquired;
judging whether a metadata coding group corresponding to the equipment to be collected exists in a preset metadata coding table or not;
if not, generating a target metadata coding group corresponding to the equipment to be acquired according to the sub-equipment information and the parameter information to be acquired;
and acquiring the current data of the equipment to be acquired, and correspondingly storing the current data and the target metadata coding group.
2. The data acquisition method according to claim 1, wherein the step of determining whether the metadata encoding group corresponding to the device to be acquired exists in a preset metadata encoding table specifically comprises:
extracting the sub-equipment type and the sub-equipment serial number from the sub-equipment information, and taking the sub-equipment type, the sub-equipment serial number and the parameter information to be acquired as reference information;
and searching whether a metadata coding group corresponding to the reference information exists in a preset metadata coding table.
3. The data collection method of claim 2, wherein after the step of determining whether the metadata encoding group corresponding to the device to be collected exists in the preset metadata encoding table, the data collection method further comprises:
if yes, the sub-device information and the parameter information to be collected are used as target information, and a metadata coding group corresponding to the target information is searched in a preset metadata coding table;
and acquiring the current data of the equipment to be acquired, and correspondingly storing the current data and the metadata coding group.
4. The data acquisition method according to any one of claims 1 to 3, wherein the step of acquiring current data of the device to be acquired and storing the current data in correspondence with the target metadata encoding set specifically comprises:
acquiring current data of the equipment to be acquired, and establishing a corresponding relation between the current data and the target metadata coding group;
and generating a data storage table based on the current data, the target metadata coding group and the corresponding relation, and storing the data storage table.
5. The data collection method according to any one of claims 1 to 3, wherein before the step of collecting current data of the device to be collected and storing the current data in correspondence with the target metadata encoding set, the data collection method further comprises:
searching a network transmission protocol corresponding to the equipment to be acquired, and carrying out communication protocol adaptation according to the network transmission protocol;
correspondingly, the step of acquiring the current data of the device to be acquired and correspondingly storing the current data and the target metadata encoding group specifically includes:
and after the adaptation is successful, acquiring the current data of the equipment to be acquired, and correspondingly storing the current data and the target metadata coding group.
6. The data acquisition method according to any one of claims 1 to 3, wherein the step of acquiring the sub-device information of the device to be acquired and the parameter information to be acquired specifically includes:
performing composition analysis on the equipment to be acquired to obtain a composition topological graph of the equipment to be acquired;
and extracting the equipment information of the topological graph to obtain the information of the sub-equipment and the information of the parameter to be acquired.
7. The data collection method of any one of claims 1-3, wherein after the step of collecting current data of the device to be collected and storing the current data in correspondence with the target metadata encoding set, the data collection method further comprises:
when a data query request is received, determining a query metadata code according to the data query request;
and searching target data corresponding to the query metadata codes, and aggregating the target data to obtain a data aggregation result.
8. A data acquisition device, characterized in that the data acquisition device comprises: memory, a processor and a data acquisition program stored on the memory and executable on the processor, the data acquisition program when executed by the processor implementing the steps of the data acquisition method according to any one of claims 1 to 7.
9. A storage medium, characterized in that the storage medium has stored thereon a data acquisition program which, when executed by a processor, implements the steps of the data acquisition method according to any one of claims 1 to 7.
10. A data acquisition device, characterized in that the data acquisition device comprises: the device comprises an information acquisition module, a code judgment module, a code generation module and a data storage module;
the information acquisition module is used for acquiring the sub-equipment information of the equipment to be acquired and the parameter information to be acquired;
the encoding judgment module is used for judging whether a metadata encoding group corresponding to the equipment to be acquired exists in a preset metadata encoding table;
the code generation module is used for generating a target metadata code group corresponding to the equipment to be acquired according to the sub-equipment information and the parameter information to be acquired if the target metadata code group is not acquired;
and the data storage module is used for acquiring the current data of the equipment to be acquired and correspondingly storing the current data and the target metadata coding group.
Background
In the prior art, different types of acquisition templates are often configured through different types of data sources to realize data acquisition of multiple data sources.
However, in the above method, when a new type of data source appears, the acquisition template needs to be re-developed, which results in a large workload and high implementation difficulty.
The above is only for the purpose of assisting understanding of the technical aspects of the present invention, and does not represent an admission that the above is prior art.
Disclosure of Invention
The invention mainly aims to provide a data acquisition method, data acquisition equipment, a storage medium and a data acquisition device, and aims to solve the technical problems that in the prior art, different types of acquisition templates are configured for different types of data sources to realize data acquisition of multiple data sources, the workload is high, and the realization difficulty is high.
In order to achieve the above object, the present invention provides a data acquisition method, including the steps of:
acquiring sub-equipment information and parameter information of equipment to be acquired;
judging whether a metadata coding group corresponding to the equipment to be collected exists in a preset metadata coding table or not;
if not, generating a target metadata coding group corresponding to the equipment to be acquired according to the sub-equipment information and the parameter information to be acquired;
and acquiring the current data of the equipment to be acquired, and correspondingly storing the current data and the target metadata coding group.
Optionally, the step of determining whether a metadata encoding group corresponding to the device to be acquired exists in a preset metadata encoding table specifically includes:
extracting the sub-equipment type and the sub-equipment serial number from the sub-equipment information, and taking the sub-equipment type, the sub-equipment serial number and the parameter information to be acquired as reference information;
and searching whether a metadata coding group corresponding to the reference information exists in a preset metadata coding table.
Optionally, after the step of determining whether the metadata encoding group corresponding to the device to be acquired exists in the preset metadata encoding table, the data acquisition method further includes:
if yes, the sub-device information and the parameter information to be collected are used as target information, and a metadata coding group corresponding to the target information is searched in a preset metadata coding table;
and acquiring the current data of the equipment to be acquired, and correspondingly storing the current data and the metadata coding group.
Optionally, the step of acquiring current data of the device to be acquired and storing the current data in correspondence with the target metadata encoding group specifically includes:
acquiring current data of the equipment to be acquired, and establishing a corresponding relation between the current data and the target metadata coding group;
and generating a data storage table based on the current data, the target metadata coding group and the corresponding relation, and storing the data storage table.
Optionally, before the step of acquiring the current data of the device to be acquired and storing the current data in correspondence with the target metadata encoding group, the data acquisition method further includes:
searching a network transmission protocol corresponding to the equipment to be acquired, and carrying out communication protocol adaptation according to the network transmission protocol;
correspondingly, the step of acquiring the current data of the device to be acquired and correspondingly storing the current data and the target metadata encoding group specifically includes:
and after the adaptation is successful, acquiring the current data of the equipment to be acquired, and correspondingly storing the current data and the target metadata coding group.
Optionally, the step of obtaining the sub-device information of the device to be acquired and the parameter information to be acquired specifically includes:
performing composition analysis on the equipment to be acquired to obtain a composition topological graph of the equipment to be acquired;
and extracting the equipment information of the topological graph to obtain the information of the sub-equipment and the information of the parameter to be acquired.
Optionally, after the step of acquiring the current data of the device to be acquired and storing the current data in correspondence with the target metadata encoding group, the data acquisition method further includes:
when a data query request is received, determining a query metadata code according to the data query request;
and searching target data corresponding to the query metadata codes, and aggregating the target data to obtain a data aggregation result.
Furthermore, to achieve the above object, the present invention further provides a data acquisition device, which includes a memory, a processor, and a data acquisition program stored on the memory and operable on the processor, wherein the data acquisition program is configured to implement the steps of the data acquisition method as described above.
In addition, to achieve the above object, the present invention further provides a storage medium having a data acquisition program stored thereon, wherein the data acquisition program, when executed by a processor, implements the steps of the data acquisition method as described above.
In addition, in order to achieve the above object, the present invention further provides a data acquisition apparatus, including: the device comprises an information acquisition module, a code judgment module, a code generation module and a data storage module;
the information acquisition module is used for acquiring the sub-equipment information of the equipment to be acquired and the parameter information to be acquired;
the encoding judgment module is used for judging whether a metadata encoding group corresponding to the equipment to be acquired exists in a preset metadata encoding table;
the code generation module is used for generating a target metadata code group corresponding to the equipment to be acquired according to the sub-equipment information and the parameter information to be acquired if the target metadata code group is not acquired;
and the data storage module is used for acquiring the current data of the equipment to be acquired and correspondingly storing the current data and the target metadata coding group.
The invention discloses a method for acquiring sub-equipment information and parameter information of equipment to be acquired, judging whether a preset metadata coding table has a metadata coding group corresponding to the equipment to be acquired, if not, generating a target metadata coding group corresponding to the equipment to be acquired according to the sub-equipment information and the parameter information to be acquired, acquiring current data of the equipment to be acquired, and correspondingly storing the current data and the target metadata coding group; compared with the existing mode that different types of data sources are configured with different types of acquisition templates to realize data acquisition of multiple data sources, the method and the device can generate the target metadata encoding group according to the sub-device information and the parameter information to be acquired of the equipment to be acquired when the metadata encoding group corresponding to the equipment to be acquired does not exist in the preset metadata encoding table, and correspondingly store the acquired current data and the target metadata encoding group, so that the uniform format collection of the different types of data sources is realized, and the data acquisition workload is reduced.
Drawings
FIG. 1 is a schematic diagram of a data acquisition device in a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a first embodiment of a data acquisition method according to the present invention;
FIG. 3 is a schematic flow chart of a data acquisition method according to a second embodiment of the present invention;
FIG. 4 is a topological diagram of the devices to be collected according to an embodiment of the data collection method of the present invention;
FIG. 5 is a schematic flow chart of a data acquisition method according to a third embodiment of the present invention;
fig. 6 is a block diagram of a first embodiment of the data acquisition device according to the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a data acquisition device in a hardware operating environment according to an embodiment of the present invention.
As shown in fig. 1, the data acquisition apparatus may include: a processor 1001, such as a Central Processing Unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), and the optional user interface 1003 may further include a standard wired interface and a wireless interface, and the wired interface for the user interface 1003 may be a USB interface in the present invention. The network interface 1004 may optionally include a standard wired interface, a WIreless interface (e.g., a WIreless-FIdelity (WI-FI) interface). The Memory 1005 may be a Random Access Memory (RAM) or a Non-volatile Memory (NVM), such as a disk Memory. The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the configuration shown in fig. 1 does not constitute a limitation of the data acquisition apparatus and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in FIG. 1, memory 1005, identified as one type of computer storage medium, may include an operating system, a network communication module, a user interface module, and a data collection program.
In the data acquisition device shown in fig. 1, the network interface 1004 is mainly used for connecting to a background server and performing data communication with the background server; the user interface 1003 is mainly used for connecting user equipment; the data acquisition device calls a data acquisition program stored in the memory 1005 through the processor 1001 and executes the data acquisition method provided by the embodiment of the invention.
Based on the hardware structure, the embodiment of the data acquisition method is provided.
Referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of the data acquisition method of the present invention, and provides the first embodiment of the data acquisition method of the present invention.
Step S10: and acquiring the sub-equipment information and the parameter information of the equipment to be acquired.
It should be understood that the main body of the method of the embodiment may be a computing service device with data processing, network communication and program running functions, such as a server and a computer, or other electronic devices capable of implementing the same or similar functions, which is not limited in this embodiment. In this embodiment and other embodiments, a data acquisition device is taken as an example for explanation.
It should be noted that the device to be acquired may be determined by a data acquisition request sent by the terminal device. The terminal device can establish communication connection with the data acquisition device in advance, and a user can initiate a data acquisition request through the terminal device.
The sub-device information of the device to be acquired may be information such as a sub-device type and a sub-device serial number constituting the device to be acquired. For example, when the device to be acquired is a server, the types of the child devices are CPU, memory, disk, and the like.
The parameter information to be acquired of the device to be acquired can be the utilization rate, the highest frequency and the lowest frequency of a CPU, the total memory capacity and the memory utilization rate of a memory, the utilization rate and the total capacity of a disk and a partition format.
It should be understood that the acquiring of the sub-device information of the device to be acquired and the parameter information to be acquired may be acquiring a device model of the device to be acquired, searching for sub-device information corresponding to the device model, and acquiring parameter information to be acquired corresponding to the sub-device.
Further, in consideration of practical application, some users may modify the device, which may result in inaccuracy of the sub-device information determined according to the device model and the parameter information to be acquired. In order to overcome the above drawback, step S10 includes:
and performing composition analysis on the equipment to be acquired to obtain a composition topological graph of the equipment to be acquired, and extracting equipment information from the composition topological graph to obtain sub-equipment information and parameter information to be acquired.
Step S20: and judging whether a metadata coding group corresponding to the equipment to be collected exists in a preset metadata coding table.
It should be noted that the preset metadata encoding table may be preset by a manager of the data acquisition device, or may be generated by the data acquisition device in a self-adaptive manner according to the obtained historical metadata encoding set, which is not limited in this embodiment.
It should be understood that the judgment of whether the metadata encoding group corresponding to the device to be acquired exists in the preset metadata encoding table is to judge whether the uniform format metadata encoding group corresponding to the device to be acquired exists in the preset metadata encoding table, so as to subsequently determine whether an individual target metadata encoding needs to be generated for the device to be acquired.
It can be understood that, the judgment of whether the metadata coding group corresponding to the device to be acquired exists in the preset metadata coding table may be to acquire a device identifier of the device to be acquired, match the device identifier of the device to be acquired with a device identifier corresponding to each metadata coding group in the preset metadata coding table, and when the matching is successful, judge that the metadata coding group corresponding to the device to be acquired exists in the preset metadata coding table; and when the matching fails, judging that the preset metadata coding table does not have a metadata coding group corresponding to the equipment to be collected.
Further, in practical applications, it may be considered that the preset metadata encoding table does not store the metadata encoding group corresponding to the device to be acquired, but stores the metadata encoding group corresponding to the child device of the device to be acquired. In the above manner, only the information of the device to be acquired is considered, and therefore, the situation of data acquisition through the metadata encoding group corresponding to the sub-device cannot be identified. In order to overcome the above drawback, step S20 includes:
and extracting the type and the serial number of the sub-equipment from the information of the sub-equipment, taking the type and the serial number of the sub-equipment and the parameter information to be acquired as reference information, and searching whether a metadata coding group corresponding to the reference information exists in a preset metadata coding table.
Step S30: and if not, generating a target metadata coding group corresponding to the equipment to be acquired according to the sub-equipment information and the parameter information to be acquired.
It should be understood that, when the metadata encoding table corresponding to the reference information does not exist in the preset metadata encoding table, it is indicated that the uniform format encoding table corresponding to the device to be acquired is not stored in the preset metadata encoding table, and a uniform format encoding group corresponding to the device to be acquired needs to be separately generated.
It can be understood that the generation of the target metadata encoding group corresponding to the device to be acquired according to the sub-device information and the parameter information to be acquired may be extracting a sub-device type number and a sub-device serial number from the sub-device information, and generating the target metadata encoding group corresponding to the device to be acquired according to the sub-device type number, the sub-device serial number and the parameter information to be acquired. The format of the target metadata encoding group is as follows:
sub-device type number/sub-device serial number/data acquisition parameter
For convenience of understanding, the following description is given by way of example, but the present invention is not limited thereto. The target metadata encoding group is shown below, where- -xxx (e.g., device serial number, device type, etc.) is used to interpret the specific meaning of the target metadata encoding group.
100/0/10001- -equipment Serial number
100/0/10002- -type of equipment
100/0/10003- -device Address
100/0/10004- -current time of device
1001/1/10001- -CPU/core 1/real time value of usage
1001/1/10002- -CPU/core 1/highest frequency
1001/1/10003- -CPU/core 1/lowest frequency
1001/2/10001- -CPU/core 2/real time value of usage
1001/2/10002- -CPU/core 2/highest frequency
1001/2/10003- -CPU/core 2/highest frequency
… omit other CPU sub-devices …
2001/1/20001- - -memory/1/total content capacity
2001/1/20002- -memory/1/content usage
3001/1/30001- -disk/1/utilization real-time value
3001/1/30002- -disk/1/total capacity
3001/1/30003- - -disk/1/partition format
It should be understood that, in this embodiment, a uniform numbering rule is used for each type of sub-device, so that uniform intercommunication of metadata encoding can be achieved, and subsequent data aggregation and number escape are facilitated.
Furthermore, in order to be able to collect the data of the device to be collected when the metadata coding table corresponding to the reference information exists in the preset metadata coding table. After the step S30, the method further includes:
if yes, the sub-device information and the parameter information to be collected are used as target information, and a metadata coding group corresponding to the target information is searched in a preset metadata coding table;
and acquiring the current data of the equipment to be acquired, and correspondingly storing the current data and the metadata coding group.
It can be understood that, when the metadata encoding table corresponding to the reference information exists in the preset metadata encoding table, it is indicated that the uniform format encoding table corresponding to the device to be acquired is stored in the preset metadata encoding table, and a uniform format encoding group corresponding to the device to be acquired does not need to be separately generated.
It should be noted that the preset metadata encoding table includes a corresponding relationship between the target information and the metadata encoding table, and the corresponding relationship between the target information and the metadata encoding table may be preset.
Step S40: and acquiring the current data of the equipment to be acquired, and correspondingly storing the current data and the target metadata coding group.
It can be understood that the step of acquiring current data of the device to be acquired and correspondingly storing the current data and the target metadata encoding group may be acquiring the current data of the device to be acquired, establishing a corresponding relationship between the current data and the target metadata encoding group, generating a data storage table based on the current data, the target metadata encoding group and the corresponding relationship, and storing the data storage table.
In the first embodiment, the method comprises the steps of acquiring sub-device information and parameter information to be acquired of a device to be acquired, judging whether a metadata coding group corresponding to the device to be acquired exists in a preset metadata coding table, if not, generating a target metadata coding group corresponding to the device to be acquired according to the sub-device information and the parameter information to be acquired, acquiring current data of the device to be acquired, and correspondingly storing the current data and the target metadata coding group; compared with the existing mode that different types of data sources are configured with different types of acquisition templates to realize data acquisition of multiple data sources, the embodiment can generate the target metadata encoding group according to the sub-device information of the device to be acquired and the parameter information to be acquired when the metadata encoding group corresponding to the device to be acquired does not exist in the preset metadata encoding table, and correspondingly stores the acquired current data and the target metadata encoding group, so that uniform format collection of different types of data sources is realized, and the data acquisition workload is reduced.
Referring to fig. 3, fig. 3 is a schematic flow chart of a second embodiment of the data acquisition method of the present invention, and the second embodiment of the data acquisition method of the present invention is proposed based on the first embodiment shown in fig. 2.
In the second embodiment, the step S10 includes:
step S101: and carrying out composition analysis on the equipment to be acquired to obtain a composition topological graph of the equipment to be acquired.
It should be noted that the composition analysis may be what sub-devices the analysis device is composed of.
It should be understood that, the obtaining of the composition topological graph of the device to be acquired may be obtaining performance data of the device to be acquired, and obtaining the composition topological graph of the device to be acquired by performing composition analysis on the device to be acquired according to the performance data.
Step S102: and extracting the equipment information of the topological graph to obtain the information of the sub-equipment and the information of the parameter to be acquired.
It should be noted that the sub-device information may include a sub-device type, a sub-device serial number, and the like.
For ease of understanding, the description will be made with reference to fig. 4, but this scheme is not limited thereto. Fig. 4 is a topological diagram of a device to be acquired, in which the device to be acquired is a server (serial number sn123), the server is composed of a CPU, a memory and a disk, the CPU is composed of a core1, a core2, a core3 and a core4, the hard disk is composed of a partition 1, a partition 2 and a partition 3, parameters corresponding to the core1-4 are a usage rate, a highest frequency and a lowest frequency, and parameters corresponding to the partition 1-3 are a usage rate, a partition total capacity and a partition format. And extracting the device information of the combined topological graph to obtain the types of the sub-devices, namely a CPU, a memory and a disk, wherein the serial numbers of the sub-devices are core1, core2, core3, core4, partition 1, partition 2 and partition 3, and the parameter information to be acquired is the utilization rate, the highest frequency and the lowest frequency corresponding to the core1-4, the usage amount and the memory capacity corresponding to the memory, and the utilization rate, the total partition capacity and the partition format corresponding to the partition 1-3.
The second embodiment obtains the composition topological graph of the device to be acquired by performing composition analysis on the device to be acquired, extracts the device information from the composition topological graph, and obtains the sub-device information and the parameter information to be acquired, thereby improving the accuracy and reliability of the sub-device information and the parameter information to be acquired.
In the second embodiment, the step S20 includes:
step S201: and extracting the sub-equipment type and the sub-equipment serial number from the sub-equipment information, and taking the sub-equipment type, the sub-equipment serial number and the parameter information to be acquired as reference information.
It should be noted that the sub-device type is used to indicate the type of the sub-device, and for example, the sub-device type may be a type such as a CPU, a memory, and a disk.
The serial number of the sub-device is used for representing the serial number value corresponding to the sub-device. For example, the sub-device number corresponding to core1 is 1, and the sub-device number corresponding to core2 is 2.
It should be understood that the extracting the sub-device type and the sub-device serial number from the sub-device information may be extracting the sub-device type and the sub-device serial number from the sub-device information according to the information identifier. Wherein the information identifier is used for identifying the information content.
Step S202: and searching whether a metadata coding group corresponding to the reference information exists in a preset metadata coding table.
It should be noted that the preset metadata encoding table may be preset by a manager of the data acquisition device, or may be generated by the data acquisition device in a self-adaptive manner according to the obtained historical metadata encoding set, which is not limited in this embodiment.
It should be understood that, when the metadata encoding table corresponding to the reference information does not exist in the preset metadata encoding table, it is indicated that the uniform format encoding table corresponding to the reference information is not stored in the preset metadata encoding table, and a uniform format encoding group corresponding to the device to be acquired needs to be separately generated.
When the metadata coding table corresponding to the reference information exists in the preset metadata coding table, the fact that the unified format coding table corresponding to the reference information is stored in the preset metadata coding table is indicated, and a unified format coding group corresponding to the equipment to be acquired does not need to be generated independently.
In the second embodiment, the sub-device type and the sub-device serial number are extracted from the sub-device information, and the sub-device type, the sub-device serial number and the parameter information to be acquired are used as reference information to search whether a metadata coding group corresponding to the reference information exists in a preset metadata coding table, so that the metadata coding group can be searched in the preset metadata coding table by using the sub-device information as the reference, and the accuracy of searching the coding group is improved.
In the second embodiment, the step S40 includes:
step S401: and acquiring the current data of the equipment to be acquired, and establishing the corresponding relation between the current data and the target metadata coding group.
It should be noted that the current data may be a specific value at the current time.
For convenience of understanding, the following description is given by way of example, but the present invention is not limited thereto. Acquiring current data of equipment to be acquired, and establishing a corresponding relation between the current data and a target metadata coding group as follows:
{ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -equipment information- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
"100/0/10001":"sn123",
100/0/10002, X86 blade machine,
"100/0/10003":"adress",
"100/0/10004":"2020-06-29 10:00:00",
-CPU sub-device information-CPU sub-device information-device information-CPU sub-device information-device information-CPU sub-device information-CPU sub-device information-CPU sub-information-CPU sub-device information-CPU sub-information-
"1001/1/10001":"80%",
"1001/1/10002":"4.2GHz",
"1001/1/10003":"2.4GHz",
"1001/2/10001":"70%",
"1001/2/10002":"3.8GHz",
"1001/2/10003":"2.1GHz",
… omit other CPU sub-devices …
-memory sub-device information
"2001/1/20001":"32G",
"2001/1/20002":"76%",
A disc sub-apparatus information
"3001/1/30001":"1.3TB",
"3001/1/30002":"2TB",
"3001/1/30003":"NFS"
}
Step S402: and generating a data storage table based on the current data, the target metadata coding group and the corresponding relation, and storing the data storage table.
It should be understood that generating the data storage table based on the current data, the target metadata encoding set and the corresponding relationship may be generating the data storage table by a preset data processing script based on the current data, the target metadata encoding set and the corresponding relationship.
It should be noted that the preset data processing script may be used to generate the data table.
For convenience of understanding, the following description is given by way of example, but the present invention is not limited thereto. The preset data processing script may be as follows:
CREATE TABLE`cn_message`.`data`(
`id`INT NOT NULL AUTO_INCREMENT,
' URL ' VARCHAR (45) NOT NULL ' - -data collection Point URL
'val' VARCHAR (45) NULL, -data acquisition
"seralNo' VARCHAR (45) NOT NULL-" device Serial number
Time' VARCHAR (45) NOT NULL data acquisition time
PRIMARY KEY(`id`));
For ease of understanding, the description is made with reference to table 1, but this scheme is not limited thereto. Table 1 is a data storage table.
Table 1 data storage table
id
url
val
seralNo
time
1
100/0/10001
sn123
sn123
2020-06-29 10:00:03
2
1001/1/10001
0.8
sn123
2020-06-29 10:00:03
3
1001/1/10002
4.2GHz
sn123
2020-06-29 10:00:03
4
1001/1/10003
2.4GHz
sn123
2020-06-29 10:00:03
5
1001/1/10001
0.7
sn124
2020-06-29 10:00:03
6
1001/1/10002
3.8GHz
sn124
2020-06-29 10:00:03
7
1001/1/10003
2.1GHz
sn124
2020-06-29 10:00:03
The second embodiment collects the current data of the device to be collected, establishes the corresponding relation between the current data and the target metadata coding group, generates a data storage table based on the current data, the target metadata coding group and the corresponding relation, and stores the data storage table, so that the data can be stored in the form of the data table, and a user can conveniently inquire the data.
Referring to fig. 5, fig. 5 is a schematic flow chart of a data acquisition method according to a third embodiment of the present invention, and the data acquisition method according to the third embodiment of the present invention is proposed based on the first embodiment shown in fig. 2.
In the third embodiment, before the step S40, the method further includes:
step S410: and searching a network transmission protocol corresponding to the equipment to be acquired, and carrying out communication protocol adaptation according to the network transmission protocol.
It should be noted that the network transport protocol may be HTTP, TCP, or UDP.
It should be understood that the searching for the network transmission protocol corresponding to the device to be acquired may be searching for the network transmission protocol corresponding to the device to be acquired in a preset protocol table. The preset protocol table comprises the corresponding relation between the equipment to be acquired and the network transmission protocol, and the corresponding relation between the equipment to be acquired and the network transmission protocol can be pre-entered by a manager of the data acquisition equipment.
It should be noted that, the communication protocol adaptation may be to adjust its configuration according to the communication protocol, so as to be able to receive data uploaded by different protocols.
Accordingly, the step S40 includes:
step S40': and after the adaptation is successful, acquiring the current data of the equipment to be acquired, and correspondingly storing the current data and the target metadata coding group.
It can be understood that after the adaptation is successful, the data acquisition device is stated to be capable of supporting the network transmission protocol corresponding to the device to be acquired. Therefore, the current data of the device to be acquired can be directly acquired.
In the third embodiment, the network transmission protocol corresponding to the device to be acquired is searched, the communication protocol adaptation is performed according to the network transmission protocol, after the adaptation is successful, the current data of the device to be acquired is acquired, and the current data and the target metadata coding group are correspondingly stored, so that the communication protocol adaptation can be performed before the data are acquired, and the reliability of data acquisition is ensured.
In the third embodiment, after the step S40, the method further includes:
step S50: when a data query request is received, determining a query metadata code according to the data query request.
It should be noted that the data query request may be input by a user through a user interaction interface of the data acquisition device, or may be input by the user through a terminal device, where the terminal device establishes a communication connection with the data acquisition device in advance, and this embodiment does not limit this.
It should be appreciated that determining the query metadata encoding from the data query request may be extracting the query metadata encoding from the data query request.
Step S60: and searching target data corresponding to the query metadata codes, and aggregating the target data to obtain a data aggregation result.
It should be noted that the aggregation may be based on a preset aggregation rule to count the average CPU utilization rate and the total amount of managed device contents. The preset aggregation rule may be an average value, a maximum value, and a minimum value.
In a particular implementation, for example, statistical CPU usage may be calculated using the following sql:
select avg(val)from data where url='1001/1/10001';
it will be appreciated that aggregation may also be performed periodically or in real time based on data acquired to obtain value information.
In the third embodiment, when a data query request is received, the query metadata code is determined according to the data query request, the target data corresponding to the query metadata code is searched, and the target data is aggregated to obtain a data aggregation result, so that a user can query aggregated data conveniently, and the user can obtain valuable information conveniently.
In addition, an embodiment of the present invention further provides a storage medium, where the storage medium stores a data acquisition program, and the data acquisition program, when executed by a processor, implements the steps of the data acquisition method described above.
In addition, referring to fig. 6, an embodiment of the present invention further provides a data acquisition apparatus, where the data acquisition apparatus includes: the device comprises an information acquisition module 10, a code judgment module 20, a code generation module 30 and a data storage module 40;
the information obtaining module 10 is configured to obtain sub-device information of a device to be collected and parameter information to be collected.
It should be noted that the device to be acquired may be determined by a data acquisition request sent by the terminal device. The terminal device can establish communication connection with the data acquisition device in advance, and a user can initiate a data acquisition request through the terminal device.
The sub-device information of the device to be acquired may be information such as a sub-device type and a sub-device serial number constituting the device to be acquired. For example, when the device to be acquired is a server, the types of the child devices are CPU, memory, disk, and the like.
The parameter information to be acquired of the device to be acquired can be the utilization rate, the highest frequency and the lowest frequency of a CPU, the total memory capacity and the memory utilization rate of a memory, the utilization rate and the total capacity of a disk and a partition format.
It should be understood that the acquiring of the sub-device information of the device to be acquired and the parameter information to be acquired may be acquiring a device model of the device to be acquired, searching for sub-device information corresponding to the device model, and acquiring parameter information to be acquired corresponding to the sub-device.
Further, in consideration of practical application, some users may modify the device, which may result in inaccuracy of the sub-device information determined according to the device model and the parameter information to be acquired. In order to overcome the above defects, the information obtaining module 10 is further configured to perform composition analysis on the device to be acquired, obtain a composition topological graph of the device to be acquired, extract device information from the composition topological graph, and obtain sub-device information and parameter information to be acquired.
The encoding judgment module 20 is configured to judge whether a metadata encoding group corresponding to the device to be acquired exists in a preset metadata encoding table.
It should be noted that the preset metadata encoding table may be preset by a manager of the data acquisition device, or may be generated by the data acquisition device in a self-adaptive manner according to the obtained historical metadata encoding set, which is not limited in this embodiment.
It should be understood that the judgment of whether the metadata encoding group corresponding to the device to be acquired exists in the preset metadata encoding table is to judge whether the uniform format metadata encoding group corresponding to the device to be acquired exists in the preset metadata encoding table, so as to subsequently determine whether an individual target metadata encoding needs to be generated for the device to be acquired.
It can be understood that, the judgment of whether the metadata coding group corresponding to the device to be acquired exists in the preset metadata coding table may be to acquire a device identifier of the device to be acquired, match the device identifier of the device to be acquired with a device identifier corresponding to each metadata coding group in the preset metadata coding table, and when the matching is successful, judge that the metadata coding group corresponding to the device to be acquired exists in the preset metadata coding table; and when the matching fails, judging that the preset metadata coding table does not have a metadata coding group corresponding to the equipment to be collected.
Further, in practical applications, it may be considered that the preset metadata encoding table does not store the metadata encoding group corresponding to the device to be acquired, but stores the metadata encoding group corresponding to the child device of the device to be acquired. In the above manner, only the information of the device to be acquired is considered, and therefore, the situation of data acquisition through the metadata encoding group corresponding to the sub-device cannot be identified. In order to overcome the above defect, the encoding judgment module 20 is further configured to extract the sub-device type and the sub-device serial number from the sub-device information, use the sub-device type, the sub-device serial number, and the parameter information to be acquired as reference information, and search whether a metadata encoding group corresponding to the reference information exists in a preset metadata encoding table.
And the code generating module 30 is configured to generate a target metadata code group corresponding to the device to be acquired according to the sub-device information and the parameter information to be acquired if the target metadata code group is not acquired.
It should be understood that, when the metadata encoding table corresponding to the reference information does not exist in the preset metadata encoding table, it is indicated that the uniform format encoding table corresponding to the device to be acquired is not stored in the preset metadata encoding table, and a uniform format encoding group corresponding to the device to be acquired needs to be separately generated.
It can be understood that the generation of the target metadata encoding group corresponding to the device to be acquired according to the sub-device information and the parameter information to be acquired may be extracting a sub-device type number and a sub-device serial number from the sub-device information, and generating the target metadata encoding group corresponding to the device to be acquired according to the sub-device type number, the sub-device serial number and the parameter information to be acquired. The format of the target metadata encoding group is as follows:
sub-device type number/sub-device serial number/data acquisition parameter
For convenience of understanding, the following description is given by way of example, but the present invention is not limited thereto. The target metadata encoding group is shown below, where- -xxx (e.g., device serial number, device type, etc.) is used to interpret the specific meaning of the target metadata encoding group.
100/0/10001- -equipment Serial number
100/0/10002- -type of equipment
100/0/10003- -device Address
100/0/10004- -current time of device
1001/1/10001- -CPU/core 1/real time value of usage
1001/1/10002- -CPU/core 1/highest frequency
1001/1/10003- -CPU/core 1/lowest frequency
1001/2/10001- -CPU/core 2/real time value of usage
1001/2/10002- -CPU/core 2/highest frequency
1001/2/10003- -CPU/core 2/highest frequency
… omit other CPU sub-devices …
2001/1/20001- - -memory/1/total content capacity
2001/1/20002- -memory/1/content usage
3001/1/30001- -disk/1/utilization real-time value
3001/1/30002- -disk/1/total capacity
3001/1/30003- - -disk/1/partition format
It should be understood that, in this embodiment, a uniform numbering rule is used for each type of sub-device, so that uniform intercommunication of metadata encoding can be achieved, and subsequent data aggregation and number escape are facilitated.
Furthermore, in order to be able to collect the data of the device to be collected when the metadata coding table corresponding to the reference information exists in the preset metadata coding table. The code generating module 30 is further configured to, if yes, take the sub-device information and the parameter information to be collected as target information, and search a metadata code group corresponding to the target information in a preset metadata code table;
and acquiring the current data of the equipment to be acquired, and correspondingly storing the current data and the metadata coding group.
It can be understood that, when the metadata encoding table corresponding to the reference information exists in the preset metadata encoding table, it is indicated that the uniform format encoding table corresponding to the device to be acquired is stored in the preset metadata encoding table, and a uniform format encoding group corresponding to the device to be acquired does not need to be separately generated.
It should be noted that the preset metadata encoding table includes a corresponding relationship between the target information and the metadata encoding table, and the corresponding relationship between the target information and the metadata encoding table may be preset.
The data storage module 40 is configured to collect current data of the device to be collected, and store the current data and the target metadata encoding group correspondingly.
It can be understood that the step of acquiring current data of the device to be acquired and correspondingly storing the current data and the target metadata encoding group may be acquiring the current data of the device to be acquired, establishing a corresponding relationship between the current data and the target metadata encoding group, generating a data storage table based on the current data, the target metadata encoding group and the corresponding relationship, and storing the data storage table.
In this embodiment, it is disclosed that sub-device information and parameter information to be acquired of a device to be acquired are acquired, whether a metadata encoding group corresponding to the device to be acquired exists in a preset metadata encoding table is judged, if not, a target metadata encoding group corresponding to the device to be acquired is generated according to the sub-device information and the parameter information to be acquired, current data of the device to be acquired is acquired, and the current data and the target metadata encoding group are correspondingly stored; compared with the existing mode that different types of data sources are configured with different types of acquisition templates to realize data acquisition of multiple data sources, the embodiment can generate the target metadata encoding group according to the sub-device information of the device to be acquired and the parameter information to be acquired when the metadata encoding group corresponding to the device to be acquired does not exist in the preset metadata encoding table, and correspondingly stores the acquired current data and the target metadata encoding group, so that uniform format collection of different types of data sources is realized, and the data acquisition workload is reduced.
Other embodiments or specific implementation manners of the data acquisition device of the present invention may refer to the above method embodiments, and are not described herein again.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third, etc. do not denote any order, but rather the words first, second, third, etc. are to be interpreted as names.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention or portions thereof that contribute to the prior art may be embodied in the form of a software product, where the computer software product is stored in a storage medium (e.g., a Read Only Memory (ROM)/Random Access Memory (RAM), a magnetic disk, an optical disk), and includes several instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
- 上一篇:石墨接头机器人自动装卡簧、装栓机
- 下一篇:一种城市路网车辆出行轨迹的检索方法及系统