1. A method for synchronizing multimodal data, the method comprising:

scanning a task to be synchronized in a database to be synchronized;

disassembling the task to be synchronized to obtain a task list corresponding to the task to be synchronized, wherein the task list comprises at least one task, and each task comprises at least one data table;

and synchronizing the data in the task list from the database to be synchronized to at least one target database according to a preset sequence.

2. The method for synchronizing multimodal data as claimed in claim 1, wherein the step of synchronizing the data in the task list from the database to be synchronized to at least one target database according to the preset sequence comprises:

synchronizing the tasks in the task list from the database to be synchronized to at least one target database in a serial mode;

and in the process of synchronizing any task in the task list from the database to be synchronized to all the target databases, synchronizing the data in the data table included in the task from the database to be synchronized to all the target databases in a parallel mode.

3. The method for synchronizing multimodal data as claimed in claim 2, wherein before scanning the database to be synchronized for tasks to be synchronized, the method further comprises:

reading configuration items of a database to be synchronized, wherein the configuration items comprise data synchronization frequency and threads, and the configuration items are determined according to the service type and/or data magnitude of the database to be synchronized;

wherein, the scanning the task to be synchronized in the database to be synchronized comprises:

and scanning the tasks to be synchronized in the database to be synchronized at regular time according to the configuration items.

4. The method for synchronizing multimodal data as claimed in claim 3, wherein the step of disassembling the task to be synchronized to obtain a task list corresponding to the task to be synchronized comprises:

disassembling each task in the tasks to be synchronized in a multithreading mode to obtain a task list corresponding to the tasks to be synchronized;

after the task to be synchronized is disassembled to obtain a task list corresponding to the task to be synchronized, the method further includes:

and converting the task list into an SQL statement capable of being executed.

5. A method for synchronization of multimodal data as claimed in any of the claims 1-4, characterized in that the method further comprises:

judging whether a written task in the task list has a corresponding external transaction, wherein the written task comprises tasks synchronized to all the target databases in the task list;

when the written task is judged to have the external affair, calling an interface of the external affair, and executing the external affair;

wherein after the executing the external transaction, the method further comprises:

and marking the written task with a task completion mark for indicating that the external transaction is completed.

6. The method for synchronizing multimodal data as recited in claim 5, wherein the method further comprises:

generating a state mark according to the task synchronization condition in the task list, wherein the state mark comprises the state mark of the task in the task list and the state mark of the data table; and/or the presence of a gas in the gas,

and generating log information according to the task synchronization condition in the task list, wherein the log information comprises the log information of the tasks in the task list and the log information of the data table.

7. The method for synchronizing multimodal data as recited in claim 6, wherein the method further comprises:

acquiring all the state marks corresponding to the task list;

judging whether the task list has tasks and data tables which fail to be executed or not according to all the state marks;

when judging that the tasks and the data tables which are failed to execute exist, determining a compensation task list according to the tasks and the data tables which are failed to execute, wherein the compensation task list comprises at least one target task which is failed to execute, and each target task comprises at least one data table;

and synchronizing the data in the compensation task list from the database to be synchronized to all the target databases according to a preset sequence.

8. An apparatus for synchronizing multimodal data, the apparatus comprising:

the scanning module is used for scanning the tasks to be synchronized in the database to be synchronized;

a disassembling module, configured to disassemble the task to be synchronized to obtain a task list corresponding to the task to be synchronized, where the task list includes at least one task, and each task includes at least one data table;

and the synchronization module is used for synchronizing the data in the task list from the database to be synchronized to at least one target database according to a preset sequence.

9. An apparatus for synchronizing multimodal data, the apparatus comprising:

a memory storing executable program code;

a processor coupled with the memory;

the processor calls the executable program code stored in the memory to perform part or all of the steps of the method of synchronizing multimodal data according to any of claims 1-7.

10. A computer storage medium storing computer instructions which, when invoked, perform some or all of the steps of the method of synchronization of multimodal data as claimed in any of claims 1-7.

Background

With the rapid development of social informatization, a large amount of data information is generated, a database is required to store a large amount of data, and in the management of the database, a requirement for synchronizing the data stored in different databases exists.

Currently, a direct synchronization mode may be used to achieve synchronization between databases, in which data is randomly arranged and then the randomly arranged data is synchronized. However, practice shows that in data synchronization of a database (for example, data synchronization of a multi-model power grid), due to the fact that data models are multiple in types and large in magnitude, ordered synchronization cannot be performed on data by adopting a direct synchronization mode, and finally, data synchronization efficiency is low.

Disclosure of Invention

The technical problem to be solved by the present invention is to provide a method and an apparatus for synchronizing multi-model data, which can achieve ordered synchronization of data in synchronization between databases, thereby improving data synchronization efficiency.

In order to solve the above technical problem, a first aspect of the present invention discloses a method for synchronizing multi-model data, where the method includes:

scanning a task to be synchronized in a database to be synchronized;

disassembling the task to be synchronized to obtain a task list corresponding to the task to be synchronized, wherein the task list comprises at least one task, and each task comprises at least one data table;

and synchronizing the data in the task list from the database to be synchronized to at least one target database according to a preset sequence.

As an optional implementation manner, in the first aspect of the present invention, the synchronizing data in the task list from the database to be synchronized to at least one target database according to a preset order includes:

synchronizing the tasks in the task list from the database to be synchronized to at least one target database in a serial mode;

and in the process of synchronizing any task in the task list from the database to be synchronized to all the target databases, synchronizing the data in the data table included in the task from the database to be synchronized to all the target databases in a parallel mode.

As an optional implementation manner, in the first aspect of the present invention, before the scanning the task to be synchronized in the database to be synchronized, the method further includes:

reading configuration items of a database to be synchronized, wherein the configuration items comprise data synchronization frequency and threads, and the configuration items are determined according to the service type and/or data magnitude of the database to be synchronized;

wherein, the scanning the task to be synchronized in the database to be synchronized comprises:

and scanning the tasks to be synchronized in the database to be synchronized at regular time according to the configuration items.

As an optional implementation manner, in the first aspect of the present invention, the disassembling the task to be synchronized to obtain a task list corresponding to the task to be synchronized includes:

disassembling each task in the tasks to be synchronized in a multithreading mode to obtain a task list corresponding to the tasks to be synchronized;

after the task to be synchronized is disassembled to obtain a task list corresponding to the task to be synchronized, the method further includes:

and converting the task list into an SQL statement capable of being executed.

As an alternative implementation, in the first aspect of the present invention, the method further includes:

judging whether a written task in the task list has a corresponding external transaction, wherein the written task comprises tasks synchronized to all the target databases in the task list;

when the written task is judged to have the external affair, calling an interface of the external affair, and executing the external affair;

wherein after the executing the external transaction, the method further comprises:

and marking the written task with a task completion mark for indicating that the external transaction is completed.

As an alternative implementation, in the first aspect of the present invention, the method further includes:

generating a state mark according to the task synchronization condition in the task list, wherein the state mark comprises the state mark of the task in the task list and the state mark of the data table; and/or the presence of a gas in the gas,

and generating log information according to the task synchronization condition in the task list, wherein the log information comprises the log information of the tasks in the task list and the log information of the data table.

As an alternative implementation, in the first aspect of the present invention, the method further includes:

acquiring all the state marks corresponding to the task list;

judging whether the task list has tasks and data tables which fail to be executed or not according to all the state marks;

when judging that the tasks and the data tables which are failed to execute exist, determining a compensation task list according to the tasks and the data tables which are failed to execute, wherein the compensation task list comprises at least one target task which is failed to execute, and each target task comprises at least one data table;

and synchronizing the data in the compensation task list from the database to be synchronized to all the target databases according to a preset sequence.

The second aspect of the present invention discloses a device for synchronizing multi-model data, wherein the device comprises:

the scanning module is used for scanning the tasks to be synchronized in the database to be synchronized;

a disassembling module, configured to disassemble the task to be synchronized to obtain a task list corresponding to the task to be synchronized, where the task list includes at least one task, and each task includes at least one data table;

and the synchronization module is used for synchronizing the data in the task list from the database to be synchronized to at least one target database according to a preset sequence.

As an optional implementation manner, in the second aspect of the present invention, a manner that the synchronization module synchronizes data in the task list from the database to be synchronized to at least one target database according to a preset sequence is specifically:

synchronizing the tasks in the task list from the database to be synchronized to at least one target database in a serial mode;

and in the process of synchronizing any task in the task list from the database to be synchronized to all the target databases, synchronizing the data in the data table included in the task from the database to be synchronized to all the target databases in a parallel mode.

As an alternative embodiment, in the second aspect of the present invention, the apparatus further comprises:

the device comprises a reading module, a processing module and a processing module, wherein the reading module is used for reading configuration items of a database to be synchronized before the scanning module scans a task to be synchronized in the database to be synchronized, the configuration items comprise data synchronization frequency and threads, and the configuration items are determined according to the service type and/or the data magnitude of the database to be synchronized;

the mode of scanning the task to be synchronized in the database to be synchronized by the scanning module is specifically as follows:

and scanning the tasks to be synchronized in the database to be synchronized at regular time according to the configuration items.

As an optional implementation manner, in the second aspect of the present invention, a manner of the parsing module parsing the task to be synchronized to obtain the task list corresponding to the task to be synchronized is specifically:

disassembling each task in the tasks to be synchronized in a multithreading mode to obtain a task list corresponding to the tasks to be synchronized;

wherein the apparatus further comprises:

and the conversion module is used for converting the task list into an SQL statement which can be executed after the disassembly module disassembles the task to be synchronized to obtain the task list corresponding to the task to be synchronized.

As an alternative embodiment, in the second aspect of the present invention, the apparatus further comprises:

the judging module is used for judging whether corresponding external affairs exist in written tasks in the task list, wherein the written tasks comprise tasks which are synchronized to all the target databases in the task list;

the calling execution module is used for calling an interface of the external transaction to execute the external transaction when the judging module judges that the written task has the external transaction;

wherein the apparatus further comprises:

and the mark generation module is used for marking the written task as a task completion mark for indicating that the external transaction is completed after the external transaction is executed by the call execution module.

As an optional implementation manner, in the second aspect of the present invention, the flag generating module is further configured to generate a status flag according to a task synchronization condition in the task list, where the status flag includes a status flag of a task in the task list and a status flag of a data table; and/or, the device further comprises:

and the log generation module is used for generating log information according to the task synchronization condition in the task list, wherein the log information comprises the log information of the tasks in the task list and the log information of the data table.

As an alternative embodiment, in the second aspect of the present invention, the apparatus further comprises:

the acquisition module is used for acquiring all the state marks corresponding to the task list;

the judging module is also used for judging whether the task list has tasks and data tables which fail to be executed according to all the state marks;

the determining module is used for determining a compensation task list according to the tasks and the data tables which fail to be executed when the judging module judges that the tasks and the data tables which fail to be executed exist, wherein the compensation task list comprises at least one target task which fails to be executed, and each target task comprises at least one data table;

and the synchronization module is further used for synchronizing the data in the compensation task list from the database to be synchronized to all the target databases according to a preset sequence.

The third aspect of the present invention discloses another synchronization apparatus for multi-model data, the apparatus comprising:

a memory storing executable program code;

a processor coupled with the memory;

the processor calls the executable program code stored in the memory to execute part or all of the steps in the multi-model data synchronization method disclosed by the first aspect of the invention.

In a fourth aspect of the present invention, a computer storage medium is disclosed, which stores computer instructions for performing some or all of the steps of the multi-model data synchronization method disclosed in the first aspect of the present invention when the computer instructions are called.

Compared with the prior art, the embodiment of the invention has the following beneficial effects:

in the embodiment of the invention, a task to be synchronized in a database to be synchronized is scanned; disassembling the task to be synchronized to obtain a task list corresponding to the task to be synchronized, wherein the task list comprises at least one task, and each task comprises at least one data table; and synchronizing the data in the task list from the database to be synchronized to at least one target database according to a preset sequence. Therefore, by implementing the embodiment of the invention, the tasks to be synchronized in the databases to be synchronized can be scanned, the tasks to be synchronized are disassembled, and the task list at the data table level is obtained.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.

FIG. 1 is a flow chart of a method for synchronizing multi-model data according to an embodiment of the present invention;

FIG. 2 is a flow chart illustrating another method for synchronizing multi-model data according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a dual database bi-directional synchronization disclosed in the embodiments of the present invention;

FIG. 4 is a schematic diagram of another one-to-many synchronization of multiple databases as disclosed in embodiments of the present invention;

FIG. 5 is a schematic structural diagram of a synchronization apparatus for multi-model data according to an embodiment of the present invention;

FIG. 6 is a schematic structural diagram of another synchronization apparatus for multi-model data according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of another multi-model data synchronization apparatus according to an embodiment of the present invention.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The terms "first," "second," and the like in the description and claims of the present invention and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or apparatus that comprises a list of steps or elements is not limited to those listed but may alternatively include other steps or elements not listed or inherent to such process, method, product, or apparatus.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.

The embodiment of the invention discloses a method and a device for synchronizing multi-model data, which can firstly scan tasks to be synchronized in a database to be synchronized, disassemble the tasks to be synchronized to obtain a task list of a data table level, and synchronize data in the task list of the data table level from the database to be synchronized to at least one target database according to a preset sequence, so that the data can be orderly synchronized in the synchronization between the databases, and the data synchronization efficiency is improved. The following are detailed below.

Example one

Referring to fig. 1, fig. 1 is a schematic flowchart illustrating a multi-model data synchronization method according to an embodiment of the present invention. As shown in fig. 1, the synchronization method of multi-model data may include the following operations:

101. and scanning the tasks to be synchronized in the database to be synchronized.

In the embodiment of the present invention, the number of the databases to be synchronized is one, the task to be synchronized in the databases to be synchronized is incremental data in the databases to be synchronized, and the incremental data may be understood as: compared with the data synchronized in the database to be synchronized last time, the incremental data of the data to be synchronized that changes in the database to be synchronized may include increased data, decreased data, modified data, and other incremental data, and the embodiment of the present invention is not limited.

In the embodiment of the invention, the database to be synchronized is connected with the target database through JDBC + mybatis technology.

102. And disassembling the tasks to be synchronized to obtain a task list corresponding to the tasks to be synchronized, wherein the task list comprises at least one task, and each task comprises at least one data table.

In the embodiment of the invention, the task to be synchronized is disassembled to obtain the task list at the data table level, the task list comprises one task or a plurality of tasks, each task is disassembled into one data table or a plurality of data tables, and the task list at the data table level is more finely and regularly disassembled for effectively clustering the data in the task list. When the incremental data magnitude of the database to be synchronized is large, the task list comprises a large number of tasks, and each task comprises a large number of data tables; when the incremental data size of the database to be synchronized is small, the task list may include a task, and the task may include only a data table.

103. And synchronizing the data in the task list from the database to be synchronized to at least one target database according to a preset sequence.

In the embodiment of the present invention, data in a task list at a data table level in a database to be synchronized is synchronized into a target database according to a preset sequence, where the preset sequence may be a data modification time sequence, a data type sequence, a data size sequence, or the like, and may also be another preset sequence capable of implementing ordered data synchronization, and the embodiment of the present invention is not limited.

In the embodiment of the present invention, as shown in fig. 3, fig. 3 is a schematic diagram of bidirectional synchronization of two databases, in the bidirectional synchronization of the two databases, one target database is used, a task to be synchronized of the database to be synchronized may be synchronized to the target database, and a task to be synchronized in the target database may also be synchronized to the database to be synchronized, so as to implement bidirectional data synchronization between the databases.

In the embodiment of the present invention, as shown in fig. 4, fig. 4 is a schematic diagram of one-to-many synchronization of multiple databases, where in the one-to-many synchronization of multiple databases, multiple target databases are provided, and tasks to be synchronized of the databases to be synchronized can be synchronized into multiple target databases at the same time, so that one database is synchronized with multiple target databases at the same time.

In the embodiment of the invention, for the whole process of the multi-model data synchronization, by taking a multi-model power grid database as an example, wherein, the database to be synchronized is a power grid database 1, the target database is a power grid database 2, in the data synchronization, the task to be synchronized in the power grid database 1 is firstly scanned, the incremental data in the task to be synchronized in the power grid database 1 is disassembled to obtain a task list of data list level corresponding to the incremental data in the task to be synchronized, wherein, the task list of the power grid database 1 comprises 100 tasks, each task in the 100 tasks comprises data tables with the number more than or equal to 1000, and synchronizing the tasks to be synchronized in the power grid database 1 to the power grid database 2 according to the data volume sequence of 100 tasks in the task list, so that the data in the power grid database 1 is synchronized to the power grid database 2. After the data in the electric network database 1 is synchronized to the electric network database 2, the data in the electric network database 2 can be synchronized to the electric network database 1, similarly, the task to be synchronized in the electric network database 2 is scanned first, the incremental data in the task to be synchronized in the electric network database 2 is disassembled, and a task list of a data table level corresponding to the incremental data in the task to be synchronized is obtained, wherein the task list of the electric network database 2 comprises 200 tasks, each task in the 200 tasks comprises data tables with the number greater than or equal to 1000, and the tasks to be synchronized in the electric network database 2 are synchronized to the electric network database 1 according to the data quantity sequence of the 200 tasks in the task list, so that bidirectional data synchronization between the electric network database 1 and the electric network database 2 containing hundreds of millions of data quantities and complex models is realized. It is worth mentioning that the bidirectional data synchronization process between the grid database 1 and the grid database 2 may be performed simultaneously.

In the embodiment of the invention, for the whole process of the multi-model data synchronization, a multi-model power grid database is used, for example, wherein a database to be synchronized is a power grid database 1, a target database is a power grid database 2, a power grid database 3 and a power grid database 4, in the data synchronization, a task to be synchronized in the power grid database 1 is scanned first, incremental data in the task to be synchronized in the power grid database 1 is disassembled, a task list of data list levels corresponding to the incremental data in the task to be synchronized is obtained, wherein the task list of the power grid database 1 comprises 100 tasks, each task in the 100 tasks comprises data lists with the number greater than or equal to 1000, the tasks to be synchronized in the power grid database 1 are synchronized into the power grid database 2, the power grid database 3 and the power grid database 4 according to the data quantity sequence of the 100 tasks in the task list, thereby realizing one-to-many data synchronization between the power grid database 1 containing hundred million-level data volume and a complex model and a plurality of power grid databases 2, 3 and 4.

It can be seen that, by implementing the multi-model data synchronization method described in the embodiments of the present invention, a task list at a data table level can be obtained by disassembling tasks to be synchronized in a database to be synchronized, and the task list at the data table level is more finely and regularly disassembled to data in the tasks to be synchronized, so that data in the task list is effectively clustered, and then the data in the task list at the data table level is synchronized into at least one target database according to a preset sequence, so that the data can be sequentially synchronized in synchronization between the databases, and thus, the data synchronization efficiency is improved.

Example two

Referring to fig. 2, fig. 2 is a schematic flowchart illustrating another method for synchronizing multi-model data according to an embodiment of the present invention. As shown in fig. 2, the synchronization method of multi-model data may include the following operations:

201. reading configuration items of the database to be synchronized, wherein the configuration items comprise data synchronization frequency and threads, and the configuration items are determined according to the service type and/or the data magnitude of the database to be synchronized.

202. And scanning the tasks to be synchronized in the database to be synchronized at regular time according to the configuration items.

In the embodiment of the invention, the task to be synchronized in the database to be synchronized can be scanned at regular time through the timer according to the configuration items of the database to be synchronized, the timer adopts a quartz timing task frame, the timing scanning frequency is determined according to the set data so as to scan the task to be synchronized in the database to be synchronized at regular time, the timing scanning frequency is determined through the configuration items so as to scan the task to be synchronized in the database to be synchronized, and the data synchronization efficiency can be further improved.

203. And disassembling each task in the tasks to be synchronized in a multithreading mode to obtain a task list corresponding to the tasks to be synchronized, wherein the task list comprises at least one task, and each task comprises at least one data table.

In the embodiment of the invention, during the disassembly of the tasks to be synchronized in the database to be synchronized, each task is disassembled one by one in a multithreading mode to form the task list at the data table level, the task list at the data table level is disassembled more finely and regularly to enable the data in the task list to be effectively clustered, and the task disassembly is carried out in the multithreading mode, so that the disassembly efficiency of the tasks to be synchronized in the database to be synchronized can be effectively improved.

In an optional embodiment, after obtaining the task list corresponding to the task to be synchronized, the method may further include the following operations:

the task list manifest is converted into SQL statements that can be executed.

In this alternative embodiment, the SQL statement is a structured query language, which is a database query and programming language for accessing data and querying, updating, and managing a relational database system.

204. And synchronizing the tasks in the task list from the database to be synchronized to at least one target database according to a serial mode, and synchronizing the data in the data table included in the task from the database to be synchronized to all the target databases according to a parallel mode in the process of synchronizing any task in the task list from the database to be synchronized to all the target databases.

In the embodiment of the invention, in the synchronization of the task list in the database to be synchronized, a plurality of tasks in the task list are synchronized in a serial mode, namely, the task 1, the task 2, the task 3 and all the tasks in the task list are synchronized in sequence, and in the synchronization of each task, the data in the data table included in the task are synchronized to all target databases from the database to be synchronized in a parallel mode, namely, the data table 1, the data table 2, the data table 3 in the task and all the data tables under the task are synchronized simultaneously.

In the embodiment of the invention, for the whole process of the multi-model data synchronization, by taking a multi-model power grid database as an example, wherein, the database to be synchronized is a power grid database 1, the target database is a power grid database 2, in the data synchronization, the configuration items of the power grid database 1 are read firstly, setting data synchronization frequency according to the result of the configuration item, scanning the task to be synchronized in the power grid database 1 at regular time through a timer, disassembling the incremental data in the task to be synchronized in the power grid database 1 to obtain a task list of a data list level corresponding to the incremental data in the task to be synchronized, wherein, the task list of the power grid database 1 comprises 100 tasks, each task in the 100 tasks comprises data tables with the number more than or equal to 1000, the 100 tasks in the task list are synchronized in sequence from the grid database 1 to the grid database 2 in a serial manner. In the synchronization of each task of the 100 tasks, the data tables with the number of more than or equal to 1000 included in each task are synchronized simultaneously in a parallel mode, so that the data of the power grid database 1 is synchronized to the power grid database 2. After the data in the electric network database 1 is synchronized to the electric network database 2, the data in the electric network database 2 can be synchronized to the electric network database 1, similarly, the configuration items in the electric network database 2 are read first, the data synchronization frequency is set according to the result of the configuration items, the tasks to be synchronized in the electric network database 2 are scanned at regular time through a timer, the incremental data in the tasks to be synchronized in the electric network database 2 are disassembled, a task list of data table levels corresponding to the incremental data in the tasks to be synchronized is obtained, wherein the task list in the electric network database 2 comprises 200 tasks, each task in the 200 tasks comprises more than or equal to 1000 data tables, the 200 tasks in the task list are sequentially synchronized from the electric network database 2 to the electric network database 1 in a serial mode, and in the synchronization of each task in the 200 tasks, and simultaneously synchronizing the data tables with the number more than or equal to 1000 included in each task in a parallel mode, thereby realizing bidirectional data synchronization between the power grid database 1 and the power grid database 2 which contain hundred million-level data volumes and complex models. It is worth mentioning that the bidirectional data synchronization process between the grid database 1 and the grid database 2 may be performed simultaneously.

In the embodiment of the invention, for the whole process of the multi-model data synchronization, a multi-model power grid database is used, for example, wherein the database to be synchronized is a power grid database 1, the target database is a power grid database 2, a power grid database 3 and a power grid database 4, in the data synchronization, configuration items of the power grid database 1 are read first, a data synchronization frequency is set according to the result of the configuration items, the tasks to be synchronized in the power grid database 1 are scanned regularly by a timer, incremental data in the tasks to be synchronized in the power grid database 1 are disassembled to obtain a task list of a data list level corresponding to the incremental data in the tasks to be synchronized, wherein the task list of the power grid database 1 comprises 100 tasks, each task in the 100 tasks comprises data lists with the number of more than or equal to 1000, and the 100 tasks in the task list are sequentially synchronized from the power grid database 1 to the power grid database 2, In the grid database 3 and the grid database 4, for synchronization of each of the 100 tasks, data tables of which the number is greater than or equal to 1000 included in each task are simultaneously synchronized in a parallel manner to synchronize data of the grid database 1 into the grid database 2, the grid database 3 and the grid database 4, thereby achieving one-to-many data synchronization between the grid database 1 and a plurality of grid databases 2, 3 and 4, which include billions of data volumes and complex models.

It can be seen that, by implementing the multi-model data synchronization method described in the embodiments of the present invention, the configuration items of the database to be synchronized are read first to scan the tasks to be synchronized in the database to be synchronized at regular time, and the tasks to be synchronized in the database to be synchronized are disassembled in a multi-thread manner, so that the data disassembly efficiency can be improved.

In another optional embodiment, the method may further comprise the operations of:

judging whether the written tasks in the task list have corresponding external transactions, wherein the written tasks comprise tasks which are synchronized to all target databases in the task list;

when judging that the written task has the external affair, calling an interface of the external affair and executing the external affair;

wherein, after executing the external transaction, the method further comprises:

the written task flag is used to indicate that the external transaction is completed.

In this alternative embodiment, the external transaction includes an abort, a suspension, a check, and the like of the task synchronization in the task list, and also includes other external transactions related to the task synchronization, which is not limited in the present invention.

In this optional embodiment, when the external transaction is an abort transaction that synchronizes tasks in the task list, the abort transaction is triggered for the written task, an interface of the abort transaction is called, the abort transaction is executed, the abort of the synchronized data is completed, and after the abort transaction is completed, the written task is marked, so that the state of the task can be obtained through the marking information.

In this optional embodiment, when the external transaction is a suspension transaction for synchronizing the tasks in the task list, the suspension transaction is triggered for the written task, an interface of the suspension transaction is called, the suspension transaction is executed, the task synchronization is automatically performed after a preset time period of suspension, and after the suspension transaction is completed, the written task is marked, so that the state of the task can be acquired through the marking information.

In this optional embodiment, when the external transaction is a verification transaction for synchronizing tasks in the task list, the verification transaction is triggered for the written task, an interface of the verification transaction is called, the verification transaction is executed, verification of the synchronized data is completed, and after the verification transaction is completed, the written task is marked, so that the state of the task can be acquired through the marking information.

Therefore, in the optional embodiment, when the written task in the task list has the external transaction, the external transaction interface is called, the external transaction is executed, and by setting the external transaction, the expansion requirement (such as suspension, verification and the like on the synchronization task) in the data synchronization process can be realized, so that the convenience of data synchronization is improved.

In yet another alternative embodiment, the method may further include the operations of:

generating state marks according to task synchronization conditions in the task list, wherein the state marks comprise state marks of tasks in the task list and state marks of the data table; and/or the presence of a gas in the gas,

and generating log information according to the task synchronization condition in the task list, wherein the log information comprises the log information of the tasks in the task list and the log information of the data table.

In this optional embodiment, the synchronous operation management of the database is implemented by monitoring information in the data synchronization process of the database, where the monitored information includes basic information and log information of task synchronization in the task list, and also includes other information in the task synchronization process in the task list, which is not limited in the present invention.

In this optional embodiment, the monitoring information in the data synchronization process may be displayed at the front end, where the displayed pages include a synchronization state display page and a synchronization log query page, and the front end may be a display panel based on Spring MVC development, and the monitoring information in the data synchronization process is displayed on the display panel.

Therefore, the optional embodiment can monitor the information in the task synchronization process in the task list, and can realize the visual presentation and the timely display of data synchronization by performing front-end display on the monitoring information, and can monitor and perform exception handling on the data synchronization state, thereby improving the management and control efficiency of data synchronization.

In yet another alternative embodiment, the method may further include the operations of:

acquiring all state marks corresponding to the task list;

judging whether the task list has tasks and data tables which fail to be executed or not according to all the state marks;

when judging that the tasks and the data tables which are failed to execute exist, determining a compensation task list according to the tasks and the data tables which are failed to execute, wherein the compensation task list comprises at least one target task which is failed to execute, and each target task comprises at least one data table;

and synchronizing the data in the compensation task list from the database to be synchronized to all target databases according to a preset sequence.

In this optional embodiment, a task and a data table which are failed to be executed are obtained through all status flags corresponding to the task list, and a compensation task list is formed again through the task and the data table which are failed to be executed, for example, a multi-model power grid database is used, wherein a database to be synchronized is a power grid database 1, a target database is a power grid database 2, during data synchronization, all status flags corresponding to the task list are inquired, the task and the data table which are failed to be executed are obtained as a data table 19 in the task 1, a data table 3 and a data table 58 in the task 9, and a data table 132 and a data table 786 in a task 76, and then the task and the data table which are failed to be executed are combined into a compensation task list, and data in the compensation task list are resynchronized according to a preset order.

Therefore, in the optional embodiment, the compensation task list can be formed again for the task and the data table which are failed to execute through all the state marks corresponding to the obtained task list, and the data synchronization can be performed again for the data, so that the integrity of the data synchronization can be improved.

EXAMPLE III

Referring to fig. 5, fig. 5 is a schematic structural diagram of a synchronization apparatus for multi-model data according to an embodiment of the present invention. As shown in fig. 5, the synchronization apparatus of multi-model data may include:

the scanning module 301 is configured to scan a task to be synchronized in a database to be synchronized;

a disassembling module 302, configured to disassemble a task to be synchronized to obtain a task list corresponding to the task to be synchronized, where the task list includes at least one task, and each task includes at least one data table;

a synchronization module 303, configured to synchronize data in the task list from the database to be synchronized to at least one target database according to a preset sequence.

It can be seen that, by implementing the apparatus described in fig. 5, the task to be synchronized in the database to be synchronized can be disassembled to obtain the task list at the data table level, and the task list at the data table level is more finely and regularly disassembled to the data in the task to be synchronized, so that the data in the task list is effectively clustered, and then the data in the task list at the data table level is synchronized to at least one target database according to the preset sequence, so that the data can be sequentially synchronized in the synchronization between the databases, and the data synchronization efficiency is improved.

In an optional embodiment, the manner for the synchronization module 303 to synchronize the data in the task list from the database to be synchronized to the at least one target database according to the preset sequence is specifically that:

synchronizing the tasks in the task list from the database to be synchronized to at least one target database in a serial mode;

in the process of synchronizing any task in the task list from the database to be synchronized to all target databases, synchronizing data in the data table included in the task from the database to be synchronized to all target databases in a parallel mode.

In an alternative embodiment, as shown in fig. 6, the apparatus may further include:

a reading module 304, configured to read configuration items of the database to be synchronized before the scanning module 301 scans the task to be synchronized in the database to be synchronized, where the configuration items include data synchronization frequency and threads, and the configuration items are determined according to the service type and/or data magnitude of the database to be synchronized;

the mode of the scanning module 301 scanning the task to be synchronized in the database to be synchronized specifically is as follows:

and scanning the tasks to be synchronized in the database to be synchronized at regular time according to the configuration items.

It can be seen that, implementing the apparatus described in fig. 6 can determine the timing scanning frequency through the configuration item, and scan the tasks to be synchronized in the database to be synchronized at the timing scanning frequency, so as to further improve the data synchronization efficiency.

In an alternative embodiment, as shown in fig. 6, fig. 6 is a schematic structural diagram of another synchronization apparatus for multi-model data disclosed in the embodiment of the present invention.

The method for disassembling the task to be synchronized by the disassembling module 302 to obtain the task list corresponding to the task to be synchronized specifically includes:

disassembling each task in the tasks to be synchronized in a multithreading mode to obtain a task list corresponding to the tasks to be synchronized;

wherein, the device still includes:

the conversion module 305 is configured to, after the disassembling module 302 disassembles the task to be synchronized to obtain a task list corresponding to the task to be synchronized, convert the task list into an SQL statement that can be executed.

It can be seen that, by implementing the apparatus described in fig. 6, during the disassembly of the tasks to be synchronized in the database to be synchronized, each task can be disassembled one by one in a multithreading manner, so as to form a task list at a data table level, and the task disassembly is performed in a multithreading manner, so that the disassembly efficiency of the tasks to be synchronized in the database to be synchronized can be effectively improved.

In an alternative embodiment, as shown in fig. 6, the apparatus may further include:

the judging module 306 is configured to judge whether a written task in the task list has a corresponding external transaction, where the written task includes tasks synchronized to all target databases in the task list;

a calling execution module 307, configured to call an interface of an external transaction to execute the external transaction when the determination module 306 determines that the written task has the external transaction;

wherein, the device still includes:

a flag generating module 308, configured to flag the written task with a task completion flag indicating that the external transaction is completed after the external transaction is executed by the call executing module 307.

It can be seen that, when there is an external transaction in the written task in the task list, the device described in fig. 6 can call the external transaction interface and execute the external transaction, and by setting the external transaction, the extended requirements (such as suspension, verification, and the like for the synchronization task) in the data synchronization process can be realized, thereby improving the convenience of data synchronization.

In an alternative embodiment, as shown in fig. 6, the flag generating module 308 is further configured to generate a status flag according to a task synchronization condition in the task list, where the status flag includes a status flag of a task in the task list and a status flag of a data table; and/or, the apparatus further comprises:

the log generating module 309 is configured to generate log information according to a task synchronization condition in the task list, where the log information includes log information of a task in the task list and log information of the data table.

Therefore, the device described in fig. 6 can monitor information in the task synchronization process in the task list, and perform front-end display on the monitored information, so that intuitive presentation and timely display of data synchronization are realized, and the management and control efficiency of data synchronization can be improved.

In an alternative embodiment, as shown in fig. 6, the apparatus may further include:

an obtaining module 310, configured to obtain all status flags corresponding to the task list;

the judging module 306 is further configured to judge whether the task list has a task and a data table that are executed unsuccessfully according to all the status flags;

a determining module 311, configured to determine, when the determining module 306 determines that the task and the data table which have failed to be executed exist, a compensation task list according to the task and the data table which have failed to be executed, where the compensation task list includes at least one target task which has failed to be executed, and each target task includes at least one data table;

the synchronization module 303 is further configured to synchronize data in the compensation task list from the databases to be synchronized to all target databases according to a preset sequence.

Therefore, the device described in fig. 6 can reform a compensation task list for the task and the data table that have failed to execute through all the status flags corresponding to the obtained task list, and perform data synchronization again for the data, so as to improve the integrity of the data synchronization.

Example four

Referring to fig. 7, fig. 7 is a schematic structural diagram of another multi-model data synchronization apparatus according to an embodiment of the present invention. As shown in fig. 7, the apparatus may include:

a memory 401 storing executable program code;

a processor 402 coupled with the memory 401;

the processor 402 calls the executable program code stored in the memory 401 for performing the steps in the synchronization method of multimodal data as described in the first embodiment or the second embodiment.

EXAMPLE five

The embodiment of the invention discloses a computer-readable storage medium which stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps in the multi-model data synchronization method described in the first embodiment or the second embodiment.

EXAMPLE six

The embodiment of the invention discloses a computer program product, which comprises a non-transitory computer readable storage medium storing a computer program, and the computer program is operable to make a computer execute the steps in the synchronization method of multi-model data described in the first embodiment or the second embodiment.

The above-described embodiments of the apparatus are merely illustrative, and the modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

Through the above detailed description of the embodiments, those skilled in the art will clearly understand that the embodiments may be implemented by software plus a necessary general hardware platform, and may also be implemented by hardware. Based on such understanding, the above technical solutions may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, where the storage medium includes a Read-Only Memory (ROM), a Random Access Memory (RAM), a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), a One-time Programmable Read-Only Memory (OTPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Compact Disc-Read-Only Memory (CD-ROM), or other disk memories, CD-ROMs, or other magnetic disks, A tape memory, or any other medium readable by a computer that can be used to carry or store data.

Finally, it should be noted that: the method and apparatus for synchronizing multi-model data disclosed in the embodiments of the present invention are only disclosed as preferred embodiments of the present invention, and are only used for illustrating the technical solutions of the present invention, rather than limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those skilled in the art; the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the spirit and scope of the technical solutions of the embodiments of the present invention.