1. A task scheduling method is applied to a task scheduling system, and the task scheduling system comprises: the system comprises a task initiating terminal, a load server and a plurality of task executing terminals; the scheduling method comprises the following steps:

the load server receives a data receiving request aiming at a target task, which is sent by the task initiating terminal; the target task is used for representing a modeling task for constructing a target model by using a plurality of different types of target training data; the data receiving request at least comprises: the task identification of the target task and the data identification of each target training data;

the load server determines a target task execution terminal aiming at the target task from the plurality of task execution terminals according to a pre-stored task processing mapping table; wherein, the task processing mapping table at least comprises: task identifiers of a plurality of different tasks, data identifiers of a plurality of training data corresponding to each task, and communication port addresses corresponding to each training data; the target training data stored on different target task execution terminals are different;

the load server sends a data transmission request aiming at each target task execution terminal to each target task execution terminal and receives a target data processing result sent by the target task execution terminal; the target data processing result is used for representing a data processing result corresponding to the target training data corresponding to the target task execution terminal under the target task;

and the load server takes each received target data processing result as a task execution result aiming at the target task and sends the task execution result to the task initiating terminal.

2. The scheduling method according to claim 1, wherein before the load server receives a data reception request for a target task sent by the task initiating terminal, the scheduling method further comprises:

the task initiating terminal responds to the starting operation aiming at the target task, and determines a task executing terminal where each target training data is located from the plurality of task executing terminals as a first task executing terminal corresponding to the target training data;

the task initiating terminal generates a task creating request corresponding to each first task execution terminal aiming at each first task execution terminal, and sends the task creating request to the first task execution terminal; wherein the task creation request includes: the task identification of the target task, the terminal identification of the first task execution terminal, the data identification of the target training data corresponding to the first task execution terminal and the data processing rule of the target training data under the target task;

after each first task execution terminal receives the task creation request, starting a target container for the target task, processing the target training data stored on the first task execution terminal through the target container, and sending container port information of the target container to the load server; wherein the container port information at least includes: the task identifier of the target task, the container port address of the target container and the data identifier of the target training data processed by the target container;

and after receiving the container port information sent by each target container, the load server stores the container port information into the task processing mapping table by taking a container port address in the container port information as a communication port address corresponding to the target training data processed by the target container.

3. The scheduling method according to claim 2, wherein the starting a target container for the target task, through which the target training data stored on the first task execution terminal is processed, comprises:

the first task execution terminal responds to the received task creation request, and generates a virtual container starting instruction aiming at the target task according to the task identifier of the target task included in the task creation request;

generating a virtual container aiming at the target task according to the virtual container starting instruction, and taking the generated virtual container as the target container;

storing the target training data stored on the first task execution terminal and the corresponding environment dependence information of the target training data under the target task into the target container in a packaging mirror image mode; the environment dependence information is used for representing a data operation environment corresponding to the target training data under the target task;

and in the target container, performing data processing on the target training data according to the data processing rule of the target training data under the target task to obtain the target data processing result corresponding to the target training data.

4. The scheduling method according to claim 2, wherein the determining a target task execution terminal for the target task from the plurality of task execution terminals comprises:

the load server determines a first task identifier from the task identifiers of the multiple different tasks included in the task processing mapping table according to the received task identifier of the target task, and takes the position of the first task identifier in the task processing mapping table as a first query area; the first task identifier is a task identifier with the highest matching degree with the task identifier of the target task in the task identifiers of the different tasks; the data identification of a plurality of training data corresponding to the first task identification, and the communication port address of each training data are all located in the first query area;

for each target training data, determining the data identifier of the training data with the highest matching degree with the data identifier of the target training data from the data identifiers of the training data in the first query area as a first data identifier;

and determining the target task execution terminal corresponding to the target training data according to the communication port address of the training data corresponding to the first data identifier.

5. The scheduling method according to claim 1, wherein when the task initiating terminal and each of the task executing terminals respectively correspond to different data encryption rules and data decryption rules, the data receiving request further includes: the data encryption rule and the data decryption rule on the task initiating terminal; each target task execution terminal sends a first encryption result of the target data processing result to the load server; the first encryption result is obtained after each target task execution terminal encrypts the target data processing result according to the data encryption rule on the target task execution terminal; the task processing mapping table further comprises: and each task executes a data encryption rule and a data decryption rule on the terminal.

6. The scheduling method according to claim 5, wherein the sending a data transmission request for each target task execution terminal to the target task execution terminal and receiving a target data processing result sent by the target task execution terminal comprises:

the load server determines a data encryption rule corresponding to each target task execution terminal according to the task processing mapping table and sends the data transmission request aiming at the target task execution terminal to the target task execution terminal according to the data encryption rule; the load server receives the first encryption result sent by the target task execution terminal aiming at the data transmission request;

and the load server determines a data decryption rule on the target task execution terminal according to the task processing mapping table, and decrypts the first encryption result according to the data decryption rule on the target task execution terminal to obtain the target data processing result.

7. The scheduling method according to claim 6, wherein the taking each received target data processing result as a task execution result for the target task and sending the task execution result to the task initiating terminal includes:

the load server determines a data encryption rule on the task initiating terminal according to the received data receiving request;

the load server encrypts each target data processing result according to a data encryption rule on the task initiating terminal to obtain a second encryption result of each target data processing result;

and the load server sends each second encryption result as the task execution result to the task initiating terminal.

8. A task scheduling apparatus, applied to a task scheduling system, the task scheduling system comprising: the system comprises a task initiating terminal, a load server and a plurality of task executing terminals; the scheduling device comprises:

a request receiving module, configured to receive, by the load server, a data receiving request for a target task sent by the task initiating terminal; the target task is used for representing a modeling task for constructing a target model by using a plurality of different types of target training data; the data receiving request at least comprises: the task identification of the target task and the data identification of each target training data;

a terminal determining module, configured to determine, by the load server, a target task execution terminal for the target task from the multiple task execution terminals according to a pre-stored task processing mapping table; wherein, the task processing mapping table at least comprises: task identifiers of a plurality of different tasks, data identifiers of a plurality of training data corresponding to each task, and communication port addresses corresponding to each training data; the target training data stored on different target task execution terminals are different;

the first transmission module is used for sending a data transmission request aiming at each target task execution terminal to each target task execution terminal through the load server and receiving a target data processing result sent by the target task execution terminal; the target data processing result is used for representing a data processing result corresponding to the target training data corresponding to the target task execution terminal under the target task;

and the second transmission module is used for taking each received target data processing result as a task execution result aiming at the target task through the load server and sending the task execution result to the task initiating terminal.

9. An electronic device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating over the bus when the electronic device is operating, the machine-readable instructions when executed by the processor performing the steps of the scheduling method of any of claims 1 to 7.

10. A computer-readable storage medium, having stored thereon a computer program for performing, when executed by a processor, the steps of the scheduling method according to any one of claims 1 to 7.

Background

With the continuous development of artificial intelligence technology, when the target technical requirements are met, more and more technicians select a machine learning mode and train the artificial intelligence model by using training data related to the target technical requirements, so that the target technical requirements are met by using the trained artificial intelligence model. Specifically, in the actual modeling process, based on the confidentiality characteristic of data, an enterprise or a department cannot place own unique training data in an open source platform which can be freely acquired by the public, so that most of the training data required by model training is distributed in different enterprises or departments in a data isolated island mode. Therefore, how to effectively realize the sharing of training data in the actual modeling process becomes a technical problem which needs to be solved urgently in the current machine learning technical field.

In the current method, technicians usually adopt a federal learning mode to allocate an external communication port for a current modeling task, determine the communication port address of an enterprise or department where training data is located according to each training data required under the modeling task, and then perform data communication according to a traditional producer consumer mode. Specifically, in a traditional producer consumer mode, the external communication port corresponds to a producer, an enterprise or department where each piece of training data is located corresponds to a consumer, the producer generates a data communication task according to each piece of training data to be interacted and a communication port address of the enterprise or department where the training data is located, then stores all the generated data communication tasks in the cache region, and waits for each consumer to extract the corresponding data communication task from the cache region. Therefore, on one hand, a producer needs to be docked with the communication ports of a plurality of consumers at the same time, and the producer is difficult to manage the external communication ports due to different safety control measures of the communication ports by different consumers; on the other hand, the same consumer may need to extract multiple data communication tasks in the cache region, at this time, the extracted multiple data communication tasks are located under the same task extraction queue of the consumer, and if an error occurs in one of the data communication tasks, the entire task extraction queue of the consumer may be abnormal, which may cause a reduction in task scheduling efficiency.

Disclosure of Invention

In view of this, an object of the present application is to provide a task scheduling method, device, apparatus, and storage medium, so as to solve the problem of docking of communication ports between a task initiating terminal and a plurality of task executing terminals, reduce the difficulty in managing the communication ports by the task initiating terminal, and facilitate improvement of task scheduling efficiency.

In a first aspect, an embodiment of the present application provides a task scheduling method, where the task scheduling method is applied to a task scheduling system, and the task scheduling system includes: the system comprises a task initiating terminal, a load server and a plurality of task executing terminals; the scheduling method comprises the following steps:

the load server receives a data receiving request aiming at a target task, which is sent by the task initiating terminal; the target task is used for representing a modeling task for constructing a target model by using a plurality of different types of target training data; the data receiving request at least comprises: the task identification of the target task and the data identification of each target training data;

the load server determines a target task execution terminal aiming at the target task from the plurality of task execution terminals according to a pre-stored task processing mapping table; wherein, the task processing mapping table at least comprises: task identifiers of a plurality of different tasks, data identifiers of a plurality of training data corresponding to each task, and communication port addresses corresponding to each training data; the target training data stored on different target task execution terminals are different;

the load server sends a data transmission request aiming at each target task execution terminal to each target task execution terminal and receives a target data processing result sent by the target task execution terminal; the target data processing result is used for representing a data processing result corresponding to the target training data corresponding to the target task execution terminal under the target task;

and the load server takes each received target data processing result as a task execution result aiming at the target task and sends the task execution result to the task initiating terminal.

Optionally, before the load server receives a data reception request for a target task sent by the task initiating terminal, the scheduling method further includes:

the task initiating terminal responds to the starting operation aiming at the target task, and determines a task executing terminal where each target training data is located from the plurality of task executing terminals as a first task executing terminal corresponding to the target training data;

the task initiating terminal generates a task creating request corresponding to each first task execution terminal aiming at each first task execution terminal, and sends the task creating request to the first task execution terminal; wherein the task creation request includes: the task identification of the target task, the terminal identification of the first task execution terminal, the data identification of the target training data corresponding to the first task execution terminal and the data processing rule of the target training data under the target task;

after each first task execution terminal receives the task creation request, starting a target container for the target task, processing the target training data stored on the first task execution terminal through the target container, and sending container port information of the target container to the load server; wherein the container port information at least includes: the task identifier of the target task, the container port address of the target container and the data identifier of the target training data processed by the target container;

and after receiving the container port information sent by each target container, the load server stores the container port information into the task processing mapping table by taking a container port address in the container port information as a communication port address corresponding to the target training data processed by the target container.

Optionally, the starting a target container for the target task, and processing the target training data stored on the first task execution terminal through the target container includes:

the first task execution terminal responds to the received task creation request, and generates a virtual container starting instruction aiming at the target task according to the task identifier of the target task included in the task creation request;

generating a virtual container aiming at the target task according to the virtual container starting instruction, and taking the generated virtual container as the target container;

storing the target training data stored on the first task execution terminal and the corresponding environment dependence information of the target training data under the target task into the target container in a packaging mirror image mode; the environment dependence information is used for representing a data operation environment corresponding to the target training data under the target task;

and in the target container, performing data processing on the target training data according to the data processing rule of the target training data under the target task to obtain the target data processing result corresponding to the target training data.

Optionally, the determining a target task execution terminal for the target task from the plurality of task execution terminals includes:

the load server determines a first task identifier from the task identifiers of the multiple different tasks included in the task processing mapping table according to the received task identifier of the target task, and takes the position of the first task identifier in the task processing mapping table as a first query area; the first task identifier is a task identifier with the highest matching degree with the task identifier of the target task in the task identifiers of the different tasks; the data identification of a plurality of training data corresponding to the first task identification, and the communication port address of each training data are all located in the first query area;

for each target training data, determining the data identifier of the training data with the highest matching degree with the data identifier of the target training data from the data identifiers of the training data in the first query area as a first data identifier;

and determining the target task execution terminal corresponding to the target training data according to the communication port address of the training data corresponding to the first data identifier.

Optionally, when the task initiating terminal and each task executing terminal respectively correspond to different data encryption rules and data decryption rules, the data receiving request further includes: the data encryption rule and the data decryption rule on the task initiating terminal; each target task execution terminal sends a first encryption result of the target data processing result to the load server; the first encryption result is obtained after each target task execution terminal encrypts the target data processing result according to the data encryption rule on the target task execution terminal; the task processing mapping table further comprises: and each task executes a data encryption rule and a data decryption rule on the terminal.

Optionally, the sending a data transmission request for each target task execution terminal to the target task execution terminal, and receiving a target data processing result sent by the target task execution terminal, includes:

the load server determines a data encryption rule corresponding to each target task execution terminal according to the task processing mapping table and sends the data transmission request aiming at the target task execution terminal to the target task execution terminal according to the data encryption rule; the load server receives the first encryption result sent by the target task execution terminal aiming at the data transmission request;

and the load server determines a data decryption rule on the target task execution terminal according to the task processing mapping table, and decrypts the first encryption result according to the data decryption rule on the target task execution terminal to obtain the target data processing result.

Optionally, the taking each received target data processing result as a task execution result for the target task, and sending the task execution result to the task initiating terminal includes:

the load server determines a data encryption rule on the task initiating terminal according to the received data receiving request;

the load server encrypts each target data processing result according to a data encryption rule on the task initiating terminal to obtain a second encryption result of each target data processing result;

and the load server sends each second encryption result as the task execution result to the task initiating terminal.

In a second aspect, an embodiment of the present application provides a task scheduling apparatus, where the task scheduling apparatus is applied to a task scheduling system, and the task scheduling system includes: the system comprises a task initiating terminal, a load server and a plurality of task executing terminals; the scheduling device comprises:

a request receiving module, configured to receive, by the load server, a data receiving request for a target task sent by the task initiating terminal; the target task is used for representing a modeling task for constructing a target model by using a plurality of different types of target training data; the data receiving request at least comprises: the task identification of the target task and the data identification of each target training data;

a terminal determining module, configured to determine, by the load server, a target task execution terminal for the target task from the multiple task execution terminals according to a pre-stored task processing mapping table; wherein, the task processing mapping table at least comprises: task identifiers of a plurality of different tasks, data identifiers of a plurality of training data corresponding to each task, and communication port addresses corresponding to each training data; the target training data stored on different target task execution terminals are different;

the first transmission module is used for sending a data transmission request aiming at each target task execution terminal to each target task execution terminal through the load server and receiving a target data processing result sent by the target task execution terminal; the target data processing result is used for representing a data processing result corresponding to the target training data corresponding to the target task execution terminal under the target task;

and the second transmission module is used for taking each received target data processing result as a task execution result aiming at the target task through the load server and sending the task execution result to the task initiating terminal.

In a third aspect, an embodiment of the present application provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the steps of the scheduling method when executing the computer program.

In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps of the scheduling method.

The technical scheme provided by the embodiment of the application can have the following beneficial effects:

the embodiment of the application provides a task scheduling method, a task scheduling device and a task scheduling storage medium, wherein the task scheduling method comprises the following steps: a load server receives a data receiving request aiming at a target task, which is sent by a task initiating terminal; the load server determines a target task execution terminal aiming at a target task from a plurality of task execution terminals according to a pre-stored task processing mapping table; the load server sends a data transmission request aiming at each target task execution terminal to each target task execution terminal and receives a target data processing result sent by the target task execution terminal; and the load server takes each received target data processing result as a task execution result aiming at the target task and sends the task execution result to the task initiating terminal. Therefore, the task initiating terminal can obtain the task execution result of the target task from the task scheduling system under the condition that the target training data does not leave the target task executing terminal. In addition, the task initiating terminal only needs to communicate with the load server in the actual data transmission process, and the task initiating terminal does not need to be in butt joint with the communication ports of the plurality of task executing terminals, so that the problem of butt joint of the communication ports between the task initiating terminal and the plurality of task executing terminals is solved, the management difficulty of the task initiating terminal on the communication ports is reduced, and the task scheduling efficiency is improved.

In order to make the aforementioned objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.

Fig. 1 is a flowchart illustrating a task scheduling method according to an embodiment of the present application;

fig. 2 is a flowchart illustrating another method for updating stored information in a task processing mapping table while initiating a target task according to an embodiment of the present application;

fig. 3 is a flowchart illustrating a method for determining a target task execution terminal according to an embodiment of the present application;

FIG. 4 is a flow chart illustrating a method for receiving a target data processing result according to an embodiment of the present application;

fig. 5 is a flowchart illustrating a method for sending a task execution result according to an embodiment of the present application;

fig. 6 is a schematic structural diagram illustrating a task scheduling apparatus according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a computer device 700 according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.

The embodiment of the application provides a task scheduling method, a task scheduling device, a task scheduling apparatus and a storage medium, which are described in the following through specific embodiments.

Referring to fig. 1, fig. 1 is a schematic flowchart illustrating a flow of a task scheduling method provided in an embodiment of the present application, where the scheduling method is applied to a task scheduling system, and the task scheduling system includes: the system comprises a task initiating terminal, a load server and a plurality of task executing terminals; the scheduling method includes steps S101-S104; specifically, the method comprises the following steps:

s101, the load server receives a data receiving request aiming at a target task, which is sent by the task initiating terminal.

In the embodiment of the application, the target task is used for representing a modeling task for constructing a target model by using various different types of target training data, the task initiating terminal is a terminal device where an initiator of the target task is located, the task executing terminal is other terminal devices except the task initiating terminal in the task scheduling system, the load server is a communication proxy server of the task initiating terminal, and the load server is used for realizing data transmission between the task initiating terminal and any one of the task executing terminals, that is, the load server is used for scheduling a data communication task between the task initiating terminal and any one of the task executing terminals in the task scheduling system.

The task initiating terminal and each task executing terminal store respective unique feature data, and the task initiating terminal and each task executing terminal share the feature data in the task scheduling system by constructing the task scheduling system on the premise of ensuring the safety and the legality of the respective stored feature data, and complete the modeling task together, so as to improve the training effect on the co-constructed model and solve the common technical requirements.

It should be noted that the task initiating terminal is a terminal device where an initiator of the target task is located, and therefore, in the task scheduling system, the task initiating terminal and the load server corresponding to the task initiating terminal are not constant. For example, the task scheduling system includes: a terminal device a of an enterprise A, a terminal device B of an enterprise B, a terminal device C of an enterprise C and a terminal device D of an enterprise D; if the enterprise A initiates a target task A1 in the task scheduling system, wherein the target task A1 is a model for realizing the face recognition function, at this time, the terminal device a of the enterprise A is equivalent to a task initiating terminal, the load server a1 of the terminal device a is equivalent to a load server in the task scheduling system, and the terminal device B of the enterprise B, the terminal device C of the enterprise C and the terminal device D of the enterprise D are all equivalent to task executing terminals; if the terminal device B of the enterprise B initiates a target task B1 in the task scheduling system, wherein the target task B1 is a model for implementing the user identity information authentication function, the terminal device B of the enterprise B is equivalent to a task initiating terminal, the load server B1 of the terminal device B is equivalent to a load server in the task scheduling system, and the terminal device a of the enterprise a, the terminal device C of the enterprise C, and the terminal device D of the enterprise D are all equivalent to task executing terminals.

In a specific embodiment, the data receiving request at least includes: a task identification of the target task and a data identification of each of the target training data.

Illustratively, taking the "target task a 1" in the above example as an example, the target task a1 is to establish a model for implementing a face recognition function, where the target task a1 corresponds to the following 3 types of training data with different types of features: target training data m1, target training data m2, and target training data m 3; at this time, terminal device a of enterprise a is a task initiating terminal, load server a1 of terminal device a is a server in the task scheduling system, and terminal device B of enterprise B, terminal device C of enterprise C, and terminal device D of enterprise D all belong to task executing terminals; the load server a1 receives a data receiving request aiming at a target task A1 sent by the terminal device a, and determines that the task identifier of the target task is A1 and the data identifier of the target training data is: m1, m2, and m 3.

And S102, the load server determines a target task execution terminal aiming at the target task from the plurality of task execution terminals according to a pre-stored task processing mapping table.

In the embodiment of the application, each target task execution terminal corresponds to one target training data in the target tasks; the task processing mapping table at least comprises: task identifiers of a plurality of different tasks, data identifiers of a plurality of training data corresponding to each task, and communication port addresses corresponding to each training data; the target training data stored on different target task execution terminals are different.

Here, the task in the task processing mapping table may be a historical task that the task initiating terminal initiated before the current time, and the plurality of training data corresponding to the task may be a plurality of historical target training data corresponding to the historical task. Specifically, as an optional embodiment, the load server may pre-establish and store the task processing mapping table according to the following method:

the load server can acquire a task identifier of each historical task and data identifiers of a plurality of historical target training data corresponding to each historical task according to a data receiving request aiming at the historical task and sent by the task initiating terminal under each historical task; for each historical target training data, the load server may obtain a communication port address of a task execution terminal where the historical target training data is located as a communication port address corresponding to the historical target training data, and thus, the task processing mapping table is pre-established and stored.

In a specific implementation manner, for the target task, as an optional embodiment, the target task execution terminal for the target task may be determined according to the following method:

the load server acquires a data identifier of each target training data according to the data receiving request;

the load server determines the data identifier of the training data with the highest matching degree with the data identifier of the target training data as the target data identifier from the task processing mapping table according to the data identifier of each target training data;

and the load server determines the communication port address corresponding to the target data identification as the target task execution terminal corresponding to the target training data according to the task processing mapping table.

Illustratively, still taking the above example as an example, the target training data corresponding to the target task a1 is: target training data m1, target training data m2, and target training data m 3; wherein, the data identification of the target training data m1 is m1, the data identification of the target training data m2 is m2, and the data identification of the target training data m3 is m 3; taking the target training data m1 as an example, if the communication port address t1 of the training data m1 and the training data m1 is stored in the task processing mapping table, it is determined that the task execution terminal b corresponding to the communication port address t1 is the target task execution terminal corresponding to the target training data m 1.

S103, the load server sends a data transmission request aiming at each target task execution terminal to each target task execution terminal and receives a target data processing result sent by the target task execution terminal.

In the embodiment of the application, the target data processing result is used for representing a data processing result of the target training data corresponding to the target task execution terminal under the target task.

Here, under the same target task, because target training data corresponding to different target task execution terminals are different, and therefore, data processing manners corresponding to each target training data under the target task are different, as an optional embodiment, when the data receiving request sent by the task initiating terminal further includes a data processing manner corresponding to each target training data under the target task, the load server may further put, into the data transmission request, the data processing manner corresponding to the target training data under the target task and send the data transmission request to each target task execution terminal together.

Illustratively, still taking the above example as an example, the target training data corresponding to the target task a1 is: target training data m1, target training data m2, and target training data m 3; the data receiving request sent by the task initiating terminal a further comprises: a data processing mode y1 corresponding to the target training data m1 under the target task A1, a data processing mode y2 corresponding to the target training data m2 under the target task A1, and a data processing mode y3 corresponding to the target training data m3 under the target task A1; if the target task execution terminal of the target training data m1 is terminal device b, the target task execution terminal of the target training data m2 is terminal device c, and the target task execution terminal of the target training data m3 is terminal device d; the load server a1 sends a data transfer request x1 to terminal device b, a data transfer request x2 to terminal device c, and a data transfer request x3 to terminal device d; the data transmission request x1 includes the data processing method y1, the data transmission request x2 includes the data processing method y2, and the data transmission request x3 includes the data processing method y 3; the load server a1 receives a target data processing result z1 sent by the terminal device b, a target data processing result z2 sent by the terminal device c and a target data processing result z3 sent by the terminal device d, wherein the target data processing result z1 is the result of data processing performed by the target training data m1 according to the data processing mode y 1; the target data processing result z2 is the result of data processing performed by the target training data m2 in the data processing manner y 2; the target data processing result z3 is a result of data processing by the target training data m3 in the data processing manner y 3.

And S104, the load server takes each received target data processing result as a task execution result aiming at the target task, and sends the task execution result to the task initiating terminal.

Illustratively, still taking the above example as an example, the load server a1 sends the received target data processing result z1, target data processing result z2 and target data processing result z3 to the task originating terminal a as a task execution result of the target task a 1.

From the above analysis, steps S101-S104 shown in fig. 1 correspond to: and under the target task, the load server realizes the specific implementation steps of data transmission between the task initiating terminal and each target task executing terminal.

Here, it should be noted that the load server is a communication proxy server of the task initiating terminal, and the load server is used for realizing data transmission between the task initiating terminal and any one of the task executing terminals. In the case of not involving data transmission, before executing step S101, the task initiating terminal may perform simple communication with any task executing terminal in the task scheduling system, where the simple communication refers to that the task initiating terminal only sends a control instruction type common interaction request to the task executing terminal, and the response of the task executing terminal to the common interaction request does not involve transmission of training data.

As can be known from the above description, before step S101 is executed, a task initiating terminal may perform simple communication without data transmission with any task executing terminal in the task scheduling system, at this time, in addition to the method for establishing a task processing mapping table by using a historical task given in step S102, specifically, in a feasible embodiment, fig. 2 illustrates a flowchart of another method for updating stored information in the task processing mapping table while initiating a target task provided in the embodiment of the present application, as shown in fig. 2, before step S101 is executed, the method further includes S201 to S204; specifically, the method comprises the following steps:

and S201, the task initiating terminal responds to the starting operation aiming at the target task, and determines the task executing terminal where each target training data is located from the plurality of task executing terminals as a first task executing terminal corresponding to the target training data.

In a specific embodiment, the task initiating terminal stores data storage information on each of the task executing terminals, and when step S201 is executed, the task initiating terminal may determine the task executing terminal where each of the target training data is located according to each of the target training data included in the target task and the pre-stored data storage information on each of the task executing terminals.

Illustratively, in response to the starting operation of the target task a1, the task initiating terminal a determines that the target training data corresponding to the target task a1 is: target training data m1, target training data m2, and target training data m 3; according to the pre-stored data storage information on each task execution terminal, if the task execution terminal where the target training data m1 is located is determined to be terminal equipment b, the task execution terminal where the target training data m2 is determined to be terminal equipment c, and the task execution terminal where the target training data m3 is determined to be terminal equipment d; it may be determined that the first task execution terminal corresponding to the target training data m1 is terminal device b, the first task execution terminal corresponding to the target training data m2 is terminal device c, and the first task execution terminal corresponding to the target training data m3 is terminal device d.

And S202, the task initiating terminal generates a task creating request corresponding to each first task execution terminal aiming at each first task execution terminal, and sends the task creating request to the first task execution terminal.

Specifically, the task creation request includes: the task identification of the target task, the terminal identification of the first task execution terminal, the data identification of the target training data corresponding to the first task execution terminal, and the data processing rule of the target training data under the target task.

For an exemplary illustration, taking the first task execution terminal in the above example as the terminal device b, the task initiating terminal a generates the task creation request w1 corresponding to the terminal device b as follows: the task identity a1 of the target task, the terminal identity b of the first task execution terminal, the data identity m1 of the target training data m1, and the data processing rule y1 of the target training data m1 under the target task a 1.

S203, after receiving the task creation request, each first task execution terminal starts a target container for the target task, processes the target training data stored on the first task execution terminal through the target container, and sends container port information of the target container to the load server.

In this embodiment of the present application, the container port information at least includes: the task identifier of the target task, the container port address of the target container, and the data identifier of the target training data processed by the target container.

Here, in a specific implementation, the first task execution terminal may start a target container for the target task by generating a Docker (Application programming interface) API (Application programming interface) start instruction, where the target container is a virtual Docker container.

Here, the container port address of the target container may be a port address randomly generated by the target container.

In this embodiment, as an optional embodiment, the starting a target container for the target task, and processing the target training data stored on the first task execution terminal through the target container includes:

the first task execution terminal responds to the received task creation request, and generates a virtual container starting instruction aiming at the target task according to the task identifier of the target task included in the task creation request;

generating a virtual container aiming at the target task according to the virtual container starting instruction, and taking the generated virtual container as the target container;

storing the target training data stored on the first task execution terminal and the corresponding environment dependence information of the target training data under the target task into the target container in a packaging mirror image mode; the environment dependence information is used for representing a data operation environment corresponding to the target training data under the target task;

and in the target container, performing data processing on the target training data according to the data processing rule of the target training data under the target task to obtain the target data processing result corresponding to the target training data.

Here, the environment-dependent information may be determined according to a data processing rule corresponding to the target training data under the target task.

Still by way of example, in the above example, taking the first task execution terminal as the terminal device b, after receiving the task creation request w1, the terminal device b generates a virtual container start instruction for the target task a1 according to the task identifier a1 of the target task included in the task creation request w1, associates the task identifier a1 with the container identifier p1 of the virtual container to be generated, and generates the target container p1 for the target task a 1; then, according to a data processing rule y1 of the target training data m1 under the target task A1, determining environment dependence information L1 corresponding to the target training data m1 under the target task A1; putting the target training data m1 and the environment dependence information L1 into a target container p1 in a packaging mirror image mode; in the target container p1, the target training data m1 is processed according to the data processing rule y1 of the target training data m1 under the target task a1, and a target data processing result z1 corresponding to the target training data m1 is obtained. The container port address r1 of a target container p1 may be randomly generated by the target container p1, and the task identifier a1 of the target task a1, the container port address r1 and the data identifier m1 of the target training data m1 may be sent to the load server a1 as the container port information of the target container p 1.

And S204, after receiving the container port information sent by each target container, the load server stores the container port information into the task processing mapping table by taking a container port address in the container port information as a communication port address corresponding to the target training data processed by the target container.

In a specific implementation manner, as an optional embodiment, the load server may store a task identifier of the target task, a data identifier of each target training data, and a container port address of a target container corresponding to each target training data into a blank area of a task processing mapping table.

For an exemplary illustration, still taking target task a1 as an example, if the target container corresponding to target training data m1 is p1, the target container corresponding to target training data m2 is p2, and the target container corresponding to target training data m3 is p3, where the container port address of target container p1 is r1, the container port address of target container p2 is r2, and the container port address of target container p3 is r 3; then, the container port information sent by each target container may be stored in the task processing mapping table according to the above-mentioned manner, and the updated task processing mapping table is obtained as shown in the following table:

。

wherein h1 in the above table is the container port address of the target container of training data s 1; h2 in the above table is the container port address of the target container for training data s 2; h3 in the above table is the container port address of the target container for training data s 3.

In a possible embodiment, on the basis of fig. 2, fig. 3 is a flowchart illustrating a method for determining a target task execution terminal according to an embodiment of the present application, and as shown in fig. 3, when step S102 is executed, the method further includes S301-S303; specifically, the method comprises the following steps:

s301, the load server determines, according to the received task identifier of the target task, a first task identifier from the task identifiers of the multiple different tasks included in the task processing mapping table, and uses a position of the first task identifier in the task processing mapping table as a first query region.

Specifically, the first task identifier is a task identifier with the highest matching degree with the task identifier of the target task, among the task identifiers of the multiple different tasks; and the data identifications of a plurality of training data corresponding to the first task identification and the communication port address of each training data are all positioned in the first query area.

For example, taking the task processing mapping table shown in the above table as an example, the load server a1 determines, according to the received task identifier a1 of the target task a1, that the task identifier a1 is the first task identifier from the task processing mapping table shown in the above table, and uses the row where the first task identifier a1 is located as the first query area.

S302, for each of the target training data, determining, from the data identifiers of the plurality of training data located in the first query area, the data identifier of the training data with the highest matching degree with the data identifier of the target training data as the first data identifier.

Illustratively, still taking the above table as an example, the first query area is determined to be the last row area in the above table, and in the first query area, the data identifier m1 is determined to be the first data identifier of the target training data m1, the data identifier m2 is determined to be the first data identifier of the target training data m2, and the data identifier m3 is determined to be the first data identifier of the target training data m 3.

And S303, determining the target task execution terminal corresponding to the target training data according to the communication port address of the training data corresponding to the first data identifier.

Illustratively, taking the first data identifier m1 in the above table as an example, it may be determined that the container port address r1 corresponding to the first data identifier m1 is the communication port address of the target training data m1, and since the container port address r1 is the container port address of the target container p1 on the terminal device b, it may be determined that the target task execution terminal corresponding to the target training data m1 is the terminal device b.

In the embodiment of the present application, as can be known by combining step S101, the security and the validity of the feature data stored in the task initiating terminal and each task executing terminal need to be ensured, so that in the process of executing steps S101 to S104, the task initiating terminal and each task executing terminal may respectively correspond to different data encryption rules and data decryption rules, so as to ensure the security of the data transmission process.

Further, in the above case, the data sent by the task initiating terminal and each of the task executing terminals need to be encrypted according to respective encryption rules, and in this case, for the above steps S101 to S104, it should be noted that:

the data receiving request further comprises: the data encryption rule and the data decryption rule on the task initiating terminal; each target task execution terminal sends a first encryption result of the target data processing result to the load server; the first encryption result is obtained after each target task execution terminal encrypts the target data processing result according to the data encryption rule on the target task execution terminal; the task processing mapping table further comprises: and each task executes a data encryption rule and a data decryption rule on the terminal.

In the following, step S103 and step S104 in the embodiment of the present application will be described in detail with respect to the above-described case:

in a possible implementation, fig. 4 is a flowchart illustrating a method for receiving a target data processing result according to an embodiment of the present application, and as shown in fig. 4, when step S103 is executed, the method further includes S401-S403; specifically, the method comprises the following steps:

s401, the load server determines a data encryption rule corresponding to each target task execution terminal according to the task processing mapping table for each target task execution terminal, and sends the data transmission request for the target task execution terminal to the target task execution terminal according to the data encryption rule.

Illustratively, taking the example that the target task execution terminal corresponding to the target training data m1 in the above step S103 is the terminal device b, the load server a1 obtains the data encryption rule nb on the terminal device b from the task processing mapping table, then encrypts the data transmission request x1 according to the obtained data encryption rule nb for the data transmission request x1 corresponding to the terminal device b, and sends the encrypted data transmission request x1 to the terminal device b.

S402, the load server receives the first encryption result sent by the target task execution terminal aiming at the data transmission request.

Illustratively, taking the example of step S401 as an example, the terminal device b decrypts the received data transmission request x1 according to the data decryption rule jb on the terminal device b to obtain the data processing manner y1 corresponding to the target training data m1 under the target task a1, the terminal device b performs data processing on the target training data m1 according to the data processing manner y1 through the target container p1 to obtain the target data processing result z1 of the target training data m1, then, the terminal device b encrypts the target data processing result z1 according to the data encryption rule nb to obtain the first encryption result of the target data processing result z1, and sends the first encryption result of the target data processing result z1 to the load server a 1.

And S403, the load server determines a data decryption rule on the target task execution terminal according to the task processing mapping table, and decrypts the first encryption result according to the data decryption rule on the target task execution terminal to obtain the target data processing result.

Illustratively, taking the example of step S402 as an example, the load server a1 obtains the data decryption rule jb on the terminal device b from the task processing mapping table, and performs decryption processing on the first encrypted result by using the data decryption rule jb to obtain the target data processing result z 1.

In a possible implementation, fig. 5 is a flowchart illustrating a method for sending a task execution result according to an embodiment of the present application, and as shown in fig. 5, when step S104 is executed, the method further includes S501-S503; specifically, the method comprises the following steps:

s501, the load server determines a data encryption rule on the task initiating terminal according to the received data receiving request.

Illustratively, taking the example of step S403 as an example, after obtaining the target data processing result z1, the load server a1 determines that the data encryption rule of the task initiating terminal a is na according to the data receiving request received in step S101.

S502, the load server encrypts each target data processing result according to the data encryption rule on the task initiating terminal to obtain a second encryption result of each target data processing result.

Illustratively, taking the example of the step S502 as an example, the load server a1 performs encryption processing on the target data processing result z1 according to the data encryption rule na of the task initiating terminal a, so as to obtain a second encryption result of the target data processing result z 1.

S503, the load server sends each second encryption result as the task execution result to the task initiating terminal.

Illustratively, if the target data processing result of the target training data m1 is z1, the target data processing result of the target training data m2 is z2, and the target data processing result of the target training data m3 is z 3; the load server a1 encrypts the target data processing result z1, the target data processing result z2 and the target data processing result z3 respectively according to the data encryption rule na of the task initiating terminal a, and sends the encrypted results to the task initiating terminal a as the task execution results of the target task a1, and the task initiating terminal a decrypts the received task execution results according to the data decryption rule ja of the task initiating terminal a, so as to obtain the target data processing result z1 of the target training data m1, the target data processing result z2 of the target training data m2 and the target data processing result z3 of the target training data m 3.

By the task scheduling method, the task initiating terminal can acquire the task execution result of the target task from the task scheduling system under the condition that the target training data does not leave the target task executing terminal. In addition, the task initiating terminal only needs to communicate with the load server in the actual data transmission process, and the task initiating terminal does not need to be in butt joint with the communication ports of the plurality of task executing terminals, so that the problem of butt joint of the communication ports between the task initiating terminal and the plurality of task executing terminals is solved, the management difficulty of the task initiating terminal on the communication ports is reduced, and the task scheduling efficiency is improved.

Based on the same inventive concept, the present application further provides a scheduling apparatus corresponding to the scheduling method, and as the principle of the scheduling apparatus in the embodiment of the present application for solving the problem is similar to the scheduling method in the embodiment of the present application, the implementation of the scheduling apparatus may refer to the implementation of the method, and repeated details are not repeated.

Referring to fig. 6, fig. 6 is a schematic structural diagram illustrating a task scheduling apparatus according to an embodiment of the present application, where the task scheduling apparatus is applied to a task scheduling system, and the task scheduling system includes: the system comprises a task initiating terminal, a load server and a plurality of task executing terminals; the scheduling device comprises:

a request receiving module 601, configured to receive, by the load server, a data receiving request for a target task sent by the task initiating terminal; the target task is used for representing a modeling task for constructing a target model by using a plurality of different types of target training data; the data receiving request at least comprises: the task identification of the target task and the data identification of each target training data;

a terminal determining module 602, configured to determine, by the load server according to a pre-stored task processing mapping table, a target task execution terminal for the target task from the multiple task execution terminals; wherein, the task processing mapping table at least comprises: task identifiers of a plurality of different tasks, data identifiers of a plurality of training data corresponding to each task, and communication port addresses corresponding to each training data; the target training data stored on different target task execution terminals are different;

a first transmission module 603, configured to send a data transmission request for each target task execution terminal to the target task execution terminal through the load server, and receive a target data processing result sent by the target task execution terminal; the target data processing result is used for representing a data processing result corresponding to the target training data corresponding to the target task execution terminal under the target task;

a second transmission module 604, configured to use each received target data processing result as a task execution result for the target task through the load server, and send the task execution result to the task initiating terminal.

Optionally, the scheduling apparatus further includes: the device comprises a first determining module, a first generating module, a first processing module and a first sending module; before the request receiving module 601 receives a data receiving request for a target task sent by the task initiating terminal through the load server:

the first determining module is configured to determine, from the plurality of task execution terminals, a task execution terminal where each target training data is located as a first task execution terminal corresponding to the target training data by the task initiating terminal responding to a starting operation for the target task;

the first generating module is configured to generate, by the task initiating terminal, a task creating request corresponding to each first task executing terminal, and send the task creating request to the first task executing terminal; wherein the task creation request includes: the task identification of the target task, the terminal identification of the first task execution terminal, the data identification of the target training data corresponding to the first task execution terminal and the data processing rule of the target training data under the target task;

the first processing module is configured to start a target container for the target task after each first task execution terminal receives the task creation request, process the target training data stored on the first task execution terminal through the target container, and send container port information of the target container to the load server; wherein the container port information at least includes: the task identifier of the target task, the container port address of the target container and the data identifier of the target training data processed by the target container;

the first sending module is configured to store, after receiving, by the load server, the container port information sent by each target container, the container port information into the task processing mapping table by using a container port address in the container port information as a communication port address corresponding to the target training data processed by the target container.

Optionally, when the target container for the target task is started and the target training data stored on the first task execution terminal is processed through the target container, the first processing module is specifically configured to:

responding to the received task creation request through the first task execution terminal, and generating a virtual container starting instruction aiming at the target task according to the task identifier of the target task included in the task creation request;

generating a virtual container aiming at the target task according to the virtual container starting instruction, and taking the generated virtual container as the target container;

storing the target training data stored on the first task execution terminal and the corresponding environment dependence information of the target training data under the target task into the target container in a packaging mirror image mode; the environment dependence information is used for representing a data operation environment corresponding to the target training data under the target task;

and in the target container, performing data processing on the target training data according to the data processing rule of the target training data under the target task to obtain the target data processing result corresponding to the target training data.

Optionally, the terminal determining module 602 is specifically configured to:

determining, by the load server, a first task identifier from the task identifiers of the plurality of different tasks included in the task processing mapping table according to the received task identifier of the target task, and taking a position of the first task identifier in the task processing mapping table as a first query region; the first task identifier is a task identifier with the highest matching degree with the task identifier of the target task in the task identifiers of the different tasks; the data identification of a plurality of training data corresponding to the first task identification, and the communication port address of each training data are all located in the first query area;

for each target training data, determining the data identifier of the training data with the highest matching degree with the data identifier of the target training data from the data identifiers of the training data in the first query area as a first data identifier;

and determining the target task execution terminal corresponding to the target training data according to the communication port address of the training data corresponding to the first data identifier.

Optionally, in the scheduling apparatus, when the task initiating terminal and each task executing terminal respectively correspond to different data encryption rules and data decryption rules, the data receiving request further includes: the data encryption rule and the data decryption rule on the task initiating terminal; each target task execution terminal sends a first encryption result of the target data processing result to the load server; the first encryption result is obtained after each target task execution terminal encrypts the target data processing result according to the data encryption rule on the target task execution terminal; the task processing mapping table further comprises: and each task executes a data encryption rule and a data decryption rule on the terminal.

Optionally, in the foregoing case, the first transmission module 603 is specifically configured to:

the load server determines a data encryption rule corresponding to each target task execution terminal according to the task processing mapping table and sends the data transmission request aiming at the target task execution terminal to the target task execution terminal according to the data encryption rule; the load server receives the first encryption result sent by the target task execution terminal aiming at the data transmission request;

and the load server determines a data decryption rule on the target task execution terminal according to the task processing mapping table, and decrypts the first encryption result according to the data decryption rule on the target task execution terminal to obtain the target data processing result.

Optionally, in the foregoing case, the second transmission module 604 is specifically configured to:

the load server determines a data encryption rule on the task initiating terminal according to the received data receiving request;

the load server encrypts each target data processing result according to a data encryption rule on the task initiating terminal to obtain a second encryption result of each target data processing result;

and the load server sends each second encryption result as the task execution result to the task initiating terminal.

As shown in fig. 7, an embodiment of the present application provides a computer device 700 for performing a scheduling method of tasks in the present application, the device includes a memory 701, a processor 702, and a computer program stored in the memory 701 and executable on the processor 702, wherein the processor 702 implements the steps of the scheduling method of tasks when executing the computer program.

Specifically, the memory 701 and the processor 702 may be general-purpose memory and processor, which are not limited specifically, and when the processor 702 runs the computer program stored in the memory 701, the scheduling method of the task can be executed.

Corresponding to the task scheduling method in the present application, an embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to perform the steps of the task scheduling method.

In particular, the storage medium can be a general-purpose storage medium, such as a removable disk, a hard disk, or the like, and the computer program on the storage medium can execute the above-mentioned task scheduling method when being executed.

In the embodiments provided in the present application, it should be understood that the disclosed system and method may be implemented in other ways. The above-described system embodiments are merely illustrative, and for example, the division of the units is only one logical functional division, and there may be other divisions in actual implementation, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of systems or units through some communication interfaces, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus once an item is defined in one figure, it need not be further defined and explained in subsequent figures, and moreover, the terms "first", "second", "third", etc. are used merely to distinguish one description from another and are not to be construed as indicating or implying relative importance.

Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present application, and are used for illustrating the technical solutions of the present application, but not limiting the same, and the scope of the present application is not limited thereto, and although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope disclosed in the present application; such modifications, changes or substitutions do not depart from the spirit and scope of the present disclosure, which should be construed in light of the above teachings. Are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.