1. A service processing method based on resource allocation is characterized by comprising the following steps:

receiving a service to be processed;

acquiring a history processing record and a service parameter of the service to be processed; the service parameter is used for representing the execution content of the service to be processed;

determining the service characteristics of the service to be processed by using the historical processing record and the service parameters; the service characteristics correspond to resource scheduling amounts; the resource scheduling amount represents an amount of resources consumed for processing the corresponding service feature;

and allocating resource groups for the service to be processed based on the service characteristics and the resource scheduling amount, so that the service to be processed is processed through the resources in the resource groups.

2. The method of claim 1, wherein the pending traffic comprises traffic running for a specified duration; and the service to be processed corresponds to a historical processing record within the specified time length.

3. The method of claim 1, wherein the business parameters comprise at least one of a number of business files, an average size of business files, a median size of business files, a business process type, a number of business processes, and a number of business file instruction statements.

4. The method of claim 1, wherein the business characteristics include at least one of business file size, step-to-process association, business statement priority, business out-degree information, and business in-degree information.

5. The method of claim 1, wherein the obtaining the historical processing record and the service parameters of the service to be processed comprises:

acquiring a service log corresponding to the service to be processed;

and determining the historical processing record and the service parameters of the service to be processed according to the service log.

6. The method of claim 1, wherein the resources in the resource group comprise at least one of computing resources, storage resources, and data resources.

7. The method of claim 1, wherein before allocating the resource group for the pending service based on the service characteristics and the resource scheduling amount, further comprising:

acquiring resource group information of each preset resource group; the resource group information is used for expressing at least one of service operation state and resource occupation condition of the resource group;

correspondingly, allocating resource groups for the service to be processed based on the service characteristics and the resource scheduling amount;

and selecting resource groups from the preset resource groups to be distributed to the service to be processed based on the service characteristics, the resource scheduling amount and the resource group information.

8. The method of claim 1, wherein the allocating a resource group for the pending service based on the service characteristics and the resource scheduling amount comprises:

training a resource allocation model based on a reinforcement learning scene; the resource allocation model is used for determining resource groups allocated for services;

and allocating resource groups for the service to be processed by utilizing the resource allocation model based on the service characteristics and the resource allocation amount.

9. The method of claim 8, wherein the training a resource allocation model based on a reinforcement learning scenario comprises:

inputting the sample service characteristics into a resource allocation model to obtain a pre-estimated reward value corresponding to each resource group;

selecting a target resource group based on the size of the pre-estimated reward value; the set of target resources corresponds to an actual reward value;

constructing a loss function based on the estimated reward value and the actual reward value;

and optimizing the resource allocation model by using the loss function and the gradient descent method.

10. The method of claim 8, wherein the to-be-processed traffic comprises traffic processed based on a master cluster; the training resource allocation model based on the reinforcement learning scene comprises the following steps:

training the resource allocation model based on the service data on the standby cluster;

and applying the trained resource allocation model to a main cluster to allocate resource groups for the service to be processed.

11. The method of claim 1, wherein prior to receiving pending traffic, further comprising:

setting a special resource group; the dedicated resource group comprises a resource group having a maximum amount of resources;

correspondingly, the allocating resource groups for the to-be-processed service based on the service features and the resource scheduling amount includes:

determining the service processing resource quantity required by the service to be processed based on the service characteristics and the resource scheduling quantity;

under the condition that the service processing resource amount is larger than a specified resource amount threshold value, allocating the special resource group to the service to be processed, so that the service to be processed is processed through the resources in the special resource group; the specified resource quantity threshold comprises the maximum resource quantity corresponding to the idle resource group except the special resource group.

12. The method of claim 1, wherein prior to receiving pending traffic, further comprising:

setting a special resource group; the dedicated resource group comprises a resource group having a maximum amount of resources;

correspondingly, the allocating resource groups for the to-be-processed service based on the service features and the resource scheduling amount includes:

determining the service priority of the service to be processed based on the service characteristics and the resource scheduling amount;

and under the condition that the service priority is higher than the designated priority, allocating the special resource group to the service to be processed, so that the processing of the service to be processed is completed through the resources in the special resource group.

13. A service processing apparatus based on resource allocation, comprising:

the service receiving module is used for receiving the service to be processed;

the parameter acquisition module is used for acquiring the historical processing record and the service parameters of the service to be processed; the service parameter is used for representing the execution content of the service to be processed;

the service characteristic determining module is used for determining the service characteristics of the service to be processed by utilizing the historical processing record and the service parameters; the service characteristics correspond to resource scheduling amounts; the resource scheduling amount represents an amount of resources consumed for processing the corresponding service feature;

and the resource group allocation module is used for allocating resource groups for the service to be processed based on the service characteristics and the resource scheduling amount so as to complete the processing of the service to be processed through the resources in the resource groups.

14. A resource allocation apparatus comprising a memory and a processor;

the memory is to store computer program instructions;

the processor is configured to execute the computer program instructions to implement the steps of: receiving a service to be processed; acquiring a history processing record and a service parameter of the service to be processed; the service parameter is used for representing the execution content of the service to be processed; determining the service characteristics of the service to be processed by using the historical processing record and the service parameters; the service characteristics correspond to resource scheduling amounts; the resource scheduling amount represents an amount of resources consumed for processing the corresponding service feature; and allocating resource groups for the service to be processed based on the service characteristics and the resource scheduling amount, so that the service to be processed is processed through the resources in the resource groups.

Background

In practical applications, when services are processed, corresponding resources are generally required to be allocated to the services to complete the processing of the services, and specifically, the resources may be storage resources, computing resources, and the like. As the number and types of services increase, the resources need to be planned to achieve effective allocation, for example, a multi-tenant mode is generally set in a large-scale hadoop cluster, that is, resources in the cluster are divided into resource sets, and the resource sets are separated from each other and do not interfere with each other. When a user or a service to be executed needs to utilize corresponding resources, the functions of running an application or a service, storing data and the like can be realized in a resource set renting mode. In practical applications, multiple users or services may use resources in the same resource set, and one user or service may also use resources of multiple tenants. The resource allocation and the service execution are effectively ensured by the resource aggregation mode.

At present, when a resource set is allocated to different services, the resource amount corresponding to the corresponding service is often allocated in advance, and the corresponding resource set is allocated to the services according to the resource amount. However, the number of services and the application requirements are constantly changing, and allocating resource sets according to the current method easily makes resources not be effectively utilized, and even the service waits too long for the resources to be effectively processed. Therefore, a solution capable of flexibly allocating resources to services to achieve effective processing of the services is needed.

Disclosure of Invention

An object of the embodiments of the present specification is to provide a service processing method, device and apparatus based on resource allocation, so as to solve a problem how to flexibly allocate resources for a service based on service requirements to implement effective processing of the service.

In order to solve the foregoing technical problem, an embodiment of the present specification provides a service processing method based on resource allocation, including: receiving a service to be processed; acquiring a history processing record and a service parameter of the service to be processed; the service parameter is used for representing the execution content of the service to be processed; determining the service characteristics of the service to be processed by using the historical processing record and the service parameters; the service characteristics correspond to resource scheduling amounts; the resource scheduling amount represents an amount of resources consumed for processing the corresponding service feature; and allocating resource groups for the service to be processed based on the service characteristics and the resource scheduling amount, so that the service to be processed is processed through the resources in the resource groups.

An embodiment of this specification further provides a service processing apparatus based on resource allocation, including: the service receiving module is used for receiving the service to be processed; the parameter acquisition module is used for acquiring the historical processing record and the service parameters of the service to be processed; the service parameter is used for representing the execution content of the service to be processed; the service characteristic determining module is used for determining the service characteristics of the service to be processed by utilizing the historical processing record and the service parameters; the service characteristics correspond to resource scheduling amounts; the resource scheduling amount represents an amount of resources consumed for processing the corresponding service feature; and the resource group allocation module is used for allocating resource groups for the service to be processed based on the service characteristics and the resource scheduling amount so as to complete the processing of the service to be processed through the resources in the resource groups.

The embodiment of the present specification further provides a resource allocation device, including a memory and a processor; the memory to store computer program instructions; the processor to execute the computer program instructions to implement the steps of: receiving a service to be processed; acquiring a history processing record and a service parameter of the service to be processed; the service parameter is used for representing the execution content of the service to be processed; determining the service characteristics of the service to be processed by using the historical processing record and the service parameters; the service characteristics correspond to resource scheduling amounts; the resource scheduling amount represents an amount of resources consumed for processing the corresponding service feature; and allocating resource groups for the service to be processed based on the service characteristics and the resource scheduling amount, so that the service to be processed is processed through the resources in the resource groups.

As can be seen from the technical solutions provided by the embodiments of the present specification, when a service to be processed is received, the embodiments of the present specification grasp specific contents of the service to be processed by obtaining a history processing record and a service parameter of the service to be processed, and then can determine service characteristics of the service to be processed, and determine resources required to be consumed for processing the service to be processed according to the service characteristics and resource scheduling amounts corresponding to the service characteristics, and further allocate corresponding resource groups to the service to be processed, so that the processing of the service to be processed can be completed by using the resources in the resource groups. By the method, the resource group allocated to the service to be processed can be determined based on the actual application condition of the service to be processed, so that flexible and effective allocation of resources is ensured, effective processing of the service is realized, and the overall processing efficiency of the service is improved.

Drawings

In order to more clearly illustrate the embodiments of the present specification or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the specification, and other drawings can be obtained by those skilled in the art without creative efforts.

Fig. 1 is a flowchart of a service processing method based on resource allocation according to an embodiment of the present disclosure;

fig. 2 is a schematic diagram of a service processing architecture according to an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of a model training process according to an embodiment of the present disclosure;

FIG. 4 is a flowchart illustrating a gray scale testing method according to an embodiment of the present disclosure;

fig. 5 is a block diagram of a service processing apparatus based on resource allocation according to an embodiment of the present disclosure;

fig. 6 is a block diagram of a resource allocation apparatus according to an embodiment of the present disclosure.

Detailed Description

The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present specification without any creative effort shall fall within the protection scope of the present specification.

In order to solve the above technical problem, an embodiment of the present specification provides a service processing method based on resource allocation. The service processing method based on resource allocation can be applied to resource allocation equipment, and the resource allocation equipment comprises but is not limited to a server, an industrial personal computer, a PC and the like. As shown in fig. 1, the service processing method based on resource allocation may include the following implementation steps.

S110: and receiving the service to be processed.

The service to be processed is a service which needs to be processed. The service to be processed not only needs the corresponding service processing logic to complete the execution of the service in the execution process, but also needs to use the corresponding resources, such as computing resources, to allocate the corresponding computing node for completing the execution process of the service to be processed, or to store resources for storing cache data and final execution result data generated in the execution process of the service to be processed, and for a part of services, the resources may also be data resources for providing data such as execution device sample data. In practical applications, other types of data may be set as needed, and are not limited to the above examples, and are not described herein again.

Therefore, after receiving the service to be processed, the service to be processed needs to be analyzed and evaluated to determine the amount of resources required for executing the service to be processed, so as to allocate corresponding resources to the service to be processed.

In some embodiments, in order to ensure that the service to be processed can be effectively analyzed in the subsequent step, the service to be processed may include a service that runs for a specified duration, and within the specified duration, the service to be processed may correspond to a history processing record, so that the service to be processed can be analyzed through the history processing record.

For example, the service to be processed may be a service that is executed in the system for three days in advance, corresponding data can be generated when the service to be processed is executed in the three days, and the data can be analyzed for the service to be processed, so that accuracy in a resource allocation analysis process is ensured.

In some embodiments, before receiving the pending traffic, a dedicated resource group may also be set. The dedicated resource group may be a resource group including the largest amount of resources, and the specific amount of resources allocated is set based on the traffic having the largest amount of resources required in the history.

The dedicated resource group may be a resource group providing resources for a specific service, for example, the dedicated resource group may be allocated to a service corresponding to a data analysis job service, so as to ensure timely and preferential processing of the specific service. Under the condition that the special resource group has the maximum resource amount, the special resource group can also be allocated to the service with larger resource demand, so as to reduce the difficulty of the service processing process and improve the service processing efficiency. In some examples, the corresponding priority may be further obtained by setting or evaluating for the service, and then the service is allocated with the corresponding resource group according to the priority of the service, and accordingly, the dedicated resource group may be allocated to the service with the higher priority.

In practical applications, the dedicated resource group may be allocated according to specific application requirements, and is not limited to the above example, and will not be described herein again.

S120: acquiring a history processing record and a service parameter of the service to be processed; the service parameter is used for representing the execution content of the service to be processed.

After receiving the service to be processed, the history processing record and the service parameter of the service to be processed may be obtained.

The history processing record may be corresponding record data generated when the to-be-processed service is executed before, and specifically, may be data of an execution time, an execution result, an execution operation, an execution step, and the like of the to-be-processed service. The historical processing records can be used for analyzing the business based on the actual processing process of the business to be processed so as to improve the accuracy of the analysis process.

The service parameter may be used to indicate the execution content of the service to be processed, and specifically may be the number of files in the service to be processed, the file type, and the specific execution step. Based on the service parameters, the specific content and the execution flow of the service can be determined, and then the service to be processed is effectively analyzed through the service parameters. For example, the service parameter may be a specific statement of the service to be processed, such as an sql statement, and the actual execution process of the statements is analyzed, so that the actual execution effect of the service to be processed can be effectively determined, and thus, corresponding resources can be accurately and effectively allocated in the subsequent process.

In some embodiments, the service parameter may include at least one of a number of service files, an average size of service files, a median size of service files, a type of service process, a number of service processes, and a number of instruction statements of service files.

In practical application, when obtaining the history processing record and the service parameter, the data acquisition function can be added aiming at data lake loading operation in the platform, and after the daily data loading is finished, the characteristics of the total number of records in the current partition of the result table, the number of files of the result table on hdfs, the average size of the files on hdfs, the maximum file size, the minimum file size, the standard deviation, the median of the file sizes and the like can be recorded. Aiming at the live data analysis operation, a resource group is set aiming at the live data analysis side operation, the total number of records of an input table and a result table of each sentence hql statement is recorded, the number of files of the input table and the result table on hdfs, the average size of the files on hdfs, the maximum file size, the minimum file size, the standard deviation and the median of the file sizes are recorded, and meanwhile, the number of stages contained in hql statements is also recorded, and the total number of map processes, reduce processes and the like are used in the execution process, so that the execution time of the service is determined based on the incidence relation between the processes and the steps.

In some embodiments, the log corresponding to the service to be processed may be extracted, and corresponding data may be obtained from the log as the historical processing record and the service parameter. Since the log is generally used for recording the specific processing procedure of the service to be processed, the data contained in the log can be effectively processed as a history processing record and a service parameter.

S130: determining the service characteristics of the service to be processed by using the historical processing record and the service parameters; the service characteristics correspond to resource scheduling amounts; the resource scheduling amount represents an amount of resources consumed to process the corresponding traffic characteristic.

After the history processing record and the service parameters are obtained, the service characteristics of the service to be processed can be determined through the history processing record and the service parameters. The service characteristics may be used to prominently represent the characteristics of the service to be processed in the execution process, for example, the service characteristics may be selected from service parameters, or the service characteristics may be obtained by summarizing the service parameters based on the history processing records. The resource amount required by the service processing can be more effectively evaluated based on the service characteristics.

The service features correspond to resource scheduling quantities, and the resource scheduling quantities can represent the resource quantities consumed by processing the service features, so that the resource quantities consumed by processing the whole service to be processed can be determined based on the resource scheduling quantities.

In some embodiments, the service characteristics may include at least one of a service file size, a relevance between a step and a process, a service statement priority, service out-degree information, and service in-degree information, where the service file size may be a size of each file included in a service to be processed in a case where one or more files are included in the service; the relevance between the steps and the processes may be a relationship between the processing steps and the processes, for example, the number of the processes and the types of the processes included in one step, in the case that the service to be processed corresponds to a plurality of processing steps and includes a plurality of processes; the priority of the business statement may be a priority difference between different statements when the business to be processed includes a plurality of business statements, for example, when the proportion of the average completion time point exceeding all hql statements within the last 30 times is higher than a specific proportion, the corresponding hql statement may be considered as a statement with a higher priority; the service out-degree information may refer to the number of other services depending on the service; the service in-degree information may refer to the number of other services on which the service depends.

Based on the service characteristics, the resource quantity consumed in the service processing process can be analyzed in a relatively strong pertinence manner, and the subsequent steps can be favorably carried out. The service features determined in practical applications may be set to other types, and are not limited to the above examples, and are not described herein again.

In some embodiments, in addition to obtaining the service characteristics, resource group information describing the resource groups may also be obtained. Specifically, the resource group information of each preset resource group may be acquired for the preset resource groups, that is, all the set resource groups. The resource group information may be used to indicate at least one of service operation status and resource occupation status of each resource group. The service operation status can be used for indicating which services are utilizing resources in the preset resource group currently; the resource occupancy may be used to indicate a proportion of resources in the preset set of resources that are utilized. In the subsequent process, the resource group allocated to the service to be processed can be further determined by combining the resource group information, so that the applicability of the selected resource group is improved.

S140: and allocating resource groups for the service to be processed based on the service characteristics and the resource scheduling amount, so that the service to be processed is processed through the resources in the resource groups.

After the service characteristics and the resource scheduling amount are determined, because the service characteristics and the resource scheduling amount can be used for reflecting the utilization degree of the service to be processed on the resources, the resources required to be consumed for processing the service to be processed can be determined through analysis, and then the resource groups allocated to the service to be processed can be determined according to the consumed resource amount.

In some embodiments, a resource allocation model may be trained in advance based on a reinforcement learning scenario, and then service characteristics and a resource scheduling amount are input into the resource allocation model to obtain a resource group allocated to the service to be processed.

The resource allocation model is used for determining resource groups allocated for services, and the effect of different resource groups on the processing of different types of services can be determined in the training process of the resource allocation model, so that the resource groups allocated for the services to be processed are determined according to the relevant parameters of the services to be processed.

In the big data hadoop cluster, hql of hundreds of thousands of data analysis are to be executed every day, the artificial labeling efficiency is too low, and the final effect of the model cannot be ensured. In general, intensive learning does not need a large number of labeled data sets, and samples for training are obtained through continuous interactive trial and error of agents and the environment, so that the time for obtaining labeled samples in a large amount is saved, and the learning process is closer to the learning process of human beings.

In some embodiments, the training process may be to input the sample business characteristics into the resource allocation model to obtain the estimated reward values corresponding to each resource group; selecting a target resource group based on the size of the pre-estimated reward value; the set of target resources corresponds to an actual reward value; constructing a loss function based on the estimated reward value and the actual reward value; and optimizing the resource allocation model by using the loss function and the gradient descent method.

Specifically, in the embodiment of the present specification, first, four primitive groups (S, a, P, R) of reinforcement learning, that is, a state (state), an action (action), a reward (reward), and a policy (policy) are defined. The state contains all the service characteristics; the action is only that the resources of the preset resource group do not reach the peak value in the current state, namely the user still has the right to use the preset resource group; the reward is obtained by hql at present, the reward obtained by the resource group can be determined after the waiting time of the hql sentences is compared with the average value of the last seven days, or the total waiting time of all programs in the operation group is compared with the average value of the last seven days after the completion of batch operation on the same day, if the comparison result reflects that the time is increased, the time is positive reward, otherwise, the time is negative reward; the policy may be to predict a prize value for each of the available predetermined set of resources, and the highest prize value may be determined for the selected set of resources.

Based on the above process, if the reward system (waiting time) corresponding to each state (current feature) is listed in a matrix form (Q table), the whole Q table is very large, and the retrieval time is long when updating. Therefore, a complete Q-Table can be generated by using the DQN algorithm, the Q value (the reward value obtained by selecting each resource group) of each observation environment is generated by the neural network, that is, the reward value of each resource group selected by the current environment is obtained by inputting the feature Features of the current environment, and the reward value is compared to select the resource group with the highest reward value.

As shown in fig. 2, which is a schematic structural diagram of a resource allocation model, the collected service features are input to a neural network of a double-layer full-connection-band Relu activation function. The neural network has a structure as shown in fig. 3, and the number of neurons in the hidden layer may be 200. Therefore, the reward value generated by selecting each resource group can be calculated and is the estimated reward value. And selecting the I-frame tuple corresponding to the highest reward value as the strategy execution of the current hql statement, and calculating the actual reward value according to the definition of the reward after the strategy execution.

After each prediction, the current State, the calculated Action and the predicted Reward value (Reward) are stored in a playback memory unit. After completing daily batch, randomly selecting a certain batch of training samples from the playback memory unit for learning and training the neural network, wherein 30% of the randomly selected training samples are selected instead of all the training samples, and machine learning based on the maximum likelihood is provided with an assumed condition: the training samples must be independently and identically distributed, if the assumption is not true, the training effect of the model is greatly reduced, and the training samples in the playback memory unit are sequences obtained by interacting with State, and have certain correlation, because the result of one statement is likely to be the input of another statement. Therefore, the solution of the present invention is to provide a playback memory unit to store a certain amount of data, which includes features, actions, and rewards, and when using the data, we randomly draw 30% of hql sentences in the playback memory unit for learning, so as to break the correlation of the sequence obtained by State interaction, i.e. solve the problem of data correlation.

In addition, the loss function loss can be defined as (actual reward value-estimated reward value) ^2, and then parameters of the neural network are updated by using a gradient descent method through back propagation of the neural network, so that the purpose of minimizing the loss function is achieved, namely the batch acceleration time trained by the neural network is closer to the actual batch acceleration time.

In some embodiments, the to-be-processed service includes a service processed based on a master cluster, and when a resource allocation model is trained, the resource allocation model may be trained based on service data on a slave cluster; and applying the trained resource allocation model to a main cluster to allocate resource groups for the service to be processed.

Specifically, the model is initially deployed on the HADOOP standby cluster, that is, the training of the neural network part in the DQN algorithm is performed on the standby cluster, because the parameters of the neural network are initialized randomly at the beginning, and it is likely that the result of assigning resource groups by using the set of network to obtain reward values is not as efficient as that manually set on the main cluster. Therefore, for insurance, the model is deployed on the HADOOP standby cluster, the resource group is assigned on the standby cluster by using the result obtained by reinforcement learning, after 3-day batch iteration, the standby cluster batch operation group waiting time and the main cluster batch operation group waiting time are compared, and if and only if the waiting time is optimized to exceed 10%, the latest network is really updated to the main cluster. The iterative update strategy of the present invention thus ensures that batch performance is improved throughout because of the increasing accuracy of automatically setting resource groups.

In some embodiments, in the case that a dedicated resource group is preset, when a resource group is allocated to a service to be processed, a service processing resource amount required by the service to be processed, that is, a resource amount required to be utilized in the service processing process to be processed, may be determined based on the service characteristics and the resource scheduling amount. Under the condition that the service processing resource amount is larger than a specified resource amount threshold value, allocating the special resource group to the service to be processed, so that the service to be processed is processed through the resources in the special resource group; the specified resource quantity threshold comprises the maximum resource quantity corresponding to the idle resource group except the special resource group.

In other embodiments, in the case that a dedicated resource group is preset, when a resource group is allocated to a service to be processed, the service priority of the service to be processed may be determined based on the service characteristics and the resource scheduling amount. Under the condition that the service priority is higher than the designated priority, allocating the special resource group to the service to be processed, so that the processing of the service to be processed is completed through the resources in the special resource group; the specified resource quantity threshold comprises the maximum resource quantity corresponding to the idle resource group except the special resource group.

It should be noted that, after determining the resource group, the processing of implementing the service by using the resource in the resource group may be completed by the resource allocation device, or the identifier of the resource group and the service to be processed may be sent to other devices, and the processing of the service is implemented by the other devices, which is not limited to this.

As shown in fig. 4, which is a schematic diagram of a scenario example of this specification, as shown in the drawing, after data is acquired and transmitted to a feature engineering, and a sentence data amount related feature, a sentence execution plan related feature, a sentence priority related feature, a sentence time related feature, a tenant feature, a cluster overall feature, and job related information are extracted, these information are merged and input to a standby cluster to implement a DQN reinforcement learning model. After the current program/environment characteristics are transmitted to the return visit memory unit, the neural network parameters are obtained by combining the output of the return visit memory unit, the predicted reward value is further obtained, and the loss function is calculated by combining the actual reward value. And performing iterative optimization on the model based on the loss function, and finally applying the trained model to the main cluster to realize resource allocation and service processing.

Based on the introduction of the above embodiment, it can be seen that, when receiving a service to be processed, the method grasps the specific content of the service to be processed by obtaining the history processing record and the service parameter of the service to be processed, and further can determine the service characteristics of the service to be processed, and determines the resources required to be consumed for processing the service to be processed according to the service characteristics and the resource scheduling amount corresponding to the service characteristics, and further allocates the corresponding resource groups to the service to be processed, so that the processing of the service to be processed can be completed by using the resources in the resource groups. By the method, the resource group allocated to the service to be processed can be determined based on the actual application condition of the service to be processed, so that flexible and effective allocation of resources is ensured, effective processing of the service is realized, and the overall processing efficiency of the service is improved.

A service processing apparatus based on resource allocation according to an embodiment of the present specification is introduced based on a service processing method based on resource allocation corresponding to fig. 1. The service processing device based on resource allocation is arranged in the resource allocation equipment. As shown in fig. 5, the service processing apparatus based on resource allocation includes the following modules.

A service receiving module 510, configured to receive a service to be processed.

A parameter obtaining module 520, configured to obtain a history processing record and a service parameter of the service to be processed; the service parameter is used for representing the execution content of the service to be processed.

A service characteristic determining module 530, configured to determine a service characteristic of the service to be processed by using the historical processing record and the service parameter; the service characteristics correspond to resource scheduling amounts; the resource scheduling amount represents an amount of resources consumed to process the corresponding traffic characteristic.

And a resource group allocation module 540, configured to allocate a resource group for the service to be processed based on the service characteristics and the resource scheduling amount, so that the processing of the service to be processed is completed through resources in the resource group.

Based on the service processing method based on resource allocation corresponding to fig. 1, an embodiment of the present specification provides a resource allocation device. As shown in fig. 6, the resource allocation device may include a memory and a processor.

In this embodiment, the memory may be implemented in any suitable manner. For example, the memory may be a read-only memory, a mechanical hard disk, a solid state disk, a U disk, or the like. The memory may be used to store computer program instructions.

In this embodiment, the processor may be implemented in any suitable manner. For example, the processor may take the form of, for example, a microprocessor or processor and a computer-readable medium that stores computer-readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, an Application Specific Integrated Circuit (ASIC), a programmable logic controller, an embedded microcontroller, and so forth. The processor may execute the computer program instructions to perform the steps of: receiving a service to be processed; acquiring a history processing record and a service parameter of the service to be processed; the service parameter is used for representing the execution content of the service to be processed; determining the service characteristics of the service to be processed by using the historical processing record and the service parameters; the service characteristics correspond to resource scheduling amounts; the resource scheduling amount represents an amount of resources consumed for processing the corresponding service feature; and allocating resource groups for the service to be processed based on the service characteristics and the resource scheduling amount, so that the service to be processed is processed through the resources in the resource groups.

It should be noted that the service processing method, device and apparatus based on resource allocation may be applied to the technical field of artificial intelligence, and may also be applied to other technical fields except the technical field of artificial intelligence, which is not limited to this.

In the 90 s of the 20 th century, improvements in a technology could clearly distinguish between improvements in hardware (e.g., improvements in circuit structures such as diodes, transistors, switches, etc.) and improvements in software (improvements in process flow). However, as technology advances, many of today's process flow improvements have been seen as direct improvements in hardware circuit architecture. Designers almost always obtain the corresponding hardware circuit structure by programming an improved method flow into the hardware circuit. Thus, it cannot be said that an improvement in the process flow cannot be realized by hardware physical modules. For example, a Programmable Logic Device (PLD), such as a Field Programmable Gate Array (FPGA), is an integrated circuit whose Logic functions are determined by programming the Device by a user. A digital system is "integrated" on a PLD by the designer's own programming without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Furthermore, nowadays, instead of manually making an Integrated Circuit chip, such Programming is often implemented by "logic compiler" software, which is similar to a software compiler used in program development and writing, but the original code before compiling is also written by a specific Programming Language, which is called Hardware Description Language (HDL), and HDL is not only one but many, such as abel (advanced Boolean Expression Language), ahdl (alternate Hardware Description Language), traffic, pl (core universal Programming Language), HDCal (jhdware Description Language), lang, Lola, HDL, laspam, hardward Description Language (vhr Description Language), vhal (Hardware Description Language), and vhigh-Language, which are currently used in most common. It will also be apparent to those skilled in the art that hardware circuitry that implements the logical method flows can be readily obtained by merely slightly programming the method flows into an integrated circuit using the hardware description languages described above.

The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

From the above description of the embodiments, it is clear to those skilled in the art that the present specification can be implemented by software plus the necessary first hardware platform. Based on such understanding, the technical solutions of the present specification may be essentially or partially implemented in the form of software products, which may be stored in a storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and include instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments of the present specification.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The description is operational with numerous first or special purpose computing system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.

This description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

While the specification has been described with examples, those skilled in the art will appreciate that there are numerous variations and permutations of the specification that do not depart from the spirit of the specification, and it is intended that the appended claims include such variations and modifications that do not depart from the spirit of the specification.