1. A heterogeneous acceleration method is applied to a longitudinal logistic regression learning process, and is characterized by comprising the following steps:

a first participant and a second participant perform plaintext operation and encryption operation according to respective feature data to respectively obtain an encrypted model parameter of the first participant and an encrypted model parameter of the second participant, wherein the feature data is data in a mini-batch subset of a data set, and the first participant and the second participant respectively hold a part of data of the mini-batch subset;

the first party or the second party performs cryptographic addition operation on the encrypted model parameter of the first party and the encrypted model parameter of the second party to obtain a forward gradient corresponding to the characteristic data;

the first party and the second party respectively carry out secret multiplication operation on the forward gradient and the respective feature data to obtain secret multiplication operation results corresponding to the respective feature data; and

performing mask accumulation operation on the secret multiplication operation result and a mask corresponding to the feature data to obtain a mask accumulation operation result corresponding to the feature data, and updating a gradient calculation result corresponding to the mini-batch subset,

the plaintext operation is used for serialization processing and corresponds to a serialization calculation task, and the accumulation calculation parts in the encryption operation, the secret addition operation, the secret multiplication operation and the mask accumulation operation are used for parallelization processing and respectively correspond to the parallelization calculation task.

2. The method of claim 1, wherein the plaintext operations are used to serialize processing and correspond to serialized computing tasks, comprising: serialized computational tasks corresponding to the plaintext operations are executed on a CPU basis.

3. The method of claim 1 or 2, wherein the cryptographic operation, the cryptographic addition operation, the cryptographic multiplication operation, and the mask accumulation operation are performed in parallel and each correspond to a parallel computing task, comprising: and executing a parallelization calculation task corresponding to the encryption operation, a parallelization calculation task corresponding to the secret addition operation, a parallelization calculation task corresponding to the secret multiplication operation and a parallelization calculation task corresponding to an accumulation calculation part in the mask accumulation operation on the basis of a GPU.

4. The method of claim 3, wherein the vector computation portion of the cryptographic operation comprises a CPU-GPU interaction, the CPU-GPU interaction comprising the GPU passing a GPU memory pointer to the CPU, the GPU memory pointer pointing to data copied from the CPU in the GPU's memory.

5. The method according to claim 1, wherein the serialized computational tasks corresponding to the plaintext operations are performed based on a first class of computational units, and the parallelized computational tasks corresponding to the cryptographic operations, the parallelized computational tasks corresponding to the cryptographic addition operations, the parallelized computational tasks corresponding to the cryptographic multiplication operations, and the parallelized computational tasks corresponding to the accumulation computation portion of the mask accumulation operations are performed based on a second class of computational units, wherein the first and second classes of computational units belong to the same heterogeneous computational architecture and are optimized for serial and parallel computations, respectively.

6. The method of claim 5, wherein the first class of computing units comprises at least one of: CPU, RISC, microchip, said second type of computing chip includes at least one of: ASIC for longitudinal logistic regression learning, FPGA optimized for matrix operation.

7. The method of claim 1, wherein the cryptographic addition operation and the accumulation calculation portion of the mask accumulation operation are each performed by one of the first or second parties and share the result of the respective operations with the other party.

8. The method of claim 1, wherein the accumulation calculation portions of the encryption operation, the secret addition operation, the secret multiplication operation, and the mask accumulation operation are each performed by a GPU optimized for vector level calculations.

9. The method of claim 8, wherein the accumulation calculation portion of the cryptographic addition operation and/or the mask accumulation operation, when executed, invokes the first party's computational resources and the second party's computational resources simultaneously.

10. A non-transitory computer readable storage medium holding computer instructions which, when executed by a processing device, cause the processing device to perform the method of any one of claims 1 to 9.

11. A heterogeneous acceleration device, characterized in that it comprises:

the model parameter calculation module is used for carrying out plaintext calculation according to local feature data to obtain a model parameter, wherein the local feature data is data in a mini-batch subset of a plurality of mini-batch subsets of a data set, the data set is used for training a federated learning model, and the local feature data is private data at the heterogeneous accelerator;

the encryption operation module is used for carrying out encryption operation on the model parameters to obtain encrypted model parameters;

a secret addition module, configured to perform a secret addition operation on the encrypted model parameters and non-local encrypted model parameters to obtain a forward gradient corresponding to the local feature data, where the non-local encrypted model parameters are provided from an external device with respect to the heterogeneous acceleration device;

the dense state multiplication module is used for carrying out dense state multiplication operation on the forward gradient and the local feature data to obtain a dense state multiplication operation result corresponding to the local feature data; and

an accumulation operation module, configured to perform a mask accumulation operation on the secret multiplication operation result, so as to update a gradient calculation result corresponding to the mini-batch subset including the local feature data,

the encryption operation module, the secret addition module, the secret multiplication module and the accumulation operation module are all optimized for vector-level parallelization calculation.

12. The heterogeneous acceleration device of claim 11, characterized in that it further comprises: and the data segmentation module is used for segmenting the data set to obtain the plurality of mini-batch subsets.

13. The heterogeneous acceleration device of claim 11, characterized in that it further comprises: and the decryption operation module is used for carrying out decryption operation on the gradient calculation result to obtain the gradient corresponding to the mini-batch subset comprising the local characteristic data.

14. The heterogeneous acceleration device of claim 11, wherein the cryptographic operation module, the secret addition module, the secret multiplication module, and the accumulation operation module are optimized for vector-level parallelized computations, comprising: the secret addition module, the secret multiplication module and the accumulation operation module respectively comprise a plurality of parallel computing units, and each computing unit of the plurality of parallel computing units optimizes matrix addition, matrix multiplication and matrix dot multiplication under the condition of ciphertext packaging.

15. The heterogeneous acceleration device of claim 11, wherein the vector computation portion of the cryptographic operations of the cryptographic operation module comprises a CPU-GPU interaction, the CPU-GPU interaction comprising the GPU passing a GPU video memory pointer to the CPU, the GPU video memory pointer pointing to data in the video memory of the GPU copied from the CPU.

16. A heterogeneous acceleration system that includes a first heterogeneous acceleration device of a first participant and a second heterogeneous acceleration device of a second participant, the first and second heterogeneous acceleration devices including a first dense-state vector calculation section and a second dense-state vector calculation section, respectively, the heterogeneous acceleration system performing a longitudinal logistic regression learning process between the first participant and the second participant according to the following method:

the first heterogeneous acceleration device performs plaintext operation and encryption operation according to private data of a first participant to obtain encrypted model parameters of the first participant, and the second heterogeneous acceleration device performs plaintext operation and encryption operation according to private data of a second participant to obtain encrypted model parameters of the second participant, wherein the private data of the first participant and the private data of the second participant are data in the same mini-batch subset of a data set used for the longitudinal logistic regression learning process;

the heterogeneous acceleration system selects the first secret vector calculation part or the second secret vector calculation part, and performs secret addition operation on the encrypted model parameters of the first participant and the encrypted model parameters of the second participant to obtain a forward gradient;

the first secret state vector calculation part and the second secret state vector calculation part respectively carry out secret state multiplication on the forward gradient and respective private data to obtain a secret state multiplication result; and

performing mask accumulation operation on the secret multiplication operation result to obtain a mask accumulation operation result, updating a gradient calculation result corresponding to the mini-batch subset,

wherein the first dense state vector calculation section and the second dense state vector calculation section are both optimized for vector-level parallelization calculation.

17. The heterogeneous acceleration system of claim 16, wherein the first dense state vector computation portion and the second dense state vector computation portion are each optimized for vector-level parallelization computations, comprising: the first dense state vector calculation part and the second dense state vector calculation part respectively comprise at least one parallel GPU, and each GPU of the at least one parallel GPU is optimized for matrix addition, matrix multiplication and matrix dot multiplication under a condition of packaging ciphertext.

Background

With the development of application fields such as artificial intelligence and big data mining analysis, the demand for data volume is more and more increased. For example, training artificial intelligence application models requires the use of large amounts of training data with appropriate data labels or feature values. High quality data often comes from application data generated and accumulated in business activities. However, application data is often distributed among different organizations and individuals, for example, transaction data is distributed among various financial institutions and medical diagnosis data is distributed among various medical institutions. Application data across industries and domains is also dispersed, for example, social attribute data and e-commerce transaction data in the internet domain are controlled by different entities. As the importance of data ownership, user privacy, data security, and the like are more emphasized, and as the laws and regulations put more strict constraints and requirements on data collection processing, organizations or individuals who grasp application data are often unwilling or do not have appropriate means to collaborate with each other, so that it is difficult for the application data grasped by each organization or individual to work together. This dilemma in data sharing and collaborative collaboration is referred to as data islanding. In order to solve the problem of cross-industry and cross-organization data cooperation, especially the key problems of privacy protection and data security, a Federal Learning (FL) concept is proposed. The federated learning refers to each participant who owns data, and under the premise that protected private data is not shared and the own data is not transmitted to the outside, the relevant information of the model is exchanged in an encryption mode, so that the collaborative optimization of the federated learning model is realized. The federated learning can be divided into horizontal federated learning with large overlap in the data feature space and small overlap in the sample space, vertical federated learning with small overlap in the data feature space and large overlap in the sample space, and federated migration learning with small overlap in both the data feature space and the sample space according to the distribution conditions of the data feature space and the sample space of the training data.

Among them, a Logistic Regression (LR) algorithm is often used to solve the problem of binary classification in the machine learning method and is widely used, for example, to estimate the probability of purchasing goods or predict the probability of illness for a user. The basic derivation process of the logistic regression algorithm is to provide a hypothesis function, which is also called a logistic regression model, then construct a loss function, and finally obtain parameter values, such as weight values, in the hypothesis function by solving the loss function. Each data in the data set can be solved for a gradient once in each iteration through a random gradient descent method, the weight value is updated through the gradient obtained through solving, and multiple iterations are repeated until the weight value meeting the requirement is solved. The solving of the gradient may use a parallelization acceleration scheme, for example, by a Central Processing Unit (CPU) or a Graphics Processing Unit (GPU). However, in the application scenario of federal learning or similar privacy calculations, raw data is scattered on each participant, and model training or logistic regression learning exchanges model-related information in an encrypted manner, for example, each participant holds a part of a data set in the scenario of longitudinal federal logistic regression learning. The goal of logistic regression learning must be achieved with the assurance that the data set is not compromised. In addition, the encrypted data is generally large integer bit width data, for example, the original data is a floating point number but is generally at least 1024 bits after encryption, and the federal learning also involves a large number of secret operations, so that a great challenge from storage resources to computational performance is provided for model training or logistic regression learning in application scenarios such as federal learning or similar privacy computation. Therefore, a heterogeneous acceleration method, a heterogeneous acceleration device and a heterogeneous acceleration system are needed, which can achieve a model training target, particularly complete longitudinal federated logistic regression learning, on the premise of ensuring that a data set is not leaked, and can well meet the requirements on storage resources and calculation performance brought by large integer bit width data and secret operation in application scenes such as federated learning or similar privacy calculation.

Disclosure of Invention

In a first aspect, an embodiment of the present application provides a heterogeneous acceleration method, which is applied to a longitudinal logistic regression learning process. The method comprises the following steps: a first participant and a second participant perform plaintext operation and encryption operation according to respective feature data to respectively obtain an encrypted model parameter of the first participant and an encrypted model parameter of the second participant, wherein the feature data is data in a mini-batch subset of a data set, and the first participant and the second participant respectively hold a part of data of the mini-batch subset; the first party or the second party performs cryptographic addition operation on the encrypted model parameter of the first party and the encrypted model parameter of the second party to obtain a forward gradient corresponding to the characteristic data; the first party and the second party respectively carry out secret multiplication operation on the forward gradient and the respective feature data to obtain secret multiplication operation results corresponding to the respective feature data; and performing mask accumulation operation on the secret multiplication operation result and a mask corresponding to the characteristic data to obtain a mask accumulation operation result corresponding to the characteristic data, and updating a gradient calculation result corresponding to the mini-batch subset. The plaintext operation is used for serialization processing and corresponds to a serialization calculation task, and the accumulation calculation parts in the encryption operation, the secret addition operation, the secret multiplication operation and the mask accumulation operation are used for parallelization processing and respectively correspond to the parallelization calculation task.

According to the technical scheme described in the first aspect, the algorithm of the longitudinal logistic regression learning process is split, corresponding operation is performed by pertinently allocating corresponding modules, the advantages of a heterogeneous computing architecture are favorably exerted to improve the resource utilization efficiency, and the resource utilization efficiency can be further improved by sharing operation results through different participants.

According to the technical solution of the first aspect, in a possible implementation manner, an embodiment of the present application further provides that the plaintext operation is used for serializing processing and corresponds to a serialized computing task, and the method includes: serialized computational tasks corresponding to the plaintext operations are executed on a CPU basis.

According to the technical solution of the first aspect, in a possible implementation manner, an accumulation calculation part in the encryption operation, the secret addition operation, the secret multiplication operation, and the mask accumulation operation is used for parallelization and corresponds to a parallelization calculation task, and includes: and executing a parallelization calculation task corresponding to the encryption operation, a parallelization calculation task corresponding to the secret addition operation, a parallelization calculation task corresponding to the secret multiplication operation and a parallelization calculation task corresponding to an accumulation calculation part in the mask accumulation operation on the basis of a GPU.

According to the technical solution of the first aspect, in a possible implementation manner, an embodiment of the present application further provides that the vector calculation part in the encryption operation includes CPU-GPU interaction, and the CPU-GPU interaction includes exchange between a CPU memory pointer and a GPU video memory pointer, where the CPU memory pointer points to unencrypted data in a GPU video memory, the GPU video memory pointer points to encrypted data in the GPU video memory, the GPU transmits the GPU video memory pointer to the CPU, and the GPU video memory pointer points to data copied from the CPU in the video memory of the GPU.

According to the technical solution of the first aspect, in a possible implementation manner, an embodiment of the present application further provides that a serialized computation task corresponding to the plaintext operation is executed based on a first class of computation units, and a parallelized computation task corresponding to the cryptographic operation, a parallelized computation task corresponding to the cryptographic addition operation, a parallelized computation task corresponding to the cryptographic multiplication operation, and a parallelized computation task corresponding to an accumulation computation part in the mask accumulation operation are all executed based on a second class of computation units, where the first class of computation units and the second class of computation units belong to a same heterogeneous computation architecture and are optimized for serial computation and parallel computation, respectively.

According to the technical solution of the first aspect, in a possible implementation manner, an embodiment of the present application further provides that the first type of computing unit includes at least one of: CPU, RISC, microchip, said second type of computing chip comprising at least one of: ASIC for longitudinal logistic regression learning, FPGA optimized for matrix operation.

According to the technical solution of the first aspect, in a possible implementation manner, an embodiment of the present application further provides that the accumulation calculating portions in the secret state addition operation and the mask accumulation operation are respectively completed by one of the first participant or the second participant and the participant shares a corresponding operation result with the other participant.

According to the technical solution of the first aspect, in a possible implementation manner, an accumulation calculation part in the encryption operation, the secret addition operation, the secret multiplication operation, and the mask accumulation operation is performed by a GPU optimized for vector-level calculation.

According to the technical solution of the first aspect, in a possible implementation manner, an embodiment of the present application further provides that the accumulation calculating part in the secret state addition operation and/or the mask accumulation operation calls the calculating resource of the first participant and the calculating resource of the second participant simultaneously when being executed.

In a second aspect, embodiments of the present application provide a non-transitory computer-readable storage medium holding computer instructions that, when executed by a processing apparatus, cause the processing apparatus to perform a method according to any one of the first aspects.

In the technical scheme described in the second aspect, the algorithm of the longitudinal logistic regression learning process is split and corresponding operation is performed by pertinently allocating corresponding modules, so that the advantages of a heterogeneous computing architecture are brought into play to improve the resource utilization efficiency, and the resource utilization efficiency can be further improved by sharing operation results through different participants.

In a third aspect, an embodiment of the present application provides a heterogeneous acceleration device. The heterogeneous acceleration apparatus includes: the model parameter calculation module is used for carrying out plaintext calculation according to local feature data to obtain a model parameter, wherein the local feature data is data in a mini-batch subset of a plurality of mini-batch subsets of a data set, the data set is used for training a federated learning model, and the local feature data is private data at the heterogeneous accelerator; the encryption operation module is used for carrying out encryption operation on the model parameters to obtain encrypted model parameters; a secret addition module, configured to perform a secret addition operation on the encrypted model parameters and non-local encrypted model parameters to obtain a forward gradient corresponding to the local feature data, where the non-local encrypted model parameters are provided from an external device with respect to the heterogeneous acceleration device; the dense state multiplication module is used for carrying out dense state multiplication operation on the forward gradient and the local feature data to obtain a dense state multiplication operation result corresponding to the local feature data; and the accumulation operation module is used for carrying out mask accumulation operation on the secret multiplication operation result so as to update the gradient calculation result corresponding to the mini-batch subset comprising the local characteristic data. The encryption operation module, the secret addition module, the secret multiplication module and the accumulation operation module are all optimized for vector-level parallelization calculation.

According to the technical scheme described in the third aspect, the algorithm of the longitudinal logistic regression learning process is split, corresponding operation is performed by allocating corresponding modules in a targeted manner, the advantages of a heterogeneous computing architecture are favorably exerted to improve the resource utilization efficiency, and the resource utilization efficiency can be further improved by sharing operation results through different participants.

According to the technical solution of the third aspect, in a possible implementation manner, an embodiment of the present application further provides that the heterogeneous acceleration apparatus further includes: and the data segmentation module is used for segmenting the data set to obtain the plurality of mini-batch subsets.

According to the technical solution of the third aspect, in a possible implementation manner, an embodiment of the present application further provides that the heterogeneous acceleration apparatus further includes: and the decryption operation module is used for carrying out decryption operation on the gradient calculation result to obtain the gradient corresponding to the mini-batch subset comprising the local characteristic data.

According to the technical solution of the third aspect, in a possible implementation manner, an embodiment of the present application further provides that the cryptographic operation module, the secret state addition module, the secret state multiplication module, and the accumulation operation module are optimized for vector-level parallelized computation, and include: the secret addition module, the secret multiplication module and the accumulation operation module respectively comprise a plurality of parallel computing units, and each computing unit of the plurality of parallel computing units optimizes matrix addition, matrix multiplication and matrix dot multiplication under the condition of ciphertext packaging.

According to the technical solution of the third aspect, in a possible implementation manner, an encryption calculation vector calculation part of the encryption calculation module further includes CPU-GPU interaction, where the CPU-GPU interaction includes exchange between a CPU memory pointer and a GPU video memory pointer, where the CPU memory pointer points to unencrypted data in a GPU video memory, the GPU video memory pointer points to encrypted data in the GPU video memory, the GPU video memory pointer transmits the GPU video memory pointer to a CPU, and the GPU video memory pointer points to data copied from the CPU in the video memory of the GPU.

In a fourth aspect, an embodiment of the present application provides a heterogeneous acceleration system. The heterogeneous acceleration system comprises a first heterogeneous acceleration device of a first participant and a second heterogeneous acceleration device of a second participant, wherein the first heterogeneous acceleration device and the second heterogeneous acceleration device respectively comprise a first dense-state vector calculation part and a second dense-state vector calculation part, and the heterogeneous acceleration system performs a longitudinal logistic regression learning process between the first participant and the second participant according to the following method: the first heterogeneous acceleration device performs plaintext operation and encryption operation according to private data of a first participant to obtain encrypted model parameters of the first participant, and the second heterogeneous acceleration device performs plaintext operation and encryption operation according to private data of a second participant to obtain encrypted model parameters of the second participant, wherein the private data of the first participant and the private data of the second participant are data in the same mini-batch subset of a data set used for the longitudinal logistic regression learning process; the heterogeneous acceleration system selects the first secret vector calculation part or the second secret vector calculation part, and performs secret addition operation on the encrypted model parameters of the first participant and the encrypted model parameters of the second participant to obtain a forward gradient; the first secret state vector calculation part and the second secret state vector calculation part respectively carry out secret state multiplication on the forward gradient and respective private data to obtain a secret state multiplication result; and performing mask accumulation operation on the secret multiplication operation result to obtain a mask accumulation operation result, and updating the gradient calculation result corresponding to the mini-batch subset. Wherein the first dense state vector calculation section and the second dense state vector calculation section are both optimized for vector-level parallelization calculation.

In the technical scheme described in the fourth aspect, the algorithm of the longitudinal logistic regression learning process is split, and corresponding operation is performed by pertinently allocating corresponding modules, so that the advantages of a heterogeneous computing architecture are brought into play to improve the resource utilization efficiency, and the resource utilization efficiency can be further improved by sharing operation results through different participants.

According to the technical solution of the fourth aspect, in a possible implementation manner, an embodiment of the present application further provides that the first dense state vector calculating unit and the second dense state vector calculating unit are optimized for vector-level parallelization calculation, and the method includes: the first dense state vector calculation part and the second dense state vector calculation part respectively comprise at least one parallel GPU, and each GPU of the at least one parallel GPU is optimized for matrix addition, matrix multiplication and matrix dot multiplication under a condition of packaging ciphertext.

Drawings

In order to explain the technical solutions in the embodiments or background art of the present application, the drawings used in the embodiments or background art of the present application will be described below.

Fig. 1 shows a flowchart of a heterogeneous acceleration method provided in an embodiment of the present application.

Fig. 2 illustrates a block diagram of a heterogeneous acceleration apparatus used in the heterogeneous acceleration method illustrated in fig. 1 according to an embodiment of the present application.

Fig. 3 is a block diagram illustrating a multi-party heterogeneous acceleration system according to an embodiment of the present disclosure.

Fig. 4 is a block diagram illustrating a multi-party heterogeneous acceleration system according to another embodiment provided in an embodiment of the present application.

Detailed Description

The embodiment of the application provides a heterogeneous acceleration method, a heterogeneous acceleration device and a heterogeneous acceleration system, and aims to solve the technical problem of how to realize a model training target, especially complete longitudinal federated logistic regression learning on the premise of ensuring that a data set is not leaked, and meanwhile, the method can well meet the requirements on storage resources and calculation performance brought by large-integer bit width data and secret operation in the application scenes of federated learning or similar privacy calculation and the like. The method comprises the following steps: a first participant and a second participant perform plaintext operation and encryption operation according to respective feature data to respectively obtain an encrypted model parameter of the first participant and an encrypted model parameter of the second participant, wherein the feature data is data in a mini-batch subset of a data set, and the first participant and the second participant respectively hold a part of data of the mini-batch subset; the first party or the second party performs cryptographic addition operation on the encrypted model parameter of the first party and the encrypted model parameter of the second party to obtain a forward gradient corresponding to the characteristic data; the first party and the second party respectively carry out secret multiplication operation on the forward gradient and the respective feature data to obtain secret multiplication operation results corresponding to the respective feature data; and performing mask accumulation operation on the secret multiplication operation result and a mask corresponding to the characteristic data to obtain a mask accumulation operation result corresponding to the characteristic data, and updating a gradient calculation result corresponding to the mini-batch subset. The plaintext operation is used for serialization processing and corresponds to a serialization calculation task, and the accumulation calculation parts in the encryption operation, the secret addition operation, the secret multiplication operation and the mask accumulation operation are used for parallelization processing and respectively correspond to the parallelization calculation task.

Embodiments of the application may be used in application scenarios including, but not limited to, machine learning model training in connection with federated learning, logistic regression learning, particularly longitudinal federated logistic regression learning, data security, privacy protection, or other application scenarios applying a privacy computing framework or algorithm.

The embodiments of the present application may be modified and improved according to specific application environments, and are not limited herein.

In order to make the technical field of the present application better understand, embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.

Generally, in application scenarios such as federal learning or similar privacy calculations, each participant performs model training locally and data does not leave the local, but establishes a shared machine learning model by exchanging model parameters. The common algorithms in the longitudinal federated learning include longitudinal linear regression, longitudinal logistic regression (Hetero-LR), longitudinal poisson regression, and other algorithms suitable for longitudinal federated learning, such as SecureBoost algorithm. It is common for longitudinal federal learning to be the case that one participant has a portion of features and labels, but another participant has another portion of features but lacks labels. The participant lacking the tag needs to be assisted by the participant with the tag to establish the joint model, and the effect of the joint model is obviously better than the result of independent modeling of the participant with the tag. Let the off-label party be the Host party of federal learning and the labeled party be the Guest party. A typical learning process of vertical logistic regression hetro-LR may be referred to in the paper "Private fed learning on vertical partial data vitamin evolution and adaptive homomorphic encryption" published in 2017, the first author being "Stephen Hardy" and for illustrative but not limiting purposes only, which paper is to be understood as part of the present disclosure. Among them, the learning process of the hetro-LR involves an arbitrator (Arbiter) as a third party, specifically including: the Guest party requests feature data held by the Host party from the Host party; the Host party sends the encrypted feature data of the Host party to the Guest party; the Guest party also encrypts the feature data held by the Guest party, performs a cryptographic operation according to the encrypted feature data of the Host party and the encrypted feature data of the Guest party to obtain a gradient factor, calculates according to the gradient factor to obtain the gradient of the encrypted Guest party and sends the gradient factor to the Host party; the Host side calculates the gradient of the encrypted Host side according to the gradient factor; the Guest party and the Host party respectively send the gradient of the encrypted Guest party and the gradient of the encrypted Host party to the judging party, the judging party performs splicing and decryption to obtain the gradient of the decrypted Guest party and the gradient of the decrypted Host party, and then the gradient of the decrypted Guest party and the gradient of the decrypted Host party are respectively sent to the Guest party and the Host party; and finally, updating the model by the Guest party and the Host party according to the gradient of the decrypted Guest party and the gradient of the decrypted Host party respectively, thereby finishing one-time iterative computation. The encrypted loss function can be calculated by a Guest party and is sent to a judging party, the judging party decrypts the loss function to obtain the decrypted loss function, and whether the model training is terminated is judged according to convergence conditions and the like. The above is the learning process of the Hetero-LR with two participants plus a arbitrator or the process of training the vertical logistic regression model. When there are three or more participants, i.e. the provider of the data involves at least three or more participants, especially in the context of longitudinal federal learning, these participants have many identical entities but different characteristics. With reference to the above learning process of hetro-LR, in a general model involving a Guest party, a Host party, and an Arbiter party, each two participating parties perform the above training process as a pair of the Host party and the Guest party, or after expansion, the training process includes exchange of encrypted feature data by more participating parties, calculation of gradient factors, and solution of respective encrypted gradients; in addition, in the learning process of the hetro-LR, there are models which relate to a single Guest party, a plurality of Host parties and an Arbiter party after expansion, and the training process also relates to the exchange of encrypted feature data, the calculation of gradient factors, the solution of respective encrypted gradients, and the like. These may be adjusted according to actual needs and application scenarios, and are not specifically limited herein.

As can be seen from the above-described learning process of the hetro-LR, both the Host party and the Guest party need to calculate the encrypted feature data and solve the encrypted gradient according to the gradient factor, and the Guest party needs to perform the secret calculation according to the encrypted feature data of each participant party to obtain the common gradient factor. Here, the encrypted feature data and the encrypted gradient are both large integer bit width data obtained after encryption, for example, data encrypted by a homomorphic encryption method. In order to improve the overall computational efficiency by utilizing parallel acceleration, the training data or the data set can be divided into several subsets (for example, into equal subsets) by a mini-batch technology in a way of parallel processing by a plurality of GPUs, and when the size of the training data or the data set is too large, gradient descent is performed on each subset. Each iterative computation needs to traverse all samples or all subsets, and complex computations including plaintext operation, encryption operation and ciphertext operation need to be performed on the same sample or the same subset, and data interaction between different computing units such as GPUs may be involved. These and other improvements and advantages of the embodiments of the present application will be described in detail below with reference to the drawings.

Referring to fig. 1, fig. 1 is a schematic flow chart illustrating a heterogeneous acceleration method according to an embodiment of the present application. The heterogeneous acceleration method 100 shown in fig. 1 is described with reference to the following equations (1) to (6).

The above equations (1) to (6) are used for traversing all data of the data set S, calculating the encrypted feature data, and solving the encrypted gradient, and are applicable to the longitudinal federal logistic regression learning and also applicable to the above learning process of the hetro-LR. Where equation (1) shows how the loss function trained with the data set S is calculated, equation (1) is described in detail below.

In formula (1), S is a data set, and the data set S is represented as (X)₁，Y₁），（X₂，Y₂），（X₃，Y₃) Up to (X)_N，Y_N). Where Y is the label quantity or a binary label such as-1 and 1, and X is the feature data, so that the data set S has N pairs (X)_i，Y_i) I is a sequence number from 1 to N, N being understood as the total number of samples of the data set S. L is_S(θ) is a loss function, θ is a weight, and θ^TX_iIs directed to the feature data X_iThe model parameters of (1). Equation (1) means that all data belonging to the data set S need to be traversed, i.e. i goes from 1 to N, and for each pair (X) is paired_i，Y_i) Corresponding iterative calculations are performed.

In longitudinal federated logistic regression learning or similar application scenarios, the data set S is dispersed among different participants, for example, in the learning process of the above-mentioned Hetero-LR, the Host party and the Guest party respectively hold a part of the data set S, and the loss function L is solved_SAnd (theta) requires that the Host party and the Guest party contribute all data respectively. For convenience of presentation, it may be assumed that the data set S is divided into a first portion and a second portion, with a first party holding the first portion of the data set S and a second party holding the second portion of the data set S. Here, the first and second parties are relative concepts for indicating a party holding private data in the context of federal learning and that party does not wish to reveal the private data to other parties. The first party may be acting as a Guest party, a Host party, an Arbiter party (Arbiter) or any suitable role. In contrast, the second party may also play any suitable role, as long as the first party and the second party maySo as to exchange the encrypted characteristic data and finally solve the loss function of the federal learning model. Equation (2) shows how the model parameter θ is scaled^TX_iThe calculation of (a) is split into secret operations that can each be performed locally by the first party and the second party, formula (2) being described in detail below.

In the formula (2), θ^TX_iIs directed to the feature data X_iModel parameter of (a), theta^TX_i(Host) is the secret operation performed locally by the Host side, namely, the encrypted model parameters of the Host side are obtained by performing plaintext operation and encryption operation according to the feature data of the Host side. Relative to the ground, θ^TX_i(Guest) is a cryptographic operation locally performed by the Guest party, that is, a plaintext operation and an encryption operation are performed according to the feature data of the Guest party to obtain an encrypted model parameter of the Guest party. The formula (2) means that the Host party and the Guest and party perform plaintext operation and encryption operation according to their respective feature data to obtain the encrypted model parameter of the Host party and the encrypted model parameter of the Guest party, respectively, and it can also be understood that the first party and the second party perform plaintext operation and encryption operation according to their respective feature data to obtain the encrypted model parameter of the first party and the encrypted model parameter of the second party, respectively. Equations (1) and (2) are directed to the entirety of the data set S. In general, the data set S contains a large number of samples, i.e. the total number N of samples of the data set S may be large. In this case, the data set S may be partitioned into several subsets (e.g., into equal number of subsets) by a mini-batch technique, with gradient descent performed for each subset. Accordingly, traversing all the data of the data set S translates to traversing all the subsets equivalently. Here, it is assumed that one of the plurality of subsets into which the data set S is divided according to the mini-batch technique is SM. Equation (3) shows how the gradient calculation of the mini-batch subset SM of the data set S is performed, and equation (3) is explained in detail below.

In equation (3), SM is the subset, and to the left of equation (3) is the loss function L of the subset SM_SMThe gradient operator of (theta) is expressed asThe inverted triangle, i.e. the gradient calculation result of the subset SM. To the right of equation (3) is a calculation formula that applies to vertical federated logistic regression learning as well as to the learning process of the above-described hetro-LR. In particular, the sequence number of the data in the subset SM is denoted by j, thereby being distinguished from the sequence number i in equation (1) used to represent the data in the data set S. Let the total number of samples in the subset SM be SM, so j is a sequence number from 1 to SM. Theta^TX_jIs directed to the feature data X_jModel parameters of, M_jIt is the mask that tells which data are overlapping, and in equation (3) the traversal is performed for all data in the subset SM and for each of them the complex calculations including the plaintext operation, the encryption operation and the crypto operation are performed. After the gradient is calculated according to equation (3), the model parameters can be updated accordingly, so that a mini-batch calculation is completed once or a subset SM calculation is completed. Reference may be made in this respect to the above-mentioned first article published in 2017 by the author "Stephen Hardy", which is understood to be part of the disclosure of the present application, and details thereof, in particular the mathematical derivation thereof, will not be described in detail herein. The specific data X in the formula (3) for the subset SM is described below in conjunction with the formulas (4) to (6)_jHow the calculation of (c) is spread out.

Equation (4) shows how the Forward Gradient (FG) is calculated, which is part of equation (3). As shown in formula (4), the FG calculation requires a secret operation performed locally by each of the first and second parties. Specifically, the FG calculation requires the first and second parties to perform plaintext operation and encryption operation according to their respective feature data to obtain the encrypted model parameters of the first party and the encrypted model parameters of the second party, respectively, and then integrate the encrypted model parameters of the first party and the encrypted model parameters of the second party through exchange to obtain the complete FG. Taking the above-mentioned learning process of the hetro-LR as an example, FG can be understood as obtaining a gradient factor shared between two parties after performing a secret operation according to feature data encrypted by the two parties. In addition, the first and second substrates are,because plaintext operation and encryption operation are involved, the FG calculation shown in formula (4) must first obtain the encrypted model parameters of each participant by performing plaintext operation and encryption operation locally by each participant, and then calculate the complete FG by integrating the respective encrypted model parameters. It should be understood that, because it is in the application scenarios of federal learning and privacy computation, in order to compute the complete FG in equation (4), a secret addition operation needs to be performed on the encrypted feature data provided by each participant, and the obtained forward gradient FG is also an encrypted state, that is, belongs to secret data. When the specific data X is obtained by calculation according to the formula (4)_jForward gradient FG of_jThereafter, the specific data X can be calculated by equation (5)_jThe result MUL of the dense multiplication_j. In the formula (5), specific data X is specified_jForward gradient FG of_jWith specific data X_jAnd performing multiplication operation. It should be understood that in the context of federal learning and privacy calculations, the multiplication of equation (5) may be a secret multiplication between two secret data or a multiplication of secret data and plaintext data, depending on whether the encryption algorithm is a fully homomorphic encryption or a semi-homomorphic encryption. Specifically, when the encryption algorithm is a fully homomorphic encryption algorithm, the multiplication operation of formula (5) is a secret multiplication operation between two secret data, that is, the forward gradient FG after encryption is applied_jWith encrypted specific data X_jAnd performing secret multiplication operation. When the encryption algorithm is a semi-homomorphic encryption algorithm, the multiplication operation of equation (5) is an encrypted forward gradient FG as the secret data_jAnd as plaintext data X_jIs multiplied between the specific data. Here, for the sake of brevity, the same symbol X is used in equation (5) and the related description below_jTo refer to particular data, it should be understood that the particular data may be either secret or clear data, depending on whether homomorphic encryption or semi-homomorphic encryption is employed. However, the multiplication result calculated by equation (5) is necessarily encrypted, that is, belongs to secret data. In the calculation according to the formula (5)Get specific data X_jThe result MUL of the dense multiplication_jThe loss function L of the subset SM in equation (3) can then be solved by equation (6)_SMA gradient operator of (θ). As described above, the data in equation (3) for a particular data X in the subset SM_jIs sequentially expanded to calculate the specific data X according to the formula (4)_jForward gradient FG of_jThen, the specific data X is calculated according to the formula (5)_jThe result MUL of the dense multiplication_jFinally, based on the dense state multiplication result MUL according to formula (6)_jSum mask M_jCompletion of specific data X_jThe mask accumulation operation of (1). Here, the mask accumulation operation of the formula (6) is mainly based on the mask M_jDetermining whether there is data overlap and performing corresponding mask accumulation operation to obtain data for specific data X_jThe mask accumulation operation result is obtained, all data in the subset SM are traversed, that is, iterative calculation or circulation with the times consistent with the total number SM of samples of the subset SM is carried out, and the mask accumulation operation aiming at the corresponding data is completed in each iterative calculation or circulation, wherein the mask accumulation operation involves secret state addition operation, secret state multiplication operation and mask accumulation operation, so that the loss function L of the subset SM is solved_SMA gradient operator of (θ). It should be appreciated that the mask accumulation operation shown in equation (6) may be performed in any suitable manner as long as the mathematical relationship shown in equation (6) is satisfied. In one possible implementation, the accumulation operation may be performed before the mask calculation. Furthermore, considering that the mask is used to indicate the data overlapping condition, it is possible for one participant to deduce the data overlapping condition of another participant by the mask, that is, to deduce which private data another participant may hold, so the mask calculation can be understood as a part of the decryption operation. On the basis of the formula (3) and the formula (6), the loss function can be obtained by combining the loss function calculation formula mentioned in the above paper, so that the model parameters can be updated by a random gradient descent method. It should be understood that for each participant, because it is in federal learning and privacy calculationsIn the application scenario of (2), the loss function L of the subset SM is solved according to equation (6)_SMAfter the gradient operator of (θ) obtains the gradient, the gradient is also in an encrypted state, that is, in a secret state, and for this purpose, each participant needs to perform a decryption operation or an arbitrator performs a decryption operation to obtain the gradient for updating the model parameter. The various steps of the heterogeneous acceleration method 100 are described below in conjunction with equations (1) through (6) above and the associated details. Specifically, the heterogeneous acceleration method 100 includes the following steps.

Step S102: the method comprises the steps that a first participant and a second participant perform plaintext operation and encryption operation according to respective feature data to obtain an encrypted model parameter of the first participant and an encrypted model parameter of the second participant respectively, wherein the feature data are data in a mini-batch subset of a data set, and the first participant and the second participant respectively hold part of data of the mini-batch subset.

Wherein, the relevant details can refer to the above description about the formula (1) and the formula (2). In the longitudinal federated logistic regression learning or similar application scenarios, the data set S is divided into a first part and a second part, a first participant holds the first part of the data set S, a second participant holds the second part of the data set S, or the first participant and the second participant respectively hold a part of data of the mini-batch subset. It should be appreciated that the first and second parties are relative concepts that indicate a party holding private data in the context of federal learning and that the party does not wish to reveal the private data to other parties. The first party may be acting as a Guest party, a Host party, an Arbiter party (Arbiter) or any suitable role. In contrast, the second party may play any suitable role, as long as the first party and the second party can exchange the encrypted feature data and finally solve the loss function of the federal learning model. It should be understood that although the heterogeneous acceleration method 100 is exemplified by two participants, namely a first participant and a second participant, but is also applicable to the case of three or more participants, since a plaintext operation and an encryption operation are involved, the encrypted model parameters of each participant must be obtained by performing the plaintext operation and the encryption operation locally by each participant. In addition, the data set S is divided into a plurality of subsets by a mini-batch technique, and the feature data is data of one of the plurality of subsets.

Step S104: and the first party or the second party performs cryptographic addition operation on the encrypted model parameter of the first party and the encrypted model parameter of the second party to obtain a forward gradient corresponding to the characteristic data.

Wherein, the relevant details can refer to the above description about the formula (3) and the formula (4). It should be understood that at step S102, the encrypted model parameters of each participant must be obtained by performing the plaintext operation and the encryption operation locally by each participant. However, at step S104, since the forward gradient is shared among the participants, such as the gradient factor shared in the learning process of the above-mentioned hetro-LR, the cryptographic addition operation of step S104 may be performed by any suitable participant, and the other participants may send the cryptographic model parameters to the participant through communication or any suitable technical means. Because the data is sent after being encrypted, the data is exposed in a secret state, and therefore, the security and privacy protection are good. And the participant who completes the secret addition operation at step S104 may share the calculated forward gradient with other participants to proceed to the next step.

Step S106: and the first party and the second party respectively carry out multiplication operation on the forward gradient and the respective characteristic data to obtain a secret multiplication operation result corresponding to the respective characteristic data.

Wherein, the relevant details can refer to the above description about the formula (3) and the formula (5). It should be appreciated that at step S102, because of the encryption operations involved, for privacy and security reasons, the encrypted model parameters of each participant must be obtained separately by each participant performing the plaintext operations and encryption operations locally. At step S104, the secret addition operation may be completed by either participant. At step S106, depending on whether the encryption algorithm employs fully homomorphic encryption or semi-homomorphic encryption, the characteristic data used for the multiplication at step S106, with reference to the specific data used for the multiplication of equation (5) above, may be either secret data (corresponding to fully homomorphic encryption) or clear data (corresponding to semi-homomorphic encryption). When the encryption algorithm employs fully homomorphic encryption, this means that not only the forward gradient is shared between the participants as secret data after encryption, but also the feature data is encrypted for communication as secret data, so that the secret multiplication operation at step S106 is a secret multiplication operation between two secret data, and thus can be performed by any suitable participant, and the other participants can send the encrypted feature data to the participant through communication or any suitable technical means. For example, in the learning process of the above-mentioned hetro-LR, the Host side sends the encrypted feature data of the Host side to the Guest side. When the encryption algorithm employs semi-homomorphic encryption, which means that the forward gradient is secret data and the feature data is plaintext data, it is suitable to perform the multiplication operation of step S106 locally, that is, the forward gradient as secret data can be shared among the participants, and each participant, for example, the first participant and the second participant, performs the multiplication operation on the forward gradient and the respective feature data respectively to obtain the secret multiplication result corresponding to the respective feature data. For exemplary purposes only, step S106 is directed to the case of a semi-homomorphic encryption algorithm, i.e., the case where the feature data participates in a multiplication operation as plaintext data, but in other exemplary embodiments, step S106 may be replaced with the case of the fully homomorphic encryption algorithm described above. It should be understood that whether the encryption algorithm is a fully homomorphic encryption algorithm or a semi-homomorphic encryption algorithm, or whether the feature data as the secret data or the feature data as the plaintext data is multiplied at step S106, the multiplication result calculated at step S106 is necessarily encrypted, that is, belongs to the secret data.

Step S108: and performing mask accumulation operation on the secret multiplication operation result and a mask corresponding to the characteristic data to obtain a mask accumulation operation result corresponding to the characteristic data, and updating a gradient calculation result of a loss function corresponding to the mini-batch subset, wherein an accumulation calculation part of the mask accumulation operation passes through the first party or the second party, and a mask calculation part of the mask accumulation operation passes through the Arbiter party or obtains a party authorized by the Arbiter party.

Wherein, the relevant details can refer to the above description about the formula (3) and the formula (6). It should be understood that at step S102, the encrypted model parameters of each participant must be obtained by performing the plaintext operation and the encryption operation locally by each participant. At step S104, the secret addition operation may be completed by either participant. At step S106, when the encryption algorithm is a semi-homomorphic encryption algorithm, it is appropriate to perform the multiplication locally. In addition, the multiplication result calculated in step S106 is necessarily encrypted, that is, belongs to secret data. It should be appreciated that the mask accumulation operation at step S108 may be performed in any suitable manner as long as the mathematical relationship shown in equation (6) is satisfied. In one possible implementation, the accumulation operation may be performed before the mask calculation. Furthermore, considering that the mask is used to indicate the data overlapping condition, it is possible for one participant to deduce the data overlapping condition of another participant by the mask, that is, to deduce which private data the other participant may hold, so the mask calculation or the mask accumulation operation shown in equation (6) can be understood as a part of the decryption operation. Specifically, for each data in the mini-batch subset of the data set, a mask accumulation operation result of each data is obtained, and a gradient calculation result shown in formula (6) is updated. And when the calculation of the mask accumulation operation result of all the data is completed, the gradient calculation of the loss function of the mini-batch subset is completed, and the gradient of the mini-batch subset is obtained. Here, traversing each data in the mini-batch subset of the data set means traversing all data in the subset SM with reference to equation (6). Thus, the calculation for specific data in the mini-batch subset sequentially comprises: performing plaintext operation and encryption operation according to the specific data to obtain encrypted model parameters at step S102; then, at step S104, a dense state addition operation is performed to obtain a forward gradient for the specific data; then, in step S106, a secret multiplication result for the specific data is obtained by performing multiplication; and then, in step S108, a mask accumulation operation is performed to obtain a mask accumulation operation result corresponding to the feature data, so as to update the gradient calculation of the loss function of the mini-batch subset. Here, the accumulation calculation part in the mask accumulation operation at step S108 is an addition between the secret data, and thus can be completed by any one of the parties. As for the mask calculation part in the mask accumulation operation, considering that the mask is used for indicating the data overlapping condition, and one participant may possibly deduce the data overlapping condition of another participant by the mask, that is, deducing which private data may be held by another participant, the mask calculation part in the mask accumulation operation is suitably accomplished by the Arbiter party or the participant authorized by the Arbiter party for data security and privacy.

Therefore, in step S102, the plaintext operation and the encryption operation performed locally by each participant result in the encrypted model parameters of each participant respectively. At step S104, a forward gradient is obtained by performing a secret addition operation by any of the participants, and the forward gradient is used as the secret data. At step S106, when the encryption algorithm is a semi-homomorphic encryption algorithm, multiplication operations are performed locally by each participant, and the multiplication result calculated at step S106 is necessarily encrypted, that is, belongs to secret data. At step S108, the accumulation calculation part of the mask accumulation operation may be performed by any of the parties and the mask calculation part of the mask accumulation operation is adapted to be performed by the Arbiter party or the party authorized by the Arbiter party. In summary, step S102 must be done locally, step S104 may be done locally or not, step S106 is also suitable to be done locally, the accumulation calculation part in step S108 may be done locally or not and the mask calculation part is suitable to be done on the Arbiter side. Thus, local completion means that a local resource is invoked and non-local completion means that a non-local resource, i.e., the computing resources of the other participants, may be invoked. By splitting the above equations (1) to (6) and explaining the above steps S102 to S108, it can be concluded that the calculations suitable for non-local completion include: a secret addition operation at step S104 and an accumulation calculation portion in a mask accumulation operation at step S108. These computations, which are suitable for non-local completion, may be accomplished by invoking computing resources of other participants. Therefore, by calling the overall idle computing resources of each participant, the various operations can be completed in a coordinated manner, so that the overall resource utilization efficiency and the computing performance of the federal learning task are improved.

In addition, as can be seen from the above equations (1) to (6) and steps S102 to S108, the plaintext operation involves a relatively complicated implementation requirement, and the four specific operator operations, i.e., the encryption operation, the decryption operation, the secret addition operation, and the secret multiplication operation, can be simplified into the vector operation and are also suitable for the accelerated processing through operator parallelization. This means that overall computational efficiency can be improved by means of heterogeneous acceleration, for example, employing a heterogeneous computing architecture. Here, heterogeneous computing (heterogeneous computing), or referred to as heterogeneous computing, refers to the way in which computing is performed using different types of instruction sets or computing units of different architectures to form a system. One type of computing unit used for heterogeneous computing is a central processing unit CPU. The CPU generally adopts an instruction set system based on a control flow, executes software layer operations and data operations of the computer through an instruction set, and is suitable for executing a serial program having an irregular data structure and a complex and flexible logic. Another computational unit for heterogeneous computation is a graphics processor GPU, which is suitable for massively parallel operations, in particular for computationally intensive and highly parallelized computational tasks. The computing units used for heterogeneous computing may also be Application-Specific Integrated circuits (ASICs), which are special-purpose hardware circuits designed and manufactured for Specific needs. The computing unit for heterogeneous computing may also be a Field Programmable Gate Array (FPGA), which provides fine-grained Programmable hardware logic computing and storing units, and the computing path structure is custom-designed in a static global reconstruction manner according to algorithm requirements. The computing unit for heterogeneous computing may also be a Reconfigurable Computing (CGRA), where the CGRA interconnects hardware resources with configured functions to form a configurable computing part, and reconstructs the computing part into different computing paths through configuration information, thereby implementing dynamic configuration of a hardware structure and simplifying interconnection configuration. Other computing units for heterogeneous computing may be Digital Signal Processing (DSP) based computing devices or any suitable technology. The heterogeneous computing architecture referred to herein may be understood to include at least two computing units that are each different in at least one of the following: instruction sets, instruction architectures, hardware architectures, control flows, data flows, reconfigurability, particularly dynamic reconfigurability, and any other suitable metrics or features, and is intended to include features that are included in the concept-related understanding of heterogeneous computing as it evolves after filing or publication.

Continuing to refer to fig. 1, in one possible implementation, the heterogeneous acceleration method 100 may employ a CPU-GPU joint acceleration method, i.e., a CPU-GPU heterogeneous computing architecture. As described above, the four specific operator operations, i.e., the encryption operation, the decryption operation, the secret addition operation, and the secret multiplication operation, can be simplified into the vector operation and are also suitable for the accelerated processing by the operator parallelization. Specifically, the plaintext operation at step S102 involves a relatively high complexity operation, and also tends to cope with changing multi-terminal data structures such as data bit width and the like, and is therefore suitable for implementation by the CPU. With regard to the subsequent operations, including the encryption operation performed at step S102, the secret addition operation performed at step S104 and obtaining a forward gradient for the specific data, the secret multiplication operation performed at step S106 and obtaining a secret multiplication result for the specific data, the accumulation calculation in the mask accumulation operation performed at step S108 and obtaining a mask accumulation operation result corresponding to the characteristic data, and the decryption operation, each of these operations is characteristic of being dense in calculation and highly parallelized, and thus suitable for implementation by the GPU. In this way, with reference to the above equations (1) to (6) and steps S102 to S108, the gradient solving algorithm used in the learning process of the aforesaid hetro-LR in the vertical federal logistic regression learning is split, and particularly, the algorithm for performing gradient calculation of the loss function of a certain mini-batch subset after dividing the data set into several subsets based on the mini-batch technology is split, so that the calculation task of the gradient solving algorithm in the vertical federal logistic regression learning is split into a more complex serial task suitable for being executed by the CPU and a highly parallel task suitable for being executed by the GPU. Therefore, various complex computing tasks included in one iteration computing in the Hetero-LR computing process are split to obtain atomic-level operator operations such as plaintext operation, encryption operation, decryption operation, secret addition operation and secret multiplication operation, the overall computing efficiency is improved through parallel acceleration, for example, a mode of parallel processing through a plurality of GPUs is facilitated, and the acceleration purpose is achieved through a heterogeneous computing framework of the CPU-GPU. Moreover, compared with a mode that is realized by a CPU, such as a currently popular open source framework FATE, the heterogeneous computing architecture of the CPU-GPU and the splitting mode of the computing task disclosed in the embodiments of the present application can avoid that the time consumed by secret computing is long and the time consumed by stopping the equation computing task (that is, one party must wait for the other party to complete computing) is too long to complete the overall computing task under the condition of multiple participants. In addition, compared with the way of realizing the whole way through the GPU, for example, the way of copying the subset of the data set to the video memory of the GPU and then simultaneously calculating through the computational resources of the GPU, the heterogeneous computation architecture of the CPU-GPU and the splitting way of the computation task disclosed in the embodiment of the present application can use the plaintext operation as a more complex serial task suitable for the CPU to execute, and use the above encryption operation, decryption operation, secret addition operation, and secret multiplication operation as a highly parallel task suitable for the GPU to execute, so that the GPU does not include a serial step and does not need to deal with operations with higher complexity and variable data structures when processing the computation task, thereby simplifying the design difficulty and hardware structure of the GPU, effectively improving the resource efficiency of the GPU, and achieving a better parallel acceleration effect. Moreover, the splitting manner of the computation task disclosed in the embodiment of the present application can also effectively reduce data interaction between the computation units responsible for different operations, because the above steps S102 to S108 need to be sequentially executed step by step, and the operation corresponding to each step can be allocated to different computation units for execution, for example, the secret addition operation, the secret multiplication operation, and the mask accumulation operation are respectively completed by the secret addition module, the secret multiplication module, and the accumulation module. In addition, the vector calculation part in the encryption operation and the decryption operation can be parallelized and accelerated by using the GPU, so that the vector calculation part in the encryption operation and the decryption operation can be parallelized and processed by the GPU. Therefore, by the aid of the splitting mode of the computing tasks and the corresponding heterogeneous computing framework of the CPU-GPU, the vector computing tasks in the ciphertext state (including vector computing parts in encryption operation and decryption operation, secret addition operation, secret multiplication operation and mask accumulation operation) can be completed by the GPU, and other computing tasks are completed by the CPU, so that the advantages of the heterogeneous computing framework of the CPU-GPU are fully exerted. In summary, the heterogeneous acceleration method 100 disclosed herein combines improvements in operator-based optimization and heterogeneous acceleration, improving overall computational efficiency and saving resources.

As described above, step S102 must be performed locally for the party holding the specific data as private data, and the calculation suitable for non-local completion can be obtained by splitting equations (1) to (6) and describing steps S102 to S108, including: a secret addition operation at step S104 and an accumulation calculation portion in a mask accumulation operation at step S108. This means that the secret addition operation at step S104 and the accumulation calculation portion in the mask accumulation operation at step S108 can be performed not only by the local GPU but also by a non-local GPU such as the GPU of another participant, so that the respective GPUs of different participants can be flexibly invoked and the related intermediate calculation results can be shared. In this way, the GPUs of the multiple participants can be regarded as a computing resource pool or a shared GPU pool, and the highly parallel tasks that can be executed by the local or non-local GPUs are executed in parallel by the GPUs that are idle in the shared GPU pool, thereby further improving the resource utilization efficiency.

In addition, the heterogeneous acceleration method 100 disclosed herein may also include improvements in data structure to further increase overall computational efficiency. As described above, the plaintext operation and the encryption operation and the decryption operation required after the gradient calculation are completed need to be completed locally, that is, the plaintext operation and the encryption operation based on the private data are completed by the computing resource of the participant holding the private data, that is, the local computing resource, and the encrypted secret data is exchanged to the local or non-local GPU, and the decryption operation is completed by the local computing resource so as to obtain the gradient for updating the model parameter. Here, since the secret data is used for interaction, there is often a large data specification, for example, the original data is a floating point number but the bit width of the encrypted secret data may be 1024 bits. Furthermore, data in the memory corresponding to the CPU is typically stored in a discrete manner, while data in the GPU memory is typically stored in a continuous manner, such as in a batch-like data format. In order to improve efficiency in data interaction, in the heterogeneous acceleration method 100, data interaction between the CPU and the GPU may be performed by directly exchanging pointers. Specifically, in a possible implementation, data splicing is performed on discrete data in a memory corresponding to the CPU, and then the spliced data is copied to a memory of the GPU, so as to obtain a GPU memory pointer, where the GPU memory pointer points to data copied from the CPU in the GPU memory. Therefore, only one data splicing operation needs to be carried out on the discrete data on the CPU side, and then the spliced data is copied to the GPU video memory, so that the mapping relation between the spliced data and the GPU video memory pointer can be established, and the data on the GPU side can be obtained through the GPU video memory pointer. The improved data interaction mode, particularly the pointer interaction mode, enables splicing operation of discrete data in subsequent CPU-GPU interaction to be unnecessary, and is beneficial to reducing loss. In another possible implementation, the discrete data on the CPU side may be rearranged so as to be continuously stored on the CPU side, but the continuously stored data still needs to be copied in its entirety in the subsequent CPU-GPU interaction. Therefore, data interaction between the CPU and the GPU can reduce loss in the aspects of subsequent data copying and the like by only needing one data splicing operation in a mode of directly exchanging pointers. Taking encryption operation and decryption operation as an example, the encryption operator and decryption operator can be configured at the bottom of the package, wherein the encryption operation and decryption operation are completed by using a matrix manner, for example, a processor which allocates computing units according to rows in the matrix to realize parallel acceleration or matrix operation through other suitable optimization. Matrix addition, matrix multiplication and matrix dot multiplication under the condition of packaging plaintext are realized by a heavy-load operator method in Python. In a CPU side, for example, in a memory of a CPU, a Python class bearer includes: the GPU is used for displaying and storing non-pointer data such as pointers and public keys, and nparray or tensor load bearing used for recovering the GPU to the CPU side. Thus, matrix addition, matrix multiplication and matrix dot multiplication under the condition of ciphertext encapsulation are realized by a heavy-load operator method in Python. The GPU memory pointer in the Python class bearer is used to point to data copied from the CPU in the GPU memory, while the nparray or tensor bearer can be used to restore data from the GPU to the CPU side. Combining the division based on the mini-batch technology, it can be known that after the data set is divided by the CPU based on the mini-batch technology to obtain a plurality of subsets, each subset can be represented as a matrix, where the line data includes numbers, eigenvectors and labels. The subset in the matrix form can be copied into a GPU video memory from a CPU side, a GPU video memory pointer pointing to the subset in the matrix form in the GPU video memory is stored in a Python class bearer of the CPU, an encryption operator and a decryption operator are formed, then the GPU video memory pointer is added to a data interval such as a data header, and therefore data interaction between the CPU and the GPU based on pointer exchange is completed. In addition, by means of reloading operators and defining an arithmetic layer, only pointers can be exchanged without relating to specific data copy, and data recovery can be realized through nparray or tensor bearing, so that loss is reduced.

It should be understood that the heterogeneous acceleration method 100 disclosed in the embodiment of the present application may employ a CPU-GPU joint acceleration method, that is, a heterogeneous computing architecture of a CPU-GPU, which is just one implementation of the heterogeneous computing architecture that the heterogeneous acceleration method 100 may employ. The heterogeneous acceleration method 100 may also adopt other combined or combined heterogeneous computing architectures, as long as appropriate computing units are configured according to the splitting manner of the computing tasks, so as to configure the serial computing units and highly parallel computing units respectively for the more complex serial tasks, thereby forming an appropriate system with the heterogeneous computing architecture as a whole. For example, plaintext operations may be performed by computational units suitable for handling complex operations and varying data specifications, but still optimized for the application scenarios of federated learning and privacy Computing, such as Reduced Instruction Set Computers (RISC) or microchips, among others. Meanwhile, the encryption operation, the decryption operation, the secret state addition operation, the secret state multiplication operation and the mask accumulation operation are executed through a computing unit suitable for processing highly parallelized tasks, such as an ASIC (application specific integrated circuit) for longitudinal federated logistic regression learning or an FPGA (field programmable gate array) optimized for matrix operation. That is, the heterogeneous acceleration method 100 may employ a heterogeneous computing architecture such as a combination of microchips suitable for processing complex serial tasks and ASICs suitable for processing highly parallelized tasks, or any other suitable heterogeneous computing architecture. These may be according to actual needs and actual application scenarios, and are not specifically limited herein.

Referring to fig. 2, fig. 2 is a block diagram illustrating a heterogeneous acceleration apparatus according to an embodiment of the present disclosure, which is used in the heterogeneous acceleration method illustrated in fig. 1. As shown in fig. 2, the heterogeneous acceleration device 200 includes a data segmentation module 202, a plaintext operation module 204, a model update module 206, an interaction calculation unit 210, and a dense vector calculation unit 220. The interactive calculation unit 210 includes an encryption operation module 212 and a decryption operation module 214. The secret vector calculation unit 220 includes a secret addition module 222, a secret multiplication module 224, and an accumulation operation module 226. Wherein the data is dividedThe module 202 is used to segment the training data or data set into several subsets (e.g., into equal subsets) by a mini-batch technique so that a gradient descent can be performed for each subset. In conjunction with fig. 1 and 2, the data partitioning module 202 is configured to receive a data set S and partition the data set S into several share subsets via a mini-batch technique. The data splitting module 202 sends the split subsets to the plaintext operation module 204 to solve the gradient for each subset, for example, the subset SM is sent to the plaintext operation module 204. The plaintext operation module 204 is used for performing plaintext operation according to the feature data, that is, calculating model parameters of the participant corresponding to the heterogeneous acceleration device 200, for example, for the feature data X_iModel parameter θ of^TX_i. Because of the private data of the participants involved, the plaintext operation block 204 must be run locally. The plaintext operation module 204 sends the calculated model parameters to the encryption operation module 212 in the interactive calculation unit 210. The encryption operation module 212 is configured to perform an encryption operation to obtain an encrypted model parameter. Therefore, the plaintext operation module 204 and the encryption operation module 212 in the interaction calculation portion 210 are used together to execute step S102 in the heterogeneous acceleration method 100 shown in fig. 1, that is, the first party and the second party perform the plaintext operation and the encryption operation according to the respective feature data to obtain the encrypted model parameter of the first party and the encrypted model parameter of the second party, respectively. Here, the first participant or the second participant may be understood as a participant corresponding to the heterogeneous acceleration device 200. As described above, the plaintext operation is suitably implemented by the CPU, and the vector calculation portion in the encryption operation can be parallelized and accelerated by the GPU. Therefore, the plaintext operation module 204 in fig. 2 is suitable for implementing corresponding plaintext operation by the CPU, and the encryption operation module 212 is suitable for implementing a vector calculation portion in the encryption operation by the GPU, so that the interaction calculation portion 210 in which the encryption operation module 212 is located is suitable for data interaction between the CPU and the GPU. The cryptographic operation module 212 obtains the encrypted model parameters, and then sends the encrypted model parameters to the cryptographic addition module 222 of the cryptographic vector calculation unit 220. The secret addition module 222 is used to execute the graphStep S104 of the heterogeneous acceleration method 100 shown in fig. 1 is to perform a cryptographic addition operation on the encrypted model parameter of the first party and the encrypted model parameter of the second party to obtain a forward gradient corresponding to the feature data. The dense state addition module 222 sends the resulting forward gradient to the dense state multiplication module 224. The dense multiplication module 224 is configured to execute step S106 in the heterogeneous acceleration method 100 shown in fig. 1, that is, perform a multiplication operation on the forward gradient corresponding to the feature data and the feature data to obtain a dense multiplication result corresponding to the feature data. It should be understood that when the encryption algorithm employs semi-homomorphic encryption, this means that the forward gradient is secret data and the feature data is plaintext data, and therefore the multiplication operation of step S106 is suitable to be performed locally, which means that the secret multiplication module 224 must be run locally. The secret multiplication module 224 sends the obtained secret multiplication result to the accumulation operation module 226. The accumulation operation module 226 is configured to perform step S108 in the heterogeneous acceleration method 100 shown in fig. 1, that is, perform a mask accumulation operation on the secret multiplication operation result corresponding to the feature data and the feature data to obtain a mask accumulation operation result corresponding to the feature data, so as to complete gradient calculation of the loss function of the mini-batch subset. In a possible embodiment, because the mask is used to indicate the data overlap condition, and one participant may speculate the data overlap condition of another participant by the mask, that is, speculate which private data may be held by another participant, the mask calculation part in the mask accumulation operation may be understood as a part of the decryption operation for data security and privacy. That is, the accumulation operation module 226 only processes the accumulation calculation part of the mask accumulation operation, and the mask calculation part of the mask accumulation operation is completed by the decryption operation module 214, or the mask calculation part of the mask accumulation operation is completed by the external part of the heterogeneous acceleration apparatus 200 (by the Arbiter party or the authorized party of the Arbiter party). It should be understood that the plaintext operation module 204, the encryption operation module 212, the secret addition module 222, and the secret multiplication moduleThe module combination composed of the module 224 and the accumulation operation module 226 may be used to perform the mask accumulation operation result of each data and complete the gradient calculation for each data in the mini-batch subset of the data set mentioned in the steps S102 to S108. The gradient obtained by the accumulation operation module 226 is the gradient in the secret state, and the accumulation operation module 226 sends the gradient in the secret state to the decryption operation module 214 of the interactive calculation unit 210. The decryption operation module 214 decrypts the gradient in the encrypted state to obtain a decrypted gradient, and then sends the decrypted gradient to the model update module 206. The model update module 206 performs model updates based on the gradient. Similar to the encryption operation module 212 adapted to implement the vector calculation part in the encryption operation by the GPU, the vector calculation part in the decryption operation can also perform parallelization acceleration by the GPU. Therefore, the decryption operation module 214 in fig. 2 is suitable for implementing the vector calculation part in the decryption operation by the GPU, and the interaction calculation part 210 in which the decryption operation module 214 is located is suitable for data interaction between the CPU and the GPU. Therefore, by the aid of the splitting mode of the computing tasks and the corresponding heterogeneous computing framework of the CPU-GPU, the vector computing tasks in the ciphertext state (including vector computing parts in encryption operation and decryption operation, secret addition operation, secret multiplication operation and mask accumulation operation) can be completed by the GPU, and other computing tasks are completed by the CPU, so that the advantages of the heterogeneous computing framework of the CPU-GPU are fully exerted.

With continued reference to fig. 2, for a specific participant's heterogeneous acceleration device 200, the plaintext operation module 204 of the heterogeneous acceleration device 200 refers to the private data of the specific participant, so the plaintext operation module 204 and the corresponding plaintext operation must be performed at the specific participant, i.e., locally. The encryption operation module 212 and the corresponding encryption operation, and the decryption operation module 214 and the corresponding decryption operation, since the encryption of the model parameter (unencrypted) of the specific participant output by the plaintext operation module 204 and the decryption of the gradient in the secret state output by the accumulation operation module 226 are involved, they must be performed at the specific participant, that is, locally. Unlike the plaintext operation module 204 adapted to perform serial processing by the CPU, i.e., processing by the CPU computational resources of the specific party, the encryption operation module 212 and the decryption operation module 214 included in the interactive computation portion 210 can process the vector computation portion thereof, i.e., processing by the GPU computational resources of the specific party, using GPU parallelization acceleration. Therefore, for a specific participant, the plaintext operation, the encryption operation, and the decryption operation are processed by the computing resources of the specific participant, that is, by the computing units included in the local CPU-GPU heterogeneous computing architecture, and the encryption operation and the decryption operation both require data interaction between the CPU and the GPU.

In contrast, the secret addition module 222, the secret multiplication module 224, and the accumulation module 226 of the secret vector calculation unit 220 respectively correspond to the forward gradient of the specific data obtained by performing the secret addition operation at step S104, the secret multiplication result obtained by performing the secret multiplication operation at step S106, and the mask accumulation result obtained by performing the mask accumulation operation at step S108, which are shown in fig. 1, to obtain the mask accumulation result corresponding to the feature data. As described above, with respect to the party holding the specific data as private data, through the splitting of the above formulas (1) to (6) and the explanation of the above steps S102 to S108, it can be found that the calculation suitable for non-local completion includes: a secret addition operation at step S104 and an accumulation calculation portion in a mask accumulation operation at step S108. Therefore, the secret addition module 222 and the accumulation calculation part responsible for the accumulation calculation module 226 included in the secret vector calculation unit 220 may be implemented by local calculation resources or non-local calculation resources, and may be used for performing calculation tasks of other participants. Thus, the secret addition module 222 and the accumulation operation module 226 responsible for accumulating the portion of computation can be understood as the contribution of the particular participant's heterogeneous acceleration device 200 to the externally shared computing resources, i.e., the shared GPU pool. In addition, assuming that the mask calculation part in the mask accumulation operation is understood as a part of the decryption calculation, the accumulation operation module 226 is only responsible for the accumulation calculation part, which means that the secret addition module 222 and the accumulation operation module 226 are both computing resources shared externally. In other words, the dense vector computation portion 220 of the heterogeneous acceleration device 200 not only parallelizes acceleration vector computations by the GPU, but may also be used to form a shared computing resource pool with similar dense vector computation portions in heterogeneous acceleration devices of other participants. Assuming that another participant collaboratively trains the federated learning model with the specific participant, the computation tasks that would have been performed by the secret addition module 222 and the accumulation module 226 included in the secret vector computation portion 220 can be performed by the computing resources of the other participant, i.e., non-local. In this way, highly parallel tasks that can be executed by local or non-local GPUs are executed in parallel by sharing the GPU that is idle in the GPU pool, thereby further improving resource utilization efficiency.

Referring to fig. 3, fig. 3 is a block diagram illustrating a multi-party heterogeneous acceleration system according to an embodiment of the present disclosure. As shown in fig. 2, the heterogeneous acceleration system 300 includes two participants, each having a heterogeneous acceleration device 310 and a heterogeneous acceleration device 320. For example, the two parties may be a pair of Host and Guest parties in the learning process of the above-described hetro-LR in the context of vertical federal learning, and no arbitrator (Arbiter) is shown in fig. 3 for the sake of brevity of presentation. As described above, in the process of performing the vertical logistic regression learning by the two participating parties, i.e., the Host party and the Guest party need to respectively calculate the encrypted feature data and solve the encrypted gradient according to the gradient factor. Here, the heterogeneous acceleration device 310 and the heterogeneous acceleration device 320 may be understood to have a structure and a function similar to those of the heterogeneous acceleration device 200 shown in fig. 2, respectively, and also be used to perform the heterogeneous acceleration method 100 shown in fig. 1. The heterogeneous acceleration device 310 includes a data dividing module 311, a plaintext operation module 312, an interaction calculation unit 313, a dense vector calculation unit 314, and a model update module 315. The structure and function of these modules can refer to the modules with the same name and related descriptions shown in fig. 2, and are not described herein again. Similarly, the heterogeneous acceleration device 320 includes a data division module 321, a plaintext operation module 322, an interaction calculation unit 323, a dense vector calculation unit 324, and a model update module 325. The structure and function of these modules can refer to the modules with the same name and related descriptions shown in fig. 2, and are not described herein again. It should be understood that the dense vector calculation section 314 of the heterogeneous acceleration device 310 and the dense vector calculation section 324 of the heterogeneous acceleration device 320 together constitute the computing resource pool 302. This means that the heterogeneous acceleration device 310 can perform GPU parallel acceleration vector calculation by its own secret vector calculation unit 314, including performing the secret addition operation of step S104, the secret multiplication operation of step S106, and the mask accumulation operation of step S108, that is, completing the vector calculation task locally; alternatively, the heterogeneous acceleration device 310 may accomplish these vector calculation tasks by the dense vector calculation section 324 of the heterogeneous acceleration device 320 of another participant. For the two participants, the GPU computing resources of the computing resource pool 302 are shared, and the idle GPUs in the shared GPU pool can be used to execute computing tasks in parallel, thereby further improving the resource utilization efficiency.

Referring to fig. 4, fig. 4 is a block diagram illustrating a multi-party heterogeneous acceleration system according to another embodiment of the present disclosure. The heterogeneous acceleration system 400 shown in fig. 4 involves three participants, each having a heterogeneous acceleration device 410 and a dense vector calculation section 412 therein, a heterogeneous acceleration device 420 and a dense vector calculation section 422 therein, a heterogeneous acceleration device 430 and a dense vector calculation section 432 therein. Each secret vector calculation unit shown in fig. 4 is configured to locally complete a vector calculation task of the heterogeneous acceleration device in which each secret vector calculation unit is located, and includes a secret addition operation in step S104, a secret multiplication operation in step S106, and a mask accumulation operation in step S108, which are shown in fig. 1, performed by each participant. The dense state vector calculation unit 412, the dense state vector calculation unit 422, and the dense state vector calculation unit 432 together form the computing resource pool 402, and for any one of the three participating parties, the vector calculation task originally completed by the local dense state vector calculation unit of the participating party can be completed by the idle computing resources in the computing resource pool 402, and thus, the idle GPUs in the computing resource pool 402 are shared to execute in parallel, thereby further improving the resource utilization efficiency.

Referring to fig. 3 and 4, fig. 3 illustrates a case where two participants (the arbitrator is not shown) are included and fig. 4 illustrates a case where three participants are included. When there are more than three participants, especially in the context of longitudinal federated learning, it may involve hundreds of participants participating together in the same longitudinal federated logistic regression learning process. These participants have many identical entities but different features. The above-mentioned learning process of the hetro-LR may be referred to, and each two participants are used as a pair of a Host party and a Guest party to perform the above-mentioned training process, or after expansion, the training process includes exchange of feature data after encryption performed by more participants, calculation of gradient factors, solution of respective gradient after encryption, and the like, which may be adjusted according to actual needs and application scenarios, and are not specifically limited herein.

Referring to fig. 1 to 4, in combination with the above formulas (1) to (6), the embodiment of the present application provides that the algorithm for gradient solution used in the vertical federal logistic regression learning in the above learning process of Hetero-LR is split, and particularly, the algorithm for gradient calculation of the loss function of a certain mini-batch subset after the data set is divided into several subsets based on the mini-batch technology is split, so that the calculation task of the gradient solution algorithm of the vertical federal logistic regression learning is divided into a more complex serial task suitable for being executed by a CPU and a highly parallel task suitable for being executed by a GPU. Specifically, various complex computing tasks included in one iteration computing in the hetro-LR computing process are split to obtain atomic-level operator operations such as plaintext operation, encryption operation, decryption operation, secret addition operation, and secret multiplication operation, which is beneficial to improving the overall computing efficiency through parallel acceleration, for example, through a mode of parallel processing by a plurality of GPUs, and is also beneficial to realizing the acceleration purpose through a heterogeneous computing architecture of the CPU-GPU. Among them, plaintext operation is a more complex serial task suitable for being executed by a CPU, and four kinds of specific operator operations, namely encryption operation, decryption operation, secret addition operation and secret multiplication operation, are suitable for being simplified into vector operation and accelerated processing through operator parallelization, such as through a plurality of GPUs. The split atomic-level operator operation has the advantages of simple structure, high utilization efficiency and the like in software layer operation and circuit implementation, and is suitable for meeting the requirements of different application scenes and hardware platforms. In addition, a plurality of atomic-level operator operations are obtained through splitting, wherein the accumulation calculation part in the dense state addition operation and the mask accumulation operation can be completed locally or non-locally, so that the calculation originally completed by local calculation resources can be coordinated and completed by the operator parallelization acceleration processing and the calculation resources of other participants, and the whole resource utilization efficiency and the calculation performance of the federal learning task are improved. And the mask calculation part in the mask accumulation operation can be listed separately and completed by the Arbiter party, thereby further improving the privacy security. In addition, the secret state vector calculation units of the participants shown in fig. 2 to fig. 4, in which the secret state addition module and the accumulation operation module may be understood as computing resources shared by the participants to the outside, the computing resources shared to the outside constitute a computing resource pool as shown in fig. 4, and the computing resources of the appropriate participants in the computing resource pool may be selected according to resource utilization, system load, and the like, so as to facilitate the learning process of the participants of the same federal learning task, for example, the above mentioned Hetero-LR with multiple Host parties, to coordinate to complete the task.

The embodiments provided herein may be implemented in any one or combination of hardware, software, firmware, or solid state logic circuitry, and may be implemented in connection with signal processing, control, and/or application specific circuitry. Particular embodiments of the present application provide an apparatus or device that may include one or more processors (e.g., microprocessors, controllers, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), etc.) that process various computer-executable instructions to control the operation of the apparatus or device. Particular embodiments of the present application provide an apparatus or device that can include a system bus or data transfer system that couples the various components together. A system bus can include any of a variety of different bus structures or combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and/or a processor or local bus that utilizes any of a variety of bus architectures. The devices or apparatuses provided in the embodiments of the present application may be provided separately, or may be part of a system, or may be part of other devices or apparatuses.

Particular embodiments provided herein may include or be combined with computer-readable storage media, such as one or more storage devices capable of providing non-transitory data storage. The computer-readable storage medium/storage device may be configured to store data, programmers and/or instructions that, when executed by a processor of an apparatus or device provided by embodiments of the present application, cause the apparatus or device to perform operations associated therewith. The computer-readable storage medium/storage device may include one or more of the following features: volatile, non-volatile, dynamic, static, read/write, read-only, random access, sequential access, location addressability, file addressability, and content addressability. In one or more exemplary embodiments, the computer-readable storage medium/storage device may be integrated into a device or apparatus provided in the embodiments of the present application or belong to a common system. The computer-readable storage medium/memory device may include optical, semiconductor, and/or magnetic memory devices, etc., and may also include Random Access Memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, a hard disk, a removable disk, a recordable and/or rewriteable Compact Disc (CD), a Digital Versatile Disc (DVD), a mass storage media device, or any other form of suitable storage media.

The above is an implementation manner of the embodiments of the present application, and it should be noted that the steps in the method described in the embodiments of the present application may be sequentially adjusted, combined, and deleted according to actual needs. In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments. It is to be understood that the embodiments of the present application and the structures shown in the drawings are not to be construed as particularly limiting the devices or systems concerned. In other embodiments of the present application, an apparatus or system may include more or fewer components than the specific embodiments and figures, or may combine certain components, or may separate certain components, or may have a different arrangement of components. Those skilled in the art will understand that various modifications and changes may be made in the arrangement, operation, and details of the methods and apparatus described in the specific embodiments without departing from the spirit and scope of the embodiments herein; without departing from the principles of embodiments of the present application, several improvements and modifications may be made, and such improvements and modifications are also considered to be within the scope of the present application.