1. A method for determining suitability of a neural network model, the method comprising:

acquiring graph data samples, wherein the graph data samples comprise training samples and testing samples;

training the selected graph neural network model according to the training sample to obtain a target graph neural network model;

inputting the test sample into the target neural network model, and obtaining a link prediction result corresponding to the test sample according to a prediction result of an incidence relation between test sample nodes output by the target neural network model;

and judging the applicability of the graph neural network model and the graph data sample according to the link prediction result.

2. The method of claim 1, wherein determining the suitability of the graph neural network model to the graph data sample based on the link prediction comprises:

when the link prediction result is smaller than a preset index threshold value, judging that the graph neural network model is not suitable for the graph data sample;

and when the link prediction result is not less than a preset index threshold value, judging the applicability of the graph neural network model and the graph data sample according to the graph signal-to-noise ratio of the test sample.

3. The method of claim 2, wherein the determining the applicability of the graph neural network model to the graph data sample according to the graph signal-to-noise ratio of the test sample when the link prediction result is not less than a preset index threshold comprises:

when the link prediction result is not less than a preset index threshold value, determining the average semantic similarity between the node and a first-order neighbor node according to the node corresponding data in the test sample and the first-order neighbor node corresponding data associated with the node;

obtaining the graph signal-to-noise ratio of the test sample according to the average semantic similarity;

when the graph signal-to-noise ratio is smaller than a preset graph signal-to-noise ratio threshold value, judging that the graph data sample is not suitable for the graph neural network model;

and when the graph signal-to-noise ratio is not less than a preset graph signal-to-noise ratio threshold value, judging that the graph data sample is suitable for the graph neural network model.

4. The method of claim 3, wherein when the graph signal-to-noise ratio is less than a preset graph signal-to-noise ratio threshold, the method further comprises:

and performing data cleaning on the graph data sample until the graph signal-to-noise ratio of the graph data sample is not less than a preset graph signal-to-noise ratio threshold value.

5. The method of claim 3, wherein when the link prediction result is less than a preset metric threshold, the method further comprises:

and performing model replacement on the selected graph neural network model based on a preset graph neural network model set until the replaced graph neural network model meets preset test conditions after model training is performed on the basis of the training sample, wherein the test conditions comprise that the link prediction result corresponding to the test sample is not less than a preset index threshold value and the graph signal-to-noise ratio corresponding to the test sample is not less than a preset graph signal-to-noise ratio threshold value.

6. The method of claim 1, wherein obtaining the graph data sample comprises:

acquiring historical behavior data of a user;

obtaining a graph data sample according to record data corresponding to user operation in the user historical behavior data, wherein the user operation comprises at least one of search switching operation and viewing operation, the record data corresponding to the search switching operation is used for representing the similarity between at least two search keywords acquired successively based on the search switching operation, and the record data corresponding to the viewing operation is used for representing the relevance between the search keywords and the search result.

7. The method according to claim 6, wherein obtaining a graph data sample according to the log data corresponding to the user operation in the user historical behavior data comprises:

taking the recorded data corresponding to the user operation as nodes, taking the user operation as a connecting edge between the nodes, and establishing a connection relation between the nodes;

determining the weight corresponding to the connecting edge according to the operation times corresponding to the user operation;

and obtaining a graph data sample according to the node, the connecting edge and the weight corresponding to the connecting edge.

8. An apparatus for determining suitability of a neural network model, the apparatus comprising:

the system comprises a sample acquisition module, a data acquisition module and a data processing module, wherein the sample acquisition module is used for acquiring graph data samples which comprise training samples and test samples;

the model training module is used for training the selected graph neural network model according to the training sample to obtain a target graph neural network model;

the model testing module is used for inputting the test sample into the target neural network model, predicting a result according to the incidence relation among the test sample nodes output by the target neural network model, and obtaining a link prediction result corresponding to the test sample;

and the applicability judging module is used for judging the applicability of the graph neural network model and the graph data sample according to the link prediction result.

9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.

10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.

Background

With the development of artificial intelligence technology, neural network models are widely applied in various fields, and deep learning is successful.

Data used in conventional machine learning is data in Euclidean space (Euclidean Domain), the most significant feature of the data in Euclidean space is a regular spatial structure, for example, a picture is a regular square grid, voice data is a one-dimensional sequence, and the data can be represented by a one-dimensional or two-dimensional matrix, and it is more efficient to perform convolution operation. Meanwhile, there is one core assumption: the samples are independent of each other. However, in real life, many data have a spatial structure without rules, that is, data in non-euclidean space, such as an abstracted graph of an electronic trading system, a recommendation system, and the like, and the connection between each node in the graph and other nodes is not fixed. The graph neural network can model data in non-Euclidean space and capture internal dependency relationship of the data. The graph neural network is irregular, chaotic. The graph neural network includes: graph volume Networks (Graph relational Networks), Graph Attention Networks (Graph Attention Networks), and the like.

Experiments show that not all tasks are suitable for being solved by the graph neural network, the graph neural network has limitations, and the effect of some tasks applying the graph neural network is not as good as that of a simple model, but a method capable of judging the applicability of the graph neural network model is lacked at present.

Disclosure of Invention

In view of the above, it is desirable to provide a method, an apparatus, a computer device, and a storage medium for determining applicability of a neural network model, which can determine applicability of the neural network model.

A method for judging applicability of a graph neural network model comprises the following steps:

acquiring a graph data sample, wherein the graph data sample comprises a training sample and a testing sample;

training the selected graph neural network model according to the training samples to obtain a target graph neural network model;

inputting the test sample into a target neural network model, and obtaining a link prediction result corresponding to the test sample according to a correlation prediction result between test sample nodes output by the target neural network model;

and judging the applicability of the graph neural network model and the graph data sample according to the link prediction result.

An apparatus for determining suitability of a neural network model, the apparatus comprising:

the system comprises a sample acquisition module, a data analysis module and a data analysis module, wherein the sample acquisition module is used for acquiring graph data samples which comprise training samples and test samples;

the model training module is used for training the selected graph neural network model according to the training samples to obtain a target graph neural network model;

the model testing module is used for inputting a test sample into the target neural network model and obtaining a link prediction result corresponding to the test sample according to a correlation prediction result between test sample nodes output by the target neural network model;

and the applicability judging module is used for judging the applicability of the graph neural network model and the graph data sample according to the link prediction result.

A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:

acquiring a graph data sample, wherein the graph data sample comprises a training sample and a testing sample;

training the selected graph neural network model according to the training samples to obtain a target graph neural network model;

inputting the test sample into a target neural network model, and obtaining a link prediction result corresponding to the test sample according to a correlation prediction result between test sample nodes output by the target neural network model;

and judging the applicability of the graph neural network model and the graph data sample according to the link prediction result.

A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:

acquiring a graph data sample, wherein the graph data sample comprises a training sample and a testing sample;

training the selected graph neural network model according to the training samples to obtain a target graph neural network model;

inputting the test sample into a target neural network model, and obtaining a link prediction result corresponding to the test sample according to a correlation prediction result between test sample nodes output by the target neural network model;

and judging the applicability of the graph neural network model and the graph data sample according to the link prediction result.

A computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the steps of:

acquiring a graph data sample, wherein the graph data sample comprises a training sample and a testing sample;

training the selected graph neural network model according to the training samples to obtain a target graph neural network model;

inputting the test sample into a target neural network model, and obtaining a link prediction result corresponding to the test sample according to a correlation prediction result between test sample nodes output by the target neural network model;

and judging the applicability of the graph neural network model and the graph data sample according to the link prediction result.

The method, the device, the computer equipment and the storage medium for judging the applicability of the graph neural network model obtain the target graph neural network model by obtaining the graph data sample, train the selected graph neural network model according to the training sample in the graph data sample to obtain the model training, test the trained model by using the test sample in the graph data sample, input the test sample into the target neural network model, obtain the link prediction result corresponding to the test sample according to the incidence relation prediction result between the test sample nodes output by the target neural network model to detect whether the trained model learns the structure of the graph data, further judge the applicability of the graph neural network model and the graph data sample according to the link prediction result to determine whether the selected graph neural network model is suitable for analyzing the graph data, the model suitable for the graph data can be quickly selected in the model selection process.

Drawings

FIG. 1 is a diagram illustrating an exemplary embodiment of a method for determining the suitability of a neural network model;

FIG. 2 is a schematic flow chart diagram illustrating a method for determining the suitability of a neural network model in accordance with one embodiment;

FIG. 3 is a schematic flow chart diagram illustrating a method for determining the applicability of the neural network model in accordance with another embodiment;

FIG. 4 is a schematic flowchart of a method for determining the suitability of a neural network model in accordance with yet another embodiment;

FIG. 5 is a diagram illustrating relationships between graph data nodes of a method for determining applicability of a graph neural network model in accordance with an embodiment;

FIG. 6 is a schematic diagram illustrating the relationship between the semantic correlation accuracy and the graph SNR for the graph neural network model applicability determination method in another embodiment;

FIG. 7 is a flowchart illustrating a method for determining the applicability of the neural network model in accordance with another embodiment;

FIG. 8 is a flowchart illustrating a method for determining the suitability of the neural network model in accordance with still another embodiment;

FIG. 9 is a flowchart illustrating a method for determining the applicability of the neural network model in accordance with another embodiment;

FIG. 10 is a schematic flowchart showing a method for determining the suitability of the neural network model in accordance with still another embodiment;

FIG. 11 is a block diagram showing the structure of an apparatus for determining the suitability of a neural network model in one embodiment;

FIG. 12 is a diagram illustrating an internal structure of a computer device according to an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.

The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

With the research and progress of artificial intelligence technology, the artificial intelligence technology is developed and applied in a plurality of fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, unmanned aerial vehicles, robots, smart medical care, smart customer service, and the like.

The scheme provided by the embodiment of the application relates to an artificial intelligence graph neural network model applicability judgment method, and can be applied to the application environment shown in FIG. 1. The terminal 102 communicates with the server 104 through a network. The server 104 acquires a graph data sample uploaded by a user through the terminal 102, wherein the graph data sample comprises a training sample and a test sample; training the selected graph neural network model according to the training samples to obtain a target graph neural network model; inputting the test sample into a target neural network model, and obtaining a link prediction result corresponding to the test sample according to a correlation prediction result between test sample nodes output by the target neural network model; and judging the applicability of the graph neural network model and the graph data sample according to the link prediction result. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the server 104 may be implemented by an independent server or a server cluster formed by a plurality of servers. In other embodiments, the method for determining the applicability of the neural network model may be implemented on a separate terminal or server having a data processing function.

In one embodiment, as shown in fig. 2, a method for determining applicability of a neural network model is provided, which is described by taking the method as an example applied to the server in fig. 1, and includes the following steps 202 to 208.

At step 202, graph data samples are obtained, wherein the graph data samples comprise training samples and testing samples.

The graph data sample refers to graph data used for training and testing a model, wherein the graph data is a data display form of a node graph for some data with correlation. The data having the association relationship may be historical behavior data of a certain user, the number of the historical behavior data is large, and the data associated with a certain historical behavior may be one item or multiple items.

For example, a user needs to search for a double-color ball, inputs the double-color ball in a search box of a terminal, and sends a search request to a server, the server feeds back related documents corresponding to the double-color ball to the terminal after responding to the search request sent by the user through the terminal, at this time, if the user clicks one of the documents A, data related to the double-color ball is the document A clicked by the user, if the user feels that none of the documents displayed on the current interface is a desired result, and re-inputs a benefit lottery in the search box, the data related to the double-color ball is the benefit lottery newly searched by the user. It will be appreciated that when both of the above events occur, then the data associated with the "two-color ball" includes both document a and the "welfare lottery". Through the graph data, the data with the incidence relation can be displayed through the connected nodes.

The training samples refer to the pattern data samples used to train the model. The test sample refers to a pattern data sample used for testing the model. Compared with graph data in the application process, the relationship between each node in the graph data sample can be accurately determined in advance, and the graph data in the application process needs to predict whether an association relationship exists between the nodes through a model.

In one embodiment, the training samples and the test samples may be proportionally assigned to the graph data samples. For example, the ratio of the training samples to the test samples is set to 9:1, and the graph data samples are distributed according to 9:1, so as to obtain the training samples and the test samples. In other embodiments, the training samples and the testing samples may be distributed according to other configuration rules. The training samples and the testing samples are obtained by distributing the graph data samples, so that the testing samples and the training samples can keep similar levels, and the accuracy of the testing result of the model can be improved.

And 204, training the selected graph neural network model according to the training samples to obtain a target graph neural network model.

In an embodiment, the training of the model may be achieved by means of supervised learning. Supervised learning is a machine learning task that infers a function from labeled training data. The result to be obtained by the training sample is determined in advance, and the model is trained by taking the result as the training target of the model. In the embodiment, the graph data sample carries labeling information, and the labeling information is an expected output result, namely an association relation between nodes which can be predicted by an expected model.

The selected graph neural network model may be one of a set of models formed from a plurality of graph neural network models, the graph neural network including: graph convolution neural networks, graph attention networks, and the like. Different graph neural network models have certain differences in graph data processing capability and principle, and therefore, graph data processing results are different. In an embodiment, the selected neural network model may be randomly selected or may be selected according to configured rules. For example, the configured rule may be selected according to a matching relationship between the application scenario of the pre-configured graph data and the graph neural network model, for example, if the application scenario of the text search matches the graph convolutional neural network, the graph convolutional neural network may be preferentially selected for training. The matching relation between the application scene and the graph neural network model can be determined based on the applicability judgment result of the historical graph neural network model and the graph data sample or based on the actual application condition of the model.

And training the selected graph neural network model according to the training samples to obtain the target graph neural network model. The training process of the graph neural network model can determine the termination condition of the training by limiting the iteration number, and can also determine the termination condition of the training by limiting the model precision and the like.

And step 206, inputting the test sample into the target neural network model, and obtaining a link prediction result corresponding to the test sample according to the incidence relation prediction result between the test sample nodes output by the target neural network model.

The target neural network model is substantially used for learning a graph structure in a training process, the server inputs a test sample into the target neural network model and processes the test sample according to a learning result of the target neural network model to predict the incidence relation among the nodes of the test sample, and data output by the target neural network model is a prediction result of the incidence relation among the nodes of the test sample.

And the server determines the difference between the prediction result and the actual result through Link prediction (Link prediction) index calculation according to the actual incidence relation between the test sample nodes and the incidence relation prediction result between the test sample nodes. The goal of the link prediction result is to obtain (weight/feature of) a new connecting edge according to the known node and connecting edge, and determine the accuracy of the obtained new connecting edge.

In an embodiment, the link predictor may be probability data with a value range of [0,1 ]. In an embodiment, the link prediction result may be at least one of AUC (Area Under the ROC Curve, an Area enclosed by coordinate axes, and a performance index for measuring the quality of the learner) or Precision.

AUC measures the accuracy of the algorithm as a whole. After training, the link prediction algorithm may obtain a similarity value Sim (similarity value of an edge) of each pair of nodes in the network. The AUC indicator is based on a comparison between a similarity value of an edge in the test set and a similarity value of an edge that is not present (i.e., the edge that is not present is used as a reference).

If Sim_Testing＞Sim_{Is absent from}The numerator of the value is added with 1 (the prediction effect is proved to be good at this moment);

if Sim_Testing＝Sim_{Is absent from}The numerator of the value is then increased by 0.5 (in this case corresponding to a random selection)

If Sim_Testing＜Sim_{Is absent from}The numerator of the value is added with 0 (the prediction effect is very poor at this moment);

the denominator of the numerical value is the number of times of comparison between the similarity value of the edge in the test set and the similarity value of the non-existing edge, and the AUC index is the ratio of the numerator of the numerical value to the denominator of the numerical value.

Precision only considers whether the edge of the first L bit is predicted accurately. After training, the link prediction algorithm obtains the similarity values between node pairs, removes the edges in the training set, only sorts the similarity values of the edges in the test set and the nonexistent edge set, and takes the first L after sorting. Assuming that N out of L belongs to the test set, the Precision value is N/L.

And step 208, judging the applicability of the graph neural network model and the graph data sample according to the link prediction result.

The link prediction result is probability data used for describing the accuracy of the prediction result, the value range is [0,1], and based on the link prediction result, a model problem or a graph data problem can be distinguished. Specifically, when the link prediction result approaches 1, the characterization model already learns the structure of the graph, the problem may be in the graph data, and when the link prediction result approaches 0, the characterization model does not learn the structure of the graph, and the problem is in the selected model.

The method for judging the applicability of the graph neural network model comprises the steps of obtaining a graph data sample, training the selected graph neural network model according to a training sample in the graph data sample to obtain a target graph neural network model, realizing the training of the model, testing the trained model by using a test sample in the graph data sample, inputting the test sample into the target neural network model, obtaining a link prediction result corresponding to the test sample according to an incidence relation prediction result between test sample nodes output by the target neural network model, detecting whether the trained model learns the structure of graph data, further judging the applicability of the graph neural network model and the graph data sample according to the link prediction result to determine whether the selected graph neural network model is suitable for analyzing the graph data, and being capable of selecting the model, the model for which the graph data applies is selected quickly.

In one embodiment, as shown in FIG. 2, the suitability of the graph neural network model to the graph data sample is determined 208 based on the link prediction results, including steps 302-304.

And step 302, when the link prediction result is smaller than a preset index threshold value, judging that the graph neural network model is not suitable for the graph data sample.

And 304, when the link prediction result is not less than the preset index threshold, judging the applicability of the graph neural network model and the graph data sample according to the graph signal-to-noise ratio of the test sample.

The preset index threshold is a limit value set based on the requirements of a specific application scene for evaluating the link prediction result. Taking the preset index threshold value as 0.5 as an example, when the link prediction result is less than 0.5, if the link prediction result is 0.2, the representation model does not learn the structure of the graph, the problem is in the selected model, at this time, it is determined that the graph neural network model is not suitable for the graph data sample, and further, for the case that the selected model has the problem, model training can be performed again by replacing the graph neural network model. When the link prediction result is not less than 0.5, if the link prediction result is 0.8, the characterization model already learns the structure of the graph, and the problem may be in the graph data.

In the above embodiment, on one hand, the determination process of the situation that the graph neural network model is not suitable for the graph data sample is simplified, and the inapplicable model can be rapidly screened according to the determination condition that the link prediction result is smaller than the preset index threshold, and on the other hand, the applicability of the graph neural network model and the graph data sample is determined through secondary detection based on the combination of the link prediction result and the graph signal-to-noise ratio, so that the effective selection of the graph neural network model suitable for the graph data analysis can be realized.

In one embodiment, when the link prediction result is not less than the predetermined index threshold, the applicability of the graph neural network model and the graph data sample is determined according to the graph signal-to-noise ratio of the test sample, i.e., step 304, which includes steps 402 to 408.

And 402, when the link prediction result is not less than the preset index threshold, determining the average semantic similarity between the node and the first-order neighbor node according to the node corresponding data in the test sample and the first-order neighbor node corresponding data associated with the node.

And step 404, obtaining the graph signal-to-noise ratio of the test sample according to the average semantic similarity.

And step 406, when the graph signal-to-noise ratio is smaller than a preset graph signal-to-noise ratio threshold value, judging that the graph data sample is not suitable for the graph neural network model.

And step 408, when the graph signal-to-noise ratio is not less than the preset graph signal-to-noise ratio threshold value, judging that the graph data sample is suitable for the graph neural network model.

The nodes in the test sample are nodes forming graph data, and the nodes are associated with the nodes through connecting edges, wherein for any node, the node directly associated with one connecting edge can be used as a first-order neighbor node associated with the node. Similarly, the node needing to be indirectly associated with the node through at least two connecting edges is the second-order neighbor node associated with the node, and so on. In an embodiment, the graph data may be a directed graph, i.e. a connecting edge from a first node to a second node has an indicated direction, wherein the indicated direction may be represented by an arrow. In an embodiment, first order neighbor nodes of a directed graph are nodes that are immediately adjacent to a node along the direction indicated by the arrow. In other embodiments, the graph data may also be an undirected graph, that is, the direct association relationship of the nodes has no explicit direction relationship, and nodes that can be connected through one connecting edge are all first-order neighbor nodes to each other.

The data corresponding to the node refers to the content for characterizing the node. For example, the data corresponding to the node in the data search scenario is the content searched by the user and the content of the clicked text. In the embodiment, a user historical behavior session is involved in a data search scene, and the session refers to a search behavior of the user within a period of time, and comprises switching a query and clicking a doc (document). Wherein, query and doc are heterogeneous. Specifically, the two texts are heterogeneous to each other, meaning that the semantic space of the text a to be matched is different from that of the text B. For example, text a corresponds to the user input query and text B corresponds to the retrieved document. The isomers are now: the query length is generally shorter, while the doc length is longer. The words that make up the query are spoken and the words that make up the doc contain more proper nouns. In the embodiment, the actions of searching, clicking and the like in the service search of the user are recorded in the background, and the data can be acquired offline to obtain the user historical action session.

In an embodiment, an average semantic similarity of a node and a first-order neighbor node may be calculated according to the text corresponding to the node in the test sample and the text corresponding to the first-order neighbor node associated with the node. The average semantic similarity may be implemented by any one of cosine similarity, euclidean distance, minkowski distance, Jaccard similarity coefficient, pearson correlation coefficient, or manhattan distance.

It should be noted that, if a certain node is a query, its associated first-order neighbor node may be a query or doc. For example, as shown in fig. 5, after the user searches for the "two-color ball" (i.e., q1), finds that there is no desired data in the selectable items provided by the corresponding page, re-enters the search for "welfare lottery" (i.e., q2), clicks on the "welfare lottery drawing" in the page option corresponding to the "welfare lottery" to jump to a new page (i.e., d1), then finds that there is no data required by the new page, then returns to the previous page by a return operation, re-enters the search for "lottery drawing number" (i.e., q3), and clicks on the "lottery drawing" in the page option corresponding to the "lottery drawing number" to jump to a new page (i.e., d 2). In the graph data, q1, q2, q3, d1, d2 are nodes in the graph data. The first-order neighbor node corresponding to q1 is q2, the first-order neighbor node corresponding to q2 is q3 and d1, and the first-order neighbor node corresponding to q3 is d 2.

Graph signal-to-noise ratio SNR_iFor characterizing the case of noisy data in graph data, where the noisy data may determine whether the relationship between a node and a first-order neighbor node is noisy data by the average semantic similarity of the two nodes.

Wherein, match-score (text)_i，text_j) Representing the average semantic similarity score, N, between node i and node j_iRefers to the number of first-order neighbor nodes of the ith node.

And when the graph signal-to-noise ratio is smaller than a preset graph signal-to-noise ratio threshold value, judging that the graph data sample is not suitable for the graph neural network model. And when the graph signal-to-noise ratio is not less than the preset graph signal-to-noise ratio threshold value, judging that the graph data sample is suitable for the graph neural network model.

As shown in fig. 6, it can be found by calculating the semantic relevance accuracy (precision) of the neighboring nodes of the graph data, which is strongly correlated with the graph signal-to-noise ratio. The reliability that the graph data sample is suitable for the neural network model of the graph is judged through the graph signal-to-noise ratio is further verified. Specifically, the semantic relevance accuracy can be obtained by marking the semantic relevance ratio in the imbedding neighbor nodes manually. The calculation formula is as follows:

wherein, label_ijRepresents the artificial label, test _ set, between the neighboring nodes i and j_iRefers to the corresponding test set for the ith node.

And if the graph signal-to-noise ratio index is lower than the preset graph signal-to-noise ratio threshold, the data distribution of the current graph data is not suitable for the graph neural network model. At this time, the data should be cleaned, the signal-to-noise ratio index of the graph should be improved, or other types of models, such as a deep neural network model, should be replaced.

In one embodiment, as shown in fig. 7, when the map signal-to-noise ratio is smaller than the preset map signal-to-noise ratio threshold, step 702 is further included.

Step 702, data cleaning is carried out on the graph data sample until the graph signal-to-noise ratio of the graph data sample is not less than a preset graph signal-to-noise ratio threshold value.

It is understood that step 702 may be performed concurrently with the determining step in step 406, either before or after the determining step.

Data cleansing refers to the process of dealing with noisy data in a sample of map data. Specifically, whether the association relationship between the nodes is noise data or not can be determined according to the semantic similarity between the nodes, when the semantic similarity between the nodes is smaller than a preset similarity threshold, the association relationship between the nodes can be determined to be noise data, and when the semantic similarity between the nodes is not smaller than the preset similarity threshold, the association data between the nodes is determined to be normal data.

Through data cleansing, noisy data in the graph data samples may be removed to adjust the data distribution of the graph data samples to make it suitable for the selected graph neural network model. If the graph signal-to-noise ratio of the graph data sample is not less than the preset graph signal-to-noise ratio threshold value after the data cleaning, the selected graph neural network model should be replaced.

In one embodiment, as shown in fig. 8, when the link prediction result is smaller than the preset index threshold, step 802 is further included. It is understood that step 802 may be performed concurrently with the determining step in step 406, either before or after the determining step.

And 802, performing model replacement on the selected graph neural network model based on a preset graph neural network model set until the replaced graph neural network model meets preset test conditions after model training is performed on the basis of a training sample.

The test conditions comprise that the link prediction result corresponding to the test sample is not less than a preset index threshold value, and the image signal-to-noise ratio corresponding to the test sample is not less than a preset image signal-to-noise ratio threshold value.

The model replacement can be realized based on a plurality of graph neural network models in the model set, and the replaced object can be any one of the model set which is not selected, or the one with the highest matching degree is selected from the model set based on the configured selection rule.

For the replaced graph neural network model, model training and testing need to be performed again based on the graph data samples, the specific training and detecting process is the same as that of the graph neural network model before replacement, and the link prediction result corresponding to the test sample and the graph signal-to-noise ratio can also be obtained in the same manner, and are not repeated.

If the replaced graph neural network model is subjected to model testing through the test sample after model training is carried out on the basis of the training sample, the link prediction result corresponding to the test sample is not smaller than the preset index threshold value, and the graph signal-to-noise ratio corresponding to the test sample is not smaller than the preset graph signal-to-noise ratio threshold value, it can be judged that the replaced graph neural network model and the graph data sample have better applicability.

By replacing the graph neural network model, training and testing the replaced graph neural network model can be convenient to find the graph neural network model suitable for analysis of the graph data sample.

In one embodiment, as shown in FIG. 9, obtaining a graph data sample includes steps 902-904.

Step 902, obtaining historical behavior data of a user.

And 904, obtaining a graph data sample according to the record data corresponding to the user operation in the user historical behavior data. The user operation can be embodied by a user historical behavior data session, wherein the session comprises a search behavior of the user within a period of time, and the search behavior comprises at least one of a search switching operation (switching query) and a viewing operation (clicking doc). The search switching operation refers to an operation that a user inputs at least two search keywords successively to search, the record data corresponding to the search switching operation comprises the at least two search keywords successively input, and the record data corresponding to the search switching operation is used for representing the similarity between the at least two search keywords successively input. For example, the search switching operation may be an operation of inputting a first search keyword by a user for searching, and then inputting a second search keyword for searching again, and at this time, the record data corresponding to the search switching operation may be used to represent the similarity between the first search keyword and the second search keyword. The viewing operation refers to an operation that after a user inputs a search keyword, a search result is triggered to enter a data detail viewing interface of the search result, the record data corresponding to the viewing operation comprises the input search keyword and a text corresponding to the search result, and the record data corresponding to the corresponding viewing operation can be used for representing the relevance between the search keyword and the search result. For each user, corresponding user historical behavior data is recorded. And constructing graph data corresponding to the user historical behavior data of the user by performing graph modeling on the session.

In one embodiment, obtaining the graph data sample according to the record data corresponding to the user operation in the user historical behavior data includes: and taking the recorded data corresponding to the user operation as nodes, and taking the user operation as a connecting edge between the nodes to establish a connection relation between the nodes. And determining the weight corresponding to the connecting edge according to the operation times corresponding to the user operation. And obtaining a graph data sample according to the nodes, the continuous edges and the weights corresponding to the continuous edges.

It can be understood that the user operation may be only a search switching operation or only a viewing operation, or may include both the search switching operation and the viewing operation, and the search switching operation and the viewing operation are simultaneously objects of data processing, so that a term associated with an input search term and a text associated with the input search term can be obtained, and an accurate search result can be obtained subsequently.

In one embodiment, taking the example of simultaneously including the search switching operation and the viewing operation, the query and the doc are taken as nodes, the query or the click doc is taken as a continuous edge, and the proportion of the repetition times qv (query volume, search times) of the query or the click doc in the user historical behavior data is taken as the weight of the continuous edge. Specifically, for a certain node a, there are 4 connected edges, where the number of repetitions of the connected edge 1 is 1, the number of repetitions of the connected edge 2 is 2, the number of repetitions of the connected edge 3 is 3, and the number of repetitions of the connected edge 4 is 4, so that the sum of all operations corresponding to the node is 1+2+3+4 equal to 10, so that the weight of the connected edge 1 is 1/10 equal to 0.1, the weight of the connected edge 2 is 2/10 equal to 0.2, the weight of the connected edge 3 is 3/10 equal to 0.3, and the weight of the connected edge 4 is 4/10 equal to 0.4. And constructing to obtain a graph data sample based on the nodes, the edges among the nodes and the weights corresponding to the edges. By calculating the weight data of the connecting edges, the correlation strength between the data can be accurately and effectively described, and the accuracy of the graph data is improved.

In one embodiment, as shown in fig. 10, a method for determining applicability of a neural network model is provided, which specifically includes the following steps 1002 to 1026.

Step 1002, obtaining historical behavior data of a user.

And 1004, taking the recorded data corresponding to the user operation as nodes, taking the user operation as a connecting edge between the nodes, and establishing a connection relation between the nodes, wherein the user operation comprises a search switching operation and a viewing operation.

Step 1006, determining the weight corresponding to the connecting edge according to the operation times corresponding to the user operation.

And step 1008, obtaining graph data samples according to the nodes, the connecting edges and the weights corresponding to the connecting edges, and dividing the graph data samples into training samples and testing samples.

And step 1010, training the selected graph neural network model according to the training samples to obtain a target graph neural network model.

Step 1012, inputting the test sample into the target neural network model, and obtaining a link prediction result corresponding to the test sample according to the incidence relation prediction result between the test sample nodes output by the target neural network model.

And 1014, when the link prediction result is smaller than the preset index threshold, judging that the graph neural network model is not suitable for the graph data sample, and entering 1026.

And step 1016, when the link prediction result is not less than the preset index threshold, determining the average semantic similarity between the node and the first-order neighbor node according to the node corresponding data in the test sample and the first-order neighbor node corresponding data associated with the node.

And step 1018, obtaining the graph signal-to-noise ratio of the test sample according to the average semantic similarity.

Step 1020, when the graph signal-to-noise ratio is smaller than a preset graph signal-to-noise ratio threshold value, judging that the graph data sample is not suitable for the graph neural network model, and entering step 1024 or step 1026.

And step 1022, when the graph signal-to-noise ratio is not less than the preset graph signal-to-noise ratio threshold, determining that the graph data sample is suitable for the graph neural network model.

And step 1024, performing data cleaning on the graph data sample until the graph signal-to-noise ratio of the graph data sample is not less than a preset graph signal-to-noise ratio threshold value.

And step 1026, performing model replacement on the selected graph neural network model based on the preset graph neural network model set until the replaced graph neural network model meets preset test conditions after model training is performed on the basis of the training sample.

The application also provides an application scenario, and the application scenario applies the method for judging the applicability of the graph neural network model. Specifically, the application of the neural network model applicability determination method in WeChat search is taken as an example for explanation.

Searching one search is a function in WeChat, and can search friend circles, articles, public numbers, novels, music, expressions and the like according to keywords. In the first search, a certain type of result search, such as a public number search, an applet search, and the like, belongs to the vertical search. The vertical search is a specialized search engine in a certain industry, is the subdivision and extension of the search engine, integrates certain special information in a library once, extracts required data from directional subsections, processes the data and returns the data to a user in a certain form. Vertical searches also include service searches. The service search may present services that satisfy the user query directly to the user. For example, when searching for a caregiver, the service search may directly provide a caregiver service menu.

The online processing part of the service search is divided into query intention identification, recall sequencing, fine ranking and posterior module. The scheme is applied to data mining of the posterior module, and the query-doc set with strong correlation is discovered offline according to user behavior data. For example, it is found from the user log that most users click on a doc under a certain query, and we consider that the correlation between the two query-doc is strong. The query-doc sets with strong correlation can be used as basic data accumulation of service search, and experience of service search is stable by ensuring recurrence of the query-doc sets.

The problem of detection of query-doc correlation can be solved by taking the graph neural network as a model, but the effect is still found to have a space for improvement, and based on the method, a method for measuring the feasibility of the graph neural network model is provided, so that guiding significance is provided for selection of a later scheme.

Firstly, according to the user historical behavior session, query and doc are used as nodes, query or click doc is switched to be used as a connecting edge, and the proportion of the repetition times of the switching query or click doc in the user historical behavior data is used as the weight of the connecting edge to obtain graph data.

Step1, training the selected Graph neural Network model (Graph convolution Network GCN, Graph constraint Network or Graph Attention Network GAT, Graph Attention Network) according to the training data in the Graph data.

Step2, calculating a Link prediction index based on test data in the graph data, and distinguishing a model problem from a graph data problem; if the Link prediction task is poor in effect, the model representing and selecting does not learn the structure of the graph, and the problem is solved in the selected model. A new model needs to be selected and Step1 is repeated.

If the Link prediction task is good in effect, the model representing the selection already learns the structure of the graph, the problem is not in the data of the model, and the Step3 is entered.

Step3, calculating the graph signal-to-noise ratio SNR of the test data_iSignal noise ratio, i.e. the average semantic similarity of a node to its first-order neighbors.

By calculating the semantic relevance accuracy precision (the ratio of semantic relevance in the artificially labeled embedding neighbor nodes), it can be found that the semantic relevance accuracy is strongly related to the graph signal-to-noise ratio.

If the graph signal-to-noise ratio SNRi is low, the current data distribution is not suitable for the graph neural network model. At this time, the data should be cleaned, the signal-to-noise ratio of the graph should be improved, or other types of models, such as a deep neural network model, should be replaced. Through the processing, whether the matching scheme of the data and the selected graph neural network has problems or not can be found as early as possible, the reason of the problems can be accurately positioned, little walking is performed and no work is done in the selection process of the model, and the model suitable for the graph data can be quickly selected.

It should be understood that, although the steps in the flowcharts referred to in the embodiments described above are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the flowcharts related to the above embodiments may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a part of the steps or stages in other steps.

In one embodiment, as shown in fig. 11, there is provided a device 1100 for determining applicability of a neural network model, which may be a part of a computer device using software modules or hardware modules, or a combination of the two modules, and specifically includes: a sample acquisition module 1102, a model training module 1104, a model testing module 1106, and an applicability determination module 1108, wherein:

a sample obtaining module 1102, configured to obtain a graph data sample, where the graph data sample includes a training sample and a test sample;

the model training module 1104 is used for training the selected graph neural network model according to the training samples to obtain a target graph neural network model;

the model testing module 1106 is used for inputting a test sample into the target neural network model, and obtaining a link prediction result corresponding to the test sample according to a correlation prediction result between test sample nodes output by the target neural network model;

the applicability determination module 1108 is configured to determine applicability of the graph neural network model and the graph data sample according to the link prediction result.

In one embodiment, the applicability determination module is further configured to determine that the graph neural network model is not applicable to the graph data sample when the link prediction result is smaller than a preset index threshold; and when the link prediction result is not less than the preset index threshold, judging the applicability of the graph neural network model and the graph data sample according to the graph signal-to-noise ratio of the test sample.

In one embodiment, the applicability judging module is further configured to determine, when the link prediction result is not less than the preset index threshold, an average semantic similarity between the node and a first-order neighbor node according to the node corresponding data in the test sample and the first-order neighbor node corresponding data associated with the node; obtaining the graph signal-to-noise ratio of the test sample according to the average semantic similarity; when the graph signal-to-noise ratio is smaller than a preset graph signal-to-noise ratio threshold value, judging that the graph data sample is not suitable for the graph neural network model; and when the graph signal-to-noise ratio is not less than the preset graph signal-to-noise ratio threshold value, judging that the graph data sample is suitable for the graph neural network model.

In one embodiment, the device for determining suitability of the graph neural network model further comprises a data cleaning module, and the data cleaning module is used for performing data cleaning on the graph data sample until the graph signal-to-noise ratio of the graph data sample is not less than a preset graph signal-to-noise ratio threshold value.

In one embodiment, the device for determining suitability of the neural network model further includes a model replacement module, where the model replacement module is configured to perform model replacement on the selected neural network model based on a preset set of neural network models until the replaced neural network model meets a preset test condition after performing model training based on a training sample, where the test condition includes that a link prediction result corresponding to the test sample is not less than a preset index threshold and a signal-to-noise ratio corresponding to the test sample is not less than a preset signal-to-noise ratio threshold.

In one embodiment, the sample acquiring module is further configured to acquire user historical behavior data; and obtaining a graph data sample according to the record data corresponding to the user operation in the user historical behavior data, wherein the user operation comprises at least one of search switching operation and viewing operation, the record data corresponding to the search switching operation is used for representing the similarity between at least two search keywords acquired successively based on the search switching operation, and the record data corresponding to the viewing operation is used for representing the relevance between the search keywords and the search result.

In one embodiment, the sample obtaining module is further configured to use the recorded data corresponding to the user operation as nodes, use the user operation as a connecting edge between the nodes, and establish a connection relationship between the nodes; determining the weight corresponding to the connecting edge according to the operation times corresponding to the user operation; and obtaining a graph data sample according to the nodes, the connecting edges and the weights corresponding to the connecting edges.

The device for judging the applicability of the graph neural network model obtains the target graph neural network model by obtaining the graph data sample and training the selected graph neural network model according to the training sample in the graph data sample to realize the training of the model, tests the trained model by using the testing sample in the graph data sample, inputs the testing sample into the target neural network model, obtains the link prediction result corresponding to the testing sample according to the incidence relation prediction result between the testing sample nodes output by the target neural network model to detect whether the trained model learns the structure of the graph data, further judges the applicability of the graph neural network model and the graph data sample according to the link prediction result to determine whether the selected graph neural network model is suitable for analyzing the graph data, and can realize the selection of the model, the model for which the graph data applies is selected quickly.

The specific definition of the device for determining the applicability of the neural network model may refer to the above definition of the method for determining the applicability of the neural network model, and is not described herein again. The modules in the device for determining the applicability of the neural network model in the figure can be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 12. The computer device includes a processor 1210, a memory 1220, and a network interface 1250 connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory 1220 of the computer device includes a non-volatile storage medium 1230, an internal memory 1240. The non-volatile storage media 1230 stores an operating system 1232, computer programs 1234, and a database 1236. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for recording data operated by a user and constructed graph data. The network interface 1250 of the computer device is used to communicate with an external terminal through a network connection. The computer program is executed by a processor to implement a method for determining suitability of a neural network model.

Those skilled in the art will appreciate that the architecture shown in fig. 12 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is further provided, which includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method embodiments when executing the computer program.

In an embodiment, a computer-readable storage medium is provided, in which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.

In one embodiment, a computer program product or computer program is provided that includes computer instructions stored in a computer-readable storage medium. The computer instructions are read by a processor of a computer device from a computer-readable storage medium, and the computer instructions are executed by the processor to cause the computer device to perform the steps in the above-mentioned method embodiments.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware related to instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above examples only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.