1. A method of semantic recognition, the method comprising:

acquiring a text to be identified;

acquiring a candidate text set in a label text index library according to the text to be identified; the annotation text index library comprises a plurality of annotation texts with semantic annotation information;

acquiring a label text which is most matched with the sentence to be recognized from the candidate text set as a target text;

and marking the text to be recognized according to the semantic marking information corresponding to the target text to obtain a semantic marking result.

2. The method of claim 1, further comprising:

and obtaining a semantic recognition result of the text to be recognized according to the semantic labeling result.

3. The method according to claim 1, wherein the candidate text set is obtained from a label text index library according to the text to be identified; the annotated text index library comprises a plurality of annotated texts with semantic annotation information, and comprises the following steps:

searching in a label text index library by taking a text to be identified as an index to obtain an inverted list;

sorting the labeled texts in the inverted list according to the similarity with the texts to be recognized;

and taking the labeled text with the similarity greater than the preset similarity as a candidate text set.

4. The method according to claim 3, wherein the step of using the labeled text with similarity greater than the preset similarity as the candidate text set comprises:

and adding the labeled texts with the similarity greater than the preset similarity into the candidate text set in sequence according to the sequence from the large similarity to the small similarity, wherein the number of the similar texts in the candidate text set does not exceed the preset number.

5. The method of claim 1, wherein the annotated text index library comprises a plurality of annotated texts,

the labeled text is obtained by converting the preprocessed text into an index format;

the preprocessed text is obtained by preprocessing the labeled corpus labeled with the slot information.

6. The method of claim 1, wherein the obtaining, as a target text, a tagged text that best matches the sentence to be recognized from the candidate text set comprises:

acquiring a first feature vector representation of a text to be recognized;

acquiring a second characteristic vector representation of the label text in the candidate text set;

and inputting the first characteristic vector representation and the second characteristic vector representation into a trained text matching model, and obtaining a label text which is most matched with the text to be recognized as a target text.

7. The method of claim 6, wherein obtaining the first feature vector representation of the text to be recognized comprises:

performing word segmentation processing on the text to be recognized to obtain a word vector of the text to be recognized;

and carrying out average word vector processing on the word vectors of the sentences to be recognized to obtain the sentence vectors of the texts to be recognized.

8. The method of claim 6, wherein obtaining the second feature vector representation of the annotation text in the candidate text set comprises:

performing word segmentation processing on the labeled texts in the candidate text set to obtain word vectors of the labeled texts;

and carrying out average word vector processing on the word vectors of the labeled texts to obtain sentence vectors of the labeled texts.

9. The method of claim 1, further comprising:

when a bad case response occurs, acquiring a text to be identified corresponding to the bad case;

performing semantic annotation on a text to be recognized;

and adding the text to be identified corresponding to the bad case and the corresponding semantic annotation information into an annotated text index library to update the annotated text index library.

10. The method according to claim 1, wherein the labeling the text to be recognized according to the semantic labeling information corresponding to the target text to obtain a semantic labeling result, comprises:

obtaining a slot position corresponding to each word of the target text according to the semantic annotation information of the target text;

and marking words corresponding to the target text in the text to be recognized as the same slot positions.

11. A semantic recognition apparatus, the apparatus comprising:

the text to be recognized acquisition module is used for acquiring a text to be recognized;

the candidate text set acquisition module is used for acquiring a candidate text set in a label text index library according to the text to be identified; the annotation text index library comprises a plurality of annotation texts with semantic annotation information;

a target text acquisition module, configured to acquire, in the candidate text set, a tagged text that is most matched with the sentence to be recognized as a target text;

and the semantic annotation module is used for annotating the text to be identified according to the semantic annotation information corresponding to the target text to obtain a semantic annotation result.

12. An electronic device, characterized in that the electronic device comprises:

one or more processors;

a memory;

one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the method of any of claims 1-10.

13. A computer-readable storage medium, having stored thereon program code that can be invoked by a processor to perform the method according to any one of claims 1 to 9.

Background

Artificial Intelligence (AI) is a theory, method and technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. Natural Language Processing (NLP) is an important direction in artificial intelligence, and various theories and methods for realizing efficient communication between a person and a computer using natural Language are mainly studied.

At present, a commonly used semantic recognition scheme usually needs to recognize entities with specific meanings, such as song names, person names, place names and the like, in texts through a machine learning model, and training of the machine learning model is long in time consumption and affects the efficiency of semantic recognition.

Disclosure of Invention

In view of the above, embodiments of the present application provide a semantic recognition method, apparatus, electronic device and computer-readable storage medium to improve the above problem.

In a first aspect, an embodiment of the present application provides a semantic identification method, where the method includes:

acquiring a text to be identified;

acquiring a candidate text set in a label text index library according to the text to be identified; the annotation text index library comprises a plurality of annotation texts with semantic annotation information;

acquiring a label text which is most matched with the sentence to be recognized from the candidate text set as a target text;

and marking the text to be recognized according to the semantic marking information corresponding to the target text to obtain a semantic marking result.

In a second aspect, an embodiment of the present application provides a semantic recognition apparatus, including:

the text to be recognized acquisition module is used for acquiring a text to be recognized;

the candidate text set acquisition module is used for acquiring a candidate text set in a label text index library according to the text to be identified; the annotation text index library comprises a plurality of annotation texts with semantic annotation information;

a target text acquisition module, configured to acquire, in the candidate text set, a tagged text that is most matched with the sentence to be recognized as a target text;

and the semantic annotation module is used for annotating the text to be identified according to the semantic annotation information corresponding to the target text to obtain a semantic annotation result.

In a third aspect, an embodiment of the present application provides an electronic device, including: one or more processors; a memory; one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the semantic recognition method provided in the first aspect above.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where a program code is stored in the computer-readable storage medium, and the program code may be called by a processor to execute the semantic recognition method provided in the first aspect.

According to the scheme provided by the embodiment of the application, when the text to be recognized is obtained, the candidate text set is obtained in the labeled text index library based on the text to be recognized, and the labeled text which is most matched with the sentence to be recognized is obtained in the candidate text set and serves as the target text, so that the text to be recognized is labeled according to the semantic labeling information corresponding to the target text, and the semantic labeling result is obtained. Therefore, the text to be recognized can be labeled through the semantic labeling information of the matched target text, so that the semantic recognition result of the text to be recognized can be quickly obtained without extracting the entity through the entity model, the response speed of semantic recognition is improved, and delay is reduced.

These and other aspects of the embodiments of the present application will be more readily apparent from the following description of the embodiments.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic diagram illustrating an application environment according to an embodiment of the present application;

FIG. 2 illustrates a semantic platform system diagram according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a corpus callout page according to an embodiment of the present application;

FIG. 4 is a flow chart illustrating a semantic recognition method according to an embodiment of the present application;

fig. 5 is a schematic flow chart illustrating steps S221 to S223 of a semantic recognition method according to an embodiment of the present application;

fig. 6 is a schematic flow chart illustrating steps S231 to S233 of a semantic identification method according to an embodiment of the present application;

FIG. 7 is a diagram illustrating a structure of a text matching model according to an embodiment of the present application;

FIG. 8 is a diagram illustrating a structure of a text matching model according to another embodiment of the present application;

fig. 9 is a schematic structural diagram illustrating a semantic recognition device according to an embodiment of the present application;

fig. 10 is a block diagram illustrating a structure of an electronic device according to an embodiment of the present application;

fig. 11 illustrates a storage unit for storing or carrying program codes for implementing a semantic recognition method according to an embodiment of the present application.

Detailed Description

In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.

Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.

The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

Natural Language Processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that enable efficient communication between humans and computers using natural language. Natural language processing is a science integrating linguistics, computer science and mathematics. Therefore, the research in this field will involve natural language, i.e. the language that people use everyday, so it is closely related to the research of linguistics. Natural language processing techniques typically include text processing, semantic understanding, machine translation, robotic question and answer, knowledge mapping, and the like.

Among them, with the development of text processing technology in artificial intelligence technology, many scenes involving semantic recognition based on text processing technology and natural language processing technology have appeared. Such as a smart question and answer scenario. In the intelligent question-answering scene, a user can input a question which the user desires to know in a text or voice mode, and the intelligent question-answering system can inquire a corresponding answer according to the question input by the user and feed the answer back to the user. For another example, in a news information search scenario, a user may input a topic desired to be known in the form of text voice, and the search system may search for corresponding information according to the topic and feed the corresponding information back to the user. When the user inputs the text in the form of voice, the system can convert the voice into the text and then further process the text.

However, the inventors have studied the relevant text recognition method in the semantic recognition scene and found that the relevant text recognition method has a problem that the delay time is to be improved. In a related semantic recognition method, entities of texts input by a user are extracted through a machine learning model, while the traditional model solution involves model retraining when bad cases are encountered, along with more and more platform functions, more and more labeled corpora are provided, and when more and more complex models are used, the training time of the models is prolonged, so that when the models are used, the bad cases on the line cannot be solved quickly, and great delay is caused.

Therefore, in order to improve the above problems, the inventors provide a semantic identification method, a semantic identification device, an electronic device, and a storage medium, in which a text to be identified is obtained first, then a candidate text set is obtained in a tagged text index library according to the text to be identified, a tagged text that is most matched with a sentence to be identified is obtained in the candidate text set as a target text, and thus the text to be identified is tagged according to semantic tagging information corresponding to the target text, and a semantic tagging result is obtained. Therefore, in the process of performing semantic recognition on the text to be recognized, the text to be recognized does not need to be input into the model for entity extraction, but the labeling information of the labeled corpus can be directly utilized, so that the time consumption of the model for entity extraction in the text processing process is reduced, and the efficiency of performing semantic recognition is improved.

Before further detailed description of the embodiments of the present application, an application environment designed in the embodiments of the present application will be described.

As shown in fig. 1, fig. 1 is a schematic diagram of an application environment according to an embodiment of the present application. The system includes a client 110 and a server 120. The client 110 is configured to collect a text to be recognized input by a user, and then send the collected text to be recognized to the server 120. After receiving the text to be recognized, the server 120 further obtains a candidate text set according to the text to be recognized, and then obtains a target text, thereby executing the semantic recognition method provided by the embodiment of the present application. The server 120 returns the semantic recognition result to the client 110 when obtaining the semantic recognition result of the text to be recognized by the semantic recognition method.

It should be noted that fig. 1 is an exemplary application environment, and the method provided in the embodiment of the present application may also be executed in other application environments. For example, the semantic recognition methods provided by the embodiments of the present application may all be performed by the client 110.

It should be noted that the server 120 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, a middleware service, a domain name service, a security service, a CDN (Content Delivery Network), a big data and an artificial intelligence platform. The electronic device in which the client 110 is located may be, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, and the like.

As shown in fig. 2, fig. 2 is a semantic platform system diagram provided in the embodiment of the present application. As shown in fig. 2, the skilled expert labels corpora in the corpus labeling system, and the obtained initialized corpora are stored in the database. Further, the offline model training module may obtain the labeled corpus from the database as training data to perform offline model training, and publish the trained model. In the application, the offline model training module acquires a large number of similar labeled corpora from the database to train the model. The annotated text index library acquires the annotated corpus from the database to construct an index library, and it can be understood that the annotated corpus used for constructing the index library also needs to be subjected to format conversion according to the rules of the index library. The client sends a text query request to the server, the labeled text index library indexes according to the text to be identified to obtain a candidate text set, and the model processing module obtains the candidate text set and outputs a target text which is most matched with the text query request sent by the user through model processing. And the semantic annotation module labels the text to be recognized according to the semantic labeling information of the target text, then the semantic recognition module performs semantic recognition on the labeled text to be recognized, and finally, the recognition result is fed back to the client.

As shown in fig. 3, fig. 3 is a schematic view of a page labeled with corpora according to an embodiment of the present application. As shown in fig. 3, when the corpus labeling system labels corpuses, the skilled expert labels corpuses according to the intention slot position. For example, for a string of linguistic data "i want to see under the championship", a skilled expert may edit the linguistic data on a linguistic data labeling system, and mark a slot of the string of linguistic data, where the slot is a "resource", and an entity corresponding to the slot is "under the championship". The database comprises a plurality of corpora labeled by the corpus labeling system, and the corpora with labeling information are stored in the database and are used for providing index texts for the index database to import and providing the index texts for the offline model training module to perform offline model training.

The following description will be made of terms related to the embodiments of the present application.

Short text query (query): a request statement input by a user in the intelligent assistant usually contains only one intention expectation of the user. For example: "the first Liudebua ice rain"; "give me a story of fool-watch mountain"; "i want to see the movie without a break" and so on.

Slot (slot): in the task-based dialogue system, slot design under specific intentions is used for expressing important information in the query of a user. If a user creates a music skill play intent to identify a query of "i want to listen to ice and rain" he will design a slot of "singer" in liu de hua and song "in ice and rain".

Es (elastic search): the open-source search indexing tool supports functions of distributed expansion, real-time indexing and the like, supports self-development of retrieval plug-ins, and is used for creating index retrieval for platform labeling query in the scheme.

Word units (Token), before any actual processing of the input text, need to be segmented into language units such as words, punctuation, numbers, or pure character numbers. These units are called word units.

Word Embedding is a technique of converting a Word expressed in a natural language into a vector or matrix form that can be understood by a computer.

Sentence vector (sequence Embedding) is a technique of combining all word vectors of a Sentence into an understandable vector or matrix form.

Embodiments of the present application will be described in detail below with reference to the accompanying drawings.

Referring to fig. 4, fig. 4 is a flowchart illustrating a semantic recognition method according to an embodiment of the present disclosure.

And step S210, acquiring a text to be recognized.

It should be noted that, in the embodiment of the present application, the text to be recognized is a text that needs to be subjected to semantic recognition. The text to be recognized may be a query text input by a user or a string of Speech, and when the user inputs a string of Speech, the Speech of the user may be converted into the text through Speech Recognition (ASR), which facilitates processing by the server to obtain a semantic Recognition result of the text to be recognized. The server can be a Web server or other servers, the client can be a mobile phone, a tablet computer, a notebook computer, a wearable device, an intelligent sound box and other devices capable of interacting with the server, and the text to be recognized input by the user can be transmitted to the server through an intelligent assistant on the client. It can be understood that the present application is not limited thereto, and there may be a plurality of ways of obtaining the text to be recognized.

As an implementation manner of the present invention, the semantic recognition method provided in the embodiment of the present invention may be triggered by a semantic recognition request triggered by a client, and if a text to be recognized that needs to be subjected to semantic recognition is carried in the semantic recognition request, the semantic recognition request may be analyzed to obtain the text carried in the semantic recognition request as the text to be recognized.

As another mode of the present invention, it is not necessary to carry the text to be subjected to semantic recognition matching in the aforementioned semantic recognition request. At this time, after receiving the semantic recognition request, response information may be returned to the client that sent the semantic recognition request, and the client sends the text that needs to be subjected to semantic recognition after receiving the response information, so as to use the received text that needs to be subjected to semantic recognition as the text to be recognized.

S220, acquiring a candidate text set in a label text index library according to the text to be identified; the annotated text index library comprises a plurality of annotated texts with semantic annotation information.

It should be noted that the annotation text may be derived from a database, where a large amount of annotation corpora having semantic annotation information are stored in the database, and the semantic annotation information includes corresponding slots in the annotation corpora. The labeled corpora can be edited by a skill expert in advance, and the corresponding slot position of the corpora is labeled. For example, for a string of "blank lattice of singing in Yang Zong", the annotating personnel can edit the corpus to annotate the slot position of the corpus, the slot position is "singer" and "song name", the entity corresponding to the slot position "singer" is "Yang Zong", and the entity corresponding to the slot position "song name" is "blank lattice". In some embodiments, the semantic annotation information further includes an intent classification corresponding to the annotation corpus. For example, for a string of linguistic data of "blank lattice of Yang Zong sing", the annotating person may edit the linguistic data to mark the intent of the linguistic data as the playing intent. It is to be understood that the present invention is not so limited and that the semantic annotation information can also include other annotation information that can be used for semantic recognition.

It can be understood that, in order to recall more candidate corpuses, as an implementation manner of the present invention, the labeled corpus may be preprocessed before the labeled corpus is used to construct the labeled text index library, and as an implementation manner, the labeled corpus labeled with the slot position information may be processed with stop words, that is, words similar to "i", "do", "yes", "next", "what", "got", "bar", "o", "? "," wool "," most "," [ lambda ], "ng", "what", "still", "it", etc., have no specific meaning, but often appear as stop words in the user query text. The preprocessed text is obtained through stop word processing, and more candidate corpora can be recalled conveniently through preprocessing, so that the overall recall rate is improved.

It can be understood that, in order to construct the annotated text index library, as an embodiment of the present invention, the preprocessed text further needs to be converted into a format suitable for the index library to be retrieved according to the rule of the index library to be constructed, and the preprocessed text is converted into the annotated text through the index format.

As an embodiment of the present invention, in the process of obtaining the candidate text set, a Search mode is adopted to Search for a tagged text most similar to a text to be identified from the tagged text, specifically, an Elastic Search engine is adopted as a bottom Search engine, a format of the preprocessed text is converted according to an index rule, an index is built, and the preprocessed text can be inserted into the index in batch by calling a plug-in of the ES service. Because the ES supports the indexing function, the delay of the ES is generally 500 milliseconds, so that a piece of markup corpus added by a user in real time is used for temporarily solving the operation of an online bad case, for example, when a bad case response occurs, a text to be recognized corresponding to the bad case is acquired, semantic annotation is performed on the text to be recognized, and the text to be recognized corresponding to the bad case and corresponding semantic annotation information are added to the markup text index library to update the markup text index library. Meanwhile, the server sends a data change message to the ES service and pushes the latest change corpus, so that the data change is reflected to the ES index in real time, and a user can conveniently obtain an updated result in real time.

As an implementation mode of the invention, a candidate text set is obtained in a tagged text index library according to a text to be identified, wherein the tagged text index library comprises a plurality of tagged texts with semantic tagged information. The data in the labeled text index library is adjusted based on the index rule of the index library, so that the uniform data format is facilitated, and the index is quickly carried out.

Specifically, the text to be recognized is used as an index and is retrieved from the labeled text index database to obtain the inverted list. In this embodiment, the text to be recognized is used as an index, the labeled text related to the text to be recognized is retrieved from the labeled text index library, and all the labeled texts in the hit list appear in the inverted list, so as to ensure the recall rate. In one embodiment, the annotated text in the inverted list is recalled as a candidate set for subsequent similarity ranking calculations, and is used for data screening in the next step. In this embodiment, a candidate text set similar to the text to be similar is obtained by indexing, and then the candidate text set is further screened, so as to obtain the labeled text most matched with the text to be identified.

As an embodiment of the present invention, the tagged texts in the inverted list may be sorted according to the similarity with the text to be recognized. As another embodiment of the invention, the label text with similarity greater than the preset similarity can be used as the candidate text set.

It can be understood that the number of candidate text sets is uncertain, and the number of the labeled texts in the hit inverted list is different for different texts to be recognized. When the data volume is large, the index library may be overtime, so that the delay of the whole system is affected. Specifically, the labeled texts with the largest similarity can be selected preferentially, and are selected in sequence, and assuming that the number of the candidate text sets is controlled to be a preset number, the inverted list is sorted from large to small according to the similarity, and the labeled texts with the preset number are selected as the candidate text sets. The preset number can be set according to actual needs, and the application is not limited to this.

Step S230, obtaining a label text that is most matched with the sentence to be recognized in the candidate text set as a target text.

And acquiring a candidate text set through the labeled text index library, so that a batch of labeled texts similar to the text to be identified can be acquired. And then screening a target text which is most matched with the text to be recognized from the candidate text set, so that a semantic recognition result of the text to be recognized can be accurately obtained. In this embodiment, further, the annotation text that is most matched with the sentence to be recognized is continuously obtained from the candidate text set. The target text can be obtained in various ways, for example, matching can be performed through various similar models, and the labeled text which is most matched with the sentence to be recognized is obtained from the candidate text set. As one embodiment, there may be multiple choices for the similarity model. The similar model can be obtained by performing off-line training on a large number of similar labeled corpora. Similar notations such as "q 1 ═ i help me to play liu de hua's water of forgetting to play liu de hua" and "q 2 ═ i want to listen to liu de hua's water of forgetting". The large amount of similar linguistic data can be obtained by labeling the linguistic data in advance by a technical expert, and the labeled linguistic data is stored in a database. When the similar model is trained in an off-line mode, a large number of prestored similar labeled texts can be obtained from the database for training the similar model. The labeled text most similar to the text to be recognized is used as the target text, and the target text is most similar to the labeled text and has similar word composition, so that the semantic labeling information of the text to be recognized can be obtained by labeling the text to be recognized which is most matched with the target text with the same semantic labeling information by utilizing the semantic labeling information already possessed by the target text, and further, the semantic recognition result of the text to be recognized is obtained. And the semantic annotation information and the semantic recognition result of the text to be recognized can be obtained without entity extraction of the text to be recognized, so that delay caused by adopting a model can be greatly saved, and the efficiency of semantic recognition is greatly improved.

And S240, labeling the text to be recognized according to the semantic labeling information corresponding to the target text to obtain a semantic labeling result.

In this embodiment, the semantic annotation information includes annotation information related to semantic recognition, which is obtained by annotating an annotation text in advance by an annotator. In some embodiments, the semantic annotation information includes slot position information corresponding to the annotation text, for example, in the annotation text "weather today", the slot position corresponding to "weather" is "date", and the slot position corresponding to "weather" is "weather". In some embodiments, the semantic annotation information also includes a corresponding intent classification in the annotation text, e.g., an intent classification corresponding to the annotation text being "weather today" is query weather. In step S230, a target text that is most matched with the sentence to be recognized is obtained from the candidate text set through similarity model matching. And the labeled texts in the candidate text set are all texts labeled by labeling personnel and provided with semantic labeling information, the text to be identified can be labeled by referring to the semantic labeling information of the target text which is most matched with the text to be identified, and the semantic identification is carried out through the labeling result. As an implementation mode of the invention, the slot position corresponding to each word of the target text is obtained according to the semantic annotation information of the target text. For example, in the target text "i want to listen to the water of forgetting to liu deluxe", the slot corresponding to the entity "liu deluxe" is "singer", and the slot corresponding to the entity "water of forgetting" is "song". And further labeling the word corresponding to the target text in the text to be recognized as the same slot position, for example, in the text "forgetting to do me to play liu de hua", the word corresponding to the target text is "liu de hua", the word is labeled as the same slot position "singer", the word corresponding to the target text is also "forgetting to do me", and the word is labeled as the same slot position "song". And for example, the corresponding intention of the target text is obtained according to the semantic labeling information of the target text. For example, the target text "i want to listen to the water of forgetfulness in Liu De" corresponds to an intent to play. And marking the intention of the text to be recognized as the same play intention as the target text. Therefore, a semantic labeling result of the text to be recognized, namely a slot position and an intention corresponding to the labeled text can be obtained. The text to be recognized and the target text have high similarity and similar entities, the text to be recognized is directly labeled by using the semantic labeling information of the target text, and the step of extracting the entities from the text to be recognized can be saved.

In some embodiments, the semantic recognition method of the embodiments of the present invention may further include the steps of: and obtaining a semantic recognition result of the text to be recognized according to the semantic labeling result.

And the server acquires a semantic recognition result of the recognized text according to the semantic labeling result, transmits the semantic recognition result to the client, provides a corresponding response and completes one-time interaction.

As an implementation manner of the invention, when the target text of the text to be recognized, i.e. the text helping me play the forgetting water of liu de hua, is the forgetting water of liu de hua i want to listen to, the text to be recognized is labeled by the labeling slot position information of the target text, the entity extraction of the text to be recognized is not needed, and the words corresponding to the target text in the text to be recognized can be labeled as the same slot position. By marking the slot position information of the target text into the text to be recognized, the obtained marking result of the text to be recognized is that the slot position corresponding to the entity Liudebua is a singer, the slot position corresponding to the entity forgetting water is a song, the intention corresponding to the target text is a play intention, the text to be recognized adopts the intention corresponding to the target text, namely the play intention, and the play intention comprises the slot position 'singer' and the slot position 'song', so that the semantic recognition result of the text to be recognized, namely 'the forgetting water for playing Liudebua', is 'the forgetting water for playing Liudebua'. And transmitting the semantic recognition result to the client, and opening the music player by the client according to the semantic recognition result to play the forgetting water of Liudebua for the user to complete one-time interaction.

Referring to fig. 5, fig. 5 is a schematic flow chart illustrating steps S221 to S223 of a semantic recognition method according to an embodiment of the present application.

Specifically, step S220 is to obtain a candidate text set in the annotated text index library according to the text to be identified. The annotated text index library comprises a plurality of annotated texts with semantic annotation information, and comprises the following steps:

and step S221, the text to be identified is taken as an index to be searched in the labeled text index database to obtain the inverted list.

As an implementation mode, based on the ES retrieval grammar, the inverted list is obtained by retrieving the text to be recognized as an index in the annotation text index database. The inverted list is a list of all the labeled texts related to the text to be recognized, and in order to ensure recall rate, the labeled texts in the inverted list are generally recalled and used as a candidate set for subsequent similarity ranking calculation.

And S222, sequencing the labeled texts in the inverted list according to the similarity of the texts to be recognized.

In order to obtain a more effective candidate text set and find a target text more matching with the problem to be recognized, the similarity of the texts to be recognized may be sorted according to the size, and preferably, the labeled text with the high similarity is selected as the candidate text set.

And step S223, taking the labeled text with the similarity greater than the preset similarity as a candidate text set.

It can be understood that the number of the labeled texts in the candidate text set cannot be determined, and in some embodiments, the number of the labeled texts hit in the inverted list is not large, and the number of the labeled texts in the candidate text set is small. In other embodiments, the inverted list hits a very large number of labeled texts, and the number of labeled texts in the candidate text combination is large. When the number of the labeled texts in the candidate text set reaches a certain number, the overtime of the system retrieval is triggered, so that the delay of the whole system is influenced. By properly controlling the size of the candidate text set and screening the most similar labeled texts with proper quantity as the candidate text set, the target text can be ensured to be obtained, and the system retrieval is prevented from overtime.

Referring to fig. 6, fig. 6 is a flowchart illustrating steps S231 to S233 of a semantic recognition method according to an embodiment of the present application.

Specifically, step S230, acquiring a labeled text that is most matched with the sentence to be recognized from the candidate text set as a target text, includes:

s231, obtaining a first feature vector representation of the text to be recognized.

As an embodiment of the present invention, acquiring a first feature vector identifier of a text to be recognized includes: and performing word segmentation processing on the sentence to be recognized to obtain a word vector of the sentence to be recognized. And carrying out average word vector processing on the word vectors of the sentences to be recognized to obtain the sentence vectors of the sentences to be recognized.

Before making a detailed explanation, the meaning of the word vector and the sentence vector will be explained first. In this embodiment, the Word vector (Word embedding) is a vector corresponding to a Word or a phrase in the text content. Where a word vector characterizes the meaning of a single word or phrase itself. A Sentence vector (sequence Embedding) is a vector corresponding to the entire text content, and the Sentence vector represents the meaning expressed by the entire text. For example, the word vectors corresponding to the text "we go to motion" may include the word vector corresponding to "i", the word vectors corresponding to "go", the word vectors corresponding to "motion", and the word vectors corresponding to "motion". And the sentence vector corresponding to the "we go to move" is a vector obtained by performing linear transformation on the word vector corresponding to each word, wherein the linear transformation vector can represent the overall meaning of the text "we go to move". The overall meaning can be understood as meaning expressed by the text obtained by combining each word in the text content in the whole.

As an embodiment of the present invention, the word vector corresponding to the text to be recognized includes a word vector corresponding to each word in the text to be recognized. For example, if the text to be recognized is "i want to listen to zhou jilun", the word vector corresponding to "i want to listen to zhou jilun" may include the word vector corresponding to "i", "want" and "listen" and "week" and "jijg" and "lun".

Specifically, there are various methods for constructing word vectors by segmenting words of the text to be recognized, and optionally, in this embodiment, the word vectors of the text to be recognized may be obtained through a specified model. For example, the method may be implemented by a word2vec model or a fasttext model, but the present invention is not limited thereto, and other models may be used to obtain a word vector of a text to be recognized. As an implementation manner, in the process of obtaining the word vector through the specified model, each word in the text to be recognized may be converted into a one-dimensional vector, and an initial one-dimensional vector corresponding to each word in the text to be recognized is obtained. It should be noted that the initial one-dimensional vector here can be understood as an initial word vector of each word in the text to be recognized. Then, the initial one-dimensional vector corresponding to each word is input into the specified model, so that a word vector corresponding to the text to be recognized output by the specified model is obtained, and then the word vectors corresponding to the text to be recognized are obtained by combining the word vectors corresponding to each word output by the specified model. Optionally, the word vector corresponding to the text to be recognized obtained by the combination may be in a sequence form.

It should be noted that, compared with the initial one-dimensional vector corresponding to each word in the text to be recognized, the word vector corresponding to each word in the text to be recognized output by the specified model can fuse more meanings of the context of the text to be recognized, so that the word vector corresponding to each word in the text to be recognized output by the specified model can express the corresponding meaning more accurately. Illustratively, the first text content is "play onion", and the onion therein may be understood as a kind of vegetable or fruit, and may also be understood as a song, and it is known in combination with the context that for onion, the user takes the action of "play" instead of "eat" or other keywords related to vegetables and fruits, so that the actually expressed intention corresponding to the meaning of speech may determine that the onion is a song, and the user actually intends to play music onion. Then, in obtaining the initial one-dimensional vector, the one-dimensional vector corresponding to the onion may represent a vegetable, and after the designated model is processed and output, the word vector corresponding to the onion may represent a song.

Further, as an embodiment of the present invention, a term vector of a sentence to be recognized is obtained by performing average term vector processing on a term vector of the sentence to be recognized. Specifically, the resulting word vectors are averaged to represent the first feature vector representation of the entire text to be recognized.

S232, obtaining a second feature vector representation of the label text in the candidate text set.

As an embodiment of the present invention, the obtaining of the second feature vector representation of the annotation text includes: and performing word segmentation processing on the labeled text to obtain a word vector of the labeled text. Specifically, a word vector is constructed by segmenting the tagged text, and the word vector in this embodiment can be implemented by word2vec or fasttext. Further, average word vector processing is carried out on the word vectors of the labeled texts, and sentence vectors of the labeled texts are obtained. Specifically, the obtained word vectors are averaged to represent the second feature vector representation of the whole annotation text. It is to be understood that the data of the labeled text in the candidate text set may include one or more. When the number of the labeled texts in the candidate text set includes a plurality of labeled texts, a second feature vector representation of each labeled text needs to be obtained respectively.

In this embodiment, the labeled text in the candidate text set is a text for matching with the text to be recognized. As an embodiment of the present invention, in the process of matching the tagged text in the candidate text set with the text to be recognized, matching is performed based on the sentence vector of the text to be recognized and the sentence vector of the tagged text in the candidate text set. And correspondingly acquiring sentence vectors of the labeled texts in the candidate text set after the texts to be identified are acquired. In this way, when the sentence vectors of the labeled texts in the candidate text set need to be acquired, the sentence vectors of the labeled texts in the candidate text set can be acquired by directly reading from the designated storage area, and the sentence vectors of the labeled texts in the candidate text set do not need to be acquired in a calculation way.

And S233, inputting the first characteristic vector representation and the second characteristic vector representation into the trained text matching model, and obtaining a label text which is most matched with the text to be recognized as a target text.

As an implementation mode of the invention, a first feature vector of a text to be recognized and a second feature vector of a labeled text in a candidate text set are input into a trained text matching model for calculation, and a labeled text which is most matched with the text to be recognized is obtained as a similar text.

Fig. 7 shows a schematic structural diagram of a text matching model according to an embodiment of the present application.

As shown in fig. 7, fig. 7 is a schematic structural diagram of a text matching model according to an embodiment of the present application. In the implementation, a gdbt two-classification model is used as a text matching model to acquire a target text. Specifically, word vectors (word entries) of a text to be recognized are obtained, and then sentence vectors (sense entries) of the text to be recognized are obtained by combining all word vectors of the text to be recognized. Similarly, sentence vector representation of the labeled text is obtained, and then the two sentence vectors are transmitted to a classifier of gdbt for 0/1 classification and judgment. And finally, obtaining a label text which is most matched with the text to be recognized as a target text by matching the sentence vector of the text to be recognized with the sentence vectors in all candidate text sets. The gdbt secondary classification model is created by taking similar labeled text pairs stored in a database as training data and obtaining the gdbt secondary classification model through offline training, wherein a logistic regression algorithm can be adopted for modeling, and the logistic regression algorithm is a two-classification/multi-classification modeling algorithm, is a commonly used method in a linear classification model, is simple and stable in calculation and high in speed, and supports a large number of characteristic dimensions. And the first feature vector representation and the second feature vector representation are input into the trained text matching model, and the label text which is most matched with the text to be recognized can be obtained as the target text. It is to be understood that the present application is not limited thereto, and the target text may be obtained by other similar models.

Fig. 8 is a schematic structural diagram of a text matching model according to another embodiment of the present application.

As shown in fig. 8, fig. 8 is a schematic structural diagram of another text matching model according to an embodiment of the present application. In the embodiment, a bert model is used as a text matching model to obtain the target text. bert is called Bidirectional Encoder reproduction from transformations, is a language model training method using massive texts, and can be widely applied to various natural language processing tasks, such as text classification, text matching, machine reading understanding and the like. The Bert model is obtained by using an Encoder in a transform model as a feature extraction module, the Bert model comprises a plurality of trms, and the Trm is the feature extraction module in the Bert model, which may be an Encoder in the transform model. Inputting the text to be recognized and the labeled text in the candidate text set into a BERT model in a format of [ CLS ] + sentence 1+ [ SEP ] + sentence 2, sentence 1 (for example, the text to be recognized) is composed of Tok1, …, TokN, where TokN (for example, the labeled text) may be a mark generated in an input character stream, sentence 2 is composed of Tok ' 1, …, Tok ' M, E1, …, EN is Tok1, …, word vector input of TokN, E1 ', …, EM ', word vector input of Tok ' 1, …, Tok ' M, T1, …, TN is Tok1, …, output of TokN model, i.e., Tok1, …, representation of TokN, T ' 1, …, T ' M, T ' 1, …, output of Tok ' M, i.e., input of Tok ' 1, [ SEP ] is CLS ], where the input of Tok ' M is [ CLS ], [ CLS ] is [ C ] s ], [ s ] is input of Tok's ', [ s ] is CLS ], [ s ] is input of Tok ' 1, and [ s ], [ s ] is represented as [ s ], [ s ] is represented by CLS ], [ s ] is represented by the input of Tok's ', [ s ] s [ s ', [ s ] M, and [ s ] is represented by Tok ', [ s ] and is represented by Tok, [ SEP ] is represented by (T [ SEP ]). And processing the text to be recognized and the labeled text in the candidate text set by the bert model to obtain a target text which is most matched with the text to be recognized in the candidate text set.

The semantic annotation method of the embodiment of the application can not only quickly match the target text, but also label the text to be annotated by using the semantic annotation information of the target text, thereby quickly obtaining the semantic identification result of the text to be annotated. And when a bad case on the line is encountered, the semantic annotation method of the embodiment of the application can quickly solve the problem. The bad case on line means that the system cannot recognize the text to be recognized input by the user. As an implementation manner of the present application, when the weather of the user is all the day, and the query system cannot identify the weather, that is, when a bad case occurs, in this embodiment, the query may be added on the semantic platform, and the corresponding slot information is labeled, the newly added labeled corpus is updated to the system ES index library in less than 1s, when the query of the user requests again, the corresponding target text may be found, and then the pre-labeled slot information is used to complete semantic identification of the text to be identified by the user, and the whole process is completed in seconds. Compared with the existing semantic recognition method adopting the model to extract the entity, when a bad case is encountered, if the bad case is caused by an entity extraction error, the entity model needs to be updated. The process of updating the model takes a long time, and the whole process takes hours. Compared with the scheme in the prior art, the technical scheme of the application can simplify the solving steps when the bad case is solved, and the bad case is quickly solved.

According to the scheme provided by the embodiment of the application, when the text to be recognized is obtained, the candidate text set is obtained in the labeled text index base based on the text to be recognized, and the labeled text which is most matched with the sentence to be recognized is obtained in the candidate text set and serves as the target text, so that the text to be recognized is labeled according to the semantic labeling information corresponding to the target text, the semantic labeling result is obtained, and the semantic recognition result of the text to be recognized is obtained according to the semantic labeling result. Therefore, the text to be recognized can be labeled through the matched target text, so that the semantic recognition result of the text to be recognized can be quickly obtained without extracting the entity through the entity model, the response speed of semantic recognition is improved, delay is reduced, and the accuracy is high.

Referring to fig. 9, which shows a schematic structural diagram of a semantic recognition apparatus 300 according to an embodiment of the present application, the semantic recognition apparatus may include: a text to be recognized acquisition module 310, a candidate text set acquisition module 320, a target text acquisition module 330, and a semantic labeling module 340.

A to-be-recognized text obtaining module 310, configured to obtain a to-be-recognized text. In the embodiment of the application, the text to be recognized is the text needing semantic recognition. The text to be recognized may be a query text input by a user or a string of Speech, and when the user inputs a string of Speech, the Speech of the user may be converted into the text through Speech Recognition (ASR), which facilitates processing by the server to obtain a semantic Recognition result of the text to be recognized.

A candidate text set obtaining module 320, configured to obtain a candidate text set in the labeled text index library according to the text to be identified.

The annotation text index library comprises a plurality of annotation texts with semantic annotation information. The labeled text can be from a database, a large number of labeled linguistic data are stored in the database, and the labeled linguistic data refer to linguistic data labeled with slot position information. The labeled corpus can be the corpus edited by a skill expert in advance, and the slot position information corresponding to the corpus is labeled.

And the target text acquiring module 330 is configured to acquire, as a target text, a labeled text that is most matched with the sentence to be recognized from the candidate text set.

And acquiring a candidate text set through the labeled text index library, so that a batch of labeled texts similar to the text to be identified can be acquired. And then screening a target text which is most matched with the text to be recognized from the candidate text set, so that a semantic recognition result of the text to be recognized can be accurately obtained. Specifically, according to the semantic annotation information already possessed by the target text, the text to be recognized which is most matched with the target text is annotated with the same semantic annotation information, so that the semantic annotation information of the text to be recognized can be obtained, and further the semantic recognition result of the text to be recognized is obtained. And the semantic annotation information and the semantic recognition result of the text to be recognized can be obtained without entity extraction of the text to be recognized, so that delay caused by adopting a model can be greatly saved, and the efficiency of semantic recognition is greatly improved.

And the semantic labeling module 340 is configured to label the text to be identified according to the semantic labeling information corresponding to the target text, and obtain a semantic labeling result.

The text to be recognized and the target text have high similarity and similar entities, the text to be recognized is directly labeled by using the semantic labeling information of the target text, and a semantic labeling result of the text to be recognized can be obtained.

In some embodiments, the semantic recognition apparatus may further include a semantic recognition module 350. The semantic recognition module 350 is configured to obtain a semantic recognition result of the text to be recognized according to the semantic labeling result.

And the server acquires a semantic recognition result of the recognized text according to the semantic labeling result, transmits the semantic recognition result to the client, provides a corresponding response and completes one-time interaction.

According to the scheme provided by the embodiment of the application, when the text to be recognized is obtained, the candidate text set is obtained in the labeled text index base based on the text to be recognized, and the labeled text which is most matched with the sentence to be recognized is obtained in the candidate text set and serves as the target text, so that the text to be recognized is labeled according to the semantic labeling information corresponding to the target text, the semantic labeling result is obtained, and the semantic recognition result of the text to be recognized is obtained according to the semantic labeling result. Therefore, the text to be recognized can be labeled through the matched target text, so that the semantic recognition result of the text to be recognized can be quickly obtained without extracting the entity through the entity model, the response speed of the semantic recognition is improved, and the delay is reduced.

Referring to fig. 10, which shows a block diagram of an electronic device according to an embodiment of the present application, the electronic device 400 includes a memory 410, a processor 420, and a computer program stored in the memory 410 and running on the processor 420, and the processor 420 executes the computer program to implement the method described in the foregoing method embodiment.

Processor 420 may include one or more processing cores. The processor 420 connects various parts throughout the electronic device 400 using various interfaces and lines, performs various functions of the electronic device 700 and processes data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 410, and calling data stored in the memory 410. Alternatively, the processor 420 may be implemented in hardware using at least one of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 420 may integrate one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing display content; the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 420, but may be implemented by a single communication chip.

The Memory 410 may include a Random Access Memory (RAM) or a Read-Only Memory (Read-Only Memory). The memory 410 may be used to store instructions, programs, code sets, or instruction sets. The memory 410 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (e.g., fetch, select, fetch, control, etc.), instructions for implementing various method embodiments described below, and the like. The stored data area may also store data created by the electronic device 400 during use (e.g., text to be recognized, tagged text, target text, semantic tagging results, semantic recognition results, etc.

Referring to fig. 11, a computer-readable storage medium 500 according to an embodiment of the present application is shown, wherein a program code is stored in the computer-readable storage medium, and the program code can be invoked by a processor to execute the method described in the foregoing method embodiment.

It should be noted that the computer readable storage medium 500 shown in the embodiments of the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium 500 may be, for example, a system, apparatus or device, including, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM), a flash Memory, an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium 500 includes a non-volatile computer-readable storage medium. The computer readable storage medium 500 has a storage unit for program code 510 for performing any of the method steps of the method described above. The program code can be read from or written to one or more computer program products. The program code 510 may be compressed, for example, in a suitable form.

Alternatively, computer readable storage medium 500 may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with a computer program embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. Each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present application may be implemented by software, or may be implemented by hardware, and the described units may also be disposed in a processor. Wherein the names of the elements do not in some way constitute a limitation on the elements themselves.

It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the application. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains.

It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not necessarily depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.