1. A method of recording a user's point of interest, performed on an in-vehicle host, comprising:

determining whether the audio collected by the vehicle-mounted host contains one of a plurality of preset trigger words;

in response to the detected one of the plurality of trigger words, determining whether the one of the plurality of trigger words is a second trigger word;

and if the second trigger word is the second trigger word, generating a keyword for recording the user attention point, wherein the second keyword is a keyword related to the preset user attention point.

2. The method of claim 1, wherein the step of detecting whether the audio collected by the on-board host includes one of a plurality of preset trigger words is performed after the on-board host system is loaded and the microphone authority is obtained.

3. The method of claim 1 or 2, further comprising:

in response to the detected one of the plurality of trigger words, judging whether the one of the plurality of trigger words is a first trigger word;

and if the first trigger word is the first trigger word, starting voice interaction, wherein the first awakening word is a preset voice interaction awakening word.

4. The method of claim 3, wherein the second wake-up word and the first wake-up word are non-near words.

5. The method of claim 1 or 2, further comprising:

and if the network is connected, sending the trigger record to a cloud storage.

6. The method of claim 5, wherein the step of generating a trigger recording for recording the user's point of interest and not initiating a voice interaction further comprises at least one of:

recording the content of the detected second trigger word;

recording the time when the second trigger word is detected;

recording the GPS position when the second trigger word is detected; or

Recording vehicle specification data when the second trigger word is detected; the vehicle specification data includes at least one of:

the vehicle-mounted host type, user account information, navigation application data, entertainment application data, driving state data and vehicle body sensor data.

7. The method of claim 1 or 2, further comprising:

in response to the detected one of the plurality of trigger words, determining whether the one of the plurality of trigger words is a third trigger word;

and if the third trigger word is present, responding to and executing the corresponding operation instruction.

8. The method of claim 1, further comprising obtaining new keyword content from a cloud for iterative updating of a local second trigger word lexicon constructed from at least one second trigger word.

9. An in-vehicle host, comprising:

at least one processing unit; and

a memory coupled with the at least one processing unit, the memory containing instructions stored therein, which when executed by the at least one processing unit, cause the apparatus to perform the steps of the method of any of claims 1 to 8.

10. A vehicle equipped with the in-vehicle main unit according to claim 9.

Background

In the internet of vehicles, in order to better serve users, the users are generally required to be imaged, so that corresponding contents required by the users can be provided more accurately.

The traditional user portrait can only obtain the registration information of the user when the vehicle-mounted host registers the user and part of application data when the user uses the vehicle-mounted host, which causes the problem that the user portrait is not accurate enough.

On the other hand, the user is required to actively input data many times, which brings redundant operations, excessive man-machine interaction steps and very time waste.

Therefore, it is important to provide a method for collecting user attention content and portraying the user, which is accurate and reduces user intervention.

Disclosure of Invention

The embodiment of the invention provides a method for monitoring trigger words and acquiring data of interest points. Based on the mode of the invention, the user attention point can be recorded off line in a simple and efficient mode, so that the user portrait can be accurately depicted.

In a first aspect of the present invention, a method for recording a user's attention point is provided, which is performed at a local end, and includes: detecting whether locally acquired audio contains preset trigger words or not; responding to the detected trigger word, and judging whether the trigger word is a second trigger word; if the second trigger word is present, a trigger record is generated and voice interaction is not started.

In a second aspect of the present invention, there is provided an in-vehicle host machine including: at least one processing unit; and a memory coupled to the at least one processing unit, the memory containing instructions stored therein, which when executed by the at least one processing unit, cause the apparatus to perform the steps of the method according to the first aspect.

In a third aspect of the present invention, there is provided a vehicle on which the in-vehicle host machine realized according to the second aspect is mounted.

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the invention, nor is it intended to be used to limit the scope of the invention.

Drawings

The above and other objects, features and advantages of the present invention will become more apparent by describing in more detail exemplary embodiments thereof with reference to the attached drawings, in which like reference numerals generally represent like parts.

FIG. 1 shows a schematic diagram of a scenario in which an embodiment of the invention can be implemented;

FIG. 2 shows a flow diagram of a method according to an embodiment of the invention;

FIG. 3 illustrates a schematic diagram of a plurality of trigger records generated according to one embodiment of the present invention;

FIG. 4 illustrates a diagram of multiple second trigger attributes associating different applications, according to one embodiment of the invention;

FIG. 5 shows a schematic block diagram of a device capable of implementing one embodiment of the present invention.

Detailed Description

The principles of the present invention will be described below with reference to a number of exemplary embodiments shown in the drawings. While the preferred embodiments of the invention have been illustrated in the accompanying drawings, it is to be understood that these embodiments are described merely for the purpose of enabling those skilled in the art to better understand the invention for practicing the invention, and are not intended to limit the scope of the invention in any way.

The term "include" and variations thereof as used herein is meant to be inclusive in an open-ended manner, i.e., "including but not limited to". Unless specifically stated otherwise, the term "or" means "and/or". The term "based on" means "based at least in part on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first," "second," and the like may refer to different or the same object. Other explicit and implicit definitions are also possible below.

As mentioned above, the user data available from the conventional user portrait has great limitations, and many manufacturers may conduct product-specific research to obtain accurate data, which results in high labor cost, and needs deep participation of users, and does not benefit the needs of large-scale popularization and research, which is not good for efficient and low-cost user portrait.

To at least partially solve one or more of the above problems and other potential problems, an embodiment of the present invention provides a method for recording a user's attention point, which can be implemented by monitoring a trigger word, and specifically, executed on a local side, including: detecting whether locally acquired audio contains preset trigger words or not; responding to the detected trigger word, and judging whether the trigger word is a second trigger word; if the trigger word is the second trigger word, generating the trigger record for recording the user attention point does not perform other operations, that is, only the local background generates the trigger record, and the local front-end user is unaware because no operation perceivable by other users is performed, specifically, generating the trigger record for recording the user attention point records the detected specific trigger word, where the specific trigger word is the second trigger word, and hereinafter, the second trigger word will be used to describe the case where the acquired trigger word is the second trigger word, and those skilled in the art can take one action and another action, and such description is to reduce the confusion of stuttering voice, and is intended to describe the key point of the present invention: responding to the detected trigger word, and judging whether the trigger word is a second trigger word; and if the word is the second trigger word, generating a trigger record for recording the attention point of the user and not executing other operations. In this way, a manufacturer only needs to sort and preset keywords in the key information which needs to be investigated in advance as second trigger words, so that the local trigger is utilized to record the times related to the keywords needed by the manufacturer in the voice of the user, and the data information of the point of interest of the user is obtained.

Specifically, a trigger record generated may be stored locally. In yet another embodiment, the data can be directly uploaded to the cloud.

Those skilled in the art understand that since the background noise of the environment in the vehicle is relatively low, the application effect of the voice interaction can be maximally exhibited here, and therefore, the voice triggering and then starting the voice interaction in the vehicle is a commonly-used man-machine interaction mode. The method and the device utilize the function of triggering word monitoring, particularly utilize the function of monitoring triggering words executed at a local end without a cloud end, thereby realizing the first-time response and quickly finishing the collection work. In other words, the second trigger word is a preset keyword for recording the number of triggered times.

In a preferred embodiment, after the local terminal loads the system, the local terminal obtains the microphone permission, and starts the step of detecting whether the locally acquired audio contains a preset trigger word. The purpose of this is to be able to record the number of triggers all the way through, and thus obtain more accurate data information.

In yet another embodiment, in carrying out the present invention, further comprising: responding to the detected trigger word, and judging whether the trigger word is a first trigger word; and if the first trigger word is present, starting voice interaction. In this embodiment, the first trigger word is a preset voice interaction trigger word. Those skilled in the art understand that, since the first trigger word plays a role in starting the voice interaction portal, manufacturers have a great drive to improve the trigger accuracy of the first trigger word, and therefore, in some cases, when the first trigger word is set, the sensitivity of the voice recognition system to the first trigger word is often properly improved.

Therefore, in a preferred embodiment, in order to avoid distortion of the collected data, the pronunciation approximation problem between the second trigger word and the first trigger word should be avoided. For example, the set second trigger word is close to the first trigger word, so that the voice recognition system recognizes the voice content actually acquired by the microphone as the first trigger word, and the triggering times of the second trigger word acquired according to the method of the present invention are less than the times of the second trigger word mentioned by the voice of the actual user. Such nearing words may include, for example: the front nasal sound and the rear nasal sound are different in tone, and the specific conditions include that the first trigger word "hello click", the second trigger word "hello hong kong", and the judgment and selection of the nearsighted word are the prior art, and are not described herein again. Considering that the setting of the first trigger word is often bound with the brand and the model of the device, that is, a specific brand or model product is often corresponding to a specific first trigger word, in order to avoid the problem of pronunciation approximation between the second trigger word and the first trigger word, the first trigger word needs to be locked and the second trigger word needs to be adjusted.

Further, the first trigger word needs to be reset or even masked, and those skilled in the art can determine the trigger word according to the needs, which is not described herein.

In another embodiment, the trigger word of the present invention further includes a third trigger word, and the third trigger word may be a preset plurality of keyword phrases corresponding to different operation instructions, including: the voice passwords for directly executing the operation instructions, such as 'window opening', 'flameout', 'air conditioner opening' and the like, are also the third trigger word based on the invention, voice interaction is not required to be started in advance, but the instructions are directly and basically preset locally, and the local end of the vehicle-mounted host computer directly responds to the execution operation based on the collected third trigger word, so that the user experience is fast.

Having initially described the general context of the present invention, in order to better explain the essence of the invention, a number of different embodiments of the invention will now be described in detail with reference to fig. 1-5.

Referring initially to FIG. 1, FIG. 1 illustrates a schematic diagram of a scenario 100 in which an embodiment of the present invention can be implemented. The interior scene 100 of the vehicle 110 includes an on-board host 112, a cloud 140, and users 130 and 190, which include a first trigger 120 and a second trigger 180 in their daily utterances.

In fig. 1, the on-board host 112 may receive information from the cloud 140 and may upload local information data to the cloud 140. Such information may be, for example, information related to the second trigger 180. In fig. 1, there may be a conversation between users 130 and 190, or users 130 or 190 may speak alone, and although the scenario 100 in fig. 1 is particularly suitable for taking place in the car cabin, it should not be understood that the speech source is limited to the car users 130 and 190, and considering that the car is a closed environment, and has the advantage of low background noise, the scenario applied in the present invention can obtain better sound reception effect. At the same time, however, the first trigger word 120 and the second trigger word 180 may also be included in an off-board speech source (not shown) via a microphone (not shown), for example, the speech content of the off-board user communicating with the user 130 via telephone is transmitted to the inside of the vehicle via a microphone, and those skilled in the art understand that the speech from the user 130 and the speech from the microphone are equivalent for a speech recognition system without voiceprint recognition.

As shown, the on-board host 112 is connected to the cloud 140 via a network, and the cloud 140 should not be regarded as one or more specific devices, but may be understood as a server and/or a server group located at a remote location and providing background computing capability, and may also be a program running in a logical container in a certain physical server, at this time, the on-board host 112 may obtain the second trigger from the cloud 140, and those skilled in the art understand that the on-board host 112 may also be configured to obtain the second trigger from other terminals (not shown): the information is directly obtained by devices such as a mobile phone, a PAD and the like through near field communication: such as a second trigger 180. In some embodiments, a sensor group (not shown) for collecting status signals associated with vehicle 110 may also be included in vehicle 110.

The first trigger word 120 is configured as a voice interaction trigger word, and those skilled in the art understand that the first trigger word 120 and the second trigger word 180 are non-near-sound words, because the setting of the first trigger word 120 is the prior art, and the process executed after the method of the present invention monitors the first trigger word 120 is also the prior art, further details regarding the setting of the first trigger word 120 and the execution process after monitoring are omitted here.

The main process flow of the present invention is described below with reference to fig. 1 and 2. Therein, FIG. 2 shows a flow diagram of a method 200 according to an embodiment of the invention. Method 200 is performed by in-vehicle host 112 of FIG. 1, and the following description of method 200 should not be construed as limiting the invention, as method 200 may include additional acts not shown and/or may omit acts shown. In the present embodiment, generating a record specifically employs recording specific contents in a point of interest data table.

In an embodiment, step 201 is first executed to locally monitor the trigger word, and those skilled in the art understand that monitoring not only monitors the trigger word described in the present invention, but also monitors all audio input contents, and matches the corresponding audio input contents according to a locally preset trigger word rule. The local monitoring in this step is realized by using the technology of off-line monitoring the local wakeup word in the prior art, without uploading the audio content to the cloud for processing, and directly comparing locally.

Then, according to the monitored content, the following different steps are executed, wherein if the first trigger word 120 is monitored, the step 202 is executed, and voice interaction is started; if a second trigger word is monitored, judging whether one of the trigger words is the second trigger word; if the word is the second trigger word, step 203 is executed, a trigger record for recording the user attention point is generated, and no other operation is executed. Specifically, the monitoring of the first trigger word 120 and the execution of the step 202 to start the voice interaction, and the subsequent voice interaction process are widely implemented in the prior art and are not the key points of the present invention, and therefore, are not described herein again. However, fig. 2 is a schematic diagram and does not show the logical relationship after step 202, that is, after step 202, in the process of performing voice interaction according to the present invention, step 201 is further executed to monitor the trigger word, so that the acquisition function according to the present invention is not interrupted by the voice interaction process. Those skilled in the art will appreciate that this maximizes the accuracy of the data collected by the present invention.

In contrast, in step 201, if the second trigger word 180 is monitored, step 203 is performed, and the data is recorded in a silent manner into the locally created point of interest data table. Since the second trigger word 180 is not set to be used as a password for initiating voice interaction, and the first trigger word 120 also often has a requirement for separate record storage times, different trigger words need to be isolated, so that the permission of the user to modify the first trigger word 120 needs to be limited to avoid confusion in the use process of the user. Preferably, the setting authority of the first trigger word 120 is implemented only by a preset, and in yet another embodiment, the setting of the first trigger word 120 may be modified by a subsequent finer iteration. Further, in a preferred embodiment, the second trigger word 180 may be added by the user, and the user may freely add the required second trigger word 180 by analyzing the word frequency of the environment in the vehicle, and a person skilled in the art may determine the optional word pool of the second trigger word 180 as needed to avoid the second trigger word 180 set by the user colliding with the first trigger word 120. By strictly distinguishing the second trigger word 180 from the first trigger word 120, the present invention can ensure one of the advantages of the present invention, namely timeliness, only by using the local monitoring module.

In the process of implementing the invention, in order to maximally collect data that can be collected in the vehicle, the method of the invention may acquire the microphone permission and start monitoring when the on-board host 112 loads the system. Considering that the power consumption of the on-board host 112 is very small in the vehicle 110, the start-up monitoring is acceptable, and the background operation monitoring does not affect the operation smoothness of the on-board host 112 due to the improvement of the performance of the current processing unit. Of course, in another embodiment, the listening module may be activated according to instructions from a user or other devices.

In a preferred embodiment, the first trigger word 120 is a voice interaction trigger word preset by a manufacturer, such as "HEY, SIRI", "smallness", "hello-click", "love classmate", and the like, which are common in the prior art, and it should be noted that the setting of the first trigger word 120 should avoid common phrases in life, and considering that to avoid frequent false triggering and the real triggering requirement of the user, the phrase length of the voice trigger word should be moderate, a large amount of research and user feedback display, and two to four short-tone phrases are suitable.

Further, the second trigger word 180 is a preset user focus keyword, and specifically, the second trigger word 180 is used for assuming that the user focus is predicted, and the collection of different focus keywords is realized by presetting a plurality of different keyword contents of the second trigger word 180.

In a specific embodiment, after step 203 is executed, the method further includes uploading the point of interest data table to the cloud 140 in time for calling according to a network connection condition, and an API interface may be generally opened to obtain data for a third-party analysis platform, so as to implement an information analysis function. In another embodiment, the generated records are directly sent to the cloud, and the generated records are only cached locally and do not have local storage, so that the method is more efficient. The data acquired by the method is undoubtedly more real and effective and has higher pertinence.

The setting and acquisition of the second trigger word 180 are described below with reference to fig. 3 and 4. Specifically, fig. 3 is a schematic diagram of a plurality of records generated according to an embodiment of the present invention, and more specifically, the records may be recorded in the form of a form by a person skilled in the art, where the form includes at least a listening item, and the records listen to the content of the second trigger word; a time item, recording the time when the second trigger word is heard; a location item recording a real-time GPS location; and the vehicle specification data item records real-time vehicle specification data. In the example in fig. 3, the phrase "doctor", "tumor", "hospital", "appointment", "operation", "rehabilitation" is intercepted and set as the recorded segment of the second trigger word 180, and referring to the table, the position information is not successfully acquired, and the position may be the underground and other non-information positions, so that the invention can record the acquired information in the non-information area, and the time monitored by each keyword is recorded according to the specified format, and the total of 7 times of the second trigger word 180 contents is acquired in the time period from 03 minutes to 07 minutes in 10 am on 1 month 2 day 1 month 2 year 2020, and in the embodiment in fig. 3, the model of the vehicle-mounted host 112 is recorded as ACV-9, the user information is in the unregistered state, the XX hospital of the navigation destination, and as can be known by combining fig. 2 and fig. 3, the content of the monitored item is preset manufacturer, and the time item, the position item, the contents of the vehicle specification data items are information that is automatically generated by the in-vehicle host 112 according to the execution of the program. Moreover, the second trigger words 180 in the embodiment of fig. 3 are all double-word phrases, and it is seen that different from the first trigger word 120, because only silent recording is performed on the second trigger word 180, a shorter entry length can be completely adopted, and there is no fear that frequent triggering caused by too short triggering is likely to occur like the first trigger word 120, and the possibility that excessive monitoring may occur due to too short entry length of the second trigger word 180 is acceptable to some extent.

The example in fig. 3 is a preferred embodiment, and in different implementation cases, the step of generating a trigger record for recording the user's attention point and not initiating the voice interaction may include one of the following steps: recording content for indicating the content of the detected second trigger word; recording time for indicating the time of the detected second trigger word; recording the position, which is used for indicating the GPS position when the second trigger word is detected; or recording vehicle specification data for indicating the vehicle specification data when the second trigger word is detected; the vehicle specification data includes at least one of: the vehicle-mounted host type, user account information, navigation application data, entertainment application data, driving state data and vehicle body sensor data. The particular recording step or steps to be performed may be adjusted according to actual requirements.

Through the records generated by the present invention, all the monitoring records within a period of time are derived to obtain the triggering frequency of each different second trigger 180, for example, within 4 minutes shown in fig. 3, the second trigger 180 of "operation" is triggered by 2 words, and the other 6 second triggers 180 are triggered by 1 time. Those skilled in the art will appreciate that fig. 3 depicts the partial records to the public by way of example only, and in the actual case, a larger number of partial second trigger words 180 may be found as a result of a larger number of data records. The vehicle specification data shown in the embodiment of fig. 3 includes three data items of the in-vehicle host 112 model, the user account information, and the navigation application data. Generally, with the user account information data, the portrait can be better analyzed for a specific user, so that the portrait is separated from the specific vehicle-mounted host 112 and is bound with the user identity, and the subsequent information pushing according to the user portrait can depend on the user account information itself rather than a certain vehicle-mounted host specifically limited to. In a preferred embodiment, user account information, the model number of the vehicle host 112, and a unique activation authentication code of the vehicle host 112 may be included, thereby achieving more accurate identity confirmation.

The different effects of the different settings of the second trigger 180 are described below with reference to fig. 4. In one embodiment, the second trigger 180 selects a keyword having a higher relevance to the in-vehicle scene 100, such as the words "refuel", "eat", "hospital", "movie", etc., which are strongly relevant to the in-vehicle scene 100, and the attribute classification of these keywords includes interest point, car owner consumption, and travel service. When the second trigger word 180 is preset, a manager can select and determine the corresponding keyword attribute of the keyword content of the second trigger word 180 at the same time, and corresponding application mapping relationships are set between the selectable keyword attribute and the application program in the vehicle-mounted host, and those skilled in the art understand that when the strongly related keyword similar to the in-vehicle scene 100 is set as the second trigger word 180, the record items can be appropriately reduced, for example, the vehicle specification data item in the record only contains vehicle body sensor data, but does not contain navigation application data, entertainment application data, driving state data, and the like, so as to compress similar substantial redundant data. The embodiment shown in fig. 4 exemplifies a single mapping relationship, and those skilled in the art may adopt one-to-many or many-to-many attribute-application relationships according to the actual requirement of data acquisition accuracy, which is not described herein.

In another embodiment, if the local storage resources are rich, the navigation application data, the entertainment application data, the driving state data and the like can be recorded at the same time to be used as information secondary confirmation to form mutual evidence and ensure the accuracy of the data. Those skilled in the art can adopt different schemes according to actual needs.

Of course, in many cases, the second trigger 180 is often completely unrelated to the in-vehicle scene 100, such as "make public welfare", "donation", "blood type", "brain", and so on, in which case, if more vehicle specification data items can be recorded together, the complementarity of the data can be better perfected in breadth, and the scene when the keyword is triggered is depicted by using the content of the second trigger 180 and the corresponding multiple vehicle specification data items. In one embodiment, the vehicle specification data items may include a plurality of vehicle body sensor data, such as seat status, door status, temperature and air pressure data, etc. it is understood by those skilled in the art that the vehicle host model, user account information, navigation application data, entertainment application data, driving status data, and vehicle body sensor data exemplified in the present invention are only used as an example reference, and other different vehicle specification data may be recorded together according to specific actual requirements, which is not limited herein.

In a further preferred embodiment, the on-board host 112 further performs the step of replacing the second trigger word 180, specifically, in the process of communicating with the cloud 140, obtaining new keyword content from the cloud to perform iterative update on the local second trigger word. Usually, tens, hundreds or even more second keywords 180 may be set locally according to the requirement of recording the user focus, when the collection requirement changes, at least one second trigger word 180 required may be uploaded to the cloud end in a background management system (not shown in the present invention), and when the vehicle-mounted host 112 accesses the cloud end 140 or the cloud end 140 actively issues data to the vehicle-mounted host 112, the at least one second trigger word 180 in the cloud end 140 may be transmitted to the vehicle-mounted host 112 for replacing or updating the second trigger word 180 locally in the vehicle-mounted host 112.

With continued reference to FIG. 5, FIG. 5 illustrates a schematic block diagram of a device 500 capable of implementing embodiments of the present invention. For example, the in-vehicle host 112 as shown in FIG. 1 may be implemented by the device 500. As shown, device 500 includes a Central Processing Unit (CPU)501 that may perform various appropriate actions and processes in accordance with computer program instructions stored in a Read Only Memory (ROM)502 or loaded from a storage unit 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The CPU 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.

A number of components in the device 500 are connected to the I/O interface 505, including: an input unit 506 such as a microphone or the like; an output unit 507 such as various types of displays, speakers, and the like; a storage unit 508 such as a hard disk or the like; and a communication unit 509 such as a network card, modem, wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.

The various processes and processes described above, such as method 200, may be performed by processing unit 501. For example, in some embodiments, the method 200 may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 500 via the ROM 502 and/or the communication unit 509. When the computer program is loaded into RAM 503 and executed by CPU 501, one or more of the acts of method 200 described above may be performed.

The present invention may be methods, apparatus, systems and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied therein for carrying out aspects of the present invention.

The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present invention may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present invention are implemented by personalizing an electronic circuit, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA), with state information of computer-readable program instructions, which can execute the computer-readable program instructions.

Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processing unit of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Having described embodiments of the present invention, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.