1. An ITOF ranging system, comprising: the system comprises a transmitter, a collector and a processing circuit; wherein the content of the first and second substances,

the transmitter configured to transmit a signal beam toward an object to be measured;

the collector is configured to collect the optical signal reflected by the measured object;

the processing circuit is connected with the emitter and the collector and used for acquiring the electric charge amount corresponding to the optical signal reflected back by the measured object, determining sampling signal data according to the electric charge amount and calculating the reflectivity of the measured object according to the sampling signal data.

2. A method of calculating reflectance of an object, comprising:

acquiring the electric charge amount corresponding to the optical signal reflected by the measured object, and determining sampling signal data according to the electric charge amount;

and calculating the reflectivity of the measured object according to the sampling signal data.

3. The method of claim 2, wherein calculating the reflectivity of the object from the sampled signal data comprises:

acquiring the tap exposure times, the illumination incident angle, the measurement distance of the measured object and the peak power of a light source emission light signal;

and calculating the reflectivity of the measured object according to the sampling signal data, the tap exposure times, the illumination incidence angle, the distance of the measured object, the peak power of the light signal emitted by the light source and a prestored reflectivity calculation rule.

4. The method of claim 3, wherein the pre-stored reflectance calculation rule is:

wherein, Re is the reflectivity of the measured object; c_sSampling the signal data; n is the exposure times of the tap in the integration time of single frame measurement; theta is the illumination incident angle; l is the measurement distance of the measured object; p_tIs the peak power of the laser; k is a radical of₁Is a first predetermined coefficient.

5. The method of claim 2, further comprising, after calculating the reflectance of the object from the sampled signal data:

determining ambient light data from the amount of charge;

calculating ambient light irradiance from the ambient light data and the reflectivity.

6. The method of claim 5, wherein the calculating ambient light irradiance from the ambient light data and the reflectivity comprises:

and calculating to obtain the ambient light irradiance according to the ambient light data, the sampling signal data, the lens focal length of the collector, the illumination incidence angle, the reflectivity and a pre-stored ambient light irradiance calculation rule.

7. An apparatus for calculating reflectance of an object, comprising:

the device comprises a first acquisition unit, a second acquisition unit and a control unit, wherein the first acquisition unit is used for acquiring the electric charge amount corresponding to an optical signal reflected by a measured object and determining sampling signal data according to the electric charge amount;

and the first calculating unit is used for calculating the reflectivity of the measured object according to the sampling signal data.

8. The apparatus for calculating reflectance of an object to be measured according to claim 7, further comprising:

a second acquisition unit configured to determine ambient light data from the amount of charge;

a second calculation unit for calculating ambient light irradiation from the ambient light data and the reflectivity.

9. The device for calculating reflectance of a measured object according to claim 7, wherein the first calculating unit is specifically configured to:

acquiring the tap exposure times, the illumination incident angle, the measurement distance of the measured object and the peak power of a light source emission light signal;

and calculating the reflectivity of the measured object according to the sampling signal data, the tap exposure times, the illumination incidence angle, the distance of the measured object, the peak power of the light signal emitted by the light source and a prestored reflectivity calculation rule.

10. An apparatus for calculating reflectance of an object, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of calculating reflectance of an object as claimed in any one of claims 2 to 6 when executing the computer program.

Background

ToF (Time-of-Flight) ranging is a technique for achieving accurate ranging by measuring the round-trip Time-of-Flight of light pulses between a transmitting/receiving device and a target object. The technique of directly measuring the optical time of flight in the ToF technique is called dtofs (direct-ToF); the measurement technique of periodically modulating the emitted light signal, measuring the phase delay of the reflected light signal relative to the emitted light signal, and calculating the time of flight from the phase delay is called the iToF (index-TOF) technique. According to the difference of the modulation and demodulation type, the modulation and demodulation method can be divided into a Continuous Wave (CW) modulation and demodulation method and a Pulse Modulated (PM) modulation and demodulation method.

When distance measurement is carried out, the time flight measurement system only can provide the time of light flight, further calculates the distance and restores the distance into 3D point cloud data according to the emission angle. The 3D point cloud data can only restore the three-dimensional information of the measured object, and can not acquire the information of other dimensions of the measured object.

Disclosure of Invention

In order to overcome the problems in the prior art, the embodiment of the invention provides an ITOF distance measuring system and a method for calculating the reflectivity of a measured object.

In order to achieve the above purpose, the technical solution of the embodiment of the present invention is realized as follows:

an ITOF ranging system comprising: the system comprises a transmitter, a collector and a processing circuit; wherein the content of the first and second substances,

the transmitter configured to transmit a signal beam toward an object to be measured;

the collector is configured to collect the optical signal reflected by the measured object;

the processing circuit is connected with the emitter and the collector and used for acquiring the electric charge amount corresponding to the optical signal reflected by the measured object, determining sampling signal data according to the electric charge amount and calculating the reflectivity of the measured object according to the sampling signal data.

The other technical scheme of the embodiment of the invention is as follows:

a method of calculating reflectance of an object, comprising:

acquiring the electric charge amount corresponding to the optical signal reflected by the measured object, and determining sampling signal data according to the electric charge amount;

and calculating the reflectivity of the measured object according to the sampling signal data.

Further, the calculating the reflectivity of the measured object according to the sampling signal data includes:

acquiring the tap exposure times, the illumination incident angle, the measurement distance of the measured object and the peak power of a light source emission light signal;

and calculating the reflectivity of the measured object according to the sampling signal data, the tap exposure times, the illumination incidence angle, the distance of the measured object, the peak power of the light signal emitted by the light source and a prestored reflectivity calculation rule.

Further, the pre-stored reflectivity calculation rule is:

wherein, Re is the reflectivity of the measured object; c_sSampling the signal data; n is the exposure times of the tap in the integration time of single frame measurement; theta is the illumination incident angle; l is a measurement distance; p_tIs the peak power of the laser; k is a radical of₁Is a first predetermined coefficient.

Further, after the calculating the reflectivity of the measured object according to the sampling signal data, the method further includes:

determining ambient light data from the amount of charge;

calculating ambient light irradiance from the ambient light data and the reflectivity.

Further, said calculating ambient light irradiance from said ambient light data and said reflectivity, comprises:

and calculating to obtain the ambient light irradiance according to the ambient light data, the sampling signal data, the lens focal length of the collector, the illumination incidence angle, the reflectivity and a pre-stored ambient light irradiance calculation rule.

The other technical scheme of the embodiment of the invention is as follows:

an apparatus for calculating reflectance of an object, comprising:

the device comprises a first acquisition unit, a second acquisition unit and a control unit, wherein the first acquisition unit is used for acquiring the electric charge amount corresponding to an optical signal reflected by a measured object and determining sampling signal data according to the electric charge amount;

and the first calculating unit is used for calculating the reflectivity of the measured object according to the sampling signal data.

Further, the apparatus for calculating the reflectivity of the measured object further includes:

a second acquisition unit configured to determine ambient light data from the amount of charge;

a second calculation unit for calculating an ambient light illuminance from the ambient light data and the reflectivity.

Further, the first calculating unit is specifically configured to:

acquiring the tap exposure times, the illumination incident angle, the measurement distance of the measured object and the peak power of a light source emission light signal;

and calculating the reflectivity of the measured object according to the sampling signal data, the tap exposure times, the illumination incidence angle, the distance of the measured object, the peak power of the light signal emitted by the light source and a prestored reflectivity calculation rule.

The embodiment of the invention adopts another technical scheme that:

an apparatus for calculating reflectance of an object to be measured, comprising a memory, a processor and a computer program stored in the memory and executable on the processor; wherein, the processor implements the method for calculating the reflectivity of the measured object according to any one of the above embodiments when executing the computer program.

Compared with the prior art, the embodiment of the invention determines the sampling signal data through the electric charge quantity, calculates the reflectivity of the measured object through the sampling signal data, provides the fourth-dimensional information of the measured object except for the 3D point cloud data, namely the reflectivity, realizes 4D measurement, and provides more comprehensive information to express the measured object.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

For a better understanding and practice, the invention is described in detail below with reference to the accompanying drawings.

Drawings

FIG. 1 is a schematic diagram illustrating an ITOF ranging system in accordance with an exemplary embodiment of the present invention;

FIG. 2 is a schematic flow chart illustrating a method for calculating reflectance of an object under test according to an exemplary embodiment of the invention;

FIG. 3 is a flowchart illustrating steps S103-S104 of a method for calculating reflectivity of an object according to an exemplary embodiment of the invention;

FIG. 4 is a schematic diagram of an apparatus for calculating reflectance of an object according to an exemplary embodiment of the present invention;

fig. 5 is a schematic diagram of an apparatus for calculating reflectance of an object to be measured according to an exemplary embodiment of the present invention.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present invention.

Referring to fig. 1, fig. 1 is a schematic diagram illustrating a distance measuring system according to an exemplary embodiment of the present invention, the distance measuring system including: the system comprises a transmitter, a collector and a processing circuit; wherein the content of the first and second substances,

a transmitter 11 configured to transmit a signal beam toward a measured object;

a collector 12 configured to collect the optical signal reflected by the measured object;

and the processing circuit 13 is connected with the emitter and the collector and is used for acquiring the electric charge amount corresponding to the optical signal reflected by the measured object, determining sampling signal data according to the electric charge amount and calculating the reflectivity of the measured object according to the sampling signal data.

Specifically, the emitter 11 is configured to emit a signal beam 30 to the object to be measured 20, and an optical signal 40 reflected by the object to be measured is received by the collector; emitter 11 and collector 12 may be disposed on a substrate, specifically, may be disposed on the same substrate, or may be disposed on different substrates.

Collector 12 includes an image sensor 121, a filter unit 123, and a receiving optical element 122; the receiving optical element 122 is used to image the spot beam reflected by the measured object onto the image sensor 121; the filter unit 123 is used to suppress background light noise in the remaining wavelength bands different from the wavelength of the light source; the image sensor 121 may be an image sensor array composed of a Charge Coupled Device (CCD), a Complementary Metal Oxide Semiconductor (CMOS), etc., and the size of the array represents the resolution of the depth camera, such as 320 × 240, etc.

Generally, the image sensor 121 comprises at least one pixel, each pixel comprising at least one tap (tap for storing and reading or discharging charge signals generated by incident photons under control of a corresponding electrode), such as 2 taps, which are sequentially switched in a certain order within a single frame period (or within a single exposure time) to collect corresponding light signals to receive the light signals and convert them into electrical signals, reading the charge signal data.

In an alternative embodiment, each pixel comprises at least one tap for storing and reading or draining the electrical signal generated by the incident photons under control of the corresponding electrode, the ambient light data and the sampled signal data being calculated from the amount of charge accumulated by the tap over the integration time.

In an alternative embodiment, each pixel comprises three taps, which are switched in sequence within a single frame period to acquire the respective light signals, and one of which is used to acquire the ambient light signal.

The processing circuit 13 may be a stand-alone dedicated circuit, such as a dedicated SOC chip, an FPGA chip, an ASIC chip, etc. including a CPU, a memory, a bus, etc., or may include a general-purpose processing circuit, such as when the TOF distance measuring system is integrated into an intelligent terminal, such as a mobile phone, a television, a computer, etc., and the processing circuit of the terminal may be at least a part of the processing circuit 13.

In an alternative embodiment, the processing circuit 13 is configured to provide a modulation signal (emission signal) required when the light source emits laser light, and the light source emits a pulse light beam to the object to be measured under the control of the modulation signal; further, the processing circuit 13 supplies a demodulation signal (acquisition signal) of a tap in each pixel of the image sensor 121, and the tap acquires the amount of electric charge generated by the optical signal reflected back by the object to be measured under the control of the demodulation signal.

The embodiment of the method for calculating the reflectivity of the object to be measured will be described in detail later, and the processing circuit 13 will calculate the reflectivity of the object to be measured and the ambient light illuminance according to the method.

Referring to fig. 2, fig. 2 is a schematic flowchart illustrating a method for calculating reflectivity of an object to be measured according to an exemplary embodiment of the present invention, where the method is performed by a device for calculating reflectivity of an object to be measured (hereinafter referred to as a device), and includes the following steps:

s101: and acquiring the electric charge amount corresponding to the optical signal reflected by the measured object, and determining the sampling signal data according to the electric charge amount.

In this implementation, the distance measuring system is an ITOF ranging system, the collector of which comprises an image sensor comprising a plurality of pixels, each pixel comprising at least one tap for storing and reading or discharging an electrical signal generated by an incident photon under control of a respective electrode, the sampled signal data being calculated from the amount of charge accumulated by the tap over an integration time.

The device determines sampled signal data from the amount of charge. In an alternative embodiment, each pixel comprises 3 taps, collects the light signal during the integration time and outputs the amount of charge a_1-3Wherein two taps are used to collect the reflected optical signal, the amount of charge A collected by the two taps₁、A₂Characterizing the collected sampled signal data.

In an optional embodiment, when each pixel includes a plurality of taps, the sine waveform of the reflected signal collected by the collector can be fitted according to the output charge amount of the plurality of taps, the device fits the charge amount to obtain a sine wave fitting curve corresponding to the optical signal, and the sampled signal data is determined according to the sine wave fitting curve. For example, the fitted sinusoid is: and y, a + b cost + c sint, and determining the amplitude and the direct current quantity according to the fitted curve, wherein the amplitude is used for representing the sampled signal data.

In an alternative embodiment, the amplitude of the sine wave fitted curve isThe sampled signal data is represented as

S102: and calculating the reflectivity of the measured object according to the sampling signal data.

The apparatus calculates the reflectivity of the object to be measured from the sampled signal data. The device stores reflectivity calculation rules in advance, namely, the corresponding relation between the sampling signal data and the reflectivity, and the reflectivity of the measured object is calculated according to the corresponding relation between the sampling signal data and the reflectivity.

The corresponding relation between the sampled signal data and the reflectivity can be derived according to the relational expression between the sampled signal data and the reflectivity. The data of the sampling signals collected by the collector is influenced by factors such as the exposure times of taps, the illumination incident angle, the measurement distance of the measured object, the peak power of the light source emission signal beam and the like besides the reflectivity of the measured object, so that the corresponding relation between the data of the sampling signals and the reflectivity when other factors are fixed is calibrated, and the calculation rule of the reflectivity is deduced. When the ITOF ranging system is used for ranging, the equipment can acquire information such as tap exposure times, illumination incidence angles, measuring distances of the measured object, peak power of signal beams emitted by the light source and the like, and calculate the reflectivity of the measured object according to a prestored reflectivity calculation rule.

In an alternative embodiment, the pre-stored reflectivity calculation rule may be:

wherein, Re is the reflectivity of the measured object; c_sSampling the signal data; n is the exposure times of the tap in the integration time of single frame measurement; theta is the illumination incident angle; l is the measuring distance of the measured object; p_tPeak power of the signal beam emitted for the light source; k is a radical of₁Is a first predetermined coefficient, is a constant determined according to the design of the system, and the constant k is a constant determined according to different system designs₁A change will occur.

It is to be understood that the correspondence between the sampled signal data and the reflectance is not limited to the above-described relational expression, and the above-described relational expression does not limit the correspondence between the sampled signal data and the reflectance.

In order to obtain the environmental information at the same time when the distance measurement is performed, the ambient light illuminance can be calculated by obtaining the ambient light data in the present embodiment. After step S102, steps S103 to S104 may be further included, and as shown in fig. 3, steps S103 to S104 are specifically as follows:

s103: determining ambient light data from the amount of charge.

The collector of the ITOF ranging system comprises a plurality of pixels, each pixel comprises at least one tap, and the ambient light data is calculated according to the electric charge amount corresponding to the electric signal accumulated by the tap in the integration time.

In an alternative embodiment, each pixel comprises 3 taps, collects the light signal during the integration time and outputs the amount of charge a_1-3Wherein two taps are used to collect the reflected optical signal, the amount of charge A collected by the two taps₁、A₂Characterizing the acquired sampled signal data, another tap A₃For collecting ambient light signal, the collected charge amount A is output by the tap₃Characterizing ambient light data.

In an optional embodiment, when each pixel comprises a plurality of taps, the sine waveform of the reflected signal collected by the collector can be fitted according to the output charge amount of the taps, and the device fits the charge amount to obtain a sine wave fitting curve corresponding to the optical signal; ambient light data is determined from the sine wave fitted curve. For example, the fitted sinusoid is: and y, a + b cost + c sint, and determining the amplitude and the direct current quantity according to the fitted curve, wherein the direct current quantity is used for representing the ambient light data.

In an alternative embodiment, the sine wave fitting curve has a DC component ofThe ambient light data is represented as

S104: calculating ambient light irradiance from the ambient light data and the reflectivity.

The apparatus calculates the ambient light illuminance from the ambient light data and the reflectivity and from calculation rules pre-stored in the apparatus for the ambient light illuminance.

Specifically, the ambient light illuminance may be calculated by the device according to the ambient light data, the sampling signal data, the focal length of the lens of the collector, the light incidence angle, the reflectivity, and the calculation rule of the ambient light illuminance prestored in the device. Specifically, the pre-stored calculation rule of the ambient light illuminance is:

wherein, C_sSampling the signal data; c_nIs ambient light data; theta is the illumination incident angle; l is the measuring distance of the measured object; f represents the focal length of the lens of the collector; k is a radical of₂Is a second predetermined coefficient, k₃Is a third predetermined coefficient, and the second predetermined coefficient and the third predetermined coefficient are constants determined according to the design of the system, and the constants will change for different system designs.

According to the embodiment of the invention, the sampling signal data is determined through the electric charge quantity, the reflectivity of the measured object is calculated through the sampling signal data, the fourth-dimensional information of the measured object except for the 3D point cloud data, namely the reflectivity, is provided, 4D measurement is realized, and more comprehensive information is provided to express the measured object.

Referring to fig. 4, fig. 4 is a schematic structural diagram of an apparatus for calculating reflectance of a measured object according to an exemplary embodiment of the present invention. The included units are used for executing steps in the embodiments corresponding to fig. 2 and fig. 3, and refer to the related description in the embodiments corresponding to fig. 2 and fig. 3. For convenience of explanation, only the portions related to the present embodiment are shown. Referring to fig. 4, the apparatus 4 for calculating the reflectance of the measured object includes:

a first obtaining unit 410, configured to obtain an electric charge amount corresponding to an optical signal reflected by an object to be measured, and determine sampling signal data according to the electric charge amount;

the first calculating unit 420 is configured to calculate the reflectivity of the measured object according to the sampling signal data.

Further, the apparatus 4 for calculating the reflectivity of the measured object further includes:

a second acquisition unit configured to determine ambient light data from the amount of charge;

a second calculation unit for calculating an ambient light illuminance from the ambient light data and the reflectivity.

Further, the first calculating unit 420 is specifically configured to:

acquiring the tap exposure times, the illumination incident angle, the measurement distance of the measured object and the peak power of a light source emission light signal;

and calculating the reflectivity of the measured object according to the sampling signal data, the tap exposure times, the illumination incidence angle, the distance of the measured object, the peak power of the light signal emitted by the light source and a prestored reflectivity calculation rule.

Further, the second calculating unit is specifically configured to:

and calculating to obtain the ambient light irradiance according to the ambient light data, the sampling signal data, the lens focal length of the collector, the illumination incidence angle, the reflectivity and a pre-stored ambient light irradiance calculation rule.

Referring to fig. 5, fig. 5 is a schematic diagram of an apparatus for calculating reflectance of an object to be measured according to an exemplary embodiment of the present invention. As shown in fig. 5, the apparatus 5 for calculating the reflectance of an object to be measured of this embodiment includes: a processor 50, a memory 51 and a computer program 52, such as a reflectivity determination program, stored in said memory 51 and executable on said processor 50. The processor 50, when executing the computer program 52, implements the steps in the above-described embodiments of the method for calculating the reflectivity of the measured object, such as the steps S101 to S102 shown in fig. 2. Alternatively, the processor 50, when executing the computer program 52, implements the functions of the modules/units in the above-mentioned device embodiments, such as the functions of the units 410 to 420 shown in fig. 4.

Illustratively, the computer program 52 may be partitioned into one or more modules/units that are stored in the memory 51 and executed by the processor 50 to implement the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions for describing the execution of the computer program 52 in the apparatus 5 for calculating reflectance of an object to be measured. For example, the computer program 52 may be divided into a first acquisition module, a first computation module, each module functioning as follows:

the first acquisition module is used for acquiring the electric charge amount corresponding to the optical signal reflected by the measured object and determining sampling signal data according to the electric charge amount;

and the first calculation module is used for calculating the reflectivity of the measured object according to the sampling signal data.

The device 5 for calculating the reflectivity of the measured object may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will appreciate that fig. 5 is merely an example of the device 5 for calculating reflectance of the measured object, and does not constitute a limitation of the device 5 for calculating reflectance of the measured object, and may include more or less components than those shown, or some components in combination, or different components, for example, the device 5 for calculating reflectance of the measured object may further include an input-output device, a network access device, a bus, etc.

The Processor 50 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 51 may be an internal storage unit of the device 5 for calculating the reflectivity of the measured object, such as a hard disk or a memory of the device 5 for calculating the reflectivity of the measured object. The memory 51 may also be an external storage device of the device 5 for calculating the reflectivity of the measured object, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the device 5 for calculating the reflectivity of the measured object. Further, the memory 51 may also include both an internal storage unit of the device 5 for calculating the reflectance of the measured object and an external storage device. The memory 51 is used for storing the computer program and other programs and data required by the apparatus for calculating the reflectance of the measured object. The memory 51 may also be used to temporarily store data that has been output or is to be output.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice. The present invention is not limited to the above-described embodiments, and various modifications and variations of the present invention are intended to be included within the scope of the claims and the equivalent technology of the present invention if they do not depart from the spirit and scope of the present invention.