1. An audio repair method, comprising:

acquiring an audio frame signal to be processed;

detecting a clipping interval existing in the audio frame signal; the wave interception interval comprises a plurality of continuous wave interception points, and the wave interception points are signal points with amplitude values exceeding a preset threshold value;

judging whether the detected chopping interval meets a specified interpolation condition or not according to a non-chopping point adjacent to the chopping interval;

and if so, repairing the chopping interval by adopting a specified interpolation method based on the non-chopping point.

2. The method according to claim 1, wherein the step of obtaining the audio frame signal to be processed comprises:

performing framing processing on audio to be processed to obtain an initial frame signal;

converting the initial frame signal into a specified data type if the initial data type of the initial frame signal is not the specified data type;

and taking the converted initial frame signal as an audio frame signal to be processed.

3. The method of claim 2, further comprising:

and converting the data type of the audio frame signal after the repair processing into the original data type.

4. The method according to claim 1, wherein the specified interpolation condition includes a parabolic interpolation condition, and the step of determining whether the clipping interval satisfies the specified interpolation condition according to a non-clipping point adjacent to the clipping interval includes:

detecting whether at least two non-clipping points exist before a first clipping point of the clipping interval and without crossing other clipping intervals; and detecting whether there are at least two non-clipping points after the last clipping point of the clipping interval and without crossing other clipping intervals;

and if the detection results are yes, determining that the chopping interval meets the parabolic interpolation condition.

5. The method of claim 4, wherein the specified interpolation method comprises a parabolic interpolation method, and wherein the step of repairing the clipping region using the specified interpolation method based on the non-clipping point comprises:

calculating the absolute value of a first slope between two non-chopping points which are nearest to the first chopping point of the chopping interval and the absolute value of a second slope between two non-chopping points which are nearest to the last chopping point of the chopping interval;

if the absolute value of the first slope is smaller than or equal to the absolute value of the second slope, selecting two non-chopping points which are nearest to the first chopping point of the chopping interval and one non-chopping point which is nearest to the last chopping point of the chopping interval to perform parabolic interpolation calculation, and repairing the chopping interval based on the obtained first interpolation calculation result;

and if the absolute value of the first slope is larger than the absolute value of the second slope, selecting two non-chopping points which are nearest to the last chopping point of the chopping interval and one non-chopping point which is nearest to the first chopping point of the chopping interval to perform parabolic interpolation calculation, and repairing the chopping interval based on the obtained second interpolation calculation result.

6. The method according to any one of claims 1 to 5, further comprising:

and smoothing the repaired wave interception interval, a first specified length interval positioned before the first wave interception point of the wave interception interval and a second specified length interval positioned after the last wave interception point of the wave interception interval by adopting a preset smoothing filtering algorithm.

7. The method of claim 1, further comprising:

and performing dynamic range compression processing on the repaired audio frame signal so as to enable the amplitude range of the repaired audio frame signal to be located in a specified amplitude range interval.

8. The method according to claim 7, wherein the step of performing dynamic range compression processing on the repaired audio frame signal comprises:

converting the repaired audio frame signal into a logarithmic domain for representation through a logarithmic table look-up method;

and carrying out dynamic range compression processing on the converted audio frame signal.

9. The method of claim 1, further comprising:

and if the chopping interval does not meet the specified interpolation condition, performing dynamic range compression processing on the audio frame signal so as to enable the amplitude ranges of the audio frame signal to be located in the specified amplitude range interval.

10. An audio restoration device, comprising:

the signal acquisition module is used for acquiring an audio frame signal to be processed;

the clipping detection module is used for detecting clipping intervals existing in the audio frame signals; the wave interception interval comprises a plurality of continuous wave interception points, and the wave interception points are signal points with amplitude values exceeding a preset threshold value;

the judging module is used for judging whether the detected chopping interval meets a specified interpolation condition or not according to a non-chopping point adjacent to the chopping interval;

and the wave interception repairing module is used for repairing the wave interception interval by adopting a specified interpolation method based on the non-wave interception point when the judgment result of the judging module is yes.

11. An electronic device, characterized in that the electronic device comprises:

a processor;

a memory for storing the processor-executable instructions;

the processor is configured to read the executable instructions from the memory and execute the instructions to implement the audio repair method of any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that the storage medium stores a computer program for executing the audio restoration method of any of the preceding claims 1-9.

Background

Audio transmission is involved in scenes such as audio and video conferences, live webcasts, network courses and the like, but audio interception (framing) may occur due to various reasons such as audio equipment, recording modes, recording occasions and the like, so that audio distortion is caused. For example, when the microphone device used by the user has a high gain and the volume setting of the playback device is too high, the magnitudes of the near-end speech signal and the far-end echo signal collected by the microphone are likely to exceed the threshold value, and a clipping phenomenon of the audio waveform occurs, that is, the waveform portion exceeding the threshold value is clipped.

In the related art, a relatively complex mode is mostly adopted for audio restoration, such as a sparse synthesis and sparse analysis model method, a least mean square autoregressive model method, a minimized high-order derivative energy method and the like, but the mode has a very large calculation amount and long restoration time, and is difficult to achieve the rapid and real-time audio restoration effect required by scenes such as audio and video conferences, live webcasts, network courses and the like.

Disclosure of Invention

To solve the above technical problem or at least partially solve the above technical problem, the present disclosure provides an audio repair method, apparatus, device, and medium.

The embodiment of the present disclosure provides an audio repairing method, which includes: acquiring an audio frame signal to be processed; detecting a clipping interval existing in the audio frame signal; the wave interception interval comprises a plurality of continuous wave interception points, and the wave interception points are signal points with amplitude values exceeding a preset threshold value; judging whether the detected chopping interval meets a specified interpolation condition or not according to a non-chopping point adjacent to the chopping interval; and if so, repairing the chopping interval by adopting a specified interpolation method based on the non-chopping point.

Optionally, the step of acquiring the audio frame signal to be processed includes: performing framing processing on audio to be processed to obtain an initial frame signal; converting the initial frame signal into a specified data type if the initial data type of the initial frame signal is not the specified data type; and taking the converted initial frame signal as an audio frame signal to be processed.

Optionally, the method further includes: and converting the data type of the audio frame signal after the repair processing into the original data type.

Optionally, the step of determining whether the clipping interval meets the specified interpolation condition according to a non-clipping point adjacent to the clipping interval includes: detecting whether at least two non-clipping points exist before a first clipping point of the clipping interval and without crossing other clipping intervals; and detecting whether there are at least two non-clipping points after the last clipping point of the clipping interval and without crossing other clipping intervals; and if the detection results are yes, determining that the chopping interval meets the condition of parabolic interpolation.

Optionally, the step of repairing the clipping interval by using a parabolic interpolation method based on the non-clipping point includes: calculating the absolute value of a first slope between two non-chopping points which are nearest to the first chopping point of the chopping interval and the absolute value of a second slope between two non-chopping points which are nearest to the last chopping point of the chopping interval; if the absolute value of the first slope is smaller than or equal to the absolute value of the second slope, selecting two non-chopping points which are nearest to the first chopping point of the chopping interval and one non-chopping point which is nearest to the last chopping point of the chopping interval to perform parabolic interpolation calculation, and repairing the chopping interval based on the obtained first interpolation calculation result; and if the absolute value of the first slope is larger than the absolute value of the second slope, selecting two non-chopping points which are nearest to the last chopping point of the chopping interval and one non-chopping point which is nearest to the first chopping point of the chopping interval to perform parabolic interpolation calculation, and repairing the chopping interval based on the obtained second interpolation calculation result.

Optionally, the method further includes: and smoothing the repaired wave interception interval, a first specified length interval positioned before the first wave interception point of the wave interception interval and a second specified length interval positioned after the last wave interception point of the wave interception interval by adopting a preset smoothing filtering algorithm.

Optionally, the method further includes: and performing dynamic range compression processing on the repaired audio frame signal so as to enable the amplitude range of the repaired audio frame signal to be located in a specified amplitude range interval.

Optionally, the step of performing dynamic range compression processing on the repaired audio frame signal includes: converting the repaired audio frame signal into a logarithmic domain for representation through a logarithmic table look-up method; and carrying out dynamic range compression processing on the converted audio frame signal.

Optionally, the method further includes: and if the chopping interval does not meet the specified interpolation condition, performing dynamic range compression processing on the audio frame signal so as to enable the amplitude ranges of the audio frame signal to be located in the specified amplitude range interval.

An embodiment of the present disclosure further provides an audio repair apparatus, including: the signal acquisition module is used for acquiring an audio frame signal to be processed; the clipping detection module is used for detecting clipping intervals existing in the audio frame signals; the wave interception interval comprises a plurality of continuous wave interception points, and the wave interception points are signal points with amplitude values exceeding a preset threshold value; the judging module is used for judging whether the detected chopping interval meets a specified interpolation condition or not according to a non-chopping point adjacent to the chopping interval; and the wave interception repairing module is used for repairing the wave interception interval by adopting a specified interpolation method based on the non-wave interception point when the judgment result of the judging module is yes.

An embodiment of the present disclosure further provides an electronic device, which includes: a processor; a memory for storing the processor-executable instructions; the processor is used for reading the executable instructions from the memory and executing the instructions to realize the audio repairing method provided by the embodiment of the disclosure.

The embodiment of the disclosure also provides a computer-readable storage medium, which stores a computer program for executing the audio repairing method provided by the embodiment of the disclosure.

In the technical scheme provided by the embodiment of the disclosure, whether the chopping interval meets the specified interpolation condition can be judged according to the non-chopping point adjacent to the chopping interval in the detected audio signal, and when the condition is met, the chopping interval is repaired by adopting the specified interpolation method based on the non-chopping point. According to the method provided by the embodiment of the disclosure, interpolation restoration is performed on the chopping interval by using the non-chopping point adjacent to the chopping interval, and the restoration time can be greatly shortened due to the low calculation amount of the interpolation method, so that the quick and real-time audio restoration effect can be achieved.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.

In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present disclosure, the drawings used in the description of the embodiments or prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.

Fig. 1 is a schematic flowchart of an audio repairing method according to an embodiment of the present disclosure;

fig. 2 is a schematic flow chart of another audio repair method provided in the embodiment of the present disclosure;

fig. 3 is a schematic diagram of a dynamic range compression process provided by an embodiment of the present disclosure;

fig. 4 is a block diagram illustrating an audio repair apparatus according to an embodiment of the disclosure;

fig. 5 is a block diagram of another audio repair apparatus provided in the embodiment of the present disclosure;

fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.

Detailed Description

In order that the above objects, features and advantages of the present disclosure may be more clearly understood, aspects of the present disclosure will be further described below. It should be noted that the embodiments and features of the embodiments of the present disclosure may be combined with each other without conflict.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may be practiced in other ways than those described herein; it is to be understood that the embodiments disclosed in the specification are only a few embodiments of the present disclosure, and not all embodiments.

The audio clipping phenomenon may cause audio distortion, and in a clipping position of audio, a portion of a continuous waveform exceeding a threshold (such as ± 32767) may be clipped, and the clipped portion may also be referred to as a clipping interval or a clipping region, and a high-frequency noise component like white noise may easily occur. For the near-end signal, split noise occurs to affect the hearing; for a far-end echo signal, the characteristics of a high-frequency noise component appearing in a chopping interval and a reference signal are not consistent, the far-end echo signal is a nonlinear signal, and the linear echo cancellation effect is also affected, so that audio restoration needs to be performed on the chopping interval. The sparse synthesis and sparse analysis model method, the least mean square autoregressive model method, the minimized high-order derivative energy method and the like adopted in the related technology all need to involve a large amount of matrix calculation, the calculation amount is very large, the time required for repair is long, the related technology cannot be suitable for real-time repair of scenes such as audio and video conferences, live webcasts, network courses and the like due to the characteristics of high calculation amount and high time delay, and in order to solve the problems, the disclosed embodiment provides an audio repair method, device, equipment and medium. For ease of understanding, reference may be made specifically to the following.

Fig. 1 is a flowchart of an audio repairing method provided by an embodiment of the present disclosure, where the method may be performed by an audio repairing apparatus, where the apparatus may be implemented by software and/or hardware, and may be generally integrated in an electronic device. As shown in fig. 1, the method mainly includes the following steps S102 to S108:

step S102, obtaining the audio frame signal to be processed.

In some embodiments, the to-be-processed audio frame signal may be actively acquired in real time, or may be passively received from an external device or uploaded by a user. If a segment of audio to be repaired is acquired, the audio to be repaired may be subjected to framing processing, such as dividing a frame at a specified time interval of every 10ms, and each audio frame signal includes a plurality of signal points (which may also be referred to as data points or sampling points). In practical applications, the audio frame signal obtained by framing can be converted into a specified data type for subsequent processing.

Step S104, detecting a wave-chopping interval existing in the audio frame signal; the wave interception interval comprises a plurality of continuous wave interception points, and the wave interception points are signal points with amplitude values exceeding a preset threshold value.

For example, the chopping points may be detected first, and then the chopping intervals may be determined according to the positions of the chopping points. Generally, a clipping point is clipped (i.e., amplitude clipping or clipping) due to the amplitude exceeding a predetermined threshold, and a plurality of consecutive clipping points can be regarded as a clipping interval. For example, the preset threshold may be set to 32760, and when detecting the clipping interval, signal points having an absolute value of amplitude greater than 32760 in the audio frame signal may be detected first, and the time coordinate of the first clipping point and the number of subsequent consecutive clipping points may be recorded, so that the clipping interval may be determined. In practical applications, it is possible to traverse all the clipping intervals possessed by the current audio frame signal and record information of the clipping intervals, such as storing the start coordinates of all the clipping intervals and the number of clipping points in a two-dimensional matrix for subsequent processing.

Step S106, judging whether the detected chopping interval meets the specified interpolation condition or not according to the non-chopping point adjacent to the chopping interval; if yes, go to step S108; if not, the process is ended. The term "adjacent" is understood to mean "within a specified range of distance", that is, a non-chopping point adjacent to a chopping block is understood to mean a non-chopping point within a specified range of distance from the chopping block. Specifically, the non-chopping point adjacent to the chopping interval may include: 1) before and adjacent to (within a predetermined distance range) a first clipping point of the clipping interval, and 2) after and adjacent to (within a predetermined distance range) a last clipping point of the clipping interval. The first chopping point of the chopping interval is the starting end of the chopping interval, and the last chopping point of the chopping interval is the ending end of the chopping interval. In some embodiments, the range length of the "designated range" can be flexibly set according to requirements, and in other embodiments, the "designated range" can be set as a range between two adjacent chopping intervals, that is, a non-chopping section between two adjacent chopping intervals.

The aforementioned specified interpolation condition includes, but is not limited to, a parabolic interpolation condition, a cubic spline interpolation condition, or a sinusoidal interpolation condition, and is not limited herein. It can be understood that, if a signal point is interpolated, a corresponding interpolation condition needs to be satisfied, and the interpolation condition is a precondition that a corresponding interpolation method can be used for the signal point, and specifically, the interpolation condition is a precondition that a corresponding interpolation method can be used to interpolate a clipping interval based on a non-clipping point adjacent to the clipping interval. Taking the example that the specified interpolation condition is a parabolic interpolation condition, the parabolic interpolation condition is a condition that can interpolate the clipping interval by using a parabolic interpolation method for the non-clipping point adjacent to the clipping interval. The parabolic interpolation method may be called a quadratic interpolation method or a third-order lagrange interpolation method, and is a polynomial interpolation method that can be applied only when a certain condition is satisfied. Such as at least three points, can achieve parabolic interpolation. Therefore, only when the non-clipping points adjacent to the clipping interval meet the conditions (such as at least a specified number of effective non-clipping points are arranged at the outer sides of the two ends of the clipping interval), the effective non-clipping points can be applied to carry out parabolic interpolation calculation to obtain the signal point amplitude of the clipping interval, so that the original waveform of the clipping interval is fitted and repaired.

And step S108, repairing the chopping interval by adopting a specified interpolation method based on the non-chopping point.

When the specified interpolation condition includes a parabolic interpolation condition, the specified interpolation method includes a parabolic interpolation method; when the specified interpolation condition comprises a cubic spline interpolation condition, the specified interpolation method comprises a cubic spline interpolation method; when the specified interpolation condition includes a sine interpolation condition, the specified interpolation method includes a sine interpolation method. Similarly, if the specified interpolation condition includes another interpolation condition other than the interpolation condition described above, the specified interpolation method includes an interpolation method corresponding to the other interpolation condition, and is not limited herein.

If the clipping interval meets the parabolic interpolation condition, the non-clipping point adjacent to the clipping interval can be adopted to carry out parabolic interpolation repair on the clipping interval. The core calculation amount of the parabola interpolation method only relates to a quadratic parabola equation, and the calculation amount is far lower than that of a repairing mode needing a large number of matrix operations in the related technology, so that the repairing time can be greatly shortened.

In summary, in the above manner provided by the embodiment of the present disclosure, the non-intercept point adjacent to the intercept interval is used to perform parabolic interpolation restoration on the intercept interval, and the computation amount of the parabolic interpolation method is very low, so that the restoration time can be greatly shortened, and a fast and real-time audio restoration effect can be achieved.

Most of the audio to be repaired is short integer type, because the short type audio frame signal is most likely to have a problem exceeding a preset threshold, which results in clipping the audio frame signal, and in order to facilitate audio repair processing, the short type audio frame signal may be converted into float floating point type, which can perform repair restoration on a clipping point clipped by the amplitude exceeding the preset threshold, and therefore, in some embodiments, the foregoing step S102 may be performed with reference to the following steps a to c:

step a, performing framing processing on the audio to be processed to obtain an initial frame signal. The number of initial frame signals may be plural. For example, one frame is divided at a specified time interval of every 10ms or the like, and each audio frame signal includes a plurality of signal points, such as 160 signal points or 480 signal points. It should be noted that the above 160 signal points or 480 signal points included in 10ms or each frame are only exemplary illustrations and should not be considered as limitations, and in practical applications, the audio framing process can be performed according to actual needs.

And b, if the original data type of the initial frame signal is not the specified data type, converting the initial frame signal into the specified data type. In some embodiments, the specified data type may be a float type.

And c, taking the converted initial frame signal as an audio frame signal to be processed. That is, each of the initial frame signals converted into the float type may be individually made as an audio frame signal to be processed.

If the data type conversion has been performed for the audio frame signal before the audio repair is performed, after the repair process, the method further includes: and converting the data type of the audio frame signal after the repair processing into the original data type. For example, if the short type audio frame signal has been converted into the float type audio frame signal before the audio repair operation is performed, the float type audio frame signal may be further converted back into the short type audio frame signal after the repair is completed, so that the data types before and after the audio repair are consistent.

In some embodiments, a parabolic interpolation method with a smaller calculation amount may be selected from a plurality of interpolation methods to perform interpolation restoration on the clipping interval, so as to further reduce the calculation amount required by audio restoration. At this time, the specified interpolation condition includes a parabolic interpolation condition, and the specified interpolation method includes a parabolic interpolation method; it can be understood that the parabolic interpolation method generally requires at least three points, such as at least three points at non-clipping points adjacent to a clipping interval, to apply the parabolic interpolation method, and in order to better implement the parabolic interpolation and make the result of the parabolic interpolation more reliable, the embodiments of the present disclosure provide an implementation manner for determining whether the clipping interval satisfies a parabolic interpolation condition according to the non-clipping points adjacent to the clipping interval, including: detecting whether at least two non-chopping points exist before a first chopping point of a chopping interval and under the condition of not crossing other chopping intervals; and detecting whether there are at least two non-clipping points after the last clipping point of the clipping interval and without crossing other clipping intervals; and if the detection results are yes, determining that the chopping interval meets the parabolic interpolation condition. The first clipping point of the clipping section may be regarded as a starting point (or a left end point) of the clipping section, and the last clipping point of the clipping section may be regarded as an ending point (or a right end point) of the clipping section. For convenience of understanding, the following descriptions are respectively made according to the following cases that the current clipping interval to be processed is the first clipping interval (that is, only the other clipping intervals are arranged behind the current clipping interval, and no other clipping intervals are arranged in front of the current clipping interval), the middle clipping interval (that is, all other clipping intervals are arranged in front of the current clipping interval), and the last clipping interval (that is, only the other clipping intervals are arranged in front of the current clipping interval, and no clipping interval is arranged behind the current clipping interval) in the audio frame signal:

if the current clipping interval is the first clipping interval of the audio frame signal, whether at least two non-clipping points exist before the current clipping interval is detected, whether at least two non-clipping points exist between the current clipping interval and the next clipping interval is detected, and if the detection results are all yes, the fact that the clipping interval meets the condition of parabolic interpolation is determined.

If the current chopping interval is the middle chopping interval of the audio frame signal, whether at least two non-chopping points exist between the current chopping interval and the adjacent previous chopping interval or not and whether at least two non-chopping points exist between the current chopping interval and the adjacent next chopping interval or not are detected, and if the detection results are all yes, the fact that the chopping interval meets the parabolic interpolation condition is determined.

If the current clipping interval is the last clipping interval of the audio frame signal, whether at least two non-clipping points exist after the current clipping interval is detected, whether at least two non-clipping points exist between the current clipping interval and the adjacent previous clipping interval is detected, and if the detection results are all yes, the clipping interval is determined to meet the condition of parabolic interpolation.

By setting the above conditions, at least two non-chopping points are arranged on the outer sides of the two ends of the chopping interval, so that the parabolic interpolation result is more reliable by selecting a proper non-chopping point for parabolic interpolation in a subsequent step. If the detected chopping interval meets the parabolic interpolation condition based on the adjacent non-chopping point of the chopping interval, the chopping interval can be further repaired by adopting a parabolic interpolation method based on the non-chopping point, in order to improve the parabolic interpolation effect and ensure the reasonability and reliability of the waveform after the chopping repair, the slope evaluation can be further carried out on the adjacent non-chopping point, and therefore a more appropriate non-chopping point is selected for parabolic interpolation. In one embodiment, the specific step of repairing the clipping interval by using the parabolic interpolation method based on the non-clipping point can be realized by referring to the following (1) to (3):

(1) the absolute value of a first slope between two non-intercept points that are nearest to the first intercept point of the intercept interval and the absolute value of a second slope between two non-intercept points that are nearest to the last intercept point of the intercept interval are calculated. Assuming that two non-chopping points closest to the first chopping point of the chopping interval are P11 and P12, and the absolute value of the slope of the connecting line of P11 and P12 is K1 (first slope); two non-chopping points which are nearest to the last chopping point of the chopping interval are P21 and P22, and the absolute value of the slope of the connecting line of P21 and P22 is K2 (second slope);

(2) and if the absolute value of the first slope is less than or equal to the absolute value of the second slope, selecting two non-chopping points which are nearest to the first chopping point of the chopping interval and one non-chopping point which is nearest to the last chopping point of the chopping interval to perform parabolic interpolation calculation, and repairing the chopping interval based on the obtained first interpolation calculation result. That is, if K1 is less than or equal to K2, three points P11, P12, and P21 are selected.

(3) And if the absolute value of the first slope is larger than that of the second slope, selecting two non-chopping points which are nearest to the last chopping point of the chopping interval and one non-chopping point which is nearest to the first chopping point of the chopping interval to perform parabolic interpolation calculation, and repairing the chopping interval based on the obtained second interpolation calculation result. That is, if K1 is greater than K2, three points, P12, P21, and P22 are selected.

It can be understood that, through the above manner, the non-clipping point with the lower absolute value of the slope is preferentially selected to perform parabolic interpolation calculation to repair the clipping interval, so that the rationality of the repaired waveform can be ensured as much as possible, and the probability of the waveform energy of the repaired waveform in the original clipping position, which is caused by interpolation using the non-clipping point with the higher slope, is reduced as much as possible, thereby better ensuring the audio repairing effect.

On the basis of the embodiment of the audio frequency repairing method, the appointed interpolation condition is the parabolic interpolation condition, and the appointed interpolation method is the parabolic interpolation method. The method can be implemented with reference to the following steps S202 to S212:

step S202, an audio frame signal to be processed is acquired. For example, the audio to be repaired may be subjected to framing, data type conversion, and the like, resulting in an audio frame signal with a specified data type.

In step S204, a clipping interval existing in the audio frame signal is detected.

Step S206, judging whether the detected chopping interval meets the condition of parabolic interpolation according to a non-chopping point adjacent to the chopping interval; if yes, go to step S208; if not, step S210 is performed.

In step S208, the clipping interval is repaired by a parabolic interpolation method based on the non-clipping point, and then step S212 is performed.

It is understood that, the related contents of the foregoing embodiments can be referred to in steps S204 to S208, and are not described herein again.

Step S210, performing smoothing processing on the repaired clipping interval, the first specified length interval located before the first clipping point of the clipping interval, and the second specified length interval located after the last clipping point of the clipping interval by using a preset smoothing filtering algorithm.

The smoothing processing is carried out on the chopping interval by adopting the smoothing filtering algorithm, so that the high-frequency pseudo noise of the chopping interval can be effectively reduced, the waveform restoration effect is improved, and the shaking problem possibly existing in the signal points on the two sides of the chopping interval is fully considered in the mode, so that the smoothing processing is carried out on the restored chopping interval, the smoothing processing is also carried out on the specified length intervals on the two sides of the chopping interval, the lengths of the first specified length interval and the second specified length interval can be the same or different, the smoothing processing can be flexibly set according to the requirements, for example, the first specified length interval and the second specified length interval respectively contain the specified number of signal points, and if the specified number is four, the smoothing filtering operation is carried out on the chopping interval and the four points in front and back by adopting the preset smoothing filtering algorithm. In practical applications, the predetermined smoothing filter algorithm may be, for example, a five-point cubic smoothing filter algorithm, and other smoothing filter algorithms may also be used, which is not limited herein.

Step S212, performing dynamic range compression processing on the current audio frame signal, so that the amplitude ranges of the audio frame signal are all within the specified amplitude range interval. The dynamic range compression processing can be called as dynamic energy compression processing, and aims to limit waveform energy and ensure that the amplitude range of a processed signal is within a specified amplitude range, such as-32768-32767, so that the problems of wave-chopping energy overflow and the like possibly existing in a repaired waveform are avoided, and the repaired waveform can be prevented from secondary wave-chopping through further dynamic energy compression.

The current audio frame signal involved in step S212 may be an audio frame signal repaired by a parabolic interpolation method, or an audio frame signal that has not been repaired by a parabolic interpolation method because the detected audio frame signal does not satisfy the parabolic interpolation condition. That is, if the clipping interval meets the condition of parabolic interpolation, then performing dynamic range compression processing on the repaired audio frame signal so as to enable the amplitude range of the repaired audio frame signal to be located in the specified amplitude range interval; and if the chopping interval does not meet the specified interpolation condition, subsequently, directly carrying out dynamic range compression processing on the audio frame signal so as to enable the amplitude range of the audio frame signal to be located in the specified amplitude range interval.

In order to further shorten the operation time, the step of performing dynamic range compression processing on the repaired audio frame signal comprises the following steps: converting the repaired audio frame signal into a logarithmic domain for representation through a logarithmic table look-up method; and carrying out dynamic range compression processing on the converted audio frame signals. In practical application, a log table can be pre-recorded, log operation can be involved in the process of dynamic range compression processing, corresponding results can be directly obtained in a table look-up mode, and compared with direct calculation, the log table look-up method is more convenient and time-saving, so that time is shortened as far as possible, and a rapid and real-time audio frequency restoration effect is achieved.

For convenience of understanding, an embodiment of the present disclosure provides an implementation of dynamic range compression, and referring to a schematic diagram of a dynamic range compression process shown in fig. 3, a main flow may be implemented with reference to steps 1 to 6:

step 1, converting an input signal x [ n ] into a dB domain (logarithmic domain), using the following formula:

x_dB[n]＝20×log₁₀|x[n]|

in the step, the operation amount of dB calculation can be further reduced by adopting a log table look-up method, and the operation time is shortened.

Step 2, in the gain computer part, an upper limit T of the energy threshold may be set, in a specific implementation provided in the embodiment of the present disclosure, the value of T may be-4.3 dB, W is the length of the pass band window, which may be 2dB, and then static characteristics (static characteristics) x is calculated in the following manner_sc：

Step 3, in the gain computer part, x is known_scThe logarithmic gain (gain) value g is then further calculated_c：

g_c[n]＝x_SC[n]-x_dB[n].

Step 4, smoothing the gain value to obtain a smoothed gain value g_s：

In a specific implementation provided by the embodiments of the present disclosure, T_AAttack time, which may have a value of 0.0004; t is_RFor release time, its value may be 0.004: fs is the signal sampling frequency; alpha is alpha_AIs a first coefficient, α_RIs the second coefficient.

Step 5, log field gain value (i.e., g)_s) Conversion to the linear domain glin:

step 6: outputting a dynamic range compression result: y [ n ]]＝x[n]×g_lin[n].。

Through the mode, the energy of the repaired waveform can be effectively compressed and adjusted, and the problem of secondary wave interception is avoided.

In summary, the audio repairing method provided by the embodiment of the disclosure can greatly reduce the amount of computation by repairing the clipping interval through the parabolic interpolation method, shorten the repairing time, and help to achieve a quick and real-time repairing effect, so that the audio repairing method is better applied to scenes such as audio and video conferences, live webcasts, and network courses. In practical application, the calculated amount of the audio frequency repairing method provided by the embodiment of the disclosure is only 1% -0.1% of the calculated amount of the mainstream clipping repairing method through experiments, and the calculated amount requirement of real-time communication is completely met.

In addition, smoothing processing is performed on the chopping section and the front-rear length section, so that high-frequency pseudo noise can be sufficiently reduced, and the waveform after restoration can be further improved. And the energy of the repaired waveform can be effectively adjusted by adopting a log domain dynamic range compression method, so that the problem of secondary wave interception is avoided.

The audio repairing method provided by the embodiment of the disclosure can be applied to various occasions where the wave-clipping is required to be repaired, and can also be used for repairing the audio in advance, and inputting the repaired audio into a subsequent audio processing module for subsequent processing, for example, the audio repairing method of the embodiment of the disclosure is used for performing wave-clipping repair on an input signal of a linear echo cancellation algorithm AEC of a common audio communication algorithm, and inputting the signal after wave-clipping repair to the AEC for subsequent audio processing, so that the processing effect obtained after the AEC algorithm is processed based on the repaired signal is better.

Corresponding to the foregoing audio repairing method, the embodiment of the present disclosure further provides an audio repairing apparatus, which may be implemented by software and/or hardware, and may be generally integrated in an electronic device, and may implement a fast and real-time audio repairing effect by executing the audio repairing method provided by the embodiment of the present disclosure. Referring to fig. 4, a block diagram of an audio repair apparatus is shown, including:

a signal obtaining module 402, configured to obtain an audio frame signal to be processed;

a clipping detection module 404, configured to detect a clipping interval existing in the audio frame signal; the wave interception interval comprises a plurality of continuous wave interception points, and the wave interception points are signal points with amplitude values exceeding a preset threshold value;

a judging module 406, configured to judge whether the detected clipping interval meets a specified interpolation condition according to a non-clipping point adjacent to the clipping interval;

and the clipping repairing module 408 is configured to repair the clipping interval by using a specified interpolation method based on the non-clipping point when the determination result of the determining module is yes.

According to the device provided by the embodiment of the disclosure, interpolation restoration is performed on the chopping interval by using the non-chopping point adjacent to the chopping interval, and the restoration time can be greatly shortened due to the low calculation amount of the interpolation method, so that a rapid and real-time audio restoration effect can be achieved.

In some embodiments, the signal acquisition module 402 is configured to: performing framing processing on audio to be processed to obtain an initial frame signal; if the original data type of the initial frame signal is not the specified data type, converting the initial frame signal into the specified data type; and taking the converted initial frame signal as an audio frame signal to be processed.

In some embodiments, the apparatus further includes a data conversion module, configured to convert the data type of the audio frame signal after the repair processing into the original data type.

In some embodiments, the specified interpolation condition includes a parabolic interpolation condition, and the determining module 406 is specifically configured to: detecting whether at least two non-chopping points exist before a first chopping point of a chopping interval and under the condition of not crossing other chopping intervals; and detecting whether there are at least two non-clipping points after the last clipping point of the clipping interval and without crossing other clipping intervals; and if the detection results are yes, determining that the chopping interval meets the parabolic interpolation condition.

In some embodiments, the prescribed interpolation includes a parabolic interpolation, and the clipping repair module 408 is specifically configured to: calculating the absolute value of a first slope between two non-chopping points which are nearest to the first chopping point of the chopping interval and the absolute value of a second slope between two non-chopping points which are nearest to the last chopping point of the chopping interval; if the absolute value of the first slope is smaller than or equal to the absolute value of the second slope, selecting two non-chopping points which are nearest to the first chopping point of the chopping interval and one non-chopping point which is nearest to the last chopping point of the chopping interval to perform parabolic interpolation calculation, and repairing the chopping interval based on the obtained first interpolation calculation result; and if the absolute value of the first slope is larger than that of the second slope, selecting two non-chopping points which are nearest to the last chopping point of the chopping interval and one non-chopping point which is nearest to the first chopping point of the chopping interval to perform parabolic interpolation calculation, and repairing the chopping interval based on the obtained second interpolation calculation result.

In some embodiments, referring to the structural block diagram of another audio repair apparatus shown in fig. 5, on the basis of fig. 4, the apparatus further includes a smoothing module 502 and a first compression module 504, where the smoothing module 502 is configured to: and smoothing the repaired wave interception interval, a first specified length interval positioned before the first wave interception point of the wave interception interval and a second specified length interval positioned after the last wave interception point of the wave interception interval by adopting a preset smoothing filtering algorithm. The first compression module 504 is configured to: and performing dynamic range compression processing on the repaired audio frame signals so as to enable the amplitude ranges of the repaired audio frame signals to be located in the specified amplitude range interval. In some embodiments, the first compression module 504 is specifically configured to: converting the repaired audio frame signal into a logarithmic domain for representation through a logarithmic table look-up method; and carrying out dynamic range compression processing on the converted audio frame signals.

In some embodiments, the above apparatus further comprises: and the second compression module is used for carrying out dynamic range compression processing on the audio frame signal if the chopping interval does not meet the specified interpolation condition so as to enable the amplitude range of the audio frame signal to be located in the specified amplitude range interval.

The audio repairing device provided by the embodiment of the disclosure can execute the audio repairing method provided by any embodiment of the disclosure, and has corresponding functional modules and beneficial effects of the executing method.

It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatus embodiments may refer to corresponding processes in the method embodiments, and are not described herein again.

An embodiment of the present disclosure provides an electronic device, including: a processor; a memory for storing processor-executable instructions; the processor is used for reading the executable instructions from the memory and executing the instructions to realize any one of the audio repairing methods.

Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. As shown in fig. 6, the electronic device 600 includes one or more processors 601 and memory 602.

The processor 601 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 600 to perform desired functions.

Memory 602 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium and executed by the processor 601 to implement the audio repair methods of the embodiments of the present disclosure described above and/or other desired functions. Various contents such as an input signal, a signal component, a noise component, etc. may also be stored in the computer-readable storage medium.

In one example, the electronic device 600 may further include: an input device 603 and an output device 604, which are interconnected by a bus system and/or other form of connection mechanism (not shown).

The input device 603 may also include, for example, a keyboard, a mouse, and the like.

The output device 604 may output various information including the determined distance information, direction information, and the like to the outside. The output devices 604 may include, for example, a display, speakers, a printer, and a communication network and remote output devices connected thereto, among others.

Of course, for simplicity, only some of the components of the electronic device 600 relevant to the present disclosure are shown in fig. 6, omitting components such as buses, input/output interfaces, and the like. In addition, electronic device 600 may include any other suitable components depending on the particular application.

In addition to the above-described methods and apparatus, embodiments of the present disclosure may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the audio repair method provided by embodiments of the present disclosure.

The computer program product may write program code for carrying out operations for embodiments of the present disclosure in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.

Furthermore, embodiments of the present disclosure may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform the audio repair method provided by embodiments of the present disclosure.

The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Embodiments of the present disclosure also provide a computer program product comprising a computer program/instructions that, when executed by a processor, implement an audio repair method in embodiments of the present disclosure.

It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

The foregoing are merely exemplary embodiments of the present disclosure, which enable those skilled in the art to understand or practice the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.