|Publication number||US7742914 B2|
|Application number||US 11/073,820|
|Publication date||Jun 22, 2010|
|Filing date||Mar 7, 2005|
|Priority date||Mar 7, 2005|
|Also published as||US20060200344|
|Publication number||073820, 11073820, US 7742914 B2, US 7742914B2, US-B2-7742914, US7742914 B2, US7742914B2|
|Inventors||Daniel A. Kosek, Robert Crawford Maher|
|Original Assignee||Daniel A. Kosek|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (57), Non-Patent Citations (23), Referenced by (23), Classifications (12), Legal Events (2)|
|External Links: USPTO, USPTO Assignment, Espacenet|
1. Field of the Invention
The present invention relates to the field of digital signal processing, and more specifically, to a spectral noise reduction method and apparatus that can be used to remove the noise typically associated with analog signal environments.
2. Description of the Related Art
When an analog signal contains unwanted additive noise, enhancement of the perceived signal-to-noise ratio before playback will produce a more coherent, and therefore more desirable, signal. An enhancement process that is single-ended, that is, one that operates with no information available at the receiver other than the noise-degraded signal itself, is preferable to other methods. The reason it is preferable is because complementary noise reduction schemes, which require cooperation on the part of the broadcaster and the receiver, require both the broadcaster and the receiver to be equipped with encoding and decoding gear, and the encoding and decoding levels must be carefully matched. These considerations are not present with single-ended enhancement processes.
A composite “noisy” signal contains features that are noise and features that are attributable to the desired signal. In order to boost the desired signal while attenuating the background noise, the features of the composite signal that are noise need to be distinguished from the features of the composite signal that are attributable to the desired signal. Next, the features that have been identified as noise need to be removed or reduced from the composite signal. Lastly, the detection and removal methods need to be adjusted to compensate for the expected time-variant behavior of the signal and noise.
Any single-ended enhancement method also needs to address the issue of signal gaps—or “dropouts”—which can occur if the signal is lost momentarily. These gaps can occur when the received signal is lost due to channel interference (for example, lightning, cross-talk, or weak signal) in a radio or transmission or decoding errors in the playback system. The signal enhancement process must detect the signal dropout and take appropriate action, either by muting the playback or by reconstructing an estimate of the missing part of the signal. Although muting the playback does not solve the problem, it is often used because it is inexpensive to implement, and if the gap is very short, it may be relatively inaudible.
Several single-ended methods of reducing the audibility of unwanted additive noise in analog signals have already been developed. These methods generally fall into two categories: time-domain level detectors and frequency-domain filters. Both of these methods are one-dimensional in the sense that they are based on either the signal waveform (amplitude) as a function of time or the signal's frequency content at a particular time. By contrast, and as explained more fully below in the Detailed Description of Invention section, the present invention is two-dimensional in that it takes into consideration how both the amplitude and frequency content change with time.
Accordingly, it is an object of the present invention to devise a process for improving the signal-to-noise ratio in audio signals. It is a further object of the present invention to develop an intelligent model for the desired signal that allows a substantially more effective separation of the noise and the desired signal than current single-ended processes. The one-dimensional (or single-ended) processes used in the prior art are described more fully below, as are the discrete Fourier transform and Fourier transform magnitude—two techniques that play a role in the present invention.
A. Time-Domain Level Detection
The time-domain method of noise elimination or reduction uses a specified signal level, or threshold, that indicates the likely presence of the desired signal. The threshold is set (usually manually) high enough so that when the desired signal is absent (for example, when there is a pause between sentences or messages), there is no hard hiss. The threshold, however, must not be set so high that the desired signal is affected when it is present. If the received signal is below the threshold, it is presumed to contain only noise, and the output signal level is reduced or “gated” accordingly. As used in this context, the term “gated” means that the signal is not allowed to pass through. This process can make the received signal sound somewhat less noisy because the hiss goes away during the pause between words or sentences, but it is not particularly effective. By continuously monitoring the input signal level as compared to the threshold level, the time-domain level detection method gates the output signal on and off as the input signal level varies. These time-domain level detection systems have been variously referred to as squelch control, dynamic range expander, and noise gate.
In simple terms, the noise gate method uses the amplitude of the signal as the primary indicator: if the input signal level is high, it is assumed to be dominated by the desired signal, and the input is passed to the output essentially unchanged. On the other hand, if the received signal level is low, it is assumed to be a segment without the desired signal, and the gain (or volume) is reduced to make the output signal even quieter.
The difference between the time-domain methods and the present invention is that the time-domain methods do not remove the noise when the desired signal is present. Instead, if the noisy signal exceeds the threshold, the gate is opened, and the signal is allowed to pass through. Thus, the gate may open if there is a sudden burst of noise, a click, or some other loud sound that causes the signal level to exceed the threshold. In that case, the output signal quality is good only if the signal is sufficiently strong to mask the presence of the noise. For that reason, this method only works if the signal-to-noise ratio is high.
The time-domain method can be effective if the noisy input consists of a relatively constant background noise and a signal with a time-varying amplitude envelope (i.e., if the desired signal varies between loud and soft, as in human speech). Changing the gain between the “pass” (or open) mode and the “gate” (or closed) mode can cause audible noise modulation, which is also called “gain pumping.” The term “gain pumping” is used by recording engineers and refers to the audible sound of the noise appearing when the gate opens and then disappearing when the gate closes. Furthermore, the “pass” mode simply allows the signal to pass but does not actually improve the signal-to-noise ratio when the desired signal is present.
The effectiveness of the time-domain detection methods can be improved by carefully controlling the attack and release times (i.e., how rapidly the circuitry responds to changes in the input signal) of the gate, causing the threshold to vary automatically if the noise level changes, and splitting the gating decision into two or more frequency bands. Making the attack and release times somewhat gradual will lessen the audibility of the gain pumping, but it does not completely solve the problem. Multiple frequency bands with individual gates means that the threshold can be set more optimally if the noise varies from one frequency band to another. For example, if the noise is mostly a low frequency hum, the threshold can be set high enough to remove the noise in the low frequency band while still maintaining a lower threshold in the high frequency ranges. Despite these improvements, the time-domain detection method is still limited as compared to the present invention because the noise gate cannot distinguish between noise and the desired signal, other than on the basis of absolute signal level.
B. Frequency-Domain Filtration
The other well-known procedure for signal enhancement involves the use of spectral subtraction in the frequency domain. The goal is to make an estimate of the noise power as a function of frequency, then subtract this noise spectrum from the input signal spectrum, presumably leaving the desired signal spectrum.
For example, consider the signal spectrum shown in
The example signal of
In a prior art spectral subtraction system, the receiver estimates the noise level as a function of frequency. The noise level estimate is usually obtained during a “quiet” section of the signal, such as a pause between spoken words in a speech signal. The spectral subtraction process involves subtracting the noise level estimate, or threshold, from the received signal so that any spectral energy that is below the threshold is removed. The noise-reduced output signal is then reconstructed from this subtracted spectrum.
An example of the noise-reduced output spectrum for the noisy signal of
The spectral subtraction process can cause various audible problems, especially when the actual noise level differs from the estimated noise spectrum. In this situation, the noise is not perfectly canceled, and the residual noise can take on a whistling, tinkling quality sometimes referred to as “musical noise” or “birdie noise.” Furthermore, spectral subtraction does not adequately deal with changes in the desired signal over time, or the fact that the noise itself will generally fluctuate rapidly from time to time. If some signal components are below the noise threshold at one instant in time but then peak above the noise threshold at a later instant in time, the abrupt change in those components can result in an annoying audible burble or gargle sound.
Some prior art improvements to the spectral subtraction method have been made, such as frequently updating the noise level estimate, switching off the subtraction in strong signal conditions, and attempting to detect and suppress the residual musical noise. None of these techniques, however, has been wholly successful at eliminating the audible problems.
C. Discrete Fourier Transform and Fourier Transform Magnitude
The discrete Fourier transform (“DFT”) is a computational method for representing a discrete-time (“sampled” or “digitized”) signal in terms of its frequency content. A short segment (or “data frame”) of an input signal, such as a noisy audio signal treated in this invention, is processed according to the well-known DFT analysis formula (1):
where N is the length of the data frame, x[n] are the N digital samples comprising the input data frame, X[k] are the N Fourier transform values, j represents the mathematical imaginary quantity (square-root of −1), e is the base of the natural logarithms, and ejθ=cos(θ)+jˇsin(θ), which is the relationship known as Euler's formula.
The DFT analysis formula expressed in equation (1) can be interpreted as producing N equally-spaced samples between zero and the digital sampling frequency for the signal x[n]. Because the DFT formula involves the imaginary number j, the X[k] spectral samples will, in general, be mathematically complex numbers, meaning that they will have a “real” part and an “imaginary” part.
The inverse DFT is computed using the standard inverse transform, or “Fourier synthesis” equation (2):
Equation 2 shows that the data frame x[n] can be reconstructed, or synthesized, from the DFT data X[k] without any loss of information: the signal can be reconstructed from its Fourier transform, at least within the limits of numerical precision. This ability to reconstruct the signal from its Fourier transform allows the signal to be converted from the discrete-time domain to the frequency domain (Fourier) and vice versa.
In order to estimate the signal power in a particular range of frequencies, such as when attempting to distinguish between the background noise and the desired signal, this information can be obtained by calculating the spectral magnitude of the DFT by the standard Pythagorean formula (3):
where Re( ) and Im( ) indicate taking the mathematical real part and imaginary part, respectively. Although the input signal x[n] cannot, in general, be reconstructed from the DFT magnitude, the magnitude information can be used to find the distribution of signal power as a function of frequency for that particular data frame.
The present invention covers a method of reducing noise in an audio signal, wherein the audio signal comprises spectral components, comprising the steps of: using a furrow filter to select spectral components that are narrow in frequency but relatively broad in time; using a bar filter to select spectral components that are broad in frequency but relatively narrow in time; wherein there is a relative energy distribution between the output of the furrow and bar filters, analyzing the relative energy distribution between the output of the furrow and bar filters to determine the proportion of spectral components selected by each filter that will be included in an output signal; and reconstructing the audio signal based on the analysis above to generate the output signal. The furrow filter is used to identify discrete spectral partials, as found in voiced speech and other quasi-periodic signals. The bar filter is used to identify plosive and fricative consonants found in speech signals. The output signal that is generated as a result of the method of the present invention comprises less broadband noise than the initial audio signal. In the preferred embodiment, the audio signal is reconstructed using overlapping inverse Fourier transforms.
An optional enhancement to the method of the present invention includes the use of a second pair of time-frequency filters to improve intelligibility of the output signal. More specifically, this second pair of time-frequency filters is used to obtain a rapid transition from a steady-state voiced speech segment to adjacent fricatives or gaps in speech without temporal smearing of the audio signal. The first pair of time-frequency filters described in connection with the main embodiment of the present invention is referred to as the “long-time” filters, and the second pair of time-frequency filters that is included in the enhancement is referred to as the “short-time” filters. The long-time filters tend not to respond as rapidly as the short-time filters to input signal changes, and they are used to enhance the voiced features of a speech segment. The short-time filters do respond rapidly to input signal changes, and they are used to locate where new words start. Transient monitoring is used to detect sudden changes in the input signal, and resolution switching is used to change from the short-time filters to the long-time filters and vice versa.
Each pair of filters (both short-time and long-time) comprise a furrow filter and a bar filter, and another optional enhancement to the method of the present invention includes monitoring the temporal relationship between the furrow filter output and the bar filter output so that the fricative components are allowed primarily at boundaries between intervals with no voiced signal present and intervals with voice components. This monitoring ensures that the fricative phoneme(s) of the speech segment is/are not mistaken for undesired additive noise.
In an alternate embodiment, the present invention covers a method of reducing noise in an audio signal, wherein the audio signal comprises spectral components, comprising the steps of: segmenting the audio signal into a plurality of overlapping data frames; multiplying each data frame by a smoothly tapered window function; computing the Fourier transform magnitude for each data frame; and comparing the resulting spectral data for each data frame to the spectral data of the prior and subsequent frames to determine if the data frame contains predominantly coherent or predominantly random material. The predominantly coherent material is indicated by the presence of distinct characteristic features in the Fourier transform magnitude, such as discrete harmonic partials or other repetitive structure. The predominantly random material, on the other hand, is indicated by a spread of spectral energy across all frequencies. Furthermore, the criteria used to compare the resulting spectral data for each frame are consistently applied from one frame to the next in order to emphasize the spectral components of the audio signal that are consistent over time and de-emphasize the spectral components of the audio signal that vary randomly over time.
The present invention also covers a noise reduction system for an audio signal comprising a furrow filter and a bar filter, wherein the furrow filter is used to select spectral components that are narrow in frequency but relatively broad in time, and the bar filter is used to select spectral components that are broad in frequency but relatively narrow in time, wherein there is a relative energy distribution between the output of the furrow and bar filters, and said relative energy distribution is analyzed to determine the proportion of spectral components selected by each filter that will be included in an output signal, and wherein the audio signal is reconstructed based on the analysis of the relative energy distribution between the output of the furrow and bar filters to generate the output signal. As with the method claims, the furrow filter is used to identify discrete spectral partials, as found in voiced speech and other quasi-periodic signals, and the bar filter is used to identify plosive and fricative consonants found in speech signals. The output signal that exits the system comprises less broadband noise than the audio signal that enters the system. In the preferred embodiment, the audio signal is reconstructed using overlapping inverse Fourier transforms.
An optional enhancement to the system of the present invention further comprises a second pair of time-frequency filters, which are used to improve intelligibility of the output signal. As stated above, this second pair of time-frequency filters is used to obtain a rapid transition from a steady-state voiced speech segment to adjacent fricatives or gaps in speech without temporal smearing of the audio signal. As with the method claims, the second pair of “short-time” filters responds rapidly to input signal changes and is used to locate where new words start. The first pair of “long-time” filters tends not to respond as rapidly as the short-time filters to input signal changes, and they are used to enhance the voiced features of a speech segment. Transient monitoring is used to detect sudden changes in the input signal, and resolution switching is used to change from the short-time filters to the long-time filters and vice versa.
Another optional enhancement to the system of the present invention, wherein each pair of filters comprises a furrow filter and a bar filter, includes monitoring the temporal relationship between the furrow filter output and the bar filter output so that the fricative components are allowed primarily at boundaries between intervals with no voiced signal present and intervals with voice components. As stated above, this monitoring ensures that the fricative phoneme(s) of the speech segment is/are not mistaken for undesired additive noise.
The current state of the art with respect to noise reduction in analog signals involves the combination of the basic features of the noise gate concept with the frequency-dependent filtering of the spectral subtraction concept. Even this method, however, does not provide a reliable means to retain the desired signal components while suppressing the undesired noise. The key factor that has been missing from prior techniques is a means to distinguish between the coherent behavior of the desired signal components and the incoherent behavior of the additive noise. The present invention involves performing a time-variant spectral analysis of the incoming noisy signal, identifying features that behave consistently over a short-time window, and attenuating or removing features that exhibit random or inconsistent fluctuations.
The method employed in the present invention includes a data-adaptive, multi-dimensional (frequency, amplitude and time) filter structure that works to enhance spectral components that are narrow in frequency but relatively long in time, while reducing signal components (noise) that exhibit neither frequency nor temporal correlation. The effectiveness of this approach is due to its ability to distinguish the quasi-harmonic characteristics and the short-in-time but broad-in-frequency content of fricative sounds found in typical signals such as speech and music from the uncorrelated time-frequency behavior of broadband noise.
The major features of the signal enhancement method of the present invention include:
A. Basic Method: Reducing Noise Through the Use of Two-Dimensional Filters in the Time-Frequency Domain
The present invention entails a time-frequency orientation in which two separate 2-D (time vs. frequency) filters are constructed. One filter, referred to as a “furrow” filter, is designed so that it preferentially selects spectral components that are narrow in frequency but relatively broad in time (corresponding to discrete spectral partials, as found in voiced speech and other quasi-periodic signals). The other 2-D filter, referred to as a “bar” filter, is designed to pass spectral components that are broad in frequency but relatively narrow in time (corresponding to plosive and fricative consonants found in speech signals). The relative energy distribution between the output of the furrow and bar 2-D filters is used to determine the proportion of these constituents in the overall output signal. The broadband noise, lacking a coordinated time-frequency structure, is therefore reduced in the composite output signal.
In the case of single-ended noise reduction, the received signal s(t) is assumed to be the sum of the desired signal d(t) and the undesired noise n(t): s(t)=d(t)+n(t). Because only the received signal s(t) can be observed, the above equation is analogous to a+b=5, one equation with two unknowns. Thus, it is not possible to solve the equation using a simple mathematical solution. Instead, a reasonable estimate has to be made as to which signal features are most likely to be attributed to the desired portion of the received signal and which signal features are most likely to be attributed to the noise. In the present invention, the novel concept is to treat the signal as a time-variant spectrum and use the consistency of the frequency versus time information to separate out what is desired signal and what is noise. The desired signal components are the portions of the signal spectrum that tend to be narrow in frequency and long in time.
In the present invention, the furrow and bar filters are used to distinguish between the coherent signal, which is indicated by the presence of connected horizontal tracks on a spectrogram (with frequency on the vertical axis and time on the horizontal axis), and the unwanted broadband noise, which is indicated by the presence of indistinct spectral features. The furrow filter emphasizes features in the frequency vs. time spectrum that exhibit the coherent property, whereas the bar filter emphasizes features in the frequency vs. time spectrum that exhibit the fricative property of being short in time but broad in frequency. The background noise, being both broad in frequency and time, is minimized by both the furrow and bar filters.
There is a fundamental signal processing tradeoff between resolution in the time dimension and resolution in the frequency dimension. Obtaining very narrow frequency resolution is accomplished at the expense of relatively poor time resolution, and conversely, obtaining very short time resolution can only be accomplished with broad frequency resolution. In other words, this fundamental mathematical uncertainty principle dictates that the tradeoff cannot be used to create a set of filters that offer a variety of time and frequency resolutions.
The 2-D filters of the present invention are placed systematically over the entire frequency vs. time spectrogram, the signal spectrogram is observed through the frequency vs. time region specified by the filter, and the signal spectral components with the filter's frequency vs. time resolution are summed. This process emphasizes features in the signal spectrum that are similar to the filter in frequency vs. time, while minimizing signal spectral components that do not match the frequency vs. time shape of the filter.
This 2-D filter arrangement is depicted in
In an alternate embodiment, the furrow and bar structures are not implemented as 2-D digital filters; instead, a frame-by-frame analysis and recursive testing procedure can also be used in order to minimize the computation rate. In this alternate embodiment, the noisy input signal is segmented into a plurality of overlapping data frames. Each frame is multiplied by a smoothly tapered window function, the Fourier transform magnitude (the spectrum) for the frame is computed, and the resulting spectral data for that frame is examined and compared to the spectral data of the prior frame and the subsequent frame to determine if that portion of the input signal contains predominantly coherent material or predominantly random material.
The resulting collection of signal analysis data can be viewed as a spectrogram: a representation of signal power as a function of frequency on the vertical axis and time on the horizontal axis. Spectral features that are coherent appear as connected horizontal lines, or tracks, when viewed in this format. Spectral features that are due to broadband noise appear as indistinct spectral components that are spread more or less uniformly over the time vs. frequency space. Spectral features that are likely to be fricative components of speech are concentrated in relatively short time intervals but relatively broad frequency ranges that are typically correlated with the beginning or the end of a coherent signal segment, such as would be caused by the presence of voiced speech components.
In this alternate embodiment, the criteria applied to select the spectral features are retained from one frame to the next in order to accomplish the same goal as the furrow and bar 2-D filters, namely, the ability to emphasize the components of the signal spectrum that are consistent over time and de-emphasize the components that vary randomly from one moment to the next.
B. First Optional Enhancement: Using Parallel Filter Sets to Match the Processing Resolution to the Time-Variant Signal Characteristics
To further enhance the effectiveness of the present invention, a second pair of time-frequency filters may be used in addition to the furrow and bar filter pair described above. The latter pair of filters are “long-time” filters, whereas the former (or second) pair of filters are “short-time” filters. A short-time filter is one that will accept sudden changes in time. A long-time filter, on the other hand, is one that tends to reject sudden changes in time. This difference in filter behavior is attributable to the fact that there is a fundamental trade-off in signal processing between time resolution and frequency resolution. Thus, a filter that is very selective (narrow) in frequency will need a long time to respond to an input signal. For example, a very short blip in the input will not be enough to get a measurable signal in the output of such a filter. Conversely, a filter that responds to rapid input signal changes will need to be broader in its frequency resolution so that its output can change rapidly.
In the present invention, a short-time window (i.e., one that is wider in frequency) is used to locate where new words start, and a long-time window (i.e., one that is narrower in frequency) is used to track what happens during a word. The short-time filters enhance the effectiveness of the present invention by allowing the system to respond rapidly as the input signal changes. By using two separate pairs of filters—one for narrow frequency with relatively poor time resolution and the other for broad frequency with relatively good time resolution—the present invention obtains the optimal signal.
More specifically, the parallel short-time filters are used to obtain a rapid transition from the steady-state voiced speech segments to the adjacent fricatives or gaps in the speech without temporal smearing of the signal. The presence of a sudden change in the input signal is detected by the system, and the processing is switched to use the short-time (broad in frequency) filters so that the rapid change (e.g., a consonant at the start of a word) does not get missed. Once the signal appears to be in a more constant and steady-state segment, the system returns to using the long-time (tighter frequency resolution) filters to enhance the voiced features and reject any residual noise.
This approach provides a useful enhancement because the transitions from voiced to unvoiced speech, which can be discerned better with the short-time filters than the long-time filters, contribute to the intelligibility of the recovered speech signal. Moreover, the procedure for transient monitoring (i.e., detecting sudden changes in the input signal) and resolution switching (changing from the short-in-time but broad-in-frequency set of filters to the broad-in-time but narrow-in-frequency filters) has been used successfully in a wide variety of perceptual audio coders, such as MPEG-1, Layer 3 (MP3).
An example of the use of parallel filters is provided in Table 1. Using a signal sample frequency of 48,000 samples per second (48 kHz), a set of four time-length filters is created to observe the signal spectrum: 32 samples, 64 samples, 128 samples, and 2048 samples, corresponding to 667 microseconds, 1.33 milliseconds, 2.667 milliseconds, and 42.667 milliseconds, respectively. The shortest two durations correspond to the bar filter type, and the longer two durations correspond to the furrow filter type. Using a smoothly tapered time window function such as a hanning window (w[n]=0.5-0.5 cos(2πn/M), 0≦n≦M (total window length is M+1)), the fundamental frequency vs. time tradeoff yields frequency resolution as shown in Table 1 below, based on a normalized radian frequency resolution of 8π/M for the hanning window.
By way of comparison, a male talker with speech fundamental frequency 125 Hz corresponds to 8 ms (384 samples at 48 k Hz); therefore, the long furrow filter covers several fundamental periods and will resolve the individual partials. A female talker with speech fundamental frequency 280 Hz corresponds to 3.6 ms (171 samples at 48 k Hz), which is closer to the short furrow length. The bar filters are much shorter in time and will, therefore, detect spectral features that are short in duration as compared to the furrow filters. Although specific filter characteristics are provided in this example, many other tradeoffs are possible because the duration of the filter and its frequency resolution can be adjusted in a reciprocal manner (duration multiplied by bandwidth is a constant, due to the uncertainty principle).
A graphic representation of the short and long furrow and bar filters expressed in Table 1 is shown in
C. Second Optional Enhancement: Improving Intelligibility by Monitoring the Temporal Relationship Between Voiced Segments and Fricative Segments
The effectiveness of the furrow and bar filter concept may be enhanced in the context of typical audio signals such as speech by monitoring the temporal relationship between the voiced segments (furrow filter output) and the fricative segments (bar filter output) so that the fricative components are allowed primarily at boundaries between (i) intervals with no voiced signal present and (ii) intervals with voiced components. This temporal relationship is important because the intelligibility of speech is tied closely to the presence and audibility of prefix and suffix consonant phonemes. The behavior of the time-frequency filters includes some knowledge of the phonetic and expected fluctuations of natural speech, and these elementary rules are used to aid noise reduction while enhancing the characteristics of the speech.
D. Overview of the Present Invention
As described above, the present invention provides the means to distinguish between the coherent behavior of the desired signal components and the incoherent (uncorrelated) behavior of the additive noise. In the present invention, a time-variant spectral analysis of the incoming noisy signal is performed, features that behave consistently over a short-time window are identified, and features that exhibit random or inconsistent fluctuations are attenuated or removed. The major features of the present invention are:
The present invention detects the transition from a coherent segment of the signal to an incoherent segment, assesses the likelihood that the start of the incoherent segment is due to a fricative speech sound, and either allows the incoherent energy to pass to the output if it is attributed to speech, or minimizes the incoherent segment if it is attributed to noise. The effectiveness of this approach is due to its ability to pass the quasi-harmonic characteristics and the short-in-time but broad-in-frequency content of fricative sounds found in typical signals such as speech and music, as opposed to the uncorrelated time-frequency behavior of the broadband noise. An example of the time-frequency behavior of a noisy speech signal is depicted in
Several notable and typical features are shown in
As discussed above, the present invention utilizes two separate 2-D filters. The furrow filter preferentially selects spectral components that are narrow in frequency but relatively broad in time (corresponding to discrete spectral partials, as found in voiced speech and other quasi-periodic signals), while the bar filter passes spectral components that are broad in frequency but relatively narrow in time (corresponding to plosive and fricative consonants found in speech signals). This 2-D filter arrangement is depicted in
For each block, the data is multiplied by a suitable smoothly tapered window function 110 to avoid the truncation effects of an abrupt (rectangular) window, and passed through a fast Fourier transform (“FFT”) 120. The FFT computes the complex discrete Fourier transform of each windowed data block. The FFT length can be equal to the block length, or optionally the windowed data can be zero-padded to a longer block length if more spectral samples are desired.
The blocks of raw FFT data 130 are stored in queue 140 containing the current and a plurality of past FFT blocks. The queue is a time-ordered sequence of FFT blocks that is used in the two-dimensional furrow and bar filtering process, as described below. The number of blocks stored in queue 140 is chosen to be sufficiently long for the two-dimensional furrow and bar filtering,
Simultaneously, the FFT data blocks 130 are sent through magnitude computation 150, which entails computing the magnitude of each complex FFT sample. The FFT magnitude blocks are stored in queue 200 and form a sequence of spectral “snapshots,” ordered in time, with the spectral information of each FFT magnitude block forming the dependent variable.
The two-dimensional (complex FFT spectra vs. time) raw data in queue 140 is processed by the two-dimensional filters 160 (long furrow), 170 (short furrow), 180 (short bar), and 190 (long bar), yielding filtered two-dimensional data 230, 240, 250, and 260, respectively.
Evaluation block 210 processes the FFT magnitude data from queue 200 and the filtered two-dimensional data 230, 240, 250, and 260, to determine the current condition of the input signal. In the case of speech input, the evaluation includes an estimate of whether the input signal contains voiced or unvoiced (fricative), whether the signal is in the steady-state or undergoing a transition from voiced to unvoiced or from unvoiced to voiced, whether the signal shows a transition to or from a noise-only segment, and similar calculations that interpret the input signal conditions. For example, a steady-state voiced speech condition could be indicated by harmonics in the FFT magnitude data 200 and more signal power present in the long furrow filter output 230 than in the short bar filter output 250.
The evaluation results are used in the filter weighting calculation 220 to generate mixing control weights 270, 280, 290, and 300, which are each scalar quantities between zero and one. The control weights 270, 280, 290, and 300 are sent to multipliers 310, 320, 330, and 340, respectively, to adjust the proportion of the two-dimensional output data 230, 240, 250, and 260 that are additively combined in summer 350 to create the composite filtered output FFT data 360. The control weights select a mixture of the four filtered versions of the signal data such that the proper signal characteristics are recovered from the noisy signal. The control weights 270, 280, 290, and 300 are calculated such that their sum is equal to or less than one. If the evaluation block 210 detects a transition from one signal state to another, the control weights are switched in smooth steps to avoid abrupt discontinuities in the output signal.
The composite filtered output FFT data blocks 360 are sent through inverse FFT block 370, and the resulting inverse FFT blocks are overlapped and added in block 380, thereby creating the noise-reduced output signal 390.
Despite the fact that the above discussion focuses on the reduction or elimination of noise from analog signals, the present invention can also be applied to a signal that has already been digitized (like a .wav or .aiff file of a music recording that happens to contain noise). In that case, it is not necessary to perform the analog-to-digital conversion. Because the processing of the present invention is performed on a digitized signal, the present invention is not dependent on an analog-to-digital conversion.
E. Practical Applications
In contrast to the prior art methods for noise reduction and signal enhancement, the filter technology of the present invention effectively removes broadband noise (or static) from analog signals while maintaining as much of the desired signal as possible. The present invention can be used in connection with AM radio, particularly for talk radio and sports radio, and especially in moving vehicles or elsewhere when the received signal is of low or variable quality. The present invention can also be applied in connection with shortwave radio, broadcast analog television audio, cell phones, and headsets used in high-noise environments like tactical applications, aviation, fire and rescue, police and manufacturing.
The problem of a low signal-to-noise ratio is particularly acute in the area of AM radio. Analog radio broadcasting uses two principal methods: amplitude modulation (AM) and frequency modulation (FM). Both techniques take the audio signal (speech, music, etc.) and shift its frequency content from the audible frequency range (0 to 20 kHz) to a much higher frequency that can be transmitted efficiently as an electromagnetic wave using a power transmitter and antenna. The radio receiver reverses the process and shifts the high frequency radio signal back down to the audible frequency range so that the listener can hear it. By assigning each different radio station to a separate channel (non-overlapping high frequency range), it is possible to have many stations broadcasting simultaneously. The radio receiver can select the desired channel by tuning to the assigned frequency range.
Amplitude modulation (AM) means that the radio wave power at the transmitter is rapidly made larger and smaller (“modulated”) in proportion to the audio signal being transmitted. The amplitude of the radio wave conveys the audio program; therefore, the receiver can be a very simple radio frequency envelope detector. The fact that the instantaneous amplitude of the radio wave represents the audio signal means that any unavoidable electromagnetic noise or interference that enters the radio receiver causes an error (audible noise) in the received audio signal. Electromagnetic noise may be caused by lightning or by a variety of electrical components such as computers, power lines, and automobile electrical systems. This problem is especially noticeable when the receiver is located in an area far from the transmitter because the received signal will often be relatively weak compared to the ambient electromagnetic noise, thus creating a low signal-to-noise-ratio condition.
Frequency modulation (FM) means that the instantaneous frequency of the radio wave is rapidly shifted higher and lower in proportion to the audio signal to be transmitted. The frequency deviation of the radio signal conveys the audio program. Unlike AM, the FM broadcast signal amplitude is relatively constant while transmitting, and the receiver is able to recover the desired frequency variations while effectively ignoring the amplitude fluctuations due to electromagnetic noise and interference. Thus, FM broadcast receivers generally have less audible noise than AM radio receivers.
It should be clear to those skilled in the art of digital signal processing that there are many similar methods and processing rule modifications that can be envisaged without altering the key concept of this invention, namely, the use of a 2-D filter model to separate and enhance the desired signal components from those of the noise. Although a preferred embodiment of the present invention has been shown and described, it will be apparent to those skilled in the art that many changes and modifications may be made without departing from the invention in its broader aspects. The appended claims are therefore intended to cover all such changes and modifications as fall within the true spirit and scope of the invention.
The term “amplitude” means the maximum absolute value attained by the disturbance of a wave or by any quantity that varies periodically. In the context of audio signals, the term “amplitude” is associated with volume.
The term “demodulate” means to recover the modulating wave from a modulated carrier.
The term “frequency” means the number of cycles completed by a periodic quantity in a unit time. In the context of audio signals, the term “frequency” is associated with pitch.
The term “fricative” means a primary type of speech sound of the major languages that is produced by a partial constriction along the vocal tract which results in turbulence; for example, the fricatives in English may be illustrated by the initial and final consonants in the words vase, this, faith and hash.
The term “hertz” means a unit of frequency or cycle per second.
The term “Hz” is an abbreviation for “hertz.”
The term “kHz” is an abbreviation for “kilohertz.”
The term “modulate” means to vary the amplitude, frequency, or phase of a wave, or vary the velocity of the electrons in an electron beam in some characteristic manner.
The term “modulated carrier” means a radio-frequency carrier wave whose amplitude, phase, or frequency has been varied according to the intelligence to be conveyed.
The term “phoneme” means a speech sound that is contrastive, that is, perceived as being different from all other speech sounds.
The term “plosive” means a primary type of speech sound of the major languages that is characterized by the complete interception of airflow at one or more places along the vocal tract. For example, the English words par, bar, tar, and car begin with plosives.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US4074069 *||Jun 1, 1976||Feb 14, 1978||Nippon Telegraph & Telephone Public Corporation||Method and apparatus for judging voiced and unvoiced conditions of speech signal|
|US4701953||Jul 24, 1984||Oct 20, 1987||The Regents Of The University Of California||Signal compression system|
|US4736432||Dec 9, 1985||Apr 5, 1988||Motorola Inc.||Electronic siren audio notch filter for transmitters|
|US5377277||Nov 17, 1993||Dec 27, 1994||Bisping; Rudolf||Process for controlling the signal-to-noise ratio in noisy sound recordings|
|US5432859||Feb 23, 1993||Jul 11, 1995||Novatel Communications Ltd.||Noise-reduction system|
|US5459814||Mar 26, 1993||Oct 17, 1995||Hughes Aircraft Company||Voice activity detector for speech signals in variable background noise|
|US5566103||Aug 1, 1994||Oct 15, 1996||Hyatt; Gilbert P.||Optical system having an analog image memory, an analog refresh circuit, and analog converters|
|US5615142||May 2, 1995||Mar 25, 1997||Hyatt; Gilbert P.||Analog memory system storing and communicating frequency domain information|
|US5649055||Sep 29, 1995||Jul 15, 1997||Hughes Electronics||Voice activity detector for speech signals in variable background noise|
|US5706395||Apr 19, 1995||Jan 6, 1998||Texas Instruments Incorporated||Adaptive weiner filtering using a dynamic suppression factor|
|US5742694||Jul 12, 1996||Apr 21, 1998||Eatwell; Graham P.||Noise reduction filter|
|US5794187||Jul 16, 1996||Aug 11, 1998||Audiological Engineering Corporation||Method and apparatus for improving effective signal to noise ratios in hearing aids and other communication systems used in noisy environments without loss of spectral information|
|US5859878||Aug 31, 1995||Jan 12, 1999||Northrop Grumman Corporation||Common receive module for a programmable digital radio|
|US5909193||Aug 31, 1995||Jun 1, 1999||Northrop Grumman Corporation||Digitally programmable radio modules for navigation systems|
|US5930687||Sep 30, 1996||Jul 27, 1999||Usa Digital Radio Partners, L.P.||Apparatus and method for generating an AM-compatible digital broadcast waveform|
|US5950151 *||Feb 12, 1996||Sep 7, 1999||Lucent Technologies Inc.||Methods for implementing non-uniform filters|
|US5963899||Aug 7, 1996||Oct 5, 1999||U S West, Inc.||Method and system for region based filtering of speech|
|US6001131||Feb 24, 1995||Dec 14, 1999||Nynex Science & Technology, Inc.||Automatic target noise cancellation for speech enhancement|
|US6072994||Aug 31, 1995||Jun 6, 2000||Northrop Grumman Corporation||Digitally programmable multifunction radio system architecture|
|US6091824||Sep 26, 1997||Jul 18, 2000||Crystal Semiconductor Corporation||Reduced-memory early reflection and reverberation simulator and method|
|US6097820||Dec 23, 1996||Aug 1, 2000||Lucent Technologies Inc.||System and method for suppressing noise in digitally represented voice signals|
|US6115689 *||May 27, 1998||Sep 5, 2000||Microsoft Corporation||Scalable audio coder and decoder|
|US6157908||Jan 27, 1998||Dec 5, 2000||Hm Electronics, Inc.||Order point communication system and method|
|US6182035 *||Mar 26, 1998||Jan 30, 2001||Telefonaktiebolaget Lm Ericsson (Publ)||Method and apparatus for detecting voice activity|
|US6249757||Feb 16, 1999||Jun 19, 2001||3Com Corporation||System for detecting voice activity|
|US6263307||Apr 19, 1995||Jul 17, 2001||Texas Instruments Incorporated||Adaptive weiner filtering using line spectral frequencies|
|US6351731||Aug 10, 1999||Feb 26, 2002||Polycom, Inc.||Adaptive filter featuring spectral gain smoothing and variable noise multiplier for noise reduction, and method therefor|
|US6363345||Feb 18, 1999||Mar 26, 2002||Andrea Electronics Corporation||System, method and apparatus for cancelling noise|
|US6415253||Feb 19, 1999||Jul 2, 2002||Meta-C Corporation||Method and apparatus for enhancing noise-corrupted speech|
|US6424942||Oct 25, 1999||Jul 23, 2002||Telefonaktiebolaget Lm Ericsson (Publ)||Methods and arrangements in a telecommunications system|
|US6453285||Aug 10, 1999||Sep 17, 2002||Polycom, Inc.||Speech activity detector for use in noise reduction system, and methods therefor|
|US6480610||Sep 21, 1999||Nov 12, 2002||Sonic Innovations, Inc.||Subband acoustic feedback cancellation in hearing aids|
|US6493689||Dec 29, 2000||Dec 10, 2002||General Dynamics Advanced Technology Systems, Inc.||Neural net controller for noise and vibration reduction|
|US6512555||Aug 16, 1999||Jan 28, 2003||Samsung Electronics Co., Ltd.||Radio receiver for vestigal-sideband amplitude-modulation digital television signals|
|US6591234||Jan 7, 2000||Jul 8, 2003||Tellabs Operations, Inc.||Method and apparatus for adaptively suppressing noise|
|US6661837||Mar 8, 1999||Dec 9, 2003||International Business Machines Corporation||Modems, methods, and computer program products for selecting an optimum data rate using error signals representing the difference between the output of an equalizer and the output of a slicer or detector|
|US6661847||Oct 29, 1999||Dec 9, 2003||International Business Machines Corporation||Systems methods and computer program products for generating and optimizing signal constellations|
|US6694029||Sep 14, 2001||Feb 17, 2004||Fender Musical Instruments Corporation||Unobtrusive removal of periodic noise|
|US6718306 *||Oct 17, 2000||Apr 6, 2004||Casio Computer Co., Ltd.||Speech collating apparatus and speech collating method|
|US6745155 *||Nov 6, 2000||Jun 1, 2004||Huq Speech Technologies B.V.||Methods and apparatuses for signal analysis|
|US6751602||Nov 5, 2002||Jun 15, 2004||General Dynamics Advanced Information Systems, Inc.||Neural net controller for noise and vibration reduction|
|US6757395||Jan 12, 2000||Jun 29, 2004||Sonic Innovations, Inc.||Noise reduction apparatus and method|
|US6804640||Feb 29, 2000||Oct 12, 2004||Nuance Communications||Signal noise reduction using magnitude-domain spectral subtraction|
|US6859540||Jul 28, 1998||Feb 22, 2005||Pioneer Electronic Corporation||Noise reduction system for an audio system|
|US6862558 *||Feb 13, 2002||Mar 1, 2005||The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration||Empirical mode decomposition for analyzing acoustical signals|
|US6910011 *||Aug 16, 1999||Jun 21, 2005||Haman Becker Automotive Systems - Wavemakers, Inc.||Noisy acoustic signal enhancement|
|US7233899 *||Mar 7, 2002||Jun 19, 2007||Fain Vitaliy S||Speech recognition system using normalized voiced segment spectrogram analysis|
|US7243060 *||Apr 2, 2003||Jul 10, 2007||University Of Washington||Single channel sound separation|
|US7574352 *||Sep 13, 2002||Aug 11, 2009||Massachusetts Institute Of Technology||2-D processing of speech|
|US20020055839 *||Sep 11, 2001||May 9, 2002||Michihiro Jinnai||Method for detecting similarity between standard information and input information and method for judging the input information by use of detected result of the similarity|
|US20030187637||Mar 29, 2002||Oct 2, 2003||At&T||Automatic feature compensation based on decomposition of speech and noise|
|US20030194002||Apr 15, 2002||Oct 16, 2003||Corless Mark W.||Run-time coefficient generation for digital filter with slewing bandwidth|
|US20030195910||Apr 15, 2002||Oct 16, 2003||Corless Mark W.||Method of designing polynomials for controlling the slewing of adaptive digital filters|
|US20040002852 *||Jul 1, 2002||Jan 1, 2004||Kim Doh-Suk||Auditory-articulatory analysis for speech quality assessment|
|US20040054527 *||Sep 13, 2002||Mar 18, 2004||Massachusetts Institute Of Technology||2-D processing of speech|
|US20050123150 *||Feb 3, 2003||Jun 9, 2005||Betts David A.||Method and apparatus for audio signal processing|
|US20060074642 *||Jan 4, 2005||Apr 6, 2006||Digital Rise Technology Co., Ltd.||Apparatus and methods for multichannel digital audio coding|
|1||*||A. Drygajlo and B. Carnero, "Integrated speech enhancement and coding in time-frequency domain," in ICASSP'97, (Munich, Germany), pp. 1183-1186, Apr. 1997.|
|2||*||D.A. Heide, G.S. Kang, Speech enhancement for bandlimited speech, in: Proceedings of the ICASSP, vol. 1, Seattle, WA, USA, May 1998, pp. 393-396.|
|3||*||D.M. Nadeu, and J. Hernando. Time and frequency filtering of filter-bank energies for robust HMM speech recognition, Speech Communication, vol. 34, No. 1-2, pp. 93-114, Apr. 2001.|
|4||*||Drullman, R., Festen, J.M., Plomp, R.,1994. Effect of temporal envelope smearing on speech reception. J. Acoust. Soc. Amer. 95, 1053-1064.|
|5||*||G. Whipple, "Low residual noise speech enhancement utilizing time-frequency filtering," Proc. ICASSP'94, pp. I-5/I-8.|
|6||*||H. Kirchauer, F. Hlawatsch, and W. Kozek, "Time-frequency formulation and design of nonstationaryWiener filters," in Proc. ICASSP, 1995, pp. 1549-1552.|
|7||*||J. J D. Gibson, B. Koo, and S. D. Gray, "Filtering of colored noise for speech enhancement and coding," IEEE Trans. Acoust., Speech, Signal Processing, vol. 39, pp. 1732-1742, 1991.|
|8||*||J. L. Shen, J. W. Hung, and L. S. Lee, "Robust entropy-based endpoint detection for speech recognition in noisy environments," presented at the ICSLP, 1998.|
|9||Jae S. Lim, and Alan V. Oppenheim, Enhancement and Bandwith Compresion of Noisy Speech., Proceedings of the IEEE, Dec. 1979, pp. 1586-1604, vol. 67, No. 12., Institute of Electrical and Electronics Engineers, Inc., Piscataway, NJ.|
|10||*||Kawahara, H., Masuda-Katsuse, I., and de Cheveigné, A. (1999b). "Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequency-based F0 extraction: Possible role of a repetitive structure in sounds," Speech Commun. 27, 187-207.|
|11||*||Kingsbury et al., 1998 B.E.D Kingsbury, N Morgan and S Greenberg, Robust speech recognition using the modulation spectrogram, Speech Communication 25 (1998), pp. 117-132.|
|12||L.R. Rabiner, and R.W. Schafer, Digital Processing of Speech Signals, 1978, pp. 1-54, Prentice-Hall, Inc., Englewood Cliffs, NJ.|
|13||*||Lin and Goubran, 2003 Lin, Z., Goubran, R.A., 2003. Musical noise reduction in speech using two-dimensional spectrogram enhancement. In: Proc. 2nd IEEE Internat. Workshop on Haptic, Audio and Visual Environments and Their Applications, pp. 61-64.|
|14||*||Macho, D., Nadeu, C., Hernando, J., Padrell, J., 1999a. Time and frequency filtering for speech recognition in real noise conditions. In: Proceedings of the Workshop on Robust Methods for Speech Recognition in Adverse Conditions, Tampere, pp. 111-114.|
|15||Mark Kahrs, and Karheinz Brandenburg, eds., Applications of Digital Signal Processing to Audio and Acoustics, 1998, Kluwer Academic Publishers Group, Norwell, MA.|
|16||Mark R. Weiss, and Ernest Aschkenasy, Widebamd Speech Enhancement (Addition, Final Tech. Rep, RADC-TR-81-53, DTIC ADA 100462, May 1981, Rome Air Development Center, Griffiss Air Force Base, NY.|
|17||Pavan K. Ramarapu, and Robert C. Maher, Methods for Reducing Audible Artifacts in a Wavelet-Based Broad-Band Denoising System, J. Audio Eng. Soc., Mar. 1998, pp. 178-189, vol. 46, No. 3., Audio Engineering Society, Inc., New York, NY.|
|18||Robert C. Maher, A Method for Extrapolation of Missing Digital Audio Data, J. Audio Eng. Soc., May 1994, pp. 350-357, vol. 42, No. 5., Audio Engineering Society, Inc., New York, NY.|
|19||Robert C. Maher, Digital Mathods for Noise Removal and Quality Enhancement of Audio Signals, Seminar Presentaion, Creative, Advanced Technology Center Scotts Valley, CA, Apr. 2, pp. 1-194.|
|20||Robert J. McAulay and Marilyn L. Malpass, Speech Enhancement Using a Soft-Decision Noise Suppression Filter, IEEE Transactions on Accoustics, Speech, and Signal processing, Apr. 1980, pp. 137-145, vol. ASSP-28, No. 2., Institute of Electrical and Electronics Engineers, Inc., Piscataway, NJ.|
|21||*||S. E. Bou-Ghazale and K. Assaleh, "A robust endpoint detection of speech for noisy environments with application to automatic speech recognition," in Proc. IEEE Int. Conf. Acoust. Speech, Signal Process., May 2002, pp. 3808-3811.|
|22||Steven F. Boll, Suppression of acoustic noise in speech using spectral subtraction, IEEE Transactions on Accoustics, Speech, and Signal processing, Apr. 1, 1979, pp. 113-120, vol. ASSP-27, No. 2., Institute of Electrical and Electronics Engineers, Inc., Piscataway, NJ.|
|23||*||T. F. Quatieri and R. B. Dunn, "Speech enhancement based on auditory spectral change," in Proc. IEEE ICASSP, vol. 1, May 2002, pp. 257-260.|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US8000962 *||May 19, 2006||Aug 16, 2011||Nuance Communications, Inc.||Method and system for using input signal quality in speech recognition|
|US8190430||Aug 9, 2011||May 29, 2012||Nuance Communications, Inc.||Method and system for using input signal quality in speech recognition|
|US8280727 *||May 11, 2010||Oct 2, 2012||Fujitsu Limited||Voice band expansion device, voice band expansion method, and communication apparatus|
|US8280731 *||Mar 14, 2008||Oct 2, 2012||Dolby Laboratories Licensing Corporation||Noise variance estimator for speech enhancement|
|US8438030 *||Nov 25, 2009||May 7, 2013||General Motors Llc||Automated distortion classification|
|US8554552 *||Oct 30, 2009||Oct 8, 2013||Samsung Electronics Co., Ltd.||Apparatus and method for restoring voice|
|US8676574 *||Nov 10, 2010||Mar 18, 2014||Sony Computer Entertainment Inc.||Method for tone/intonation recognition using auditory attention cues|
|US8756061 *||Apr 1, 2011||Jun 17, 2014||Sony Computer Entertainment Inc.||Speech syllable/vowel/phone boundary detection using auditory attention cues|
|US8793126 *||Apr 14, 2011||Jul 29, 2014||Huawei Technologies Co., Ltd.||Time/frequency two dimension post-processing|
|US8874390 *||Mar 23, 2011||Oct 28, 2014||Hach Company||Instrument and method for processing a doppler measurement signal|
|US9020822||Oct 19, 2012||Apr 28, 2015||Sony Computer Entertainment Inc.||Emotion recognition using auditory attention cues extracted from users voice|
|US9031293||Oct 19, 2012||May 12, 2015||Sony Computer Entertainment Inc.||Multi-modal sensor based emotion recognition and emotional interface|
|US9159325 *||Dec 31, 2007||Oct 13, 2015||Adobe Systems Incorporated||Pitch shifting frequencies|
|US9251783||Jun 17, 2014||Feb 2, 2016||Sony Computer Entertainment Inc.||Speech syllable/vowel/phone boundary detection using auditory attention cues|
|US20060265223 *||May 19, 2006||Nov 23, 2006||International Business Machines Corporation||Method and system for using input signal quality in speech recognition|
|US20100100386 *||Mar 14, 2008||Apr 22, 2010||Dolby Laboratories Licensing Corporation||Noise Variance Estimator for Speech Enhancement|
|US20100114570 *||Oct 30, 2009||May 6, 2010||Jeong Jae-Hoon||Apparatus and method for restoring voice|
|US20100318350 *||May 11, 2010||Dec 16, 2010||Fujitsu Limited||Voice band expansion device, voice band expansion method, and communication apparatus|
|US20110125500 *||May 26, 2011||General Motors Llc||Automated distortion classification|
|US20110257979 *||Oct 20, 2011||Huawei Technologies Co., Ltd.||Time/Frequency Two Dimension Post-processing|
|US20120116756 *||Nov 10, 2010||May 10, 2012||Sony Computer Entertainment Inc.||Method for tone/intonation recognition using auditory attention cues|
|US20120245863 *||Sep 27, 2012||Hach Company||Instrument and method for processing a doppler measurement signal|
|US20120253812 *||Apr 1, 2011||Oct 4, 2012||Sony Computer Entertainment Inc.||Speech syllable/vowel/phone boundary detection using auditory attention cues|
|U.S. Classification||704/205, 704/E21.01, 704/E19.046, 704/E21.009, 704/E15.006, 704/E21.002, 704/248, 704/208|
|Cooperative Classification||G10L21/0208, G10L21/0232|
|Aug 11, 2005||AS||Assignment|
Owner name: KOSEK, DANIEL A.,MONTANA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MAHER, ROBERT CRAWFORD;REEL/FRAME:016391/0987
Effective date: 20050809
|Dec 20, 2013||FPAY||Fee payment|
Year of fee payment: 4