|Publication number||US6741960 B2|
|Application number||US 09/751,302|
|Publication date||May 25, 2004|
|Filing date||Dec 28, 2000|
|Priority date||Sep 19, 2000|
|Also published as||US20020052736|
|Publication number||09751302, 751302, US 6741960 B2, US 6741960B2, US-B2-6741960, US6741960 B2, US6741960B2|
|Inventors||Hyoung Jung Kim, In Sung Lee, Jong Hark Kim, Man Ho Park, Byung Sik Yoon, Song In Choi, Dae Sik Kim|
|Original Assignee||Electronics And Telecommunications Research Institute|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (8), Non-Patent Citations (3), Referenced by (39), Classifications (8), Legal Events (6)|
|External Links: USPTO, USPTO Assignment, Espacenet|
The present invention relates to a speech coding and more particularly to the speech coder and coding method using harmonic-noise speech coding algorithm capable of achieving more improved speech quality by using cepstrum analysis method and LPC (Linear Prediction Coefficient) analysis method for the mixed signal of voiced/unvoiced sound which is not represented well in the generally used harmonic coding algorithm.
As harmonic model is generally based on sinusoidal analysis and synthesis in the low rate speech coder, noise component with non-stagnant characteristic is not represented well. Therefore, the method for modeling noise component observed in the spectrum of real speech signal has been required.
In order to meet these demands, the research for the harmonic speech coding model such as MELP (Mixed Excitation Linear Prediction) algorithm or MBE (Mixed Band Excitation) algorithm which are known as guaranteeing good speech quality, has been progressed, in which the characteristic of said algorithms is that speech can be observed by dividing the speech into each band and then analyzing it.
However, said algorithms analyze with fixed bandwidth the sound in which voiced/unvoiced sound signal is multiply mixed. And due to the binary decision structure, which is deciding voiced/unvoiced sound at each band, also have limitation on effective representation. And particularly, in the case that voiced/unvoiced sounds are mixed simultaneously or the mixed signal is distributed on the band border, there is a disadvantage that the spectral distortion is occurred.
These disadvantages are caused by using the signal modeling method utilizing only the frequency peak value of the harmonic model in the mixed signal of voiced/unvoiced sound. These cases are caused by insufficient representation of the mixed signal of voiced/unvoiced sound of the low rate model. Recently, in order to solve these disadvantages, the studies on the coding methods for the mixed signal of voiced/unvoiced sound have been actively progressed.
The object of coding for mixed signal of voiced/unvoiced sound is to represent effectively voiced sound spectral part and unvoiced sound spectral part in frequency domain. And there are two coding methods in recent analysis method. The first coding method is dividing into two parts of voiced/unvoiced bands after defining frequency transitional point and the second coding method is differing mixing level of voice/unvoiced sound during synthesis after defining probability value of voiced sound from total spectral information.
As an example of the second coding method, there is U.S. Pat. No. 5,774,837 entitled “Speech Coding System And Method Using Voicing Probability Determination” written by Suat Yeldener and Joseph Gerard Aguilar. The patent described the technology which, in order to analyze and synthesize the mixed signal of voiced/unvoiced sound, analyzes the spectral of voiced sound and modified linear prediction parameter of unvoiced sound and by using the analyzed results synthesizes the mixed signal according to the degree of the probability value of the voiced sound computed from the pitch and parameter which are extracted from the spectrum of the inputted speech signal.
However, above mentioned prior art or technologies extract unvoiced sound by dividing spectral of the mixed signal of voiced/unvoiced sound into two sections and as the analysis and synthesis of the in putted speech signal are based on the probability value, it is impossible to do sound analysis and synthesis effectively through real spectral values of all sections.
According to a first aspect of the present invention, a harmonic-noise speech coder of the mixed signal of voiced/unvoiced sound using harmonic model is provided. The harmonic-noise coder comprises a noise spectral estimating means for coding the noise component by predicting the spectral by LPC analysis method after separating the noise component which is unvoiced sound component from the inputted LPC residual signal using cepstrum.
Also, according to a second aspect of the present invention, a harmonic-noise speech coding method of the mixed signal of voiced/unvoiced sound includes the following step: A harmonic coding step for coding voiced sound out of the mixed signal; And noise coding step for coding unvoiced sound out of the mixed signal. Preferably, the noise coding step is composed of a cepstrum analyzing step for extracting noise spectral envelope by cepstrum analyzing the mixed signal and an LPC analyzing step for extracting noise spectral envelope information from the extracted spectrum.
The embodiments of the present invention will be explained with reference to the accompanying drawings, in which:
FIG. 1 is a drawing illustrating the total block diagram of the harmonic-noise speech coder 100.
FIG. 2 is a drawing illustrating the block diagram of the harmonic coder 200 illustrated in said FIG. 1 for voiced sound component.
FIG. 3 is a drawing illustrating the all procedures for obtaining LPC parameter through cepstral-LPC noise spectral estimator.
Referring to accompanied drawings, other advantages and effects of the present invention can be more clearly understood through desirably executable examples of the coders being explained.
As described above, the present invention is related to a noise spectral estimator combining ceptrum analysis method and LPC analysis method in order to code the mixed signal of voiced/unvoiced sound and harmonic-noise speech coding combined with harmonic model.
Simply referencing the coding method according to the present invention, the noise spectral is estimated by LPC analysis method after separating the noise region using cepstrum. The estimated noise spectral is parameterized into LP coefficients.
For the mixed signal of voiced/unvoiced sound, the voiced sound uses harmonic coder and the unvoiced sound uses ceptrum LPC noise coder.
The synthesized excitation signal is obtained by adding the voiced sound which is synthesized by harmonic synthesizer and unvoiced sound component, noise which is synthesized through LPC synthesis filter.
First, referring to FIG. 1, the total block diagram of the harmonic-noise speech coder 100 is illustrated.
As shown in FIG. 1, the coder 100 according to the present invention is composed of a harmonic coder 200 and a noise coder 300 in order to code the mixed signal of voiced/unvoiced sound. The LPC residual signals become the input signal of said harmonic coder 200 and said noise coder 300 respectively.
Especially, in order to estimate the noise spectral, uses ceptrum and LPC analysis method while the open loop pitch value being input of said noise coder 300. The open loop pitch value is used as common input to said harmonic coder 200.
The other components illustrated in FIG. 1 will be referred through the detailed description of the present invention.
Referring to FIG. 2, the block diagram of the harmonic coder 200 illustrated in said FIG. 1 for voiced sound component is illustrated.
The general coding procedure of said harmonic coder 200 used in the coding method according to the present invention is described as follows. First, the LPC residual signal, the input signal is passed through the hamming window and the corrected pitch value and harmonic magnitude are extracted through the analysis of the spectrum of frequency domain. The synthesis procedure is progressed to the step for synthesizing the representative waveform of each frame obtained from Inverse Fast Fourier Transform (IFFT) waveform synthesis by overlap/add method.
From now on the extracting method of each parameter is described through more detailed description of fundamental theory.
The object of the harmonic model is LPC residual signal and the finally extracted parameters are the magnitude of the spectrum and the close loop pitch value ωo.
More concretely, the representation of the excitation signal, namely the LPC residual signal, passes detailed coding procedure on the basis of sinusoidal waveform model as following Equation 1.
Where A1 and ψ1 represent magnitude and phase of sinusoidal wave component with frequency ω1 respectively. L represents the number of sinusoidal wave. As the harmonic portion includes most of the information of the speech signal in the excitation signal of the voiced sound section, it can be approximated using appropriate spectral fundamental model.
Following Equation 2 represents the approximated model with linear phase synthesis.
Where k and Lk represent frame number and the number of harmonics of each frame respectively, ωo represents the angular frequency of the pitch, and Φk l represents the discrete phase of the kth frame and the lth harmonic.
The Ak l representing the magnitude of the kth frame harmonic is the information transmitted to the decoder, and by making the value being applied 256 DFT (Discrete Fourier Transform) of the Hamming Window to be reference model. The spectral and pitch parameter value making the value of the following Equation 3 to be minimized is determined by closed loop searching method.
Where, X(j) and B(j) represent the DFT value of the original LPC residual signal and the DFT value of the 256-point hamming window respectively, and am and bm represent the DFT indexes of the start and end the mth harmonic. X(i) means the spectral reference model.
Each parameter analyzed is used for synthesis and the method of the phase synthesis method uses general linear phase ψk(l,ω0 k−1,n) synthesis method like following Equation 4.
The linear phase is obtained by linearly interpolating the fundamental frequency according to the time of the previous fame and the present frame. Generally, the hearing sense system of man is assumed to be non-sensitive to the linear phase and to permit inaccurate or totally different discrete phase while phase continuity is preserved.
These perceptible characteristics of a man are important condition for the continuity of the harmonic model in low rate coding method. Therefore, the synthesis phase can substitute the measured phase.
These harmonic synthesis models can be implemented by the existing IFFT synthesis method and the procedure is as follows.
In order to synthesize the reference waveform, the harmonic magnitudes are extracted through inverse quantization procedure in the spectral parameter.
The phase information corresponding to each harmonic magnitude is made by using the linear phase synthesis method and then the reference waveform is made through 128-point IFFT. As the reference waveform does not include the pitch information, reformed to the circular format and then final excitation signal is obtained by sampling after interpolating to the over-sampling ratio obtained from the pitch period considering the pitch variation.
In order to guarantee the continuity between frames, the start position defined as offset is defined as following Equation 5.
Above equations represent over-sampling rate ov and sampling position Pov[n] respectively. Where N, Tp, l and k represent frame length, pitch period, number of harmonics and frame number, respectively. L means the number of over-sampled data in order to recover N samples and mod(x, y) returns the residual value after dividing x by y. Also, wrk(l) and wk(l) represent kth circular waveform and the kth reference waveform, respectively.
On the other hand, the effective modeling of the noise spectral used in the coding method according to the present invention is composed of the structure predicting noise component using cepstrum and LPC analysis method. Referring to FIG. 3, the procedure is described in detail.
The speech signal can be assumed as the model composed of several filters by analyzing the pronouncing structure of man.
In the desirably executable example according to the present invention the assumption as following Equation 6 is made, in order to obtain the noise region.
Where, s(t) is the speech signal, h(t) is the impulse response of vocal track and e(t) is excitation signal. v(t) and u(t) mean the pseudo period and the period portion of the excitation signal, respectively.
As shown in Equation 6, the speech signal can be represented as convolution of the excitation signal and the impulse response of the vocal track. The excitation signal is divided into the periodic signal and aperiodic signal. Herein the periodic signal means the voiceprint pulse train of the pitch period and the aperiodic signal means the noise-like signal by the radiation from lip or the air-flow from lung.
The Equation 6 can be transformed to the spectral region and can be represented as following Equation 7.
Where, S(w), U(w), V(w) and H(w) means the Fourier Transfer Function of s(t), u(t), v(t) and h(t) respectively. From the Equation 7, applying logarithmic arithmetic and IDFT can be represented as following Equation 8 and Equation 9 in order to obtain the cepstral coefficient.
log|S(ω)|=log|V(ω)+U(ω)|+log|H(ω)| [Equation 8]
The cepstrum obtained from said Equation 9 can concrete the voiced sound portion to three separated domains. The quefrency region, as the neighboring values of the cepstral peak in the pitch period is the portion caused by the harmonic component those can be assumed as the periodic voiced sound component. Also the high quefrency region of the right side of the peak value can be assumed as what caused mainly by noise excitation component. Finally, the low quefrency region of the left side of the peak value can be assumed as the component caused by the vocal track.
Here, the positive and negative magnitude values can be observed by transforming the cepstrum value neighboring the pitch by the harmonic component to the logarithmic spectrum domain after liftering them as many as the number of the experimental samples. The negative magnitude values become the valley portion of the mixed signal.
In reality, the harmonic components out of the spectrum of the mixed signal concentrate on the multiple of the pitch frequency and the noise components are added to the harmonic components in the mixed format. Therefore, while it is difficult to separate the aperiodic components of the neighborhood of the frequencies corresponding to the multiple of the pitch frequency, it is feasible to separate the noise component in the valley portion between the frequencies corresponding to the multiple of the pitch frequencies.
By the reason, the magnitude spectrum of the excitation signal focuses on the negative logarithmic magnitude spectrum of the extracted cepstrum.
In the coding method according to the present invention, the components of the valley portion, which is a part of the noise spectral envelope are extracted by using the cepstrum analysis method. Concretely, the spectral valley portion of the mixed signal is extracted by applying rectangular window as much as the negative region of the logarithmic magnitude extracted in the neighborhood of the pitch period.
Next, the LPC analysis method is applied to the extracted partial noise spectral components in order to predict the noise component in the harmonic region. As this is equal to the method for extracting the spectral envelope of the speech signal, it can be considered as the prediction method for estimating the noise spectral within the harmonic region.
Concretely, the extracted noise spectrum is transformed to the signal information of time axis by applying the IDFT and then the 6th LPC analysis procedure is performed in order to extract the spectral information. The extracted 6th LPC parameter is converted to the LSP parameter in order to increase the quantization effectiveness. Herein the 6th is the empirical value according to the research result of the present invention, which considered the degree of dispersion of the allocation bit and the noise spectrum component according to the low rate and the phase of the input signal is used as the phase in IDFT.
The total procedure for obtaining the LPC parameter through the cepstral-LPC noise spectral predictor is illustrated in FIG. 3.
The cepstral-LPC noise spectral predictor shown in FIG. 3 according to present invention comprises a noise coding section 310 for extracting to code unvoiced sound among the mixed signals inputted, and a gain calculating section 320 for calculating a gain value of noise component.
From the structure shown in FIG. 3, the buzz sound following low rate can be reduced and the coefficient obtained from the LPC analysis method what is called all-poll fitting can be transformed to the LSP. As various researches about said LSP is being developed now, in the coding method according to the present invention the effective quantization structure can be achieved by selecting appropriate method out of the LSP methods.
Meanwhile, the procedure for computing the gain value of the noise component excepting the information representing the spectral envelope is needed and the gain value is obtained from the ratio of the input signal and the LPC synthesis signal which is using the inversely quantizied 6th LPC value and the gaussian noise as input.
Herein, the gaussian noise is equal to the generation pattern of the gaussian noise of the speech synthesis stage and the quantization to the logarithmic scale is appropriate.
The noise spectral parameters obtained by the method are transmitted to the speech synthesis stage with the gain parameter and the spectral magnitude parameter of the harmonic coder representing the periodic component and synthesized by the overlap/add method.
The gaussian noise is generated in order to obtain the synthesis noise, the noise spectral information is added using the transmitted LPC coefficient and gain value and additionally the linear interpolation of the gain and LSP is performed.
The LPC synthesis structure can do time region synthesis by passing the LPC filter by simply making the white gaussian noise to be input without an additional phase accordance procedure between frames. Herein the gain value can be scaled considering the quantization and spectral distortion and when implementing a noise remover the LSP value can be adjusted according to the estimated value of the background noise.
Although the present invention was described on the basis of preferably executable examples, these executable examples do not limit the present invention but exemplify. Also, it will be appreciated by those skilled in the art that changes and variations in the embodiments herein can be made without departing from the spirit and scope of the present invention as defined by the following claims.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US3649765 *||Oct 29, 1969||Mar 14, 1972||Bell Telephone Labor Inc||Speech analyzer-synthesizer system employing improved formant extractor|
|US4219695 *||Oct 5, 1977||Aug 26, 1980||International Communication Sciences||Noise estimation system for use in speech analysis|
|US5749065 *||Aug 23, 1995||May 5, 1998||Sony Corporation||Speech encoding method, speech decoding method and speech encoding/decoding method|
|US5774837||Sep 13, 1995||Jun 30, 1998||Voxware, Inc.||Speech coding system and method using voicing probability determination|
|US5848387 *||Oct 25, 1996||Dec 8, 1998||Sony Corporation||Perceptual speech coding using prediction residuals, having harmonic magnitude codebook for voiced and waveform codebook for unvoiced frames|
|US5909663 *||Sep 5, 1997||Jun 1, 1999||Sony Corporation||Speech decoding method and apparatus for selecting random noise codevectors as excitation signals for an unvoiced speech frame|
|US6289309 *||Dec 15, 1999||Sep 11, 2001||Sarnoff Corporation||Noise spectrum tracking for speech enhancement|
|US6496797 *||Apr 1, 1999||Dec 17, 2002||Lg Electronics Inc.||Apparatus and method of speech coding and decoding using multiple frames|
|1||C. Laflamme et al., "Harmonic-Stochastic Excitation (HSX) Speech Coding Below 4 KBIT/S", IEEE, 1996. pp. 204-207.|
|2||Eric W.M. Yu et al., "Variable Bit Rate MBELP Speech Coding Via V/UV Distribution Dependent Spectral Quantization", IEEE, 1997. pp. 1607-1610.|
|3||Masayuki Nishiguchi et al., "Harmonic and Noise Coding of LPC Residuals with Classified Vector Quantization", IEEE, 1995. pp. 484-487.|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7376553||Jul 8, 2004||May 20, 2008||Robert Patel Quinn||Fractal harmonic overtone mapping of speech and musical sounds|
|US7773118||Mar 25, 2007||Aug 10, 2010||Fotonation Vision Limited||Handheld article with movement discrimination|
|US7831420 *||Apr 4, 2006||Nov 9, 2010||Qualcomm Incorporated||Voice modifier for speech processing systems|
|US7835907 *||Dec 21, 2005||Nov 16, 2010||Samsung Electronics Co., Ltd.||Method and apparatus for low bit rate encoding and decoding|
|US8169486 *||May 24, 2007||May 1, 2012||DigitalOptics Corporation Europe Limited||Image acquisition method and apparatus|
|US8212882||May 27, 2010||Jul 3, 2012||DigitalOptics Corporation Europe Limited||Handheld article with movement discrimination|
|US8244053||Dec 16, 2009||Aug 14, 2012||DigitalOptics Corporation Europe Limited||Method and apparatus for initiating subsequent exposures based on determination of motion blurring artifacts|
|US8264576||Dec 9, 2008||Sep 11, 2012||DigitalOptics Corporation Europe Limited||RGBW sensor array|
|US8270751||Apr 17, 2011||Sep 18, 2012||DigitalOptics Corporation Europe Limited||Method of notifying users regarding motion artifacts based on image analysis|
|US8285067||Apr 17, 2011||Oct 9, 2012||DigitalOptics Corporation Europe Limited||Method of notifying users regarding motion artifacts based on image analysis|
|US8417055||Sep 18, 2007||Apr 9, 2013||DigitalOptics Corporation Europe Limited||Image processing method and apparatus|
|US8494300||Apr 6, 2010||Jul 23, 2013||DigitalOptics Corporation Europe Limited||Method of notifying users regarding motion artifacts based on image analysis|
|US8520082 *||Oct 19, 2010||Aug 27, 2013||DigitalOptics Corporation Europe Limited||Image acquisition method and apparatus|
|US8649627||Feb 20, 2013||Feb 11, 2014||DigitalOptics Corporation Europe Limited||Image processing method and apparatus|
|US8698924||Nov 8, 2010||Apr 15, 2014||DigitalOptics Corporation Europe Limited||Tone mapping for low-light video frame enhancement|
|US8737766||Feb 20, 2013||May 27, 2014||DigitalOptics Corporation Europe Limited||Image processing method and apparatus|
|US8878967||Oct 11, 2010||Nov 4, 2014||DigitalOptics Corporation Europe Limited||RGBW sensor array|
|US8890983||Aug 28, 2013||Nov 18, 2014||DigitalOptics Corporation Europe Limited||Tone mapping for low-light video frame enhancement|
|US8989516||Dec 16, 2008||Mar 24, 2015||Fotonation Limited||Image processing method and apparatus|
|US9094648||Nov 6, 2014||Jul 28, 2015||Fotonation Limited||Tone mapping for low-light video frame enhancement|
|US9160897||Jun 11, 2008||Oct 13, 2015||Fotonation Limited||Fast motion estimation method|
|US9307212||Jul 27, 2015||Apr 5, 2016||Fotonation Limited||Tone mapping for low-light video frame enhancement|
|US20050008179 *||Jul 8, 2004||Jan 13, 2005||Quinn Robert Patel||Fractal harmonic overtone mapping of speech and musical sounds|
|US20060136198 *||Dec 21, 2005||Jun 22, 2006||Samsung Electronics Co., Ltd.||Method and apparatus for low bit rate encoding and decoding|
|US20070233472 *||Apr 4, 2006||Oct 4, 2007||Sinder Daniel J||Voice modifier for speech processing systems|
|US20070296833 *||May 24, 2007||Dec 27, 2007||Fotonation Vision Limited||Image Acquisition Method and Apparatus|
|US20080219581 *||Sep 18, 2007||Sep 11, 2008||Fotonation Vision Limited||Image Processing Method and Apparatus|
|US20080231713 *||Mar 25, 2007||Sep 25, 2008||Fotonation Vision Limited||Handheld Article with Movement Discrimination|
|US20080309769 *||Jun 11, 2008||Dec 18, 2008||Fotonation Ireland Limited||Fast Motion Estimation Method|
|US20080309770 *||Jun 18, 2007||Dec 18, 2008||Fotonation Vision Limited||Method and apparatus for simulating a camera panning effect|
|US20090167893 *||Dec 9, 2008||Jul 2, 2009||Fotonation Vision Limited||RGBW Sensor Array|
|US20100201827 *||Dec 16, 2009||Aug 12, 2010||Fotonation Ireland Limited||Method and apparatus for initiating subsequent exposures based on determination of motion blurring artifacts|
|US20100238309 *||May 27, 2010||Sep 23, 2010||Fotonation Vision Limited||Handheld Article with Movement Discrimination|
|US20100328472 *||Apr 6, 2010||Dec 30, 2010||Fotonation Vision Limited||Method of Notifying Users Regarding Motion Artifacts Based on Image Analysis|
|US20110102638 *||Oct 11, 2010||May 5, 2011||Tessera Technologies Ireland Limited||Rgbw sensor array|
|US20110115928 *||Oct 19, 2010||May 19, 2011||Tessera Technologies Ireland Limited||Image Acquisition Method and Apparatus|
|US20110199493 *||Apr 17, 2011||Aug 18, 2011||Tessera Technologies Ireland Limited||Method of Notifying Users Regarding Motion Artifacts Based on Image Analysis|
|US20110205381 *||Nov 8, 2010||Aug 25, 2011||Tessera Technologies Ireland Limited||Tone mapping for low-light video frame enhancement|
|USRE46082 *||Nov 15, 2012||Jul 26, 2016||Samsung Electronics Co., Ltd.||Method and apparatus for low bit rate encoding and decoding|
|U.S. Classification||704/219, 704/E19.026, 704/230|
|International Classification||G10L25/93, G10L19/08|
|Cooperative Classification||G10L19/08, G10L2025/935|
|May 14, 2001||AS||Assignment|
Owner name: ELECTRONICS AND TELECOMMUNICATIONS RESEARCH INSTIT
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KIM, HYOUNG JUNG;LEE, IN SUNG;KIM, JONG HARK;AND OTHERS;REEL/FRAME:011799/0711;SIGNING DATES FROM 20001226 TO 20010103
|Sep 13, 2004||AS||Assignment|
Owner name: CURITEL COMMUNICATIONS, INC., KOREA, REPUBLIC OF
Free format text: ASSIGNMENT OF FIFTY PERCENT (50%) OF THE RIGHT, TITLE AND INTEREST.;ASSIGNOR:ELECTRONICS AND TELECOMMUNICATIONS RESEARCH INSTITUTE;REEL/FRAME:015120/0875
Effective date: 20040621
|Sep 20, 2007||FPAY||Fee payment|
Year of fee payment: 4
|Oct 20, 2011||FPAY||Fee payment|
Year of fee payment: 8
|Nov 18, 2015||FPAY||Fee payment|
Year of fee payment: 12
|Sep 12, 2016||AS||Assignment|
Owner name: PANTECH CO., LTD., KOREA, REPUBLIC OF
Free format text: MERGER;ASSIGNOR:PANTECH & CURITEL COMMUNICATIONS INC.;REEL/FRAME:039695/0820
Effective date: 20091230