CA1172363A - Continuous speech recognition method - Google Patents

Abstract

CASE II

ABSTRACT OF THE DISCLOSURE

A speech recognition method for detecting and recognizing one or more keywords in a continuous audio signal is disclosed.
Each keyword is represented by a keyword template representing one or more target patterns, and each target pattern comprises statistics of each of at least one spectrum selected from plural short-term spectra generated according to a predetermined system for processing of the incoming audio. The spectra are processed by a frequency equalization and normalizing method to enhance the separation between the spectral pattern classes during later analysis. The processed audio spectra are grouped into spectral patterns, are transformed to reduce dimensionality of the patterns, and are compared by means of likelihood statistics with the target patterns of the keyword templates. A concatenation technique employing a loosely set detection threshold makes it very unlikely that a correct pattern will be rejected.

Description

BACKGROUND OF THE INVENTION
The present inven-tion rela-tes to a speech recognition method and more particularly to a me-thod for recognizing, in real time, one or more keywords in a continuous audio signal.
Various speech recognition systems have been proposed herebefore to recognize isolated utterances by comparing an unknown - isolated audio signal, suitably processed, with one or more pre-viously prepared representa-tions o~ the Xnown keywords. In this context, "keywords" is used to mean a connected group of phonemes and sounds and may be, for example, a portion o~ a syllable, a word, a phrase, etc. While many systems have met with limited success, one system, in particular, has been employed successfully, in commercial applications, to recognize isolated keywords. That system operates substantially in accordance with the method des-cribed in U.S. Patent No. 4,038,503, granted July 26, 1977, assigned to the assignee of this application, and provides a successful method for recognizing one of a restricted vocabulary of keywords provided that the boundaries of the unknown audio signal data are either silence or background noise as measured by the recognition system. That system relies upon the presumption that the interval, during which the unknown audio signal occurs, is well defined and contains a single utterance.
In a continuous audio signal, (the isolated word is one aspect of the continuous speech signal), such as continuous con-versational speech, wherein the keyword boundaries are not a priori known or marked, several methods have been devised to segment the incoming audio data, that is, to determine the boundaries of linguistic units, such as phonemes, syllables, words, sentences, etc., prior to initiation of a keyword recogni-tion process. These prior continuous speech systems, however,have achieved only a limitecl success in part because a satisfactory ~ ~t~363 1 segmenting process has not been found. Other substanti~l problerns still exist; for example, only limited vocabularies can be con-sistently recognized with a low false alarm rate, the recognition accuracy is highly sensitive to the differences between voice characteristics of different talkers, and the systems are highly sensitive to distortion in the audio signals being analyzed, such as typically occurs, for example, in audio signals transmitted over ordinary telephone communications apparatus. Thus, even though continuous speech is easily discernible and understood by the human observer, machine recognition of even a limited vo-cabulary of keywords in a continuous audio signal has yet to . achieve major success.
A speech analysis system which is effect.ive in recognizingkeywords in continuous speech is described and claimed in the applicant's copending Canadian application No. 318,438, filed December 21, 1978, entitled Continuous Speech Recognition Method..
; That system employs a method in which each keyword is charac-terized by a template consisting of an ordered sequence of one or more target-patterns and each target pattern represents a

2~ plurality of short-term keyword power spectra spaced apart in time. Together, the target patterns cover all important acoustical events in the keyword. The invention claimed in application No. 318,438 features a frequency analysis method comprising the steps of repeatedly evaluating a set of parameters deter-mining a short-term power spectrum of the audio signal at each o~ a plurality of equal duration sampling intervals, thereby generating a continuous time-ordexed sequence of short-term, audio power spectrum frames; and repeatedly selecting from the sequence of short-term power spectrum frames, one first 30 frame and at least one later occurring frame to form a multi-frame spectral pattern. The method further features the steps of comparing, preferably using a likelihood statistic, each thus formed multi-frame pattern, with each first target pattern R

~ 723~3 .`
1 of each keyword template; and deciding whether each multi-frame pattern corresponds to one of the first target pat-terns of the keyword templates. For each multi-frame pattern which, according to the deciding step, corresponds to a first target pattern of a potential candida-te keyword, the method features selecting later occurring frames to form later occurring multi-frame patterns. The method then features the steps of deciding in a similar manner whether the later m~llti-frame patterns correspond respec-tively to successive target patterns of the potential candidate keyword, and identifying a candidate keyword when a selected sequence of muIti-frame patterns corresponds r~spec-tively to the target patterns of a keyword template, designated the selected keyword template.
Even though the method claimed in copending application Serial No. 318t~38, is significantly more effective in recogni-~ing keywords in continuous speech than -the prior art systems, even that method falls short of the desired goals.
A principal object of the present invention is therefore a speech recognition method having improved effec-tiveness in recognizing keywords in a continuous, unmarked audio signal. Other objects of the invention are a method which is relatively insensi-tive to phase and amplitude distortion of the unknown audio input signal data, a method which is relatively insensitive to variations in the articulation rate of the unknown audio input signals, a method which will respond equally well to different speakers and hence different voice characteristics, a method which is reliable, and a method which will operate in real time. Yet other objects of the invention are a method which reduces the dimensionality o~
the unknown input signal.

3 ~ SUMMARY OF THE INVENI'ION
.. . .. __ ..

The inven-tion relates to a speech analysis system for ~ .

1 reco~nizing at least one predetermined keyword in an audio input signal. Each keyword is characterized by a template consisting of an ordered sequence of one or more target patterns. Each target pattern represents at least one short term keyword power spectrum. Together, the target patterns cover all important acoustical events in the keyword. The invention features a fre-quency analysis method comprising the steps of repeated]y evaluating a set of parameters determining a short-term power spe~trum of the audio signal at each of a plurality of equal duration sampling intervals, thereby generating a continuous time-ordered sequence of short-term, audio power spectrum frames; repeatedly generating a peak spectrum corresponding to the spectrum frames by a fast attack, slow decay peak detecting function and for each frame, dividing the amplitude of each frequency band by the corresponding intensity value in the corresponding peak spectrum; selecting a sequence of frame patterns; and identifying a candidate key-word when a selected sequence of frame patterns corresponds res-pectively to the target patterns of a keyword template, designated the selected keyword template.
Preferably, the method further features the steps of selecting the value of each of the peak spectrum frequency bands from the maximum of the incoming new spectrum value for that band and the previous peak spectrum value multiplied by a constant decay factor having a value less than one.
In another aspect of the invention, there is featured a pattern recognition method for identifying, in a data stream, at least one target pattern characterized by a vector of recog-nition elements xi, said elements having a statistical distribution.
This aspect features the analysis method comprising the steps of determining from plural design set pattern samples x of the target ~l7;~363 1 pattern a co~ariance matrix K and an expected value vec-tor x;
and calculating from the covariance matrix K, a plurality of eigenvectors ei having eigenvalues vi, where vi ~ vi~l. The method further features selecting unknown patterns y frorn the data stream, transforming each pattern y into a new vector W
having the form ~Wl, W2,..., Wp~ R), where Wi = ei (Y - x), p is a positive integer constant less than the number of elements of the pattern y, and R is the reconC;truction error statistic and equals ~¦Y - x ¦2 _ ~ Wj2) /2. By applying a likelihood statistic function to the vector W, it is decided whether the pattern y is identified with any one target pattern.
In particular aspects of the invention, the method further features the step of calculating a likelihood statistic according to one or the other of the equations:

r ~' = ~ 2 ~ l l -~ ln var (W.) i=l var (Wi) + (R - R) + ln var (R) ; or var (R) J

Q" = ~ l/2~(R R) _ + ln var (R)¦
var (R) where the barred variables are sample means and var ( ) is the unbiased sample variance.

23~;3 i BRIEF DESCRIPTION OF THE DRAWINGS
.
Other objects, features, and advantages of the invention will appear from the following descrip-tion of a preferred embodiment taken toge-ther with the drawings in which:
Figure 1 is a flow chart illustrating in general terms the sequence of operations performecL in accordance with the practice of the present invention;
Figure 2 is a schematic block diagram of electronic apparatus for performing certain preprocessing operations in the overall process illustrated in Figure l;
Figure 3 is a flow diagram of a digital computer pro-gram performing cer-tain procedures in the process of Figure l;and Figure 4 is a graphic tabulation of classification accuracy using different transformation procedures.
Corresponding reference charac-ters indicate corresponding parts throughout the several views of the drawings.

DESCRIPTION OF A PREFERRED EMBODIMENT

In the particular preferred embodiment which is described herein, speech recognition is performed by an overall apparatus which involves both a specially constructed elec-tronic system for effecting certain analog and digital processing of incoming audio data signals, generally speech, and a general purpose digital computer which is programmed in accordance with the present invention to effect certain other data reduction s-teps and numerical evaluations. The division of tasks between the hardware portion and the software portion of this system has been made so as to obtain an overall system which can accomplish speech recognition in real time at moderate cost. However, it should be understood that some of the tasks being performed in hardware in this particular system could well be performed in 3Li'7~3~
1 software and that some of the tasks being performed by software programming in this example might also be performed by special purpose circuitry in a different embodiment of the invention.
As indicated previously, one aspect of -the present invention is the provision of apparatus which will recognize keywords in continuous speech signals even though those signals are distorted, for example, by a telephone line. Thus, referring in particular to Figure 1, the voice input signal, indicated at 10, may be considered a voice signal, produced by a carbon element telephone transmitter and received over a telephone line encompassing any arbitrary distance or number of switching interchanges. A typical application of the invention is therefore recognizing keywords in audio data from an unknown source received over the telephone system. On the other hand, the input signal may also be any audio data signal, for example, a voice input signal, taken from a radio tele-communications link, for example, from a commercial broadcast station or from a private dedicated communications link.
As will become apparent from the description, the present method and apparatus are concerned with the recognition of speech signals containing a sequence of sounds or phonemes, or other recognizable indicia. In the description herein, and in the claims, reference is made to either "a keyword", "a sequence of target patterns", "a template pattern", or "a keyword template", the four terms being considered as generic and equivalent. This is a convenient way of expressing a recognizable sequence of audio sounds, or representations thereof, which the method and apparatus can detect. The terms should be broadly and generically construed to encompass anything from a ~ single phoneme, syllable, or sound to a series of words (in the grammatical sense) as well as a single word.

1 An analog-to-digital. (A/D) converter 13 receives the incoming analog audio signal data on line 10 and converts the signal amplitude of the incoming data to a digital form. The illustrated A/D converter is designed to convert the input signal data to a twelve-bit binary representation, the con-versions occurring at the rate of 8,000 conversions per second. The A/D converter 13 applies its output over lines 15 to an autocorrelator 17. The autocorrelator 17 processes the digital input signals to generate a short-term autocorrelation function 100 times per second and applies its output, as indicated, over lines 19. Each autocorrelation function com-prises 32 values or channels, each value being calcula-ted to a 30-bit resolutionD The autocorrelator is described in greater detail hereinafter with reference to Figure 2.
The autocorrelation functions over lines 19 are Fourier transformed by Fourier transformation apparatus 21 to obtain the corresponding short-term windowed power spectra over lines 23. The spectra are generated at the same repetition rate as the autocorrelation functions, that is, 100 per second, and each short-term power spectrum has thirty-one numerical terms having a resolution of 16 bits each. ~s will be understood, each of the thirty-one terms in the spectrum represents the signal power within a frequency band. The Fourier transformation apparatus also preferably includes a ~Iamming or similar window function to reduce spurious adjacent-band responses.
In the illustrated embodiment, the Fourier transformation as well as subsequent processing steps are performed under the control of a general purpose digital computer, appropriately programmed, utiliz:ing a peripheral array processor for speeding the arithmetic operations required repetitively according to ( 1 the present metho~. The particular computer employed is a model PDP-ll*manufactured by the Digital Equipment Corporation of Maynard, Massachuse-tts. The particular array processor employed is described in the applicant's Canadian application No. 313,111 filed October 11, 197~. The programming descxibed hereinafter with reEerence to Figure 3 is substantially predi-cated upon the capabilities and characteris-tics of these commercially available digital processing units.
The short-term windowed po~er spectra are frequency-response equalized, as indicated at 25, equalization being performed as a function of the peak amplitudes occurring in each frequency band or channel as described in greater detail hereinafter. The frequency-response equalized spectra, over lines 26, are generated at the rate of 100 per second and each spectrum has -thirty-one numerical terms evaluated to 16 bit accuracy. To facilitate the final evaluation of the incoming audio data, the frequency-response equalized and windowed spectra over lines 26 are sub~ected to an amplitude transfor~
- mation, as indicated at 35, which imposes a non-linear amplitude ~O transformation on the incoming spectra. This transformation is described in greater detail hereinafter, but it may be noted at this point that it improves the accuracy with which the unknown incoming audio signal may be matched with keywords in a reference vocabulary. In the illustrated embodiment, this transformation is performed on all of the frequency-response equalized and windowed spectra at a time prior to the comparison of the spectra with keyword templates representing the ~eywords in the reference vocabulary.

The amplitude transformed and equalized short-term spectra over lines 38 are then compared against the keyword *Trade Mark g _ .~ .

~:~7~S3 1 templates at ~0. The keyword templates, designated at 42, represent the keywords of the re~erence vocabulary in a spectral pattern with which the transformed and equalized spectra can be compared. Candidate words are thus selected according to the closeness of the comparison; and in the illustrated embodiment, the selection process is designed to minimize the liXelihood of a missed keyword while rejecting grossly inapplicable pattern sequences. The candidate words (and accumulated statistics relating to the corresponding incoming data) are applied over lines 44 for post-decision processing at ~6 to reduce the false alarm rate. The final decision is indicated at 48. The post-decision processing, which includes the use of a prosodic mask and/or an acoustic-level likelihood ratio test, improves the discrimination between eorrect detections and false alarms as deseribed in more detail below.

PreDrocessor In the apparatus illustrated in Figure 2, an auto-correlation function with its intrinsic averaging is performed digitally on the digital data s-tream generated by the analog-to-digital converter l3 from the incoming analog audio data over line 10, generally a voice signal. The converter 13 provides a digital input signal over lines 15. The digital processing funetions, as well as the input analog-to-digital conversion, are timed under the control of a elock oscillator 51. The elock oseillator provides a basic timing signal at 256,000 pulses per second, and this signal is applied to a frequency divider 52 to obtain a seeond timing signal at 8,000 pulses per second.
The slower timing signal controls the analog-to-digital converter 13 together with a latch register 53 which holds the twelve-bit results of the last conversion until the ne~t con-version is eompleted.

7~:3~3 1 The autocorrelation products are generated by a digital multiplier 56 which multiplies the number contained in a register 53 by the output of a thirty~two word shift register 58.
Shift register 58 is operated in a recirculating mode and is driven by the faster clock frequency, so that one complete circulation of the shift register data is accomplished for each analog-to-digital conversion. An input to shift register 58 is taken from register 53 once during each complete circulation cycle. One input to the digital multiplier 56 is taken directly from the latch register 53 while the other input to the multi-plier is taken (with one exception described below) from the current output of the shift register through a multiplexer 59.
The multiplications are performed at the higher clock frequency.
Thus, each value obtained from the A/D conversion is multiplied with each of the preceding 31 conversion values.
As will be understood by those skilled in the art, the signals thereby generated are equivalent to multiplying the input signal by itself, delayed in time by 32 different time increments (one of which is the zero delay). To produce the zero delay correlation, that is, the power of the signal, multiplexer 59 causes the current value of the latch register 53 to be multi~
plied by itself at the time each new value is being introduced ; into the shift register. This timing function is indicated at 60.
As will also be understood by those skilled in the art, the products from a single conversion, together with its 31 predecessors J will not be fairly representative of the energy distribution or spectrum over a reasonable sampling interval.
Accordingly, the apparatus of Figure 2 provides for averaging of these sets of products.

2;~63 1 An accumulation process, which eEfects averaging, is provided by a thirty-two word shift register 63 which is inter-connected with an adder 65 to form a set of thirty-two accumulators. Thus, each word can be recirculated after having been added to the corresponding increment from the digital multiplier. The circulation loop passes through a gate 67 which is controlled by a divide-by-N divider circuit 69 driven by the low frequency clock signal. The divider 69 divides the lower frequency clock by a factor which determines the number ~ of instantaneous autocorrelation functions which are accumulated, and thus averaged, before the shift register 63 is read out.
In the illustrated example, eighty samples are accumu-lated before being read out. In other words, N for the divide-by-N divider circuit 69 is equal to eighty. After eighty conversion samples have thus been correlated and accumulated, the divider circuit 69 triggers a computer interrupt circuit 71 over a line 72. ~t this time, the contents of the shift register 63 are successively read into the computer memory through a suitable interface circuitry 73, the thirty-two successive words in the register being presented in ordered sequence to the computer through the interface 73. As will be understood by those skilled in the art, this data transfer from a peripheral unit, the autocorrelator preprocessor, to the computer may be typically performed by a direct memory access procedure. Predicated on an averaging of eighty samples, at an initial sampling rate of 8,000 samples per second, it will be seen that 100 averaged autocorrelation functions are pro-vided to the computer every second.

While the shift register contents are being read out to the computer, the gate 67 is closed so that each of the words ~7~3s~3 1 in the shift regis~er is effectively rese-t to zero to permit the accumulation process to begin again.
Expressed in mathematical terms, the operation of the apparatus shown in Figure 2 can be described as follows.
Assuming that the analog-to-digital converter generates the time series S(t), where t = O, To, ~To, ~ and To is the sampling interval ( 8000 sec. in the illustrated embodiment), the illustrated digital correlation circuitry of Figure 2 may be considered, ignoring start-up ambiguities, to compute the autocorrelation function (j,t) = ~ S(t-kTo) S(t-(k ~ j) To) (Equation 1) k=l where j = O, 1, 2, ..., 31; t = 80 To, 160 To, ..., 80n To,...
These autocorrelation functions correspond to the correlation output on lines 19 of Figure 1.
Referring now to Figure 3, the digital correlator operates continuously to transmit to the computer a series of data blocks at the rate of one complete autocorrelation function every ten milliseconds. This is indicated at 77 (Fig. 3). Each block of data represents the autocorrelation function derived from a corresponding subinterval of time. As noted above, the illustrated autocorrelation functions are provided to the computer at the rate of one hundred, 32-word functions per second.
In the illustrated embodiment, the processing of he autocorrelation function data is performed by an appropriately programmed, special purpose digital computer. The flow chart, which includes the function provided by the computer program is given in Figure 3. Again, however, it should be pointed 3~ out that various of the steps could also be performed by hardware ~2;~63 1 rather than software and that likewise certain oE the functions performed by the apparatus of Figure 2 could additionally be performed in the so:Etware by a corresponding revision of the flow chart of Figure 3.
Although the digital correlator of Figure 2 performs some time-averaging of the autocorrelation functions generated on an instantaneous basis, the average autocorrelation functions read out to the computer may still contain some anomalous dis-continuities or unevenness which mi~ht interfere with the orderly processing and evaluation of the samples. Accordingly, each block of data, that is, each autocorrelation function ~ tj,t), is first smoothed ~ith respec-t to time. This is indicated in the f low chart of Figure 3 at 79. The preferred smoothing process is one in which the smoothed autocorrelation output ~s(j,t) is given by (j,t) = Co~(j,t) + Cl~(j,t - T) ~ C2~(j,t -~ T) (Equation 2) where ~j,t) is the unsmoothed input autocorrelation defined in Equation 1, ~s(j,t) is the smoothed autocorrelation output, i denotes the delay time, t denotes real time, and T denotes the time interval between consecutively generated autocorrelation functions (equal to .01 second in the preferred embodiment).
The weighting functions COr Cl, C2, are preferably chosen to be 1/2, 1/4, 1~4 in the illustrated embodiment, although other values could be chosen. For example, a smoothing function approximating a Gaussian impulse response with a frequency cutoff of, say, 20 Hertz could have been implemented in the compu-ter software. However, experiments indicate that the illustrated, easier to implement, smoothing function provides satisfactory results. As indicated, the smoothing function is applied separately for each value j of delay.

~ 14 -~.~.'7~ 3 1 ~s indicated at 81, a cosine Fourier transform is then applied to each time smoothed autocorrelation function, ~s (j-t), to generate a 31 point power spectrum. The power spec-trum is defined as S(f,t) ~s(' t) W (0~ + 2 ~ ~s~j,t) W (j) cos (E~uation 3) where S(f,t) is the spectral energy in a band centered at f Hz, at time t; W (i) = 12 (1 -~ cos ~) is the Hammin~ window function to reduce side lobes; ~s(j,t) is the smoothed autocorrelation function at delay j and time t; and 30 ~ looo (0.0552m + 0.438)1/ 063 H
(Equation 4) which are frequencies equally spaced on the "mel" scale of pitch. As will be understood, this corresponds to a subjective pitch (mel scale) frequency-axis spacing for fre~uencies in the bandwidth of a typical communication channel of about 300-3500 Hertz. ~s will also be understood, each point or value within each spectrum represents a corresponding band of frequencies.
While this Fourier transform can be performed completely within the conventional computer hardware, the process may be speeded considerably if an external hardware multiplier or Fast Fourier Transform (FFT) peripheral device is utilized. The construction and operation of such modules are well known in the art, however, and are not described in detail herein. Ad-vantageously built into the hardware Fast Fourier Transform peripheral device is a frequency smoothing function wherein each of the spectra are smoothed in frequency according to the preferred ~amming window weighting function W (j) defined above.
This is indicated at 83 of the block 85 which corresponds to the hardware Fourier transform implementation.

1 ~s successive smoothed power spectra are received from the Fast Fourier Transform peripheral 85, a communication channel equali~ation function is obtained by determining a (generally different) peak power spectrum for each incoming windowed power spectrum from peripheral 85, and modifying the output of the Fast ~ourier Transform apparatus accordingly, as described below. Each newly generated peak amplitude spectrum y (f,t), corresponding to an incoming windowed power spectrum S (f,t), where f is indexed over the plural frequency bands of the spectrum, is the result of a fast attack, slow decay, peak detecting function for each of the spectrum channels or bands. The windowed power spectra are normalized with respect to the respective terms of the corresponding peak amplitude spectrum. This is indicated at 87.
According to the illustrated embodiment, the values oE
the "old" peak amplitude spectrum y~f,t-T), determined prior to receiving a new windowed spectrum, are compared on a frequency band by frequency band basis wi-th the new incoming spectrum S(f,t). The new peak spectrum y(f,t) is then generated according to the following rules. The power amplitude in each band of the "old" peak amplitude spectrum is multiplied by a fixed fraction, for example, 511, in the illustrated example. This corresponds to the slow decay portion of the peak detecting function. If the power amplitude in a frequency band of the incoming spectrum S(f,t) is greater than the power amplitude in the corresponding frequency band of the decayed peak amplitude spectrum, then the decayed peak amplitude spectrum value for that (those) frequency band(s) is replaced by the spectrum value of the corresponding band of the incoming windowed spectrum. This corresponds to the fast attack portion of the i peak detectiny function. Mathematically, the peak detecting function can be e~pressed as y(f,t) = ma~ {y(f,t-T) (l-E), S(f,t)} (Equation 5) where f is indexed over each of the frequency bands, y(f,t) is the resulting peak spectrum, y(f,t-T) is the "old" or previous peak spectrum, S(f,t) is the new incoming power spectrum, and E is the decay parameter. After the peak spectrum is generated, the resulting peak amplitude spectrum is frequency smoothed at 89 by averaging each frequency band peak value with peak values corresponding to adjacent frequencies of the newly generated peak spectra, the width of the overall band of frequencies contributing to the average value being approximately equal to the typical frequency separation between formant frequencies.
As will be understood by those skilled in the speech recognition art, this separation is in the order of 1000 Hz. By averaging in this particular way, the useful information in the spectra, that is, the local variations revealing formant resonances are retained whereas overall or gross emphasis in the frequency spectrum is suppressed. The resulting smoothed peak amplitude spectrum y(f,t) is then employed to normalize and frequency equalize the just received power spectrum, S(f,t), by dividing the amplitude value of each frequency band of the incoming smoothed spectrum S(f,t), by the corresponding frequency band value in the smoothed peak spectrum y(f,t~. Mathematically, this corresponds to Sn (f,t) = S(f,t) / y(f,t) (Equation 6) wherein Sn(f,t) is the peak normalized smoothed power spectrum and f is indexed over each of the frequency bands. This step is indicated at 91. There results a sequence of frequency equalized and normalized short-term power spectra which emphasizes 7;~363 1 chan~es in the Erequency content of the incoming audio signals while suppressing any generalized lon~-term frequency emphasis or distortion. This method of frequency compensation has been found to be highly advantageous in the recognition of speech signals transmitted over frequency distorting communication links such as telephone lines, in comparison to the more usual systems of frequency compensation in which the basis for compensation is the average power le~el, either in the whole signal or in each respective frequency band.
It is useful to point out that, while successive spectra have been variously processed and equalized, the data representing the incoming audio signals still comprises spectra occurring at a rate of 100 per second.
The normalized and frequency equalized spectra indicated at 91, are subjected to an amplitude transformation, indicated at 93, which effects a non-linear scaling of the spectrum amplitude values. Designating the individual equalized and normalized spectra as Sn(f,t) (from Equation 6) where f indexes the different frequency bands of the spectrum and t denotes real time, the non-linearly scaled spectrum x(f,t) is the linear fraction function x(f,t) = Sn(f,t) - A (Equation 7A) Sn~f,t) + A
where A is the average value of the spectrum Sn(f,t) defined as follows:

31 fb=l n b (Equation 7B) where fb indexes over the frequency bands of the power spectrum.
This scaling function produces a soft threshold and gradual saturation effect for spectral intensities which deviate greatly from the short-term average A. Mathematically, for ,~ s~f~ ~ L` f~
~L IL ~ ~9~W
1 intensities near the average, the function is approximately linear; for intensities further from the average it is approxi-mately logarithmic; and at the extreme values of intensity, it is substantially constant. On a logarithmic scale, the function x~f,t) is symmetric about zero and the function exhibits threshold and saturation behavior that is suggestive of an auditory nerve firing-rate function. In practice, the overall recognition system performs signi~icantly better with this particular non-linear scaling function than it does with either a linear or a logarithmic scaling of the spectrum amplitudes.
There is thus generated a sequence of amplitude transformed, frequency-response equalized, normalized, short-term power spectra x~f,t) where t equals .01, .02, .03, .04, ....
seconds, and f = 1, ..., 31 (corresponding to the frequency bands of the generated power spectra). Thirty-two words are provided for each spectrum, and the value of ~ (Equation 7B), the average value of the spectrum values, is stored in the thirty-second word. The amplitude transformed, short-term power spectra are stored, as indicated a-t 95, in a first-in, first-out circulating memory having storage capacity, in the illustrated embodiment, for 256 thirty-two-word spectra. There is thus made available for analysis, 2.56 seconds o~ the audio input signal. This storage capacity provides the recognition system with the flexibility required to select spectra at different real times, for analysis and evaluation and thus with the ability to go forward and backward in time as the analysis requires.

Thus, the amplitude transformed power spectra for the last 2.56 seconds are stored in the circulating memory and are 3~i3 1 available as needed. In operation, in the illustrated embodiment, each amplitude transformed power spectrum is stored for 2.56 seconds. Thus, a spectrum, which enters the circulating memory at time tl, is lost or shifted from the memory 2.56 seconds later as a new amplitude transformed spectrum, corresponding to a time tl -~ 2.56, is stored.
The transformed and equalized short-term power spectra passing through the circulating memory are compared, preferably in real time, against a known vocabuLary of keywords to detect or pick out those keywords in the continuous audio data.
Each vocabularv keyword is represented by a template pattern statistically representing a plurality of processed power spectra formed into plural non-overlapping multi-frame (preferably three spectra) design set patterns. These patterns are preferably selected to best represent significant acoustical events of the keywords.
The spectra forming the design set patterns are generated for keywords spoken in various contexts using the same system described hereinabove for processing the continuous unknown speech input on line 10 as shown in Figure 3.
Thus, each keyword in the vocabulary has associated with it a generally plural sequence of design set patterns, P(i~l, P(i)2, ..., which represent~ in a domain of short-term power spectra, one designation of that i th keyword. The collection of desiyn set patterns for each keyword form the statistical basis from which the target patterns are generated.
In the illustrated embodiment of the invention, the design set patterns P(i)j can each be considered a 96 element array comprising three selected short-term power spectra arranged in a series sequence. The power spectra forming the ~ 20 -~ll7~3~;3 1 pattern should preferably be spaced at least 30 milliseconds apart to avoid spurious correlation due to time domain smoothing. In other embodiments of the invention, other sampling strategies can be implemented for choosing the spectra;
however the preferred strategy is to select spectra spaced by a constant time duration, preferably 3~ milliseconds, and to space the non-overlapping design set patterns throughout the time interval defining the keyword. Thus, a first design set pattern Pl corresponds to a portion of a keyword near its beginning, a second pattern P2 corresponds to a portion later in time, etc., and the patterns Pl, P2, ... form the statistical basis for the series or sequence of target patterns, the keyword template, against which the incoming audio data will be matched. The target patterns tl, t2, ..., each comprise the statistical data, assuming the P(i)j are comprised of independent Gaussian variables, which enab]e a likelihood sta-tistic to be generated between selected multi-frame patterns, defined below, and the target patterns. Thus, the target patterns consist of an array where the entries comprise the mean, standard deviation ~ and area normalization factor for the corresponding collection of design set pattern array entires. A more refined likelihood statistic is described below.
It will be obvious to those skilled in the art that substantially all keywords will have more than one contextual and/or regional pronunciation and hence more than one "spelling"
of design set patterns. Thus, a keyword having the patterned spelling Pl, P2, ... referred to above, can in actuality be generally expressed as p(i)l~ p(i)2, ... i = 1, 2 ! . . ~ M

where each of the p(i)j are possible alternative descriptions of the jth class of design set patterns, there being a total of M different spellings for the keyword.

~L~'7~3 1 rrhe target patterns tl, t2, ..... , ti, ... , in the most general sense, therefore, each represent plural alternative statistical spellings for the ith group or class of design set patterns. In the illustrated embodiment described herein, the term "target patterns" is thus used in the most general sense and each target pattern may therefore have more than one per-missible alternative "statistical spelling".

Processing the Stored Spectra The stored spectra, at 95, representing the incoming continuous audio data, are compared with the stored template of target patterns indicated at 96, representing keywords of the vocabulary according to the following method. Each successive transformed, frequency-response equalized spectrum is treated as a first spectrum member of a multi-frame pattern, here a three spectrum pattern which corresponds to a 96-element vector. The second and third spectrum members of the pattern, in the illustrated embodiment, correspond to spectra occurring 30 and 60 milliseconds later (in real time). In the resulting pattern, indicated at 97, then, the first selected spectrum forms the first 32 elements of the vector,the second selected spectrum forms the second 32 elements of the vector, and the third selected spectrum forms the -third 32 elements of the vector.
Preferably, each thus formed multi-frame pattern is transformed according to the following methods to reduce cross-correla-tion and decrease dimensionality, and to enhance the separation between target pattern classes. This is indicated at 99. The transformed patterns in the illustrated ~ embodiment are then applied as inputs to a statistical likeli-` hood calculation, indicated at 100, which computes a measure of the probability that the transformed pattern matches a target pattern.

~ Pattern Transformation .. _ ... .
Considering first the pattern transformation, and using matrix notation, each multi-frame pattern can be represented by a 96-by-1 column vector x = (xl, x2, ~ Xg6)~
xl, x2, ..., x32 are the elements x~f,tl) of the first spectrum frame of the pattern, X33, X34, ..., x64 are the elements x(f,t2) of the second spectrum frame of the pattern, and x65, x66, ..., x96 are the elements x(f,t3) of the third spectrum frame. Experimentally most of the elements xi of the vector x are observed to have probability distributions that are clustered symmetrically about their mean values 50 that a Gaussian probability density function closely fits the dis-tribution of each xi ranging over samples from a particular collection of design set patterns corresponding to a particular target pattern. ~Iowever, many pairs xi, x; of elements are found to be significantly correlated, so that an assumption to the effect that the elemen-ts of x are mutually independent and uncorrelated would be unwarranted. Moreover, the correlations between elements arising from different frames in the multi-frame pattern convey information about the direction of motion of for-m ~ resonances in the input speech signal, and this information remains relatively constant even though the average frequencies of the formant resonances may vary, as from talker to talker.
As is well known, the directions of motion of formant resonance frequencies are important cues for human speech perception.
As is well known, the effect of cross correlations among the elements of x can be taken into account by employing the multivariate Gaussian log likelihood statistic -L = 1/2(x-x)K (x-x) + 1/2 ln¦~ K~ (Equation 8A~

3~3 1 where x is the sample mean oE x, K is the matrix of sample ~covariances between all pairs of elements of x deEined by - (Equation 8B) ij i i ) (Xj Xj ), and 1I K 1I denotes the determinant of the matrix K. The cova~iance matrix K can be decomposed by well-known methods into an eigenvector representation K = EVE (Equation 8C) where E is the matrix of eigenvectors ei f K, and ~ is the diagonal matrix of eigenvalues vi of K. ~hese quantities are defined by the relation Kei = viei (Equation 8D) Multiplication by the matrix E corresponds to a rigid rotation in the 96-dimensional space in which the vectors x are represented. Now if a transformed vector w is defined as w = E(x-x)t (Equation 8E) then the likelihood statistic can be rewritten as -L = 1/2wV w + 1/2 ln ¦~ K¦¦
96 /w 2 i=l~ vi ~ ln vi (Equation 8F) Each eigenvalue vi is the statistical variance of the random vector x measured in the direction of eigenvector ei.
The parameters Kij and xi are determined, in the illustrated embodiment, by averaging formed multi-frame patterns, for each of the indicated statistical functions, over a number of observed design set samples. This procedure forms statistical estimates of the expected values of Kij and xi. Eowever, the number of independent parameters to be estimated is 96 mean ~ 24 ~7;Z3~3 i values plus 96x97/2 = ~65~ covariances. Since it is impractical to collec-t more than a few hundred design set pattern samples for a target pattern, the achievable number of samp]e ob-servations per statistical parameter is evidently quite small.
The effect of insufficient sample s:ize is that chance fluctuations in the parameter estimates are comparable to the parameters being estimated. These relatively large fluctuations induce a strong statistical bias on the classification accuracy of the decision processor based on equation 8F, so that although the processor may be able to classify the samples from its own design set patterns with high accuracy, the performance measured with unknown data samples will be quite poor.
It is well known that by reducing the number of statistical parameters to be estimated, the effect of small sample bias is reduced. To that end, che following method has been commonly employed to reduce the dimensionality of a statistical random vector. The eigenvectors ei defined above are ranked by decreasing order of their associated eigenvalues vi, to form a ranked matrix Er of ranked eigenvectors er so that erl is the direction of maximum variance vrl and i~l - 1 . Then the vector x-x is transformed into a vector w as in equation 8E, (using the ranked matrix Er), but only the first p elements of w are utilized to represent the pattern vector x. In this representation, sometimes termed "principal component analysisl', the effective number of statistical para-meters to be estimated would be in the order of 96p instead of 4656. To classify patterns the likelihood statistic L is computed as in equation 8F except that the summation now ranges from 1 to p instead of from 1 to 96. On applying the principal 3~3 1 component analysis method to practical data it is observed that the classification accuracy of the processor increases as p increases, until at a critical value of p the accuracy i5 a maximum; thereafter the accuracy diminishes as p is increased until the poor performance described above is observed at p=96. (See Figure 4, graph(a) (training set data~ and graph (b) (unknown input data)).
The maximum classification accuracy achieved by the principal component method is still limited by a small sample statistical bias effect, and the number of components, or dimensions, required is much larger than one would expect is really necessary to represent the data. Furthermore, it can be seen from the illustration (Figure 4) that the performance for design set pattern samples is actually worse than the performance for unknown samples, over a wide range of p.
The source of the latter two effects is found in the fact that by representing the sample space with p components of the transformed vector w, the contribution of the remaining 96-p components has been left out of the likelihood statistic 2~ L. A region where most of the pattern samples are found has thus been described, but the regions where few samples occur has not been described. The latter regions correspond to the tails of the probability distribution and thus to the reyions of overlap between the different target pattern classes. The prior art methoa thus eliminates the very information needed to make the most difficult classification decisions. ~n-fortunately these regions of overlap are of high dimensionality, so it is impractical to reverse the argument above and employ, for example, a small number of the components of w for which the variance vi is smallest instead of largest.

Z3~3 1 ~ccordi.ng to the present invention, the effect of the unutilized components wp~l, ..., wg~ is estimated by a recon-struction statistic R in the following manner. The terms dropped out of the expression for L (Equation 8F) contain the squares of the components wi, each weighted in accordance with its variance vi. All these variances can be approximated by a constant parameter c, which can then be factored out thus 96 w. 96 2 i=p+l vl Ci=p+l ' ~Equation 8G) The summation on the right is just the square of the Euclidean norm (length) of the vector w' = twp+~ , w96) (Equation 8H)-Define the vector wP to be w = (wl, O... ~ wp). (Equation 8I) Then ~ Wi2 = ¦w~ ¦2 =¦w¦2 _ ¦wPl2 (Equatlon 8J) i=p+l since the vectors w, w' and wP can be translated so as to form a right triangle. The eigenvector matrix E produces an orthogonal transformation, so the length of w is the same as the length of x-x. Therefore it is not necessary to compute all the components of w. The statistic sought, which estimates the effect of the unutilized components upon the log likelihood function L, is thus R = (¦x-x ¦2 _ I P¦~ I/2 (Equation 8K) This is the length of the difference between the observed vector x-x and the vector that would be obtained by attempting to reconstruct x-x as a linear combination of the first p ~1~7~3~3 1 eigenvectors ei f K. R thereEore has the character of a reconstruction error statistic. To utilize ~ in the likelihood function it may simply be adjoined to the set of transformed vector components to produce a new random vec-tor (wl,w2,...,wp,R) which is assumed to have independent Gaussian components. Under this assumption the new likelihood statistic is p (w.-w~ ) p ~ 1/2 ~ ln var(w ~ + M (Equation 8L) i=l var(wi) i=l where t~ M = 1/2 (R - R) ~ 1/2 ln var (R) (Equation 8M) var(R) and the barred variables are sample means and var~) denotes the unbiased sample variance. In Equation 8L the value of wl should be zero, and var(wi) should be equal to vi; however the e genvectors cannot be computed or applied with infinite arithmetic precision, so it is best to rerneasure the sample means and variances after transformation to reduce the system-atic statistical bias produced by arithmetic roundoff errors.
This remark applies also to ~quation 8F.

The measured performance of the likelihood statistic L' in the same maximum likelihood decision processor is plotted as graphs (c) and (d) of Figure 4. It can be seen that as p increases, the classification accuracy again reaches a maximum, but this time at a much smaller number p of dimensions. More-over the maxirnum accuracy achieved is noticeably higher than for the statistic L, which differs only by omission of the reconstruction error R.
As a further test of the efficacy of the reconstruction error statistic R, the same practical experiment was again 3~ repeated, but this time the likelihood function employed was Z3~3 1 simply L" = -M. (Equation 8N) That is, this time the region in which most of the sample data lie was ignored, while the reqions where relatively few samples are found was described. The maximum accuracy obtained (graphs (e) and (f) of Figure 4) is very nearly as high as for the statistic L', and the maximum occurs at a still smaller number of dimensions p=3. The result can be interpreted to mean that any data sample lying in the space of the first p eigenvectors of K can be accepted as belonging to the target pattern class, and that there is little or no benefit to be gained by making detailed probability estimates within that space.

Statistical Likelihood Calcula-tion The transformed data wi, corresponding to a formed multi-frame pattern x, are applied as inputs to the statistical likelihood calculation. This processor, as noted above, com-putes a measure of the probability that the unknown input speech, represented by the successively presented, transformed, multi-frame patterns, matches each of the target patterns of the keyword templates in the machine's vocabulary. Typically, each datum of a target pattern has a slightly skewed probability density, but nevertheless is well approximated statistically by a normal distribution having a mean value wi and a variance var(wi) where i is the sequential designation of the elements of the kth target pattern. The simplest implementation of the process assumes that the data associated with different values of i and k are uncorrelated so that the joint probability density for the datum x belonging to target pattern k is 3~3 1 (logarithmically) L(t~k) = p(x,k) = ~l/2 ln 2~(var~w~ l/2(wi _ = )2 ) var(wi) (Equation 9).
Since the logarithm is a monotonic function, this statistic is sufficient to determine whether the probability of a match with any one target pattern of a keyword template is greater than or less than the pxobability of a match with some other vocabulary target pattern, or alternatively whether the probability of a match with a particular pattern exceeds a predetermined minimum level. Each input multi-frame pattern has its s-tatistical likelihood L(t¦k) calculated for all of the target patterns o~ the keyword templates of the vocabulary.
The resulting likelihood statistics L(t¦k) are interpreted as the relative likelihood of occurrence of the target pattern named k at time t.
As will be well understood by those skilled in the art, the ranking of these likelihood statistics constitutes the speech recognition insofar as it can be performed from a single target pattern. These likelihood statistics can be utilized in various ways in an overall system, depending upon the ultimate function to be performed.
Selection of Candidate Keywords According to the preferred embodiment of the invention, if the likelihood statistic of a multi-frame pattern with res-pect to any first target pattern exceeds a predetermined threshold, the comparison being indicated at lOl, 103, the incoming data are studied further to determine first a local maximum for the li]selihood statistic corresponding to the Z3~i3 1 designated first target pattern, and second, whether other multi-frame pa-tterns exist which correspond to other patterns of the selec-ted potential candidate keywords. This is indicated at 105. Thus, the process of repetitively testing newly formed multi-spectrum frames against all first target patterns is interrupted; and a search begins for a pattern, occurring after the "first" multi~frame pattern, which best coxresponds, in a statistical likelihood sense, t;o the ne~t (second) t~get pattern of the potential candidate keyword(s).
If a "second" multi-frame pattern corresponding to the second target pattern(s) is not detected within a preset time window, the search sequence terminates, and the recognition process restarts at a time just after the end of the "first"
multi-frame pattern which identified a potential candidate keyword. Thus, after the "first" multi-frame pattern produces a likelihood score greater than the required threshold, a timing window is provided within which time a pattern matching the next target pattern in sequence corresponding to the selected potential candidate keyword(s) must appear.
~ The timing window may be variable, depending for example upon the duration of phonetic segments of the particular potential candidate keyword.
This process continues until either (1) multi-frame patterns are identified in the incoming data for all oE the target patterns of a keyword template or ~2) a target pattern cannot be associated with any pattern occurring within the allowed time window. If the search is terminated by condition (2), the search for a new "first" spectrum frame begins anew, as noted above, at the spectrum next following the end of the "first" previously :identified multi-frame pattern.

7;~363 1 At this processing level, the objective is to con-catenate possible multi-frame patterns corresponding to target patterns, and to form candidate words. (~his is indica-ted at 107). The detection thresholds are therefore set loosely so that it is very unlikely that a correct multi-frame pattern will be rejected, and here, at this acoustic processing level, discrimination between correct detec:tion and false alarms is obtained primarily by the req~lirement that a number of the pattern events must be detected jointly.

Post-Decision Processlng Processing at the acoustic level continues in this manner until the incoming audio signals terminate. However, even after a keyword is identified using the likelihood pro-bability test described above, additional post-decision pro-cessing tests (indicated at 109) are preferably used to decrease the likelihood of selecting an incorrect keyword (i.e.
to reduce the false alarm rate) while maintaining the probability of a correct detection as high as possible. ~or this reason, the output of the acoustic level processor, that i5, a candidate word selected by a concatenation process, is filtered further by a mask of prosodic relative timing windows and/or a likelihood ratio test which uses information from the acoustic level processor concerning all target pattern classes.
The Prosodic Mask As noted above, during the determination of the likeli-hood statistics, the time of occurrence of the multi~frame pattern having the local peak value of likelihood statistic relative to the active target pattern is found and in the preferred embodimen-t is recorded for each of the selected patterns ~:~7~3~3 1 corresponding to the several successive target patterns of a candidate keyword~ Those times, p-tl, pt2, ..., Ptn for each candi.date keyword are analyzed ancl evaluated according to a prede-termined prosodic mask for that keyword to determi.ne whether the time intervals between successive pattern likeli-hood peaks meet predetermined criteria. According to the method, the elapsed times between the times o:E the peak value of likelihood statistic, that is, pti-pti l' for i = 2, 3, ....
n~ are first normalized by dividiny each elapsed time i.nterval by: ptn-ptl. The resulting normalized intervals are compared with a prosodic mask, that is, a se~uence of allowable ranges of normalized interval length, for the candidate keyword, and if the interval lengths fall within the selected ranges, the candidate word is accepted.
In the illustrated embodiment the prosodic mask timing windows are determined by measuring the elapsed intervals for sample keywords spoken by as large a number of different speakers as possible. The prosodic pattern is then compared with the statistical sample keyword times using a statistical 2~ calculation wherein the mean and standard deviation for each prosodic mask (corresponding to each keyword) are derived from the keyword design set pattern samples. Thereafter, the likelihood statistic is calculated for deciding whether to accept and thus render a final decision with respect to the .candidate keyword. This likelihood statistic relates to the timing of events and is not to be confused with the likelihood statistic applied to the multi-frame patterns relative to the taryet patterns.
In another embodiment of the invention, the ranges of normalize interval duration are loosely set, but are inflexibly ~Z3~:i3 1 fixed. In this embodiment, a candidate keyword is accepted only if the normalized interval times fall within the fixed window boundaries. Thus a candidate word is acceptable only if each of the normalized times fall within the set limits.

Word-Level Likelihood Ratio Test In the preferred embodiment of -the invention, each candidate work is also tested according to a likelihood ratio test before a final decision to accept the ke~word is made~
The likelihood ratio test consists a summing a figure of merit over that sequence of selected multi~frame patterns which have been identified with the candidate keyword. The accumulated figure of merit, which is the sum of the figures of merit for each multi-frame pattern, is then compared with a decision threshold value.
The figure of merit for a detected multi-frame pattern is the difference between the best log likelihood statistic relative to any target pattern in the keyword vocabulary and the best score relative to those which are permitted choices for the target pattern. .Thus, lf the best scoring target pattern is a legal alternative for the pattern sought, the figure of merit has the value zero. However, if the best score corresponds to a target pattern not in the list of alternatives for the selected candidate word target pattern (a given target pattern may have several statistical spellings depending upon accents, etc.),then the figure of merit is the difference between the best score and the best among those that did appear in the list of alternates. The decision threshold is optimally p].aced to obtain the best balance between missed detection and false alarm rates.

1 Considering the word level likelihood ratio test from a mathematical point of view, the probability that a random multi-frame pattern x occurs, given that the input speech corresponds to target pattern class k, equals p(x¦k), read "the probability of x given k". The log likelihood statistic, then, of the input x relative to the kth reference pattern is L(x¦k) and equals ln p(x,k) as defined by Equation 9. Assuming that the detected multi-frame pattern must be caused by one of a group of n predefined target pattern classes, and assuming ~0 that either the classes occur with equal frequency or the n possible choices are considered to be equally valid, then the probability, in the sense of a relative frequency of occurrence, of observing the event x in any case is the sum of the probability densities defined by the summation:

n p(x) = ~ 1 p(xlk) n (Equation 10) of these occurrences, the proportion attributable to a given class, p~k¦x) equals:

p( ¦ ) n (Equation llA) p (x or logarithmically~ .

ln p(k¦x) = L(x¦k) ~ ln ~ p(x¦i) (Equation llB) If the decision processor is then given x, and for some reason chooses class k, then equation llA or llB above gives the probability that that choice is correct. The above equations are consequences of Bayers' rule:

~ Z3~3 1 p(x,k) - p(x¦k) p(k) = p(k¦x) p(X)V
wherein p(k) is taken to be the cons-tant nl .

If one assumes that only one class, say class m, is very likely, then equation 10 is approximated by p(x)~ max {p(x¦i) n- } p(x¦m) n (Equation 12) and we have ~(k,m,x) - L(x¦k) ~ L(x¦m) - ln p(k¦x). (Equation 13) Note that if the k~h class is the most likel~ one, then the function ~ assumes its maximum value zero. Summing o~er the set of presumed independent multi-frame patterns, the accumulated value of ~ estimates the probability that the detected word is not a false alarm. Elence, a decision threshold on the accumulated value of ~ relates directly to the trade-off between detection and flase alarm probabilities and is the basis of the likelihood ratio test. The accumulàted value of ~ then corresponds -to the figure of merit of the candidate keyword.
The realized system using the speech recognition method ~ s indicated previously, a presently preferred embodi-ment of the invention was constructed in which the signal and data manipulation, beyond that performed by the preprocessor of Figure 2, was implemented on and controlled by a Digital Equipment Ccrporation PDP-ll*computer working in combination with a special purpose processor such as that described in copending Canadian application No. 313,111.
The detailed programs which provide the functions described in relation to the flow chart of Figure 3 are set foxth in the following appendices. The program printou-ts are in *Trade Mark - 36 ~

; ,, bi3 1 MACRO-ll and FORTR~N lanc3uages provided by the Digital Equipment Corporation with its PDP-ll*computers and in the machine language of the special purpose processor.
In view of the foregoing, it may be seen that several objects of the present invention are achieved and other advantageous results have been obtained.
It will be appreciated that the continuous speech recognition method described herein includes isolated speech recognition as a special application. Other applications oE
the continuous speech method described hexein, including additions, subtractions, deletions, and other modifications of the described preferred embodiment, will be obvious to those skilled in the art, and are within the scope of the following claims.

*Trade Mark ,,

Claims (12)

The embodiments of the invention in which an exclusive property or privilege is claimed are defined as follows:

1. In a speech analysis system for recognizing at least one predetermined keyword in an audio signal, each said keyword being characterized by a template having at least one target pattern, said target patterns having an ordered sequence and each target pattern representing at least one short term power spectrum, an analysis method comprising the steps of repeatedly evaluating a set of parameters determining a short-term power spectrum of said audio signal within each of a plurality of equal duration sampling intervals, thereby to generate a continuous time ordered sequence of short-term audio power spectrum frames, repeatedly generating a peak spectrum corresponding to said short-term power spectrum frames by a fast attack,slow decay peak detecting function, and for each short-term power spectrum frame, dividing the amplitude of each frequency band by the corresponding-intensity value in the corresponding peak spectrum, thereby to generate a frequency bank equalized spectrum frame corresponding to a compensated audio signal having the same maximum short-term energy content in each of the frequency bands comprising the frame, and identifying a candidate keyword template when said selected multi-frame patterns correspond respectively to the target patterns of a said keyword template.

2. The method of claim 1 wherein generating said peak spectrum further includes the step of selecting the value of each of the peak spectrum fre-quency bands from the maximum of

Claim 2 continued (a) the current peak spectrum value multiplied by a constant decay factor having a value less than one, and (b) the incoming new spectrum frame value.

3. In a speech analysis system in which an audio signal is spectrum analyzed for recognizing at least one predetermined keyword in a continuous audio signal, each said keyword being characterized by a template having at least one target pattern, said target patterns having an ordered sequence and each target pattern representing a plurality of short-term power spectra spaced apart in real time, an analysis method comprising the steps of repeatedly evaluating a set of parameters determining a short-term power spectrum of said audio signal with each of a plurality of equal duration sampling intervals, thereby to generate a continuous time ordered sequence of short-term audio power spectrum frames, repeatedly generating a peak spectrum corresponding to said short-term power spectrum frames by a fast attack, slow decay peak detecting function, for each short-term power spectrum frame, dividing the amplitude of each frequency band by the corresponding intensity value in the corresponding peak spectrum, thereby to generate a frequency band equalized spectrum frame corresponding to a compensated audio signal having the same maximum short-term energy content in each of the frequency bands comprising the spectrum, repeatedly selecting from said sequence of equalized frames, one first frame and at least one later occurring frame to form a multi-frame pattern,

Claim 3 continued comparing each thus formed multi-frame pat-tern with each first target pattern of each keyword template, deciding whether each said multi-frame pattern corresponds to a said first target pattern of a keyword -template, for each multi-frame pattern which, according to said deciding step, corresponds -to a said first target pattern of a potential candidate keyword, selecting later occurring short-term power spectrum equalized frames to form later occurring multi-frame patterns, deciding whether said later occurring multi-frame patterns correspond respectively to successive target patterns of said potential candidate keyword template, and identifying a candidate keyword template when said selected multi-frame patterns correspond respectively to the target patterns of a said keyword template.

4 The method of claim 3 wherein generating said peak spectrum further includes the step of selecting the value of each of the peak spectrum frequency bands from the maximum of (a) the current peak spectrum value multiplied by a constant decay factor having a value less than one, and (b) the incoming new spectrum frame value.

5. In a pattern recognition system for identifying in a data stream at least one target pattern characterized by a vector of recognition elements Xi' said elements having a statistical distrubution, the analysis method comprising the steps of determining from plural design set pattern samples X
of the target pattern a covariance matrix K;

Claim 5 continued determining from said plural design set patterns an expected value vector ?;
calculating from the covariance matrix K, a plurality of eigenvectors ei having eigenvalues vi where vi ? vi + l;
selecting unknown patterns y from said data stream;
transforming each pattern y into a new vector (W1, W2, ..., Wp, R) where Wi = ei (y - ?), p is a positive integer constant less than the number of elements of the pattern y, and R is the reconstruction error statistic and equals ; and deciding, by applying a likelihood statistic function to said new vector (W1, W2, ..., Wp, R), whether said pattern y is identified with the target pattern.

6. The method of claim 5 further including the step of calculating a likelihood statistic ?' according to the equation:

where the barred variables are sample means, and var ( ) is the unbiased sample variance.

7. The method of claim 5 further including the step of calculating a likelihood statistic ?" according to the equation:

where the barred variables are sample means, and var ( ) is the unbiased sample variance.

8. In a speech analysis system for recognizing at least one predetermined keyword in a continuous audio signal, each said keyword being characterized by a template having at least one target pattern, said target patterns having an ordered sequence and each target pattern representing a plurality of short-term power spectra spaced apart in real time, an analysis method comprising the steps of for each target pattern, determining from plural design set pattern samples ? of a said target pattern having elements ?i, a covariance matrix K;
determining from said plural design set patterns an expected value vector ?;
calculating from the covariance matrix K a plurality of eigenvectors ei having eigenvalues vi where vi ? vi + 1;
repeatedly evaluating a set of parameters determining a short-term power spectrum of said audio signal within each of a plurality of equal duration sampling intervals, thereby to generate a continuous time ordered sequence of short-term audio power spectrum frames;
repeatedly selecting from said sequence of frames, one first frame and at least one later occurring frame to form a multi-frame pattern y;
transforming each multi-frame pattern y into new vectors W, represented as (W1, W2,..., Wp, R), where Wi = ei (y - ?);
p is a positive integer constant less than the number of elements of the pattern y, and R is the reconstruction error statistic and equals {¦y - ?¦2 - Wj2}1/2 ;

Claim 8 continued deciding whether each said transformed pattern corresponds to a said first target pattern of a keyword template;
for each pattern which, according to said deciding step, corresponds to a said first target pattern of a potential candidate keyword, selecting later occurring short-term power spectra to form later occurring multi-frame patterns;
deciding whether said later occurring multi-frame patterns correspond respectively to successive target patterns of said potential candidate keyword template; and identifying a candidate keyword template when said selected multi-frame patterns correspond respectively to the target patterns of a said keyword template.

9. The method of claim 8 wherein the deciding steps each include the step of calculating a likelihood statistic ?' according to the equation:
?' = where the barred variables are sample means, and var ( ) is the unbiased sample variance.

10. The method of claim 8 wherein the deciding steps each include the step of calculating a likelihood statistic ?" according to the equation:

Claim 10 continued ?" = where the barred variables are sample means, and var ( ) is the unbiased sample variance.

11. The method of claim 8 further including the steps of repeatedly generating a peak spectrum corresponding to said short-term power spectrum frames by a fast attack, slow decay peak detecting function, and for each short-term power spectrum frame dividing the amplitude of each frequency band by the corresponding intensity value in the corresponding peak spectrum, thereby to generate a frequency band equalized spectrum frame corresponding to a compensated audio signal having the same maximum short-term energy content in each of the frequency bands comprising the frame.

12. The method of claim 11 wherein generating said peak spectrum further includes the step of selecting the value of each of the peak spectrum frequency bands from the maximum of (a) the current peak spectrum value multiplied by a constant decay factor having a value less than one, and (b) the incoming new spectrum value.