CA2158849C - Speech recognition with pause detection - Google Patents
Speech recognition with pause detection Download PDFInfo
- Publication number
- CA2158849C CA2158849C CA002158849A CA2158849A CA2158849C CA 2158849 C CA2158849 C CA 2158849C CA 002158849 A CA002158849 A CA 002158849A CA 2158849 A CA2158849 A CA 2158849A CA 2158849 C CA2158849 C CA 2158849C
- Authority
- CA
- Canada
- Prior art keywords
- noise
- parameter
- pattern
- recognition
- signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 18
- 238000012545 processing Methods 0.000 claims abstract description 24
- 238000005192 partition Methods 0.000 claims abstract description 5
- 238000000034 method Methods 0.000 claims description 28
- 230000008569 process Effects 0.000 claims description 16
- 238000000638 solvent extraction Methods 0.000 claims description 8
- 230000000694 effects Effects 0.000 claims description 6
- 238000009826 distribution Methods 0.000 claims description 4
- 238000009499 grossing Methods 0.000 claims description 4
- 230000002123 temporal effect Effects 0.000 claims 3
- 238000007781 pre-processing Methods 0.000 claims 1
- 238000012360 testing method Methods 0.000 description 28
- 238000010586 diagram Methods 0.000 description 16
- 239000013598 vector Substances 0.000 description 9
- 238000004364 calculation method Methods 0.000 description 7
- 230000007704 transition Effects 0.000 description 6
- 230000009471 action Effects 0.000 description 4
- 238000012935 Averaging Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000001186 cumulative effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 102100028247 Abl interactor 1 Human genes 0.000 description 1
- 108050004693 Abl interactor 1 Proteins 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/78—Detection of presence or absence of voice signals
- G10L25/87—Detection of discrete points within a voice signal
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L17/00—Speaker identification or verification
- G10L17/02—Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computational Linguistics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Telephonic Communication Services (AREA)
- Character Discrimination (AREA)
- Indicating And Signalling Devices For Elevators (AREA)
- Machine Translation (AREA)
Abstract
A recognition system comprising: input means for receiving a speech signal; recognition processing means for processing the speech signal to indicate its similarity to predetermined patterns to be recognised, said recognition processing means being arranged repeatedly to partition the speech signal into a pattern-containing portion and, preceding and following said pattern-containing portions, noise or silence portions, and to identify a pattern corresponding to said pattern containing portion; and output means for supplying a recognition signal indicating recognition of one of said patterns, characterised by pause detection means for detecting the noise or silence portion which follows the pattern-containing portion, and means, responsive to the detection thereof, arranged to supply a signal identifying the pattern currently corresponding to the pattern portion to the output means. Also provided are similarly operating rejection means.
Description
WO94/22131 215 ~ 8 4 9 PCT/GB94/00~0 SPEECH RECOGNITION WITH PAUSE DETECTION
This invention relates to methods and apparatus for speech recognitlon. Speech recognition is used as an input means for control of machines. At present, speech recognition apparatus generally recognises isolated single words. Speech recognition apparatus is also belng developed which is intended to recognise multiple words spoken consecutively in a sentence or phrase; this is referred to as connected speech recognition.
In speech recognition, a microphone picks up a speech signal from a speaker which is then digitised and processed for recognition. However, the mlcrophone generally also picks up any background or ambient noise and the electrical system between the microphone and the speech recognition apparatus will likewise add noise (e.g. thermal noise, quantising noise and - where the speech is transmitted through a telecommunications channel - line noise). The noise may resemble parts of speech, for example unvoiced sibilant sounds. Accordingly, the correct recognition of a word depends strongly on the abi 1 i ty to distinguish the beginning and the end of the word, which correspond to the end and beginning of noise or silence. The reliability of speech recognition has been shown to depend strongly on the ldentification of the correct start and end points for speech.
One speech processing method which is intended to allow the recognition of a sequence of words using isolated word recognition technology is the "connected - for - isolated"
(CFI) technique, described in our co-pending EP patent application 93302538.9 and incorporated herein by reference.
This technique assumes that the signal from the microphone will include alternating periods of speech and noise, and attempts to recognise alternately speech and noise.
A common approach in speech recognition is to use statistical processing, making no initial assumptions about the mechanisms by which speech is produced. For example, hidden Markov modelling (HMM) techniques are used (as described in British Telecom Technology Journal, April 1988,
This invention relates to methods and apparatus for speech recognitlon. Speech recognition is used as an input means for control of machines. At present, speech recognition apparatus generally recognises isolated single words. Speech recognition apparatus is also belng developed which is intended to recognise multiple words spoken consecutively in a sentence or phrase; this is referred to as connected speech recognition.
In speech recognition, a microphone picks up a speech signal from a speaker which is then digitised and processed for recognition. However, the mlcrophone generally also picks up any background or ambient noise and the electrical system between the microphone and the speech recognition apparatus will likewise add noise (e.g. thermal noise, quantising noise and - where the speech is transmitted through a telecommunications channel - line noise). The noise may resemble parts of speech, for example unvoiced sibilant sounds. Accordingly, the correct recognition of a word depends strongly on the abi 1 i ty to distinguish the beginning and the end of the word, which correspond to the end and beginning of noise or silence. The reliability of speech recognition has been shown to depend strongly on the ldentification of the correct start and end points for speech.
One speech processing method which is intended to allow the recognition of a sequence of words using isolated word recognition technology is the "connected - for - isolated"
(CFI) technique, described in our co-pending EP patent application 93302538.9 and incorporated herein by reference.
This technique assumes that the signal from the microphone will include alternating periods of speech and noise, and attempts to recognise alternately speech and noise.
A common approach in speech recognition is to use statistical processing, making no initial assumptions about the mechanisms by which speech is produced. For example, hidden Markov modelling (HMM) techniques are used (as described in British Telecom Technology Journal, April 1988,
2 i 5 ~ 8 4 9 2 PCT/GB94/00630 vol 6 Number 2 page 105, Cox). In HMM recognition, each incoming frame of speech is compared with a number of states, to determine the likelihood of the speech frame corresponding to each of those states, and the state probabilities thus generated are compared with a number of predetermined models comprising state sequences corresponding to different words to be recognised. Whilst a word is being recognised, a number of different state sequences, and henc`è a number of different words, are simultaneously possible; the ~inal determination of which state sequence was observed is made by selecting the most likely state sequence when the whole utterance is recelved .
Some types of HMM speech recognition maintain, during recognition, a number of possible state sequences, including a current most probable sequence for defining the word which has been recognised.
In such sequential recognisers, since the decision as to the identity of the selected word is based on the sequences of states generated, the decision cannot be made until the sequence is complete. The most likely state sequence can be recalculated for each received frame, so that as soon as the end of a word can unambiguously be identified, recognition is performed by simply outputting a recognition signal corresponding to the current most likely state sequence. The recognition process will itself produce start and end points, but this is done in conjunction with the selection of the word which is recognised and not as a separate, preliminary, end pointing step.
A CFI recogniser is therefore able to automatically locate the start and end of a word, by maintaining state sequences corresponding to noise, and recognising the sequence of noise - word - noise in the speech signal. However, many words may include gaps or stops between parts of the word, which might be misrecognised as the end of a word.
Accordingly, it is important that the final identification of a word should not take place until the speaker has definitely finished speaking.
One method of achieving this is to provide a "time out"
after a predetermined time period which can unambiguously be identified as noise. However, we have found that if the period concerned is made long enough to guarantee success, the result is a delay which can be frustratingly long to the user.
One aspect of the invention therefore provides a means of detecting the end of speech for a recogniser of the type in which a most likely state sequence i9 selected.
In one aspect, the invention provides a speech recognition system comprising means for successively generating recognition outputs based on partitioning an input utterance into a speech portion and a following noise portion, characterised by means for detecting the presence of the following noise portion by testing the partitioning using a parameter derived from the input speech signal. Preferably, the or each parameter is based on the energy of the input speech signal. Preferably, the parameter comprises a parameter indicating the relative magnitudes of the speech portion and the noise portion indicated by the said partition.
Additionally or alternatively, the parameter provides a measure of the variation of the energy of the noise portion according to the partition. Preferably, in either case, the energy is smooth or averaged over several successive time periods, and preferably the averaging is non-linear so as to limit the influence of short spikes of energy differing from the running average.
Where speech recognition apparatus has recognised a word by selecting the most likely possible word, the possibility exists that the recognition was made in error, based either on a similar word (for example one not in the vocabulary of the recogniser) or noise.
Means for rejecting the recognition of certain misrecognised words are described in l'Rejection of extraneous input in speech recognition applications, using multilayer perceptrons and the trace of HMM's", Mathan and Miclet, l99l IEEE ICASSP 9l vol 1 pages 93-96, and in ~Rejection techniques 35 in continuous speech recognition using hidden Markov models", Moreno et al, Signal processing V:Theories and Applications, l990, Proc. of EUSIPCO - 90 vol 2 pages 1383-1386 (Elsevier).
Accordingly, the object of another aspect of the invention is to provide an improved means for rejecting WO94122131 5 88 ~9 PCT/GB94100630 certain words after they have been identified by a speech recognlser.
Accordingly, one aspect of the invention provides a recognition system comprising: input means for receiving a speech signal; recognition processing means for processing the speech signal to indicate its similarity to predetermined patterns to be recognised; output means for supplying a recognition signal indicating recognition of one of said patterns; and rejection means for re~cting the recognition signal under predetermined conditio ~, characterised in that the rejection means are arranged to receive at least one signal parameter derived from said speech signal which does not depend upon the output of said recognition means.
As a speech signal varies spectrally rather slowly, it is known in speech recognition to partition the speech signal into a time series of frames of duration tvpically between lO
to l00 milliseconds, comprising a plurality of speech samples.
It has been proposed ("The use of variable frame rate analysis in speech recognition", Ponting and Peeling, Computer Speech and Language (l99l) 5, 169-179) to limit the number of frames thus generated, by only generating a new frame when the speech signal has changed significantly from the previous frame.
Accordingly, a further aspect of the invention provides a pause detection means, and/or a rejection means, for use in a recogniser employing a variable frame rate.
Other aspects and embodiments of the invention are as described and/or claimed herein, with advantages which will be apparent from the following description and drawings.
The invention will now be described, by way of example only, with reference to the accompanying drawings, in which:
Figure l shows schematically an application of a recognition system according to the present invention;
Figure 2 is a block diagram showing schematically the elements of a recognition processor forming part of Figure l according to an embodiment of the invention;
Figure 3 is a block diagram indicating schematically the components of a classifier forming part of the embodiment of Figure 2;
Figure 4 is a flow diagram showing schematically the 21~88~9 operation of the classifier of Figure 3;
Figure 5 is a block diagram showing schematically the structure of a sequence parser forming part of the embodiment of Figure 2;
Figure 6 shows schematically the content of a field within a store forming part of Figure 5i Figure 7 shows schematically the content of a buffer forming part of Figure 5;
Figure 8 is a flow diagram showing schematically the operation of the sequence parser of Figure 5i Figure 9 is a block diagram indicating the structure of a pause detector forming part of the embodiment of Figure 2;
Figure 10 is a block diagram showing schematically a part of the structure of Figure 9 in greater detail;
Figure 11 is a flow diagram illustrating the operation of an averager forming part of Figure 10;
Figure 12 is a flow diagram illustrating the process of deriving a signal to noise ratio by the apparatus of Figure 10;
Figure 13 is a flow diagram illustrating the process of forming a measure of signal variance by the apparatus of Figure 10;
Figure 14 is a block diagram showing in greater detail a part of the structure of Figure 10;
Figure 15 is a block diagram showing in greater detail the combination logic forming part of Figure 10;
Figure 16 is a diagram of energy and averaged energy of a speech signal over time and indicating the correspondence with signal frames;
Figure 17 is a flow diagram illustrating the operation of a rejector forming part of Figure 2;
Figure 18 is a flow diagram corresponding to Figure 11 illustrating the process of deriving an average in a second embodiment of the invention; and Figure 19 is a diagram of energy and averaged energy against time in the embodiment of Figure 18, and corresponds to Figure 16.
WO94/22131 21 S 8 8 ~ 9 PCT/GB94/00~0 FIRST EMBODIMENT
Referring to Figure l, a telecommunications system including speech recognition generally comprises a microphone l typically forming part of a telephone handset, a telecommunications network (typically a public switched telecommunications network (PSTN)) 2, a recognition processor
Some types of HMM speech recognition maintain, during recognition, a number of possible state sequences, including a current most probable sequence for defining the word which has been recognised.
In such sequential recognisers, since the decision as to the identity of the selected word is based on the sequences of states generated, the decision cannot be made until the sequence is complete. The most likely state sequence can be recalculated for each received frame, so that as soon as the end of a word can unambiguously be identified, recognition is performed by simply outputting a recognition signal corresponding to the current most likely state sequence. The recognition process will itself produce start and end points, but this is done in conjunction with the selection of the word which is recognised and not as a separate, preliminary, end pointing step.
A CFI recogniser is therefore able to automatically locate the start and end of a word, by maintaining state sequences corresponding to noise, and recognising the sequence of noise - word - noise in the speech signal. However, many words may include gaps or stops between parts of the word, which might be misrecognised as the end of a word.
Accordingly, it is important that the final identification of a word should not take place until the speaker has definitely finished speaking.
One method of achieving this is to provide a "time out"
after a predetermined time period which can unambiguously be identified as noise. However, we have found that if the period concerned is made long enough to guarantee success, the result is a delay which can be frustratingly long to the user.
One aspect of the invention therefore provides a means of detecting the end of speech for a recogniser of the type in which a most likely state sequence i9 selected.
In one aspect, the invention provides a speech recognition system comprising means for successively generating recognition outputs based on partitioning an input utterance into a speech portion and a following noise portion, characterised by means for detecting the presence of the following noise portion by testing the partitioning using a parameter derived from the input speech signal. Preferably, the or each parameter is based on the energy of the input speech signal. Preferably, the parameter comprises a parameter indicating the relative magnitudes of the speech portion and the noise portion indicated by the said partition.
Additionally or alternatively, the parameter provides a measure of the variation of the energy of the noise portion according to the partition. Preferably, in either case, the energy is smooth or averaged over several successive time periods, and preferably the averaging is non-linear so as to limit the influence of short spikes of energy differing from the running average.
Where speech recognition apparatus has recognised a word by selecting the most likely possible word, the possibility exists that the recognition was made in error, based either on a similar word (for example one not in the vocabulary of the recogniser) or noise.
Means for rejecting the recognition of certain misrecognised words are described in l'Rejection of extraneous input in speech recognition applications, using multilayer perceptrons and the trace of HMM's", Mathan and Miclet, l99l IEEE ICASSP 9l vol 1 pages 93-96, and in ~Rejection techniques 35 in continuous speech recognition using hidden Markov models", Moreno et al, Signal processing V:Theories and Applications, l990, Proc. of EUSIPCO - 90 vol 2 pages 1383-1386 (Elsevier).
Accordingly, the object of another aspect of the invention is to provide an improved means for rejecting WO94122131 5 88 ~9 PCT/GB94100630 certain words after they have been identified by a speech recognlser.
Accordingly, one aspect of the invention provides a recognition system comprising: input means for receiving a speech signal; recognition processing means for processing the speech signal to indicate its similarity to predetermined patterns to be recognised; output means for supplying a recognition signal indicating recognition of one of said patterns; and rejection means for re~cting the recognition signal under predetermined conditio ~, characterised in that the rejection means are arranged to receive at least one signal parameter derived from said speech signal which does not depend upon the output of said recognition means.
As a speech signal varies spectrally rather slowly, it is known in speech recognition to partition the speech signal into a time series of frames of duration tvpically between lO
to l00 milliseconds, comprising a plurality of speech samples.
It has been proposed ("The use of variable frame rate analysis in speech recognition", Ponting and Peeling, Computer Speech and Language (l99l) 5, 169-179) to limit the number of frames thus generated, by only generating a new frame when the speech signal has changed significantly from the previous frame.
Accordingly, a further aspect of the invention provides a pause detection means, and/or a rejection means, for use in a recogniser employing a variable frame rate.
Other aspects and embodiments of the invention are as described and/or claimed herein, with advantages which will be apparent from the following description and drawings.
The invention will now be described, by way of example only, with reference to the accompanying drawings, in which:
Figure l shows schematically an application of a recognition system according to the present invention;
Figure 2 is a block diagram showing schematically the elements of a recognition processor forming part of Figure l according to an embodiment of the invention;
Figure 3 is a block diagram indicating schematically the components of a classifier forming part of the embodiment of Figure 2;
Figure 4 is a flow diagram showing schematically the 21~88~9 operation of the classifier of Figure 3;
Figure 5 is a block diagram showing schematically the structure of a sequence parser forming part of the embodiment of Figure 2;
Figure 6 shows schematically the content of a field within a store forming part of Figure 5i Figure 7 shows schematically the content of a buffer forming part of Figure 5;
Figure 8 is a flow diagram showing schematically the operation of the sequence parser of Figure 5i Figure 9 is a block diagram indicating the structure of a pause detector forming part of the embodiment of Figure 2;
Figure 10 is a block diagram showing schematically a part of the structure of Figure 9 in greater detail;
Figure 11 is a flow diagram illustrating the operation of an averager forming part of Figure 10;
Figure 12 is a flow diagram illustrating the process of deriving a signal to noise ratio by the apparatus of Figure 10;
Figure 13 is a flow diagram illustrating the process of forming a measure of signal variance by the apparatus of Figure 10;
Figure 14 is a block diagram showing in greater detail a part of the structure of Figure 10;
Figure 15 is a block diagram showing in greater detail the combination logic forming part of Figure 10;
Figure 16 is a diagram of energy and averaged energy of a speech signal over time and indicating the correspondence with signal frames;
Figure 17 is a flow diagram illustrating the operation of a rejector forming part of Figure 2;
Figure 18 is a flow diagram corresponding to Figure 11 illustrating the process of deriving an average in a second embodiment of the invention; and Figure 19 is a diagram of energy and averaged energy against time in the embodiment of Figure 18, and corresponds to Figure 16.
WO94/22131 21 S 8 8 ~ 9 PCT/GB94/00~0 FIRST EMBODIMENT
Referring to Figure l, a telecommunications system including speech recognition generally comprises a microphone l typically forming part of a telephone handset, a telecommunications network (typically a public switched telecommunications network (PSTN)) 2, a recognition processor
3, connected to receive a voice signal from the network 2, and a utilising apparatus 4 connected to the r`ecognition processor 3 and arranged to receive therefrom a speech recognition signal, indicating recognition or otherwise of particular words or phrases, and to take action in response thereto. For example, the utilising apparatus 4 may be a remotely operated banking terminal for effecting banking transactions.
In many cases, the utilising apparatus 4 will generate an auditory response to the speaker, transmitted via the network 2 to a loudspeaker 5 typically forming a part of the subscriber handset.
In operation, a speaker speaks into the microphone l and an analog speech signal is transmitted from the microphone l into the network 2 to the recognition processor 3, where the speech signal is analysed and a signal indicating identification or otherwise of a particular word or phrase is generated and transmitted to the utilising apparatus 4, which then takes appropriate action in the event of recognition of an expected word or phrase.
For example, the recognition processor 3 may be arranged to recognise digits 0 to 9, "yes" and "no" so as to be able to recognise personal identification numbers and a range of command words for initiating particular actions (for example, requesting statements or particular services).
Referring to Figure 2, the recognition processor 3 comprises an input 31 for receiving speech in digital form (either from a digital network or from an analog to digital converter), a frame processor 32 for partitioning the succession of digital samples into frames of contiguous samples; a feature extractor 33 for generating from the frames of samples a corresponding feature vector; a classifier 34 receiving the succession of feature vectors and operating on 2158~9 each with the plurality of models corresponding to different words, phonemes or phrases to generate recognition results;
and a parser 35 which is arranged to receive the classification results from the classifier 34 and to determine the word to which the sequence of classifier outputs indicates the greatest similarity.
Also provided is a recognition rejector 36 arranged to reject recognition of a word recognised by the parser 35 if recognition is unreliable, and a pause detector 37, arranged to detect the pause following the end of a word to enable the parser 35 to output a word recognition signal. The word recognition signal from the parser 35, or a rejection signal from the rejector 36, is output to a control signal output 38, for use in controlling the utilising apparatus 4.
Frame Generator 32 The frame generator 32 is arranged to receive speech samples at a rate of, for example, 8,000 samples per second, and to form frames comprising 256 contiguous samples, at a frame rate of 1 frame every 16ms. Preferably, each frame is windowed (i.e. the samples towards the edge of the frame are multiplied by predetermined weighting constants) using, for example, a ~mmlng window to reduce spurious artifacts, generated by the frame edges. In a preferred em.bodiment, the frames are overlapping (for example by 50~) so as to ameliorate the effects of the windowing.
Feature Extractor 33 The feature extractor 33 receives frames from the frame generator 32 and generates, in each case, a set or vector of features. The features may, for example, comprise cepstral coefficients (for example, LPC cepstral coefficients or mel frequency cepstral coefficients as described in "On the Evaluation of Speech Recognisers and Data Bases using a Reference System", Chollet & Gagnoulet, 1982 proc. IEEE
p2026), or differential values of such coefficients comprising, for each coefficient, the difference between the WO94/22131 ~ 88 ~9 8 PCT/GB94/00630 coefficient and the corresponding coefficient value in the preceding frame, as described in "On the use of Instantaneous and Transitional Spectral Information in Speaker Recognition", Soong & Rosenberg, 1988 IEEE Trans. on Accoustics, Speech and Signal Processing Vol 36 No. 6 p871. Equally, a mixture of several types of feature coefficient may be'used.
For reasons that will be discusse~ below, in this embodiment the feature extractor 33 also èxtracts a value for the energy in each frame (which energy value may, but need not, be one of the feature coefficients used in recognition).
The energy value may be generated as the sum of the squares of the samples of the frame.
Finally, the feature extractor 33 outputs a frame number, incremented for each successive frame.
The frame generator 32 and feature extractor 33 are, in this embodiment, provided by a single suitably programmed digital signal processor (DSP) device (such as the Motorola DSP'56000, the Texas Instruments TMS C 320 or similar device.
Classifier 34 Referring to Figure 3, in this embodiment, the classifier 34 comprises a classifying processor 341 and a state memory 342.
The state memory 342 comprises a state field 3421, 3422, ...., for each of the plurality of speech states. For example, each word to be recognised by the recognition processor comprises 6 or 8 states, and accordingly 6 or 8 state fields are provided in the state memory 342 for each word to be recognised. There are also provided a state field for noise/silence at the beginning of a word and a state field for a noise/silence state at the end of a word (although it might in practice be possible to provide only a single noise state).
Each state field in the state memory 342 comprises data defining a multidimensional Gaussian distribution of feature coefficient values which characterise the state in question.
For example, if there are d different feature coefficients, the data characterising a state are a constant ~ WO94122131 215 8 8 4 9 PCT/GB94100630 C, a set of d feature mean values ~i and a set of d feature deviations, ai; in other words, a total of 2d + 1 numbers.
The classification processor 34 is arranged to read each state field within the memory 342 in turn, and calculate for each, using the current input feature coefficient set, the probability that the input feature set or vector corresponds to the corresponding state. To do so, as shown in Figure 4, the processor 341 is arranged to calculate an equation P=C-- 1~ (X~ )2 2i~l a2i It is possible for a single state to be represented by several different modes or distributions; accordingly, the state memory 342 may comprise, for each state, several mode fields each corresponding to the state field described above, in which case the classification processor 341 is arranged to calculate for each mode the probability that the input vector corresponds to that mode, and then to sum the modal probabilities (weighted as appropriate).
Accordingly, the output of the classification processor 341 is a plurality of state probabilities, one for each state in the state memory 342, indicating the likelihood that the input feature vector corresponds to each state.
The classifying processor 341 may be a suitably programmed digital signal processing (DSP) device, may in particular be the same digital signal processing device as the feature extractor 33.
Parser 35 Referring to Figure 5, the parser 35 in this embo~;mPnt comprises a state sequence memory 352, a parsing processor 351, and a parser output buffer 354.
Also provided is a state probability memory 353 which stores, for each frame processed, the state probabilities output by the probability processor 341. The state sequence memory 352 comprises a plurality of state sequence fields WO94/22131 215 8 8 ~ 9 1o PCT/GB94/00630 3521, 3522, ...., each corresponding to a noise-word-noise sequence to be recognised (and one corresponding to a noise-only sequence).
Each state sequence in the state sequence memory 352 comprises, as illustrated in Figure 6, a number of states Pl, P2, PN (where N is 6 or 8) and, for each state, two probabilities; a repeat probability (Pil)y~and a transition probability to the following state~ Pi2). For a CFI
recogniser, the first and final states~àre noise states. The observed sequence of states associated with a series of frames may therefore comprise several repetitions of each state Pi in each state sequence model 3521 etc; for example:
Frame Number 1 2 3 4 5 6 7 8 9 ...................... Z Z+l State Pl Pl Pl P2 P2 P2 P2 P2 P2 .............. Pn Pn Thus, at some frame number (here, frame number 3) the observed sequence will move from the initial, noise, state to the next, speech, state; this transition marks the start of the word to be recognised. Likewise, at some frame (here frame Z) the sequence reaches the last state Pn corresponding to noise or silence following the end of the word to be recognised. Frame Z therefore corresponds to the end of the word to be recognised.
As shown in Figure 8 the parsing processor 351 is arranged to read, at each frame, the state probabilities output by the probability processor 341 and the previous stored state probabilities in the state probability memory 353 and to calculate the most likely path of states to date over time, and to compare this with each of the state sequences stored in the state sequence memory 352.
The calculation employs the well known hidden Markov model method described in the above referenced Cox paper.
Conveniently, the HMM processing performed by the parsing processor 351 uses the well known Viterbi algorithm. The parsing processor 351 may, for example, be a microprocessor such as the Intel(~ 486(~) microprocessor or the Motorola(~) 68000 microprocessor, or may alternatively be a DSP device WO94/22131 ~15 8 8 4 9 PCT/GB94/00630 (for example, the same DSP device as is employed for any of the preceding processors).
Accordingly for each state sequence (corresponding to a word) a probability score is output by the parser processor 5 351 at each frame of input speech. The identity of the most likely state sequence (and hence word recognised) may well change during the duration of the utterance by the speaker.
The parser output buffer 354 comprises a plurality of fields 3541, 3542, ... each corresponding to a word to be recognised (and one which corresponds to a noise-only sequence). Each field, as shown illustratively in Figure 7, comprises a probability score S indicating, for the current frame, the likelihood of the corresponding word being present, and two frame numbers; a first (sp_st) which indicates the 15 first frame of the word in the noise-word-noise observed sequence of frames; and a second (sp_end) which indicates the last frame of the word. Before sp_st the states in the observed sequence comprise initial noise and after sp_end, the states in the observed sequence correspond to terminal noise.
20 Naturally, the frame numbers in each of the fields 3541, 3542, .... differ from one another.
Pause Detector 37 Referring to Figure 9, the pause detector 37 comprises a signal based detector 370 and a model based detector 375.
25 The signal based detector 370 is connected to the feature extractor 33, to receive a parameter extracted from the speech signal. In this present embodiment, the parameter is the frame energy, or some parameter based on the frame energy.
The model based detector 375 is connected to the parser 30 35, to receive an indication of the current best state sequence. Specifically, the model based detector 375 is arranged to read from the parser output buffer 354 the frame - number (sp_end) of the start of final noise states, if any, in the current most probable state sequence and to subtract 35 this from the current frame number to find the length of the period following the end of the word which is currently assumed to be recognised.
WO94122131 215 88 ~9 PCTtGB94/00630 The output of the signal based pause detector 370 and the model based pause detector 375 are combined by logic 378 to generate a pause detection signal at an output 379.
Referring to Figure 10 the signal based pause detector 370 comprises a running averager 371 which maintains a running average energy level over a number of preceding energy values;
a signal to noise ratio (SNR) de~ector 372 and a noise variance (NVR) detector 373, the outputs of which are supplied to be combined by logic 378.
Also provided is a mean energy level buffer 376, connected to the output of the averager 371 to store successive mean energy values corresponding to successive frames.
Runninq Averaqer 371 The running averager 371 is arranged schematically to perform the process shown in Figure 11. In this process, in this embodiment, for each frame the energy of the frame i8 read from the feature extractor 33, and subtracted from a stored running average value to yield the difference value.
The difference value is compared with a threshold or step of predetermined absolute value. If the difference lies within +/- the step value, the running average is unaffected, but the value of the step is reduced by setting it equal to the difference divided by a constant factor or, as indicated in Figure 11, a first constant factor (upfactor) for a positive difference from the running mean and a second factor (downfactor) for a negative difference from the running mean.
If, on the other hand, the difference between the present frame input value and the stored running average exceeds the step value, then the running average is incremented or decremented by the step value depending upon the size of the difference. The step value is then updated as before.
The effect of this process is as follows. Firstly, there is a smoothing of the energy value by the process of maintaining a running average. Thus, the instantaneous running average represents a smoothed value of the energy level of the current frame taking some account of past energy ~15884~
levels.
Secondly, the presence of the threshold test introduces a non-linearity into the process such that high positive or negative energy levels, differing substantially from the 5 previous average energy level, are at first ignored. However, the threshold is subsequently enlarged so that if the high energy level is maintained, it will eventually fall within the threshold and have an effect on the running mean.
Thus, a short lived high energy level due to a noise lO spike will have little or no effect on the running mean energy level, because of the threshold stage. However, a genuinely high energy level due, for example, to a transition to speech will eventually affect the running mean energy level. The threshold is thus adaptive over time so that where incoming 15 energy levels correspond closely to the current mean, the threshold or step level progressively shrinks to a low value, but where incoming energy levels diverge from the mean, the threshold renl~l n.C initially low but then expands.
The averager 371 is thus acting to maintain an average 20 level which behaves somewhat like a running median.
SNR Detector 372 The SNR Detector 372 is arranged, at each frame, to input the frame numbers which the parser 35 has identified as the beginning and end frames of the currently most probable 25 recognised word, and to read the average energy level buffer 376, to determine a representative energy level over the frames currently identified as speech and a representative energy level over the frames current represented as noise.
In this embodiment, the representative measures comprise 30 the mean running energy level running over the noise segments and the peak average energy level over the speech segment.
The operation of the SNR detector 372 is shown in Figure 12.
If the calculated signal to noise ratio value, SNR, is greater than a predetermined threshold, the SNR pause detector 35 372 outputs a signal indicating that a pause has occurred (i.e. that speech is over). If the SNR value lies below the WO94/22131 215 88 ~ PCT/GB94/00630 threshold, a signal indicating that no pause has been recognised is output.
It is found that the SNR measure is a useful identifier of whether a correct word ending has been identified. This is partly because an erroneous recognition by the parser 35 of the start and end (and, indeed, the identity) of a word may result in speech frames being included in those frames used to calculate the mean noise level, and hence reducing the value of the SNR calculated to below the threshold so that a pause is not wrongly identified for this reason. By using the peak energy level as the characteristic energy level for speech in the SNR calculation, the reverse effect is generally avoided since the peak will generally be unaffected by wrongful identification of the start and end of the word (unless a completely erroneous recognition has taken place).
NVR Detector 373 Referring to Figure 13 the NVR Detector 373 is arranged to read the last Nl (where Nl is a predetermined constant) running average energy levels from the buffer 376, and to find the m; n; mllm and maximum values, and to calculate the ratio between the m; n; mllm and the maximum values. This ratio indicates the amount of variation of the energy level over the most recent Nl frames. If the level of variation is compared with the threshold; a high level of variation indicates the possibility that the preceding Nl frames include some speech, whereas a low level of variation compared to a predetermined threshold indicates that the last Nl frames are likely to contain only noise, and consequently the NVR detector 373 outputs a pause detection signal.
Since the energy level of the silence period following the end of speech may be low, the ratio may under some circumstances correspond to division by a very small number.
Accordingly, to avoid singularities in calculation, where the m;nlmllm average energy falls below a predetermined threshold level (for example, unity) then the ratio is calculated between the m~X; mllm and the predetermined level, rather than between the maximum and the minimum.
Other measures of the variance (for example, the difference between the maximum and minimum) could be employed, however the ratio is preferred since it takes account of gross variations in overall signal strength.
Model Based Detector 375 The model based pause detector comprises, as shown in Figure 14, first and second time out detectors 376a, 376b arranged to input from the parser 35 the frame number of the currently identified end of speech/start of end noise, and to test the difference N between this frame and the present frame against a first, relatively short, threshold N1 and a second, relatively long, threshold N2. For example, N1 is selected to be on the order of the length of a short gap within a word (i.e. 20 - 60 frames, and conveniently the same length as the test used in the NVR detector 373) and N2 is selected to be substantially longer (i.e. on the order of half a second).
Also provided is a noise score tester 377, which is arranged to read from the parser 35 the likelihood score for the end noise corresponding to the current most likely state sequence, and to test the score against a predetermined threshold, and to output a ~pause detected' signal in the event that the noise score exceeds the threshold.
Finally, a third time out detector 376c is provided, which tests the total number of frames to date (current frame number) T against a long time out N3, so as to terminate the recognition process after N3 frames if no end of speech has earlier been detected.
Combination Loqic 378 Referring to Figure 15, it will be seen that the outputs of the detectors 376b, 377, 372 and 373 are connected in an AND relationship, and the combined output of the four is connected in an OR relationship with the output of the detectors 376a and 376c.
Thus, a pause is detected either after the expiry of a long timeout (N3 frames) from the start of recognition, or after a relatively long time out (N2 frames) after the onset of noise, or after a relatively short time out (N1 frames) WO94/22131 21~ PCT/GB94/00630 following which the noise score is high, the signal to noise ratio is high and the noise variance is low.
Figure 16 illustrates the energy and average energy RM(t) over a word.
Reiector 35 ~
The rejector 36 is arranged,~after the operation of the pause detector 37, to test the level of confidence of the identification of a word by the parser 35. If the identification is suspect, it is rejected. If the identification is tentative the rejector 36 issues a "query"
signal which enables the utilising apparatus 4 to, for example, initiate a confirmatory dialogue by synthesising a phrase such as "did you say .... (the identified word)" or to ask the user to repeat the word.
Referring to Figure 17, the general operation of the rejector 36 is as follows.
Firstly, the rejector tests whether the signal corresponds to the detection of silence or noise alone. This occurs when the most likely sequence detected by the parser 35 corresponds to a sequence containing only noise states.
Silence is also detected by testing whether the SNR calculated by the SNR detector 372 lies below a very low threshold. In either case, the rejector indicates that no word (silence) has been detected, provided the test performed by the detector 376a is also met.
Secondly, the rejector performs rejection tests (discussed in greater detail below) and tests the results against relatively loose thresholds. If the relatively loose thresholds are not met, the identification is rejected.
If the relatively loose thresholds are met, the test is repeated against relatively tight thresholds. If the relatively tight thresholds are met, acceptance of the identi~ied word is indicated. If the tight thresholds are not met, a query output is generated, to enable the utilising appararus to query the user.
The tests performed by the rejector comprise:
1) A test of the probability score S generated for the most WO94/22131 215 8 8 ~ 9 PCT/GB94/00630 likely path by the parser 35 (to reject out-of-vocabulary words);
2) A test using the signal to noise ratio calculated by the SNR detector 372 (to reject noisy conditions, and out-of-vocabulary wordsi;
3) A test using the noise variance calculated by the NVR
tester 373 (to reject noisy conditions);
In many cases, the utilising apparatus 4 will generate an auditory response to the speaker, transmitted via the network 2 to a loudspeaker 5 typically forming a part of the subscriber handset.
In operation, a speaker speaks into the microphone l and an analog speech signal is transmitted from the microphone l into the network 2 to the recognition processor 3, where the speech signal is analysed and a signal indicating identification or otherwise of a particular word or phrase is generated and transmitted to the utilising apparatus 4, which then takes appropriate action in the event of recognition of an expected word or phrase.
For example, the recognition processor 3 may be arranged to recognise digits 0 to 9, "yes" and "no" so as to be able to recognise personal identification numbers and a range of command words for initiating particular actions (for example, requesting statements or particular services).
Referring to Figure 2, the recognition processor 3 comprises an input 31 for receiving speech in digital form (either from a digital network or from an analog to digital converter), a frame processor 32 for partitioning the succession of digital samples into frames of contiguous samples; a feature extractor 33 for generating from the frames of samples a corresponding feature vector; a classifier 34 receiving the succession of feature vectors and operating on 2158~9 each with the plurality of models corresponding to different words, phonemes or phrases to generate recognition results;
and a parser 35 which is arranged to receive the classification results from the classifier 34 and to determine the word to which the sequence of classifier outputs indicates the greatest similarity.
Also provided is a recognition rejector 36 arranged to reject recognition of a word recognised by the parser 35 if recognition is unreliable, and a pause detector 37, arranged to detect the pause following the end of a word to enable the parser 35 to output a word recognition signal. The word recognition signal from the parser 35, or a rejection signal from the rejector 36, is output to a control signal output 38, for use in controlling the utilising apparatus 4.
Frame Generator 32 The frame generator 32 is arranged to receive speech samples at a rate of, for example, 8,000 samples per second, and to form frames comprising 256 contiguous samples, at a frame rate of 1 frame every 16ms. Preferably, each frame is windowed (i.e. the samples towards the edge of the frame are multiplied by predetermined weighting constants) using, for example, a ~mmlng window to reduce spurious artifacts, generated by the frame edges. In a preferred em.bodiment, the frames are overlapping (for example by 50~) so as to ameliorate the effects of the windowing.
Feature Extractor 33 The feature extractor 33 receives frames from the frame generator 32 and generates, in each case, a set or vector of features. The features may, for example, comprise cepstral coefficients (for example, LPC cepstral coefficients or mel frequency cepstral coefficients as described in "On the Evaluation of Speech Recognisers and Data Bases using a Reference System", Chollet & Gagnoulet, 1982 proc. IEEE
p2026), or differential values of such coefficients comprising, for each coefficient, the difference between the WO94/22131 ~ 88 ~9 8 PCT/GB94/00630 coefficient and the corresponding coefficient value in the preceding frame, as described in "On the use of Instantaneous and Transitional Spectral Information in Speaker Recognition", Soong & Rosenberg, 1988 IEEE Trans. on Accoustics, Speech and Signal Processing Vol 36 No. 6 p871. Equally, a mixture of several types of feature coefficient may be'used.
For reasons that will be discusse~ below, in this embodiment the feature extractor 33 also èxtracts a value for the energy in each frame (which energy value may, but need not, be one of the feature coefficients used in recognition).
The energy value may be generated as the sum of the squares of the samples of the frame.
Finally, the feature extractor 33 outputs a frame number, incremented for each successive frame.
The frame generator 32 and feature extractor 33 are, in this embodiment, provided by a single suitably programmed digital signal processor (DSP) device (such as the Motorola DSP'56000, the Texas Instruments TMS C 320 or similar device.
Classifier 34 Referring to Figure 3, in this embodiment, the classifier 34 comprises a classifying processor 341 and a state memory 342.
The state memory 342 comprises a state field 3421, 3422, ...., for each of the plurality of speech states. For example, each word to be recognised by the recognition processor comprises 6 or 8 states, and accordingly 6 or 8 state fields are provided in the state memory 342 for each word to be recognised. There are also provided a state field for noise/silence at the beginning of a word and a state field for a noise/silence state at the end of a word (although it might in practice be possible to provide only a single noise state).
Each state field in the state memory 342 comprises data defining a multidimensional Gaussian distribution of feature coefficient values which characterise the state in question.
For example, if there are d different feature coefficients, the data characterising a state are a constant ~ WO94122131 215 8 8 4 9 PCT/GB94100630 C, a set of d feature mean values ~i and a set of d feature deviations, ai; in other words, a total of 2d + 1 numbers.
The classification processor 34 is arranged to read each state field within the memory 342 in turn, and calculate for each, using the current input feature coefficient set, the probability that the input feature set or vector corresponds to the corresponding state. To do so, as shown in Figure 4, the processor 341 is arranged to calculate an equation P=C-- 1~ (X~ )2 2i~l a2i It is possible for a single state to be represented by several different modes or distributions; accordingly, the state memory 342 may comprise, for each state, several mode fields each corresponding to the state field described above, in which case the classification processor 341 is arranged to calculate for each mode the probability that the input vector corresponds to that mode, and then to sum the modal probabilities (weighted as appropriate).
Accordingly, the output of the classification processor 341 is a plurality of state probabilities, one for each state in the state memory 342, indicating the likelihood that the input feature vector corresponds to each state.
The classifying processor 341 may be a suitably programmed digital signal processing (DSP) device, may in particular be the same digital signal processing device as the feature extractor 33.
Parser 35 Referring to Figure 5, the parser 35 in this embo~;mPnt comprises a state sequence memory 352, a parsing processor 351, and a parser output buffer 354.
Also provided is a state probability memory 353 which stores, for each frame processed, the state probabilities output by the probability processor 341. The state sequence memory 352 comprises a plurality of state sequence fields WO94/22131 215 8 8 ~ 9 1o PCT/GB94/00630 3521, 3522, ...., each corresponding to a noise-word-noise sequence to be recognised (and one corresponding to a noise-only sequence).
Each state sequence in the state sequence memory 352 comprises, as illustrated in Figure 6, a number of states Pl, P2, PN (where N is 6 or 8) and, for each state, two probabilities; a repeat probability (Pil)y~and a transition probability to the following state~ Pi2). For a CFI
recogniser, the first and final states~àre noise states. The observed sequence of states associated with a series of frames may therefore comprise several repetitions of each state Pi in each state sequence model 3521 etc; for example:
Frame Number 1 2 3 4 5 6 7 8 9 ...................... Z Z+l State Pl Pl Pl P2 P2 P2 P2 P2 P2 .............. Pn Pn Thus, at some frame number (here, frame number 3) the observed sequence will move from the initial, noise, state to the next, speech, state; this transition marks the start of the word to be recognised. Likewise, at some frame (here frame Z) the sequence reaches the last state Pn corresponding to noise or silence following the end of the word to be recognised. Frame Z therefore corresponds to the end of the word to be recognised.
As shown in Figure 8 the parsing processor 351 is arranged to read, at each frame, the state probabilities output by the probability processor 341 and the previous stored state probabilities in the state probability memory 353 and to calculate the most likely path of states to date over time, and to compare this with each of the state sequences stored in the state sequence memory 352.
The calculation employs the well known hidden Markov model method described in the above referenced Cox paper.
Conveniently, the HMM processing performed by the parsing processor 351 uses the well known Viterbi algorithm. The parsing processor 351 may, for example, be a microprocessor such as the Intel(~ 486(~) microprocessor or the Motorola(~) 68000 microprocessor, or may alternatively be a DSP device WO94/22131 ~15 8 8 4 9 PCT/GB94/00630 (for example, the same DSP device as is employed for any of the preceding processors).
Accordingly for each state sequence (corresponding to a word) a probability score is output by the parser processor 5 351 at each frame of input speech. The identity of the most likely state sequence (and hence word recognised) may well change during the duration of the utterance by the speaker.
The parser output buffer 354 comprises a plurality of fields 3541, 3542, ... each corresponding to a word to be recognised (and one which corresponds to a noise-only sequence). Each field, as shown illustratively in Figure 7, comprises a probability score S indicating, for the current frame, the likelihood of the corresponding word being present, and two frame numbers; a first (sp_st) which indicates the 15 first frame of the word in the noise-word-noise observed sequence of frames; and a second (sp_end) which indicates the last frame of the word. Before sp_st the states in the observed sequence comprise initial noise and after sp_end, the states in the observed sequence correspond to terminal noise.
20 Naturally, the frame numbers in each of the fields 3541, 3542, .... differ from one another.
Pause Detector 37 Referring to Figure 9, the pause detector 37 comprises a signal based detector 370 and a model based detector 375.
25 The signal based detector 370 is connected to the feature extractor 33, to receive a parameter extracted from the speech signal. In this present embodiment, the parameter is the frame energy, or some parameter based on the frame energy.
The model based detector 375 is connected to the parser 30 35, to receive an indication of the current best state sequence. Specifically, the model based detector 375 is arranged to read from the parser output buffer 354 the frame - number (sp_end) of the start of final noise states, if any, in the current most probable state sequence and to subtract 35 this from the current frame number to find the length of the period following the end of the word which is currently assumed to be recognised.
WO94122131 215 88 ~9 PCTtGB94/00630 The output of the signal based pause detector 370 and the model based pause detector 375 are combined by logic 378 to generate a pause detection signal at an output 379.
Referring to Figure 10 the signal based pause detector 370 comprises a running averager 371 which maintains a running average energy level over a number of preceding energy values;
a signal to noise ratio (SNR) de~ector 372 and a noise variance (NVR) detector 373, the outputs of which are supplied to be combined by logic 378.
Also provided is a mean energy level buffer 376, connected to the output of the averager 371 to store successive mean energy values corresponding to successive frames.
Runninq Averaqer 371 The running averager 371 is arranged schematically to perform the process shown in Figure 11. In this process, in this embodiment, for each frame the energy of the frame i8 read from the feature extractor 33, and subtracted from a stored running average value to yield the difference value.
The difference value is compared with a threshold or step of predetermined absolute value. If the difference lies within +/- the step value, the running average is unaffected, but the value of the step is reduced by setting it equal to the difference divided by a constant factor or, as indicated in Figure 11, a first constant factor (upfactor) for a positive difference from the running mean and a second factor (downfactor) for a negative difference from the running mean.
If, on the other hand, the difference between the present frame input value and the stored running average exceeds the step value, then the running average is incremented or decremented by the step value depending upon the size of the difference. The step value is then updated as before.
The effect of this process is as follows. Firstly, there is a smoothing of the energy value by the process of maintaining a running average. Thus, the instantaneous running average represents a smoothed value of the energy level of the current frame taking some account of past energy ~15884~
levels.
Secondly, the presence of the threshold test introduces a non-linearity into the process such that high positive or negative energy levels, differing substantially from the 5 previous average energy level, are at first ignored. However, the threshold is subsequently enlarged so that if the high energy level is maintained, it will eventually fall within the threshold and have an effect on the running mean.
Thus, a short lived high energy level due to a noise lO spike will have little or no effect on the running mean energy level, because of the threshold stage. However, a genuinely high energy level due, for example, to a transition to speech will eventually affect the running mean energy level. The threshold is thus adaptive over time so that where incoming 15 energy levels correspond closely to the current mean, the threshold or step level progressively shrinks to a low value, but where incoming energy levels diverge from the mean, the threshold renl~l n.C initially low but then expands.
The averager 371 is thus acting to maintain an average 20 level which behaves somewhat like a running median.
SNR Detector 372 The SNR Detector 372 is arranged, at each frame, to input the frame numbers which the parser 35 has identified as the beginning and end frames of the currently most probable 25 recognised word, and to read the average energy level buffer 376, to determine a representative energy level over the frames currently identified as speech and a representative energy level over the frames current represented as noise.
In this embodiment, the representative measures comprise 30 the mean running energy level running over the noise segments and the peak average energy level over the speech segment.
The operation of the SNR detector 372 is shown in Figure 12.
If the calculated signal to noise ratio value, SNR, is greater than a predetermined threshold, the SNR pause detector 35 372 outputs a signal indicating that a pause has occurred (i.e. that speech is over). If the SNR value lies below the WO94/22131 215 88 ~ PCT/GB94/00630 threshold, a signal indicating that no pause has been recognised is output.
It is found that the SNR measure is a useful identifier of whether a correct word ending has been identified. This is partly because an erroneous recognition by the parser 35 of the start and end (and, indeed, the identity) of a word may result in speech frames being included in those frames used to calculate the mean noise level, and hence reducing the value of the SNR calculated to below the threshold so that a pause is not wrongly identified for this reason. By using the peak energy level as the characteristic energy level for speech in the SNR calculation, the reverse effect is generally avoided since the peak will generally be unaffected by wrongful identification of the start and end of the word (unless a completely erroneous recognition has taken place).
NVR Detector 373 Referring to Figure 13 the NVR Detector 373 is arranged to read the last Nl (where Nl is a predetermined constant) running average energy levels from the buffer 376, and to find the m; n; mllm and maximum values, and to calculate the ratio between the m; n; mllm and the maximum values. This ratio indicates the amount of variation of the energy level over the most recent Nl frames. If the level of variation is compared with the threshold; a high level of variation indicates the possibility that the preceding Nl frames include some speech, whereas a low level of variation compared to a predetermined threshold indicates that the last Nl frames are likely to contain only noise, and consequently the NVR detector 373 outputs a pause detection signal.
Since the energy level of the silence period following the end of speech may be low, the ratio may under some circumstances correspond to division by a very small number.
Accordingly, to avoid singularities in calculation, where the m;nlmllm average energy falls below a predetermined threshold level (for example, unity) then the ratio is calculated between the m~X; mllm and the predetermined level, rather than between the maximum and the minimum.
Other measures of the variance (for example, the difference between the maximum and minimum) could be employed, however the ratio is preferred since it takes account of gross variations in overall signal strength.
Model Based Detector 375 The model based pause detector comprises, as shown in Figure 14, first and second time out detectors 376a, 376b arranged to input from the parser 35 the frame number of the currently identified end of speech/start of end noise, and to test the difference N between this frame and the present frame against a first, relatively short, threshold N1 and a second, relatively long, threshold N2. For example, N1 is selected to be on the order of the length of a short gap within a word (i.e. 20 - 60 frames, and conveniently the same length as the test used in the NVR detector 373) and N2 is selected to be substantially longer (i.e. on the order of half a second).
Also provided is a noise score tester 377, which is arranged to read from the parser 35 the likelihood score for the end noise corresponding to the current most likely state sequence, and to test the score against a predetermined threshold, and to output a ~pause detected' signal in the event that the noise score exceeds the threshold.
Finally, a third time out detector 376c is provided, which tests the total number of frames to date (current frame number) T against a long time out N3, so as to terminate the recognition process after N3 frames if no end of speech has earlier been detected.
Combination Loqic 378 Referring to Figure 15, it will be seen that the outputs of the detectors 376b, 377, 372 and 373 are connected in an AND relationship, and the combined output of the four is connected in an OR relationship with the output of the detectors 376a and 376c.
Thus, a pause is detected either after the expiry of a long timeout (N3 frames) from the start of recognition, or after a relatively long time out (N2 frames) after the onset of noise, or after a relatively short time out (N1 frames) WO94/22131 21~ PCT/GB94/00630 following which the noise score is high, the signal to noise ratio is high and the noise variance is low.
Figure 16 illustrates the energy and average energy RM(t) over a word.
Reiector 35 ~
The rejector 36 is arranged,~after the operation of the pause detector 37, to test the level of confidence of the identification of a word by the parser 35. If the identification is suspect, it is rejected. If the identification is tentative the rejector 36 issues a "query"
signal which enables the utilising apparatus 4 to, for example, initiate a confirmatory dialogue by synthesising a phrase such as "did you say .... (the identified word)" or to ask the user to repeat the word.
Referring to Figure 17, the general operation of the rejector 36 is as follows.
Firstly, the rejector tests whether the signal corresponds to the detection of silence or noise alone. This occurs when the most likely sequence detected by the parser 35 corresponds to a sequence containing only noise states.
Silence is also detected by testing whether the SNR calculated by the SNR detector 372 lies below a very low threshold. In either case, the rejector indicates that no word (silence) has been detected, provided the test performed by the detector 376a is also met.
Secondly, the rejector performs rejection tests (discussed in greater detail below) and tests the results against relatively loose thresholds. If the relatively loose thresholds are not met, the identification is rejected.
If the relatively loose thresholds are met, the test is repeated against relatively tight thresholds. If the relatively tight thresholds are met, acceptance of the identi~ied word is indicated. If the tight thresholds are not met, a query output is generated, to enable the utilising appararus to query the user.
The tests performed by the rejector comprise:
1) A test of the probability score S generated for the most WO94/22131 215 8 8 ~ 9 PCT/GB94/00630 likely path by the parser 35 (to reject out-of-vocabulary words);
2) A test using the signal to noise ratio calculated by the SNR detector 372 (to reject noisy conditions, and out-of-vocabulary wordsi;
3) A test using the noise variance calculated by the NVR
tester 373 (to reject noisy conditions);
4) A test of the ratio between the score generated by the parser for the most likely path and that generated for the second most likely path; and, optionally,
5) A test performed between specific known confusable words (for example, if the most likely word recognised by the parser 35 is "five", and the second most likely is "nine", the difference or the ratio between the two may be tested).
Thus, the rejector 36 can either accept a word, in which case the output of the parser 35 is passed to the output 38;
or indicate that silence is present (i.e. no word is present), in which a signal identifying silence is passed to the output 38; or reject or query the identification of a word by the parser 35, in which case the output of the parser 35 is inhibited and a corresponding "reject" or "query" control signal is passed to the output 38 to enable action by the utilising apparatus 4.
Second Embodiment In the second embodiment, the feature generator 33 is arranged to compare a newly generated set of feature coefficients with the last - output set of feature coefficients, and only to output a new set of feature coefficients when the overall difference from the earlier set is greater than a predetermined threshold. For example, the distance may be the sum of absolute differences or "city 2~ 588~9 block" distance measure, or any other convenient measure.
It is found that this technique can substantially reduce the amount of calculation required by the classifier 34 and parser 35 by, for example, on the order of 60~. Furthermore, since the HMM process makes an assumption that subsequent states are independent of each other, this embodiment may under some circumstances increase the validity of this assumption since it causes each successive set of coefficients to differ substantially from its predecessor.
In this case, it is found that the operation of the classifier 34 and parser 35 are not substantially altered.
However, the operation of the signal based pause detector 370, specifically the running averager 371, are altered as the average needs to take account of the duration of the periods between successive frames.
In this embodiment, the feature extractor 33 generates, and supplies to the pause detector 37, a number N(t) associated with each frame, which indicates the number of frames between that frame and the last frame output by the feature generator 33.
The feature extractor 33 also accumulates the energy of each frame, so as to supply a cumulative energy E(t) at each set of feature coefficients which are output, which corresponds to the sum of the energy giving rise to that set of coefficients and the energies of all the other frames between that frame and the previous frame output by the feature extractor 33.
Referring to Figure 18, in this embodiment the averager 371 reads the cumulative energy E(t) and the number of frames N(t) represented by a VFR frame, and then generates the average energy for each intervening frame by dividing E(t) by N(t). The averager then, essentially, simulates the effect of receiving N(t) successive frames each having average energy, and increments or decrements the rllnni ng average accordingly.
However, to ensure that the average running energy value used in the signal to noise ratio calculation is correct, the final averaged energy level RM(t) calculated for the VFR frame is found by averaging the N successive running averages by 21~884~
accumulating the running averages and then normalising by N(t) at the end of the calculation.
Thus, in this embodiment, the numbers stored in the output buffer 374 comprise the values RM(t) for each of the frames of the feature coefficients emitted at a variable rate by the coefficient generator 33, which correspond to the average level of the signal frames preceding the current frame.
In fact, in this embodiment, the minimum and m~x; ~llm energy levels are less clearly defined than the first embodiment because the process of cumulating energy of preceding frames carried out in the feature generator 33 acts to smooth sharp peaks or dips in the energy level of the input speech signal.
15In this embodiment, it would of course be possible instead for the averager 371 to receive and process each of the energy levels from each of the signal frames received by the feature generator 33, regardless of whether or not those frames give rise to the outputting of a feature vector for recognition. However, this would require further calculation and buffering.
In this embodiment, the pause tests calculated by the detectors 376a, 376b are calculated so as to take account of the variable rate at which coefficient vectors are generated by maintaining a current frame number calculated by accumulating the numbers of omitted frames N(t) and using this to calculate the time since the end of speech N.
Figure l9 illustrates the energy, and average energy RM(t), over a word.
30Advantaqes of the Invention F-om the foregoing embodiments, it will be seen that there are a number of advantages to aspects of the invention.
By providing a pause detector in a continuous speech recogniser which actively ~ml nes the speech signal, it is possible to provide a rapid recognition of input words, phrases or sentences. By making the pause detector Px~mlne parameters which are separate from the speech/noise model, WO94/22131 %~ 88 49 PCT/GB94/00630 assumed by the speech detector, greater robustness is ensured.
It is found that energy based measures can be particularly effecti~e in discriminating between speech and noise, in particular, a test of the difference between the signal level and the noise level (for example, a measure of the signal to noise ratio) generated on the assumption that the noise-speech-noise model used by the recogniser is correct is found to be an effective means of validating the correctness of that assumption. More particularly, a signal to noise ratio calculated between a peak value over a speech period and an average value over a noise period is found to be effective.
As the basis for pause detection, or for other purposes such as rejection of an identified word, it is found advantageous to use an averaged or smoothed measure of the signal energy; in particular, a running average measure and, more particularly, a non-linear average which provides some filtering of noise spikes is preferred. The algorithm may preferably be arranged approximately to track the median rather than the mean of the energy of the signal.
Viewed in another way, the algorithm may be arranged to increment or decrement the running average by a predetermined amount, and the predetermined amount is preferably adapted in dependence upon the difference between the input energy level and the running average.
Further, the use of a measure of the variation of signal energy (and, more specifically, variation of the smooth and averaged signal energy) is found to be a good discriminator allowing the determination of whether only noise is present;
in particular, a measure of the ratio between peak energy and min;mllm energy is generally low if only noise is present.
Accordingly, this test can be employed to validate the noise-speech-noise model generated by the recognition process.
The above tests are advantageously, but not necessarily, combined with tests based on the recogniser output itself, such as a test of the score generated by the recognition of noise, and a test of the length of time since the onset of recognised noise.
The signal based tests described above are found equally to be useful, with different thresholds, to form the basis for WO94/22131 ~ 9 PCT/GB94/00630 subsequent re~ection of recognised words under unsafe recognition conditions as described above.
Other AsPects and Embodiments of the Invention .
It will be clear from the foregoing that the described embodiments are merely examples of the invention, which is accordingly not limited thereby. In particular, various novel features of the described embodiments each have separate advantages, whether explicitly described above or clear to the skilled person herefrom, and protection is sought for each such advantageous feature in isolation and for any advantageous combination of such features.
The use of a Gaussian, continuous density classifier has been described here, but a classifier using vector quantisation could equally be employed. Similarly, other types of sequence processing (e.g. Dynamic Time Warp) could be employed.
Whilst only a 'repeat' probability and a 'transition' probability have been discussed, probabilities for transitions to next-but-one and next-but-two (etc) states (skipping transitions) are well known and could equally be employed.
Likewise, the number of states mentioned above for words and noise are purely exemplary.
Whilst particular embodiments have been described in detail, it will be realised that other embodiments are realisable using digital or analog hardware, suitably constructed or programmed.
Although a recognition syntax has been described in which isolated words (preceded and succeeded by noise) are recognised, the present invention is equally applicable to connected-word recognition. In this case, the state sequence models would represent sequences of noise-wordl-word2-....-wordN-noise, and the SNR and noise variance tests would - preferably be responslve only to the noise after the end of speech point.
Although speech recognition has been described, use of the same techniques in relation to other types of recognition (for example speaker recognition or verification) is not WO94/22131 215 8 8 ~ 9 22 PCT/GB94/00630 excluded.
The scope of protection is intended to encompass all constructions within the scope of the claims appended hereto, together with any equivalent constructions achieving substantially the same result or achieving a substantially different result using the same prlnciple or operation.
-~r ~. `
Thus, the rejector 36 can either accept a word, in which case the output of the parser 35 is passed to the output 38;
or indicate that silence is present (i.e. no word is present), in which a signal identifying silence is passed to the output 38; or reject or query the identification of a word by the parser 35, in which case the output of the parser 35 is inhibited and a corresponding "reject" or "query" control signal is passed to the output 38 to enable action by the utilising apparatus 4.
Second Embodiment In the second embodiment, the feature generator 33 is arranged to compare a newly generated set of feature coefficients with the last - output set of feature coefficients, and only to output a new set of feature coefficients when the overall difference from the earlier set is greater than a predetermined threshold. For example, the distance may be the sum of absolute differences or "city 2~ 588~9 block" distance measure, or any other convenient measure.
It is found that this technique can substantially reduce the amount of calculation required by the classifier 34 and parser 35 by, for example, on the order of 60~. Furthermore, since the HMM process makes an assumption that subsequent states are independent of each other, this embodiment may under some circumstances increase the validity of this assumption since it causes each successive set of coefficients to differ substantially from its predecessor.
In this case, it is found that the operation of the classifier 34 and parser 35 are not substantially altered.
However, the operation of the signal based pause detector 370, specifically the running averager 371, are altered as the average needs to take account of the duration of the periods between successive frames.
In this embodiment, the feature extractor 33 generates, and supplies to the pause detector 37, a number N(t) associated with each frame, which indicates the number of frames between that frame and the last frame output by the feature generator 33.
The feature extractor 33 also accumulates the energy of each frame, so as to supply a cumulative energy E(t) at each set of feature coefficients which are output, which corresponds to the sum of the energy giving rise to that set of coefficients and the energies of all the other frames between that frame and the previous frame output by the feature extractor 33.
Referring to Figure 18, in this embodiment the averager 371 reads the cumulative energy E(t) and the number of frames N(t) represented by a VFR frame, and then generates the average energy for each intervening frame by dividing E(t) by N(t). The averager then, essentially, simulates the effect of receiving N(t) successive frames each having average energy, and increments or decrements the rllnni ng average accordingly.
However, to ensure that the average running energy value used in the signal to noise ratio calculation is correct, the final averaged energy level RM(t) calculated for the VFR frame is found by averaging the N successive running averages by 21~884~
accumulating the running averages and then normalising by N(t) at the end of the calculation.
Thus, in this embodiment, the numbers stored in the output buffer 374 comprise the values RM(t) for each of the frames of the feature coefficients emitted at a variable rate by the coefficient generator 33, which correspond to the average level of the signal frames preceding the current frame.
In fact, in this embodiment, the minimum and m~x; ~llm energy levels are less clearly defined than the first embodiment because the process of cumulating energy of preceding frames carried out in the feature generator 33 acts to smooth sharp peaks or dips in the energy level of the input speech signal.
15In this embodiment, it would of course be possible instead for the averager 371 to receive and process each of the energy levels from each of the signal frames received by the feature generator 33, regardless of whether or not those frames give rise to the outputting of a feature vector for recognition. However, this would require further calculation and buffering.
In this embodiment, the pause tests calculated by the detectors 376a, 376b are calculated so as to take account of the variable rate at which coefficient vectors are generated by maintaining a current frame number calculated by accumulating the numbers of omitted frames N(t) and using this to calculate the time since the end of speech N.
Figure l9 illustrates the energy, and average energy RM(t), over a word.
30Advantaqes of the Invention F-om the foregoing embodiments, it will be seen that there are a number of advantages to aspects of the invention.
By providing a pause detector in a continuous speech recogniser which actively ~ml nes the speech signal, it is possible to provide a rapid recognition of input words, phrases or sentences. By making the pause detector Px~mlne parameters which are separate from the speech/noise model, WO94/22131 %~ 88 49 PCT/GB94/00630 assumed by the speech detector, greater robustness is ensured.
It is found that energy based measures can be particularly effecti~e in discriminating between speech and noise, in particular, a test of the difference between the signal level and the noise level (for example, a measure of the signal to noise ratio) generated on the assumption that the noise-speech-noise model used by the recogniser is correct is found to be an effective means of validating the correctness of that assumption. More particularly, a signal to noise ratio calculated between a peak value over a speech period and an average value over a noise period is found to be effective.
As the basis for pause detection, or for other purposes such as rejection of an identified word, it is found advantageous to use an averaged or smoothed measure of the signal energy; in particular, a running average measure and, more particularly, a non-linear average which provides some filtering of noise spikes is preferred. The algorithm may preferably be arranged approximately to track the median rather than the mean of the energy of the signal.
Viewed in another way, the algorithm may be arranged to increment or decrement the running average by a predetermined amount, and the predetermined amount is preferably adapted in dependence upon the difference between the input energy level and the running average.
Further, the use of a measure of the variation of signal energy (and, more specifically, variation of the smooth and averaged signal energy) is found to be a good discriminator allowing the determination of whether only noise is present;
in particular, a measure of the ratio between peak energy and min;mllm energy is generally low if only noise is present.
Accordingly, this test can be employed to validate the noise-speech-noise model generated by the recognition process.
The above tests are advantageously, but not necessarily, combined with tests based on the recogniser output itself, such as a test of the score generated by the recognition of noise, and a test of the length of time since the onset of recognised noise.
The signal based tests described above are found equally to be useful, with different thresholds, to form the basis for WO94/22131 ~ 9 PCT/GB94/00630 subsequent re~ection of recognised words under unsafe recognition conditions as described above.
Other AsPects and Embodiments of the Invention .
It will be clear from the foregoing that the described embodiments are merely examples of the invention, which is accordingly not limited thereby. In particular, various novel features of the described embodiments each have separate advantages, whether explicitly described above or clear to the skilled person herefrom, and protection is sought for each such advantageous feature in isolation and for any advantageous combination of such features.
The use of a Gaussian, continuous density classifier has been described here, but a classifier using vector quantisation could equally be employed. Similarly, other types of sequence processing (e.g. Dynamic Time Warp) could be employed.
Whilst only a 'repeat' probability and a 'transition' probability have been discussed, probabilities for transitions to next-but-one and next-but-two (etc) states (skipping transitions) are well known and could equally be employed.
Likewise, the number of states mentioned above for words and noise are purely exemplary.
Whilst particular embodiments have been described in detail, it will be realised that other embodiments are realisable using digital or analog hardware, suitably constructed or programmed.
Although a recognition syntax has been described in which isolated words (preceded and succeeded by noise) are recognised, the present invention is equally applicable to connected-word recognition. In this case, the state sequence models would represent sequences of noise-wordl-word2-....-wordN-noise, and the SNR and noise variance tests would - preferably be responslve only to the noise after the end of speech point.
Although speech recognition has been described, use of the same techniques in relation to other types of recognition (for example speaker recognition or verification) is not WO94/22131 215 8 8 ~ 9 22 PCT/GB94/00630 excluded.
The scope of protection is intended to encompass all constructions within the scope of the claims appended hereto, together with any equivalent constructions achieving substantially the same result or achieving a substantially different result using the same prlnciple or operation.
-~r ~. `
Claims (25)
1. A recognition system comprising: input means for receiving a speech signal;
recognition processing means for processing the speech signal to indicate its similarity to predetermined patterns to be recognised, said recognition processing means being arranged repeatedly to partition the speech signal into a pattern-containing portion and, preceding and following said pattern-containing portion, noise or silence portions, and to identify a pattern corresponding to said pattern containing portion; and output means for supplying a recognition signal indicating recognition of one of said patterns, characterised by pause detection means for detecting the noise or silence portion which follows the pattern-containing portion, and means, responsive to the detection thereof, arranged to supply a signal identifying the pattern currently corresponding to the pattern portion to the output means.
recognition processing means for processing the speech signal to indicate its similarity to predetermined patterns to be recognised, said recognition processing means being arranged repeatedly to partition the speech signal into a pattern-containing portion and, preceding and following said pattern-containing portion, noise or silence portions, and to identify a pattern corresponding to said pattern containing portion; and output means for supplying a recognition signal indicating recognition of one of said patterns, characterised by pause detection means for detecting the noise or silence portion which follows the pattern-containing portion, and means, responsive to the detection thereof, arranged to supply a signal identifying the pattern currently corresponding to the pattern portion to the output means.
2. A system according to Claim 1, wherein the said patterns correspond to phonemes, words, phrases or sentences.
3. A system according to Claim 2, wherein the said patterns correspond to words.
4. A system according to any one of Claims 1 to 3 in which the pause detection means are arranged to receive at least one signal parameter derived from said speech signal which does not depend upon said partitioning by said recognition processing means.
5. A system according to Claim 4, in which the a pulse detection means is arranged to process said signal parameter in accordance with said partitioning by the recognition processing means, to generate at least one measure which depends upon the accuracy of said partitioning.
6. A system according to Claim 5, in which said signal parameter has a different magnitude in the present of noise or silence to that which it has in the presence of a pattern.
7. A system according to Claim 6, in which the parameter is related to the energy of said speech signal.
8. A system according to any one of Claims 4 to 7 in which the pause detection means comprises means for smoothing said parameter over time.
9. A system according to Claim 8 in which said smoothing means comprise means for deriving a running average value of said parameter, said running average being employed in generating said measure or measures.
10. A system according to Claim 9 in which said running average means is arranged to apply a non-linear smoothing to said parameter, to reduce the effects of abrupt magnitude changes therein.
11. A system according to Claim 9 or Claim 10 in which said running average is derived to track the median of said parameter.
12. A system according to any one of Claims 5 to 7, or any of Claims 8 to 11 when appended thereto, in which the pause detection means comprises variation detecting means for deriving, within said noise or silence portion following said pattern-containing portion, a measure of the level of variation of said parameter or a parameter derived therefrom.
13. A system according to Claim 12 in which said variation detecting means is arranged to derive maximum and minimum values of said parameter or derived parameter, and to derive said measure so as to depend upon the ratio therebetween.
14. A system according to Claim 13, in which said ratio is derived so as to avoid division by a small number.
15. A system according to any one of Claims 12 to 14 in which said variation detecting means is arranged to derive said measure in dependence upon values of said parameter over a time window extending over a predetermined past portion of the speech signal lying within said following noise or silence portion.
16. A system according to any one of Claims 5 to 7, or any of Claims 8 to 15 appended thereto, in which the pause detection means comprises means for deriving a measure indicating the relative levels of said parameter, or a parameter derived therefrom, over said pattern-containing portion and over said silence or noise portions.
17. A system according to Claim 16 in which said measure is derived so as to depend upon the ratio between a first value derived from said pattern- containing portion and a second value derived from said silence or noise portion.
18. A system according to Claim 17 in which the first value comprises a maximum value of said parameter or derived parameter.
19. A system according to Claim 17 or Claim 18 in which the second value comprises an average value of said parameter or derived parameter.
20. A system according to any one of Claims 1 to 19, in which the recognition processing means is arranged to recognise noise or silence, and the pause detection means is arranged to respond to the level of confidence of said recognition of noise or silence.
21. A system according to any one of Claims 1 to 20, in which the pause detection means is arranged to respond to the duration of the silence or noise portion following said pattern-containing portion.
22. A system according to any one of Claims 1 to 21 in which the recognition processing means comprises means for storing data defining a plurality of state sequence probabilities, and for calculating the likelihood of said speech signal corresponding to each state sequence.
23. A system according to Claim 22 in which the recognition processing means comprises means for storing data defining a plurality of continuous probability distributions corresponding to different states, and means for applying said distribution data to said speech signal to calculate a measure of the correspondence between the speech signal and each said state.
24. A system according to any one of Claims 1 to 23 further comprising means for dividing said speech signal into a successive sequence of portions, and for comparing a said portion with a preceding portion, said system being arranged not to operate the recognition processing means where a said portion does not differ substantially from its predecessor.
25. A method of operating an electronic recognition system which comprises input means for receiving a speech signal, recognition processing means for processing the speech signal and output means for indicating a recognised speech pattern, the method being to detect the arrival of a point in time after the end of the speech pattern, the method comprising the steps of:
preprocessing a next occurring temporal portion of a speech signal received at the input means;
performing a recognition process on that temporal portion and preceding temporal portions to generate a pattern signal identifying a predetermined pattern to which the speech signal is recognised as corresponding; and recognising whether the point in time has occurred;
characterised in that the step of recognising whether the point in time has occurred comprises:
deriving at least one signal parameter from said speech signal which is independent of the partitioning between speech and noise performed by the recognition processing means;
deriving at least one parameter which depends upon said partitioning; and deciding whether or not said point in time has arrived taking into account both said parameters.
preprocessing a next occurring temporal portion of a speech signal received at the input means;
performing a recognition process on that temporal portion and preceding temporal portions to generate a pattern signal identifying a predetermined pattern to which the speech signal is recognised as corresponding; and recognising whether the point in time has occurred;
characterised in that the step of recognising whether the point in time has occurred comprises:
deriving at least one signal parameter from said speech signal which is independent of the partitioning between speech and noise performed by the recognition processing means;
deriving at least one parameter which depends upon said partitioning; and deciding whether or not said point in time has arrived taking into account both said parameters.
Applications Claiming Priority (5)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| EP93302302.0 | 1993-03-25 | ||
| EP93302302 | 1993-03-25 | ||
| EP93302541.3 | 1993-03-31 | ||
| EP93302541 | 1993-03-31 | ||
| PCT/GB1994/000630 WO1994022131A2 (en) | 1993-03-25 | 1994-03-25 | Speech recognition with pause detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2158849A1 CA2158849A1 (en) | 1994-09-29 |
| CA2158849C true CA2158849C (en) | 2000-09-05 |
Family
ID=26134232
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002158849A Expired - Fee Related CA2158849C (en) | 1993-03-25 | 1994-03-25 | Speech recognition with pause detection |
Country Status (9)
| Country | Link |
|---|---|
| US (1) | US5848388A (en) |
| EP (2) | EP0962913B1 (en) |
| JP (1) | JP3691511B2 (en) |
| AU (1) | AU6433094A (en) |
| CA (1) | CA2158849C (en) |
| DE (2) | DE69421911T2 (en) |
| ES (1) | ES2141824T3 (en) |
| SG (1) | SG93215A1 (en) |
| WO (1) | WO1994022131A2 (en) |
Families Citing this family (56)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB9620082D0 (en) | 1996-09-26 | 1996-11-13 | Eyretel Ltd | Signal monitoring apparatus |
| GB9625284D0 (en) * | 1996-12-04 | 1997-01-22 | Canon Kk | A data processing method and apparatus for identifying a classification to which data belongs |
| KR100302370B1 (en) * | 1997-04-30 | 2001-09-29 | 닛폰 호소 교카이 | Speech interval detection method and system, and speech speed converting method and system using the speech interval detection method and system |
| US5930748A (en) * | 1997-07-11 | 1999-07-27 | Motorola, Inc. | Speaker identification system and method |
| US6374219B1 (en) * | 1997-09-19 | 2002-04-16 | Microsoft Corporation | System for using silence in speech recognition |
| US6006182A (en) * | 1997-09-22 | 1999-12-21 | Northern Telecom Limited | Speech recognition rejection method using generalized additive models |
| US6718302B1 (en) * | 1997-10-20 | 2004-04-06 | Sony Corporation | Method for utilizing validity constraints in a speech endpoint detector |
| JP3075250B2 (en) * | 1998-03-04 | 2000-08-14 | 日本電気株式会社 | Speaker recognition method and apparatus |
| DE19824354A1 (en) * | 1998-05-30 | 1999-12-02 | Philips Patentverwaltung | Device for verifying signals |
| DE19824353A1 (en) * | 1998-05-30 | 1999-12-02 | Philips Patentverwaltung | Device for verifying signals |
| US6324509B1 (en) * | 1999-02-08 | 2001-11-27 | Qualcomm Incorporated | Method and apparatus for accurate endpointing of speech in the presence of noise |
| DE19939102C1 (en) * | 1999-08-18 | 2000-10-26 | Siemens Ag | Speech recognition method for dictating system or automatic telephone exchange |
| GB2355833B (en) * | 1999-10-29 | 2003-10-29 | Canon Kk | Natural language input method and apparatus |
| US6910035B2 (en) * | 2000-07-06 | 2005-06-21 | Microsoft Corporation | System and methods for providing automatic classification of media entities according to consonance properties |
| US7035873B2 (en) | 2001-08-20 | 2006-04-25 | Microsoft Corporation | System and methods for providing adaptive media property classification |
| US20020042709A1 (en) * | 2000-09-29 | 2002-04-11 | Rainer Klisch | Method and device for analyzing a spoken sequence of numbers |
| US20030023439A1 (en) * | 2001-05-02 | 2003-01-30 | Gregory Ciurpita | Method and apparatus for automatic recognition of long sequences of spoken digits |
| US20040104062A1 (en) * | 2002-12-02 | 2004-06-03 | Yvon Bedard | Side panel for a snowmobile |
| DE10220524B4 (en) | 2002-05-08 | 2006-08-10 | Sap Ag | Method and system for processing voice data and recognizing a language |
| EP1363271A1 (en) | 2002-05-08 | 2003-11-19 | Sap Ag | Method and system for processing and storing of dialogue speech data |
| US7356466B2 (en) * | 2002-06-28 | 2008-04-08 | Samsung Electronics Co., Ltd. | Method and apparatus for performing observation probability calculations |
| US20040044517A1 (en) * | 2002-08-30 | 2004-03-04 | Robert Palmquist | Translation system |
| US7539086B2 (en) * | 2002-10-23 | 2009-05-26 | J2 Global Communications, Inc. | System and method for the secure, real-time, high accuracy conversion of general-quality speech into text |
| JP4463526B2 (en) * | 2003-10-24 | 2010-05-19 | 株式会社ユニバーサルエンターテインメント | Voiceprint authentication system |
| BRPI0417636A (en) * | 2003-12-16 | 2007-03-27 | Speechgear Inc | system, method, and, computer readable medium |
| JP2007515019A (en) * | 2003-12-17 | 2007-06-07 | スピーチギア，インコーポレイティド | Translation tools |
| US7664642B2 (en) * | 2004-03-17 | 2010-02-16 | University Of Maryland | System and method for automatic speech recognition from phonetic features and acoustic landmarks |
| US9117460B2 (en) * | 2004-05-12 | 2015-08-25 | Core Wireless Licensing S.A.R.L. | Detection of end of utterance in speech recognition system |
| US7558389B2 (en) * | 2004-10-01 | 2009-07-07 | At&T Intellectual Property Ii, L.P. | Method and system of generating a speech signal with overlayed random frequency signal |
| US7584098B2 (en) * | 2004-11-29 | 2009-09-01 | Microsoft Corporation | Vocabulary-independent search of spontaneous speech |
| US8175877B2 (en) * | 2005-02-02 | 2012-05-08 | At&T Intellectual Property Ii, L.P. | Method and apparatus for predicting word accuracy in automatic speech recognition systems |
| EP1708172A1 (en) * | 2005-03-30 | 2006-10-04 | Top Digital Co., Ltd. | Voiceprint identification system for E-commerce |
| KR101116363B1 (en) * | 2005-08-11 | 2012-03-09 | 삼성전자주식회사 | Method and apparatus for classifying speech signal, and method and apparatus using the same |
| US7603275B2 (en) * | 2005-10-31 | 2009-10-13 | Hitachi, Ltd. | System, method and computer program product for verifying an identity using voiced to unvoiced classifiers |
| US8090582B2 (en) * | 2005-12-14 | 2012-01-03 | Mitsubishi Electric Corporation | Voice recognition apparatus |
| JP4906379B2 (en) * | 2006-03-22 | 2012-03-28 | 富士通株式会社 | Speech recognition apparatus, speech recognition method, and computer program |
| EP1933302A1 (en) * | 2006-12-12 | 2008-06-18 | Harman Becker Automotive Systems GmbH | Speech recognition method |
| US20080221882A1 (en) * | 2007-03-06 | 2008-09-11 | Bundock Donald S | System for excluding unwanted data from a voice recording |
| US8990073B2 (en) * | 2007-06-22 | 2015-03-24 | Voiceage Corporation | Method and device for sound activity detection and sound signal classification |
| DE102008039329A1 (en) * | 2008-01-25 | 2009-07-30 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | An apparatus and method for calculating control information for an echo suppression filter and apparatus and method for calculating a delay value |
| WO2010019831A1 (en) * | 2008-08-14 | 2010-02-18 | 21Ct, Inc. | Hidden markov model for speech processing with training method |
| US8838449B2 (en) * | 2010-12-23 | 2014-09-16 | Microsoft Corporation | Word-dependent language model |
| US8924213B2 (en) | 2012-07-09 | 2014-12-30 | Nuance Communications, Inc. | Detecting potential significant errors in speech recognition results |
| US9064492B2 (en) | 2012-07-09 | 2015-06-23 | Nuance Communications, Inc. | Detecting potential significant errors in speech recognition results |
| US9799328B2 (en) * | 2012-08-03 | 2017-10-24 | Veveo, Inc. | Method for using pauses detected in speech input to assist in interpreting the input during conversational interaction for information retrieval |
| US20140365068A1 (en) * | 2013-06-06 | 2014-12-11 | Melvin Burns | Personalized Voice User Interface System and Method |
| US9928851B2 (en) * | 2013-09-12 | 2018-03-27 | Mediatek Inc. | Voice verifying system and voice verifying method which can determine if voice signal is valid or not |
| US10832005B1 (en) | 2013-11-21 | 2020-11-10 | Soundhound, Inc. | Parsing to determine interruptible state in an utterance by detecting pause duration and complete sentences |
| US9311932B2 (en) * | 2014-01-23 | 2016-04-12 | International Business Machines Corporation | Adaptive pause detection in speech recognition |
| KR102446392B1 (en) * | 2015-09-23 | 2022-09-23 | 삼성전자주식회사 | Electronic device and method for recognizing voice of speech |
| US10339918B2 (en) * | 2016-09-27 | 2019-07-02 | Intel IP Corporation | Adaptive speech endpoint detector |
| US9984689B1 (en) * | 2016-11-10 | 2018-05-29 | Linearhub | Apparatus and method for correcting pronunciation by contextual recognition |
| US10636421B2 (en) | 2017-12-27 | 2020-04-28 | Soundhound, Inc. | Parse prefix-detection in a human-machine interface |
| RU2761940C1 (en) | 2018-12-18 | 2021-12-14 | Общество С Ограниченной Ответственностью "Яндекс" | Methods and electronic apparatuses for identifying a statement of the user by a digital audio signal |
| KR20210078133A (en) * | 2019-12-18 | 2021-06-28 | 엘지전자 주식회사 | Training data generating method for training filled pause detecting model and device therefor |
| US11705125B2 (en) | 2021-03-26 | 2023-07-18 | International Business Machines Corporation | Dynamic voice input detection for conversation assistants |
Family Cites Families (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| USRE31188E (en) * | 1978-10-31 | 1983-03-22 | Bell Telephone Laboratories, Incorporated | Multiple template speech recognition system |
| DE3066499D1 (en) * | 1979-08-13 | 1984-03-15 | Linear Pneumatics Inc | Method of removing stuck gaskets and pneumatic impact tool therefor |
| US4348553A (en) * | 1980-07-02 | 1982-09-07 | International Business Machines Corporation | Parallel pattern verifier with dynamic time warping |
| US4761815A (en) * | 1981-05-01 | 1988-08-02 | Figgie International, Inc. | Speech recognition system based on word state duration and/or weight |
| US4481593A (en) * | 1981-10-05 | 1984-11-06 | Exxon Corporation | Continuous speech recognition |
| US4989248A (en) * | 1983-01-28 | 1991-01-29 | Texas Instruments Incorporated | Speaker-dependent connected speech word recognition method |
| US4783804A (en) * | 1985-03-21 | 1988-11-08 | American Telephone And Telegraph Company, At&T Bell Laboratories | Hidden Markov model speech recognition arrangement |
| GB2182795B (en) * | 1985-11-12 | 1988-10-05 | Nat Res Dev | Apparatus and methods for speech analysis |
| JP2717652B2 (en) * | 1986-06-02 | 1998-02-18 | モトローラ・インコーポレーテッド | Continuous speech recognition system |
| JPH0760318B2 (en) * | 1986-09-29 | 1995-06-28 | 株式会社東芝 | Continuous speech recognition method |
| US4829578A (en) * | 1986-10-02 | 1989-05-09 | Dragon Systems, Inc. | Speech detection and recognition apparatus for use with background noise of varying levels |
| US4837831A (en) * | 1986-10-15 | 1989-06-06 | Dragon Systems, Inc. | Method for creating and using multiple-word sound models in speech recognition |
| US4803729A (en) * | 1987-04-03 | 1989-02-07 | Dragon Systems, Inc. | Speech recognition method |
| ES2018698B3 (en) * | 1987-06-16 | 1991-05-01 | Santex Ag | PROCEDURE AND DEVICE TO SATIN AND SHRINK TEXTILE GENEROS |
| US5228110A (en) * | 1989-09-15 | 1993-07-13 | U.S. Philips Corporation | Method for recognizing N different word strings in a speech signal |
| DE69128990T2 (en) * | 1990-09-07 | 1998-08-27 | Toshiba Kawasaki Kk | Speech recognition device |
| FR2677828B1 (en) * | 1991-06-14 | 1993-08-20 | Sextant Avionique | METHOD FOR DETECTION OF A NOISE USEFUL SIGNAL. |
| JP2870224B2 (en) * | 1991-06-19 | 1999-03-17 | 松下電器産業株式会社 | Voice recognition method |
| US5388183A (en) * | 1991-09-30 | 1995-02-07 | Kurzwell Applied Intelligence, Inc. | Speech recognition providing multiple outputs |
| US5390278A (en) * | 1991-10-08 | 1995-02-14 | Bell Canada | Phoneme based speech recognition |
| US5583961A (en) * | 1993-03-25 | 1996-12-10 | British Telecommunications Public Limited Company | Speaker recognition using spectral coefficients normalized with respect to unequal frequency bands |
| US5524169A (en) * | 1993-12-30 | 1996-06-04 | International Business Machines Incorporated | Method and system for location-specific speech recognition |
| US5621859A (en) * | 1994-01-19 | 1997-04-15 | Bbn Corporation | Single tree method for grammar directed, very large vocabulary speech recognizer |
-
1994
- 1994-03-25 AU AU64330/94A patent/AU6433094A/en not_active Abandoned
- 1994-03-25 WO PCT/GB1994/000630 patent/WO1994022131A2/en active IP Right Grant
- 1994-03-25 DE DE69421911T patent/DE69421911T2/en not_active Expired - Lifetime
- 1994-03-25 EP EP99202632A patent/EP0962913B1/en not_active Expired - Lifetime
- 1994-03-25 JP JP52084194A patent/JP3691511B2/en not_active Expired - Fee Related
- 1994-03-25 US US08/525,730 patent/US5848388A/en not_active Expired - Lifetime
- 1994-03-25 CA CA002158849A patent/CA2158849C/en not_active Expired - Fee Related
- 1994-03-25 ES ES94912002T patent/ES2141824T3/en not_active Expired - Lifetime
- 1994-03-25 DE DE69432570T patent/DE69432570T2/en not_active Expired - Lifetime
- 1994-03-25 SG SG9904114A patent/SG93215A1/en unknown
- 1994-03-25 EP EP94912002A patent/EP0691022B1/en not_active Expired - Lifetime
Also Published As
| Publication number | Publication date |
|---|---|
| CA2158849A1 (en) | 1994-09-29 |
| ES2141824T3 (en) | 2000-04-01 |
| EP0691022B1 (en) | 1999-12-01 |
| WO1994022131A3 (en) | 1995-01-12 |
| DE69432570T2 (en) | 2004-03-04 |
| DE69421911T2 (en) | 2000-07-20 |
| AU6433094A (en) | 1994-10-11 |
| WO1994022131A2 (en) | 1994-09-29 |
| DE69421911D1 (en) | 2000-01-05 |
| EP0962913A1 (en) | 1999-12-08 |
| JPH08508108A (en) | 1996-08-27 |
| EP0691022A1 (en) | 1996-01-10 |
| DE69432570D1 (en) | 2003-05-28 |
| SG93215A1 (en) | 2002-12-17 |
| US5848388A (en) | 1998-12-08 |
| JP3691511B2 (en) | 2005-09-07 |
| EP0962913B1 (en) | 2003-04-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2158849C (en) | Speech recognition with pause detection | |
| US8566088B2 (en) | System and method for automatic speech to text conversion | |
| EP1083542B1 (en) | A method and apparatus for speech detection | |
| US8036884B2 (en) | Identification of the presence of speech in digital audio data | |
| US5025471A (en) | Method and apparatus for extracting information-bearing portions of a signal for recognizing varying instances of similar patterns | |
| US8069039B2 (en) | Sound signal processing apparatus and program | |
| Sahoo et al. | Silence removal and endpoint detection of speech signal for text independent speaker identification | |
| JPH10319988A (en) | Speaker identifying method and speaker recognizing device | |
| KR20010102549A (en) | Speaker recognition | |
| US4937870A (en) | Speech recognition arrangement | |
| JP4696418B2 (en) | Information detection apparatus and method | |
| JP2797861B2 (en) | Voice detection method and voice detection device | |
| JP3633254B2 (en) | Voice recognition system and recording medium recording the program | |
| JP2002189487A (en) | Speech recognition device and speech recognition method | |
| KR100940641B1 (en) | Utterance verification system and method using word voiceprint models based on probabilistic distributions of phone-level log-likelihood ratio and phone duration | |
| EP0177854B1 (en) | Keyword recognition system using template-concatenation model | |
| Keshet et al. | Plosive spotting with margin classifiers. | |
| US7292981B2 (en) | Signal variation feature based confidence measure | |
| JP2002516419A (en) | Method and apparatus for recognizing at least one keyword in a spoken language by a computer | |
| JP2991288B2 (en) | Speaker recognition device | |
| Pawar et al. | Analysis of FFSR, VFSR, MFSR techniques for feature extraction in speaker recognition: a review | |
| Ying et al. | Robust voice activity detection based on noise eigenspace | |
| Nickel et al. | Robust speaker verification with principal pitch components | |
| Aye | Speech recognition using Zero-crossing features | |
| WO1997037345A1 (en) | Speech processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKLA | Lapsed |