US6442519B1 - Speaker model adaptation via network of similar users - Google Patents

Speaker model adaptation via network of similar users Download PDF

Info

Publication number
US6442519B1
US6442519B1 US09/437,646 US43764699A US6442519B1 US 6442519 B1 US6442519 B1 US 6442519B1 US 43764699 A US43764699 A US 43764699A US 6442519 B1 US6442519 B1 US 6442519B1
Authority
US
United States
Prior art keywords
speech
user
acoustic
users
domain
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
US09/437,646
Inventor
Dimitri Kanevsky
Vit V. Libal
Jan Sedivy
Wlodek W. Zadrozny
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nuance Communications Inc
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Priority to US09/437,646 priority Critical patent/US6442519B1/en
Assigned to INTERNATIONAL BUSINESS MACHINES CORPORATION reassignment INTERNATIONAL BUSINESS MACHINES CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: LIBAL, VIT V., SEDIVY, JAN
Assigned to INTERNATIONAL BUSINESS MACHINES CORPORATION reassignment INTERNATIONAL BUSINESS MACHINES CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: KANEVSKY, DIMITRI, ZADROZNY, WLODEK W.
Application granted granted Critical
Publication of US6442519B1 publication Critical patent/US6442519B1/en
Assigned to NUANCE COMMUNICATIONS, INC. reassignment NUANCE COMMUNICATIONS, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: INTERNATIONAL BUSINESS MACHINES CORPORATION
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/06Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
    • G10L15/065Adaptation
    • G10L15/07Adaptation to the speaker

Definitions

  • the present invention is related to U.S. patent application Ser. No. 08/787,031, filed Jan. 28, 1997 entitled “Speaker Recognition Using Thresholded Speaker Class Model Section or Model Adaptation” to Ittycheriah, et al. now issued as U.S. Pat. No. 5,895,447, U.S. patent application Ser. No. 08/788,471, filed Jan. 28, 1997 entitled “Text Independent Speaker Recognition for Transparent Command Ambiguity Resolution and Continuous Access Control” now U.S. Pat. No. 6,073,101 issued Jun. 6, 200, and U.S. patent application Ser. No. 08/787,029, filed Jan. 28, 1997 entitled “Speaker Model Prefetching” both to Stephane Maes now U.S. Pat. No.
  • the present invention is related to speech recognition and more particularly to speech recognition on multiple connected computer systems connected together over a network.
  • ASR Automatic speech recognition
  • Speaker independent automatic speech recognition systems such as what are normally referred to as interactive voice response systems, have a different set of problems, because they are intended to recognize speech from a wide variety of individual speakers.
  • the approach with speaker independent ASR systems is to improve recognition accuracy by assigning individual speakers or recognition system users to user clusters.
  • User clusters are groups of users with similar speech characteristics or patterns. As each speaker or user uses the system, the speaker is identified as belonging to one cluster. For each user cluster, acoustic prototypes are developed and are used for speech decoding.
  • speakers may be clustered, according to language or accent.
  • Various techniques for language identification are taught by D. Matrouf, M. Adda-Decker, L. Lamel and J. Gauvain, in “Language Identification Incorporating Lexical Information” in Proceedings of the 1998 International Conference on Spoken Language Processing (ICSLP 98), Sydney, Australia, December 1998.
  • a well known method of determining an accent from acoustic features is taught by M. Lincoln, S. Cox and S. Ringland, in “A Comparison of Two Unsupervised Approaches to Accent Identification” Proceedings of the 1998 International Conference on Spoken Language Processing (ICSLP 98), Sydney, Australia, December 1998.
  • ASR decoding system approaches are used that are based on various adaptation schemes for acoustic models. These recognition adaptation schemes use additional data that is gathered subsequent to training by the ASR system every time a user dictates to the system.
  • the speaker or user usually, interactively corrects any errors in the recognition result, and those corrected scripts are used for what is normally referred to as a supervised adaptation.
  • unsupervised ASR is used in voice response systems wherein each user calls in to a service that uses ASR to process user voice input.
  • C.H. Lee and J.L. Gauvain, “Bayesian adaptive Learning and MAP Estimation of HMM”, in Automatic Speech and Speaker Recognition , edited by Chin-Hui Lee, Frank K. Song, 1996, Kluwer academic Publishers, Boston, pp. 109-132 describe for supervised and unsupervised acoustic model adaptation methods. While it is still possible to adapt speech recognition for any new users using unsupervised adaptation, sufficient data must be collected prior to unsupervised use to insure adequate decoding accuracy for every new user.
  • the present invention is a speech recognition system, method and program product for recognizing speech input from computer users connected together over a network of computers, each computer including at least one user based acoustic model trained for a particular user.
  • Computer users on the network are clustered into classes of similar users according their similarities, including characteristics nationality, profession, sex, age, etc. Characteristics of users are collected from databases over the network and from users using the speech recognition system and distributed over the networks during or after user activities. As recognition progresses, similar language models among similar users are identified on the network.
  • the acoustic models include an acoustic model domain, with similar acoustic models being clustered according to an identified domain. Existing acoustic models are modified in response to user production activities. Update information, including information about user activities and user acoustic model data, is transmitted over the network. Acoustic models improve for users that are connected over the network as similar users use their respective voice recognition system.
  • FIG. 1 is a block diagram of users connected together over network using shared acoustic models during user production activities according to the preferred embodiment of the present invention.
  • FIG. 2 is a flow chart of the preferred embodiment speech recognition process operating over the network of FIG. 1;
  • FIG. 3 is a flowchart showing how one or more acoustic models are changed
  • FIG. 4 is an illustration of user actions and user production activities
  • FIG. 5 is a flow diagram of the user clustering step
  • FIG. 6 is a flow chart of the acoustic component comparison step
  • FIG. 7 is a flowchart illustrating supervised and unsupervised speech adaptation according to the preferred embodiment of the present invention.
  • FIG. 1 shows a speech recognition network 100 , wherein individual user utterances are analyzed to classify the user, the individual user is clustered with other similar users, and data from similar acoustic models for those clustered users are aggregated to provide an expanded or supplemented acoustic model for recognition of that individual user.
  • Computers 102 , 104 and 106 connected to the network 100 are each capable of executing an acoustic model (AM) for some type of speech recognition.
  • speech enabled interface devices 108 , 110 with embedded processors or smart controllers may be connected to the network 100 .
  • utterances or speech input may be, for example, for command/menu navigation, dictation or transcription.
  • the speech recognition network 100 is a local area network (LAN) of connected speech recognition computers 102 , 104 , 106 and speech enabled devices 108 , 110 .
  • the network 100 may be a wide area network (WAN), a connection of computers over what is commonly referred to as the internet or world wide web (www) or over an intranet, an extranet, a radio network or a telephone network, or any combination thereof
  • connected computers 102 , 104 , 106 may include what are commonly referred to as personal computers 102 , hand held computers 104 , and one or more servers 106 .
  • Hand held computers 104 may include what is known as a personal digital assistant (PDA).
  • Connected speech enabled interface devices may include, for example, cameras 108 , intelligent watches 110 and connected telephones 116 .
  • microphones 112 , 114 are shown connected to personal computer 102 and PDA 104 and are integrated into speech enabled interface devices 108 , 110 for receiving speech input from a user.
  • Personal computers 102 also may include an audio capture module that receives audio signals and converts received audio signals into digital signals.
  • Each of the speech recognition computers 102 , 104 includes an automatic speech recognition module and a local database containing acoustic models for local users. For speech recognition, each local acoustic model is the direct result of training by a specific local user.
  • a global database is maintained on at least one speech recognition server 106 .
  • the global database may include multiple acoustic models for users of connected computers, as well as individual user speech data. Further, as individual user features or characteristics are collected from local user databases in computers 102 , 104 , the collected features are aggregated in the global databases on servers 106 .
  • Interface devices 108 , 110 may avail themselves of excess capacity on servers 106 , storing local databases on a server 106 and using acoustic models from a server's global database for speech recognition.
  • the preferred embodiment of the present invention includes a dialog management unit for conducting a conversation with an individual user.
  • An audio capture module coupled to the dialog management unit captures a speech waveform from utterances spoken by the individual user during the conversation.
  • An acoustic front end coupled to the audio capture module is configured to receive and digitize the speech waveform so as to provide a digital waveform, and to extract, from the digital waveform, at least one acoustic feature.
  • the acoustic front end and audio capture module may be, for example, a microphone connected to an analog-to-digital converter located on a sound board in a personal computer or a telephone connected to an automatic interactive voice response (IVR) system.
  • the dialog management unit can include a telephone IVR system that may be, for example, the same automatic IVR system used to implement the audio capture module. Alternatively, the dialog management unit may simply be an acoustic interface to a human operator.
  • the preferred embodiment system includes at least one processing module coupled to the acoustic front end that analyzes the extracted acoustic features to determine user cluster attributes, i.e., to classify the user or speaker.
  • Each processing module includes a speaker clusterer and classifier.
  • the processing module is implemented by the processor of the IVR system. Alternatively, dedicated hardware may be used for the processing module such as an application specific integrated circuit (ASIC) or a separate general purpose computer with appropriate software.
  • the classifier can include a speaker clustering and classification module as well as a speaker classification data base. Cluster user attributes from the processing module are stored in a speaker cluster database stores.
  • a speech adaptation module transmits data to other connected user speech systems.
  • the present invention is an apparatus for collecting data associated with the voice of a user, which is then supplemented by previously collected data and used for speech recognition for the user.
  • capturing the user's speech waveform and digitizing the speech waveform acoustic features may be extracted from the digitized speech waveform.
  • the extracted features are passed to other connected systems and used to modify the speech recognition systems of similar users clustered in the same user cluster.
  • Speaker classification may be supervised or unsupervised.
  • supervised classification the classes are decided beforehand based on externally provided information. Typically, such classification employs distinctions between male and female, adult versus child, native speakers versus different potential nonnative speakers, and the like.
  • unsupervised classification there is no advanced user labeling and classes are developed on the fly with the classification information being extracted from data using very little supervision, if any, and with sounds being clustered as classes develop.
  • the processing module includes an accent identifier.
  • the accent identifier includes an accent identification module and an accent data base.
  • the accent identifier is employed for native language identification in a manner equivalent, essentially, to accent classification. Meta information about the identified native language of a speaker provides additional to definition each accent/native language model. A dialect can be determined from the user's accent.
  • a continuous speech recognizor is trained by several speakers with different accents.
  • For accent identification an accent vector is extracted from each individual user's speech and the accent vector is classified.
  • the accent vector is associated with each of the training speakers.
  • Accent vector dimensions represent the most likely component mixture associated with each state of each phoneme or phone. Then, the speakers are clustered based on the distance between corresponding accent vectors, and the clusters are identified by the accent of the member speakers.
  • model types may include what are known as acoustic prototypes, Hidden Markov Models (HMM) modeling words and phonemes or phones, acoustic ranks, acoustic decision trees.
  • HMM Hidden Markov Models
  • the preferred statistical analysis techniques include analysis of parameters such as weighted mixtures of decoding scores, thresholds that control decoding stacks, duration of phones or words, sizes of previously listed decoding alternatives, and/or the size of decoding trees.
  • each acoustic model is directed to a different speech domain. It is important to distinguish between speech domains, because the particular speech domain deeply influences the resulting speech model. Accordingly, in addition to dictation, the speech domains may include telephone speech, speaker independent speech, gender related speech, age related speech, broadcasting speech, speech partially obscured by noise, speech with music, discrete and continuous speech. Further, speech may be the result of different user production activities such as dictation or conversation that is supplemented by error correction and may be partially obscured by noise or music or by some other type of sound generation. As used herein, user production activities refers to, generally, speech or audio related activities that are intended to produce specific computer related responses.
  • a corresponding local acoustic model on each corresponding user's system e.g., PDA 104
  • PDA 104 recognizes the particular user's speech.
  • the corrections are stored locally in a local database and used to adjust and refine the local acoustic model.
  • the corrections and modifications are passed across the network 100 to the global database on one of the connected recognition servers 106 , which in turn distributes the corrections across the network to computers of other clustered similar users.
  • the preferred processing module also includes a speaker recognizor, which may be the same as the speech recognizor above.
  • the speaker recognizor is a speaker recognition module that includes a speaker prototype model, a language model and a grammar database.
  • the speaker recognizor transcribes queries from the user.
  • the speaker recognizor is a speaker-independent large-vocabulary continuous-speech recognition system.
  • the speaker recognizor is a class-dependent large-vocabulary continuous-speech recognition system.
  • Such speech recognition systems are well known in the art.
  • the output of the normal speech recognizor is complete sentences. However, finer granularity also is selectable, e.g., time alignment of the recognized words.
  • the acoustic front end extracts acoustic features that are supplied to the speaker clusterer and classifier, the speaker recognizor and the accent identifier.
  • the acoustic front end is an eight-dimension-plus-energy front end, such as are well known in the art.
  • the speech spectrum is divided into cepstral coefficients using a bank of MEL filters providing what is referred to as MEL cepstra.
  • MEL cepstra is computed over 25 ms frames with a 10 ms overlap, in combination with deriving first and second finite derivatives, typically referred to as the delta and delta-delta parameters of the speech.
  • Pitch jitter is the number of sign changes of the first derivative of pitch.
  • Shimmer is energy jitter.
  • acoustic features may be supplied from the acoustic front end to the classifier.
  • the aforementioned acoustic features, including the MEL cepstra are, essentially, raw, unprocessed features.
  • acoustic profile data for individual users previously accumulated and stored in the local databases are passed over the network 100 to the server 106 .
  • the user acoustic data are compared in step 124 in the server 106 .
  • step 126 based on that comparison, users are clustered into classes of similar users according to acoustic voice similarities.
  • different acoustic models i.e., different domains
  • acoustic model components for similar users are modified relative to user production activities.
  • similar acoustic models from different user sets located elsewhere on the network also are modified in the server 106 .
  • modified acoustic models are transmitted from the server 106 to other sites on the network 100 . So, acoustic model components, including data about users and information about user activities, are thereby synchronized in all similar acoustic models across the network.
  • FIG. 3 is a flowchart showing how one or more acoustic models are modified in step 130 .
  • the input changes to the model may be supervised input 134 or unsupervised input 132 , the output of automatic speech recognition 136 or the result of user production activities 138 as described in detail hereinbelow with reference to FIG. 4 .
  • the result of the user production activities 138 may be additional speech data (i.e., data collected from other speech related tasks such as speaker identification, speech recording, etc.) 140 or acoustic training data 142 .
  • Acoustic training data 142 is generated at each initial use by a new user.
  • Acoustic training data 142 includes, for example, acoustic prototypes, or Hidden Markov Models. Alternately, acoustic training data 142 may be employed for growing acoustic decision trees, each decision tree being based on the user's speech training data. Furthermore, acoustic training 142 may include estimating parameters that control the decoding process, estimating parameters that control signal processing and compiling a code book of user speech data. Parameters that control the decoding process may include, but are not limited to, weights of decoding score mixtures, decision thresholds that control decoding stacks, phone or word durations, decoding alternative list sizes, decoding tree sizes.
  • step 144 After receiving all acoustic input data, user acoustic model components are modified in step 144 . Then, in step 146 , acoustic prototypes are adapted for any additional or subsequent speech data produced by a user. In step 148 , HMM parameters are adapted incrementally to additional user speech data. Finally, in step 150 , new words are added to the acoustic vocabulary and new queries are added to acoustic decision trees. Additionally, adding new words in step 150 may entail modifying acoustic ranks, as well as adapting relative weights of language models and acoustic models.
  • FIG. 4 is an illustration of user actions and user production activities.
  • User production activities may include activities such as dictation 160 , conversation 162 , error correction 164 , generation of sounds 166 including noise and music. So, dictation 160 , conversation 164 and background audio 166 are provided to automatic speech recognition module 168 .
  • the automatic speech recognition module 168 generates either text 170 or passes the recognition results to dialog module 172 .
  • Error correction 164 operates on the text 170 , correcting any recognition errors, providing a supervised adaptation 174 of the input.
  • the dialog module 172 generates system commands 176 and queries 178 in response to recognition results passed to it.
  • FIG. 5 is a flow diagram of the user clustering step 126 in FIG. 1 .
  • the user's speaker characteristics including but not limited to the user's educational level, age, gender, family relationship and nationality are gathered and provided as a user profile in step 182 .
  • Network data for all users including user profiles are compared to identify similar users in step 184 .
  • an acoustic front end produces acoustic features, e.g., as the result of training.
  • corresponding acoustic features are identified in the speaker's voice.
  • acoustic features may include, for example, accent, vocal tract characteristics, voice source characteristics, fundamental frequency, running average pitch, running pitch variance, pitch jitter, running energy variance, speech rate, shimmer, fundamental frequency, variation in fundamental frequency and MEL cepstra.
  • acoustic features collected from various users are compared. Acoustic models from the same domain but from different sets or systems are compared in step 192 .
  • Common features are identified in step 194 and passed to step 184 to identify similar users. Similar users identified in step 184 are users that have one or more common characteristics or, one or more common acoustic features.
  • step 196 user clusters are identified to cluster users with one or several common features, with several similar acoustic components or with similar profile characteristics, thereby classifying such users in the same classes. Additionally, thereafter, user characteristics are recorded, collected and used for further user classification.
  • FIG. 6 is a flow chart of the acoustic component comparison step 192 of FIG. 5 .
  • step 200 acoustic vocabularies and features are provided and represented as vectors in step 202 .
  • the distance preferably the Euclidean distance between vectors is calculated. Alternately the Kulback distance may be calculated.
  • step 206 the computed distances are compared against threshold values to identify similar models, similar models being defined as having calculated values that fall below the threshold values.
  • Acoustic user vocabularies, acoustic features, acoustic user components, acoustic prototypes, Hidden Markov Models for words and phones and accent vectors are compared to determine similarities. Also, acoustic vocabularies of similar users may be analyzed to update the user acoustic vocabulary.
  • FIG. 7 is a flowchart illustrating supervised and unsupervised speech adaptation according to the preferred embodiment of the present invention, further illustrating aspects of the invention not shown in FIG. 4 .
  • Automatic speech recognition module 210 receives speech and, depending on the content of the user's speech, provides commands 212 , queries 214 or uncorrected decoded textual data 216 .
  • Commands 212 and queries 214 are passed directly to one or more applications 218 and/or to the operating system.
  • Commands 212 may direct an application operation, e.g., “open file . . . , ” “close,” “indent,” or, when passed to the operating system, may direct window navigation.
  • Queries 214 are passed to appropriate applications 218 , e.g., queries 214 are passed to a database manager for database searching.
  • Commands 212 and queries 214 that are passed to applications 218 elicit a textual output.
  • the textual output is passed to a supervisor 222 for approval.
  • the text may be, for example, used for transmission as e-mail; a decoded document for storage after some period of time that signifies supervisor approval; a decoded document that was corrected; or, a newly decoded document.
  • uncorrected decoded textual data 216 is corrected in step 224 and passed to the supervisor 222 as corrected text 226 .
  • the decoded textual data 216 is provided directly for unsupervised adaptation 228 . Accordingly, the decoded textual data 216 is either uncorrected textual data, unexecuted commands or unissued queries.

Abstract

A speech recognition system, method and program product for recognizing speech input from computer users connected together over a network of computers. Speech recognition computer users on the network are clustered into classes of similar users according their similarities, including characteristics nationality, profession, sex, age, etc. Each computer in the speech recognition network includes at least one user based acoustic model trained for a particular user. The acoustic models include an acoustic model domain, with similar acoustic models being clustered according to an identified domain. User characteristics are collected from databases over the network and from users using the speech recognition system and then, distributed over the network during or after user activities. Existing acoustic models are modified in response to user production activities. As recognition progresses, similar language models among similar users are identified on the network. Update information, including information about user activities and user acoustic model data, is transmitted over the network and identified similar language models are updated. Acoustic models improve for users that are connected over the network as similar users use their respective speech recognition system.

Description

RELATED APPLICATIONS
The present invention is related to U.S. patent application Ser. No. 08/787,031, filed Jan. 28, 1997 entitled “Speaker Recognition Using Thresholded Speaker Class Model Section or Model Adaptation” to Ittycheriah, et al. now issued as U.S. Pat. No. 5,895,447, U.S. patent application Ser. No. 08/788,471, filed Jan. 28, 1997 entitled “Text Independent Speaker Recognition for Transparent Command Ambiguity Resolution and Continuous Access Control” now U.S. Pat. No. 6,073,101 issued Jun. 6, 200, and U.S. patent application Ser. No. 08/787,029, filed Jan. 28, 1997 entitled “Speaker Model Prefetching” both to Stephane Maes now U.S. Pat. No. 6,088,669 issued Jul. 11, 2000, and (Ser. No. 09/422,383) entitled “Language Model Adaptation Via Network of Similar Users” filed Oct. 21, 1999, all assigned to the assignee of the present invention. These patents and patent applications are herein incorporated by reference in their entirety.
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention is related to speech recognition and more particularly to speech recognition on multiple connected computer systems connected together over a network.
2. Background Description
Automatic speech recognition (ASR) systems for voice dictation and the like use any of several well known approaches to for word recognition.
For example, L. R. Bahl, P. V. de Souza, P. S. Gopalakrishnan, D. Nahamoo, and M. Picheny, “Robust Methods for Using Context-dependent Features and Models in Continuous Speech Recognizer,” Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, vol. I, pp. 533-36, Adelaide, 1994, describe an acoustic ranking method useful for speech recognition. Acoustic decision trees, also useful for speech recognition are described by L. R. Bahl, P. V. de Souza, P. S. Gopalakrishnan, D. Nahamoo, and M. Picheny, in “Decision Trees for Phonological Rules in Continuous Speech,” Proceedings of the 1991 International Conference on Acoustic, Speech, and Signal Processing, Toronto, Canada, May 1991. Frederick Jelinek in Statistical Methods for Speech Recognition, The MIT Press, Cambridge, January 1999, describes identifying parameters that control decoding process.
While generally recognizing spoken words with a relatively high degree of accuracy, especially in a single user system, these prior speech recognition systems still, frequently, make inappropriate recognition errors. Generally, for single user systems, these errors can be reduced with additional user specific training. However, additional training time and increased data volume that must be handled during training are undesirable. So, for expediency, recognition accuracy is traded to minimize training time and data.
Speaker independent automatic speech recognition systems, such as what are normally referred to as interactive voice response systems, have a different set of problems, because they are intended to recognize speech from a wide variety of individual speakers. Typically, the approach with speaker independent ASR systems is to improve recognition accuracy by assigning individual speakers or recognition system users to user clusters. User clusters are groups of users with similar speech characteristics or patterns. As each speaker or user uses the system, the speaker is identified as belonging to one cluster. For each user cluster, acoustic prototypes are developed and are used for speech decoding.
For example, speakers may be clustered, according to language or accent. Various techniques for language identification are taught by D. Matrouf, M. Adda-Decker, L. Lamel and J. Gauvain, in “Language Identification Incorporating Lexical Information” in Proceedings of the 1998 International Conference on Spoken Language Processing (ICSLP 98), Sydney, Australia, December 1998. A well known method of determining an accent from acoustic features is taught by M. Lincoln, S. Cox and S. Ringland, in “A Comparison of Two Unsupervised Approaches to Accent Identification” Proceedings of the 1998 International Conference on Spoken Language Processing (ICSLP 98), Sydney, Australia, December 1998. However, the approach of Lincoln et al., if there is a very large speaker variability, as is normally the case, that variability may not be accounted for in training. Accordingly, speaker clusters that are accumulated in a normal ASR training period, generally, do not provide for all potential ASR users.
Consequently, to provide some improvement over speaker dependent methods, ASR decoding system approaches are used that are based on various adaptation schemes for acoustic models. These recognition adaptation schemes use additional data that is gathered subsequent to training by the ASR system every time a user dictates to the system. The speaker or user, usually, interactively corrects any errors in the recognition result, and those corrected scripts are used for what is normally referred to as a supervised adaptation.
See for example, Jerome R. Bellegarda, in “Context-dependent Vector Clustering for Speech Recognition,” in Automatic Speech and Speaker Recognition, edited by Chin-Hui Lee, Frank K. Song, 1996, Kluwer academic Publishers, Boston, pp. 133-153 which teaches an adaptation of acoustic prototypes in response to subsequent speech data collected from other users. Also, M. J. F. Gales and P.C. Woodland, “Mean and variance adaptation within the MLLR framework,” Computer Speech and Language (1996) 10, 249-264 teach incremental adaptation of HMM parameters derived from speech data from additional subsequent users.
The drawback with the above approaches of Bellegarda or Gales et al. is that during typical dictation sessions the user uses a relatively small number of phrases. So, it may take several user sessions to gather sufficient acoustic data to show any significant recognition accuracy improvement using such a supervised adaptation procedure. As might be expected, in the initial sessions the decoding accuracy may be very low, requiring significant interactive error correction.
Further, similar or even worse problems arise in unsupervised ASR applications when users do not correct ASR output. For example, unsupervised ASR is used in voice response systems wherein each user calls in to a service that uses ASR to process user voice input. C.H. Lee and J.L. Gauvain, “Bayesian adaptive Learning and MAP Estimation of HMM”, in Automatic Speech and Speaker Recognition, edited by Chin-Hui Lee, Frank K. Song, 1996, Kluwer academic Publishers, Boston, pp. 109-132 describe for supervised and unsupervised acoustic model adaptation methods. While it is still possible to adapt speech recognition for any new users using unsupervised adaptation, sufficient data must be collected prior to unsupervised use to insure adequate decoding accuracy for every new user.
Thus, there is a need for increasing the amount of usable acoustic data that are available for speech recognition of individual speakers in supervised and unsupervised speech recognition sessions.
SUMMARY OF THE INVENTION
It is a purpose of the invention to improve speech recognition by computers;
It is yet another purpose of the invention to expand the data available for speech recognition.
The present invention is a speech recognition system, method and program product for recognizing speech input from computer users connected together over a network of computers, each computer including at least one user based acoustic model trained for a particular user. Computer users on the network are clustered into classes of similar users according their similarities, including characteristics nationality, profession, sex, age, etc. Characteristics of users are collected from databases over the network and from users using the speech recognition system and distributed over the networks during or after user activities. As recognition progresses, similar language models among similar users are identified on the network. The acoustic models include an acoustic model domain, with similar acoustic models being clustered according to an identified domain. Existing acoustic models are modified in response to user production activities. Update information, including information about user activities and user acoustic model data, is transmitted over the network. Acoustic models improve for users that are connected over the network as similar users use their respective voice recognition system.
DESCRIPTION OF THE DRAWINGS
The foregoing and other objects, aspects and advantages will be better understood from the following detailed description of a preferred embodiment of the invention with reference to the drawings, in which:
FIG. 1 is a block diagram of users connected together over network using shared acoustic models during user production activities according to the preferred embodiment of the present invention.
FIG. 2 is a flow chart of the preferred embodiment speech recognition process operating over the network of FIG. 1;
FIG. 3 is a flowchart showing how one or more acoustic models are changed;
FIG. 4 is an illustration of user actions and user production activities;
FIG. 5 is a flow diagram of the user clustering step;
FIG. 6 is a flow chart of the acoustic component comparison step;
FIG. 7 is a flowchart illustrating supervised and unsupervised speech adaptation according to the preferred embodiment of the present invention.
DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT OF THE INVENTION
Referring now to the drawings and, more particularly, FIG. 1 shows a speech recognition network 100, wherein individual user utterances are analyzed to classify the user, the individual user is clustered with other similar users, and data from similar acoustic models for those clustered users are aggregated to provide an expanded or supplemented acoustic model for recognition of that individual user. Computers 102, 104 and 106 connected to the network 100 are each capable of executing an acoustic model (AM) for some type of speech recognition. Also, speech enabled interface devices 108, 110 with embedded processors or smart controllers may be connected to the network 100. In the preferred embodiment networked system, utterances or speech input may be, for example, for command/menu navigation, dictation or transcription.
Preferably, the speech recognition network 100 is a local area network (LAN) of connected speech recognition computers 102, 104, 106 and speech enabled devices 108, 110. Optionally, the network 100 may be a wide area network (WAN), a connection of computers over what is commonly referred to as the internet or world wide web (www) or over an intranet, an extranet, a radio network or a telephone network, or any combination thereof
As shown in FIG. 1, by way of example only, connected computers 102, 104, 106 may include what are commonly referred to as personal computers 102, hand held computers 104, and one or more servers 106. Hand held computers 104 may include what is known as a personal digital assistant (PDA). Connected speech enabled interface devices may include, for example, cameras 108, intelligent watches 110 and connected telephones 116. Further, microphones 112, 114 are shown connected to personal computer 102 and PDA 104 and are integrated into speech enabled interface devices 108, 110 for receiving speech input from a user.
Personal computers 102 also may include an audio capture module that receives audio signals and converts received audio signals into digital signals. Each of the speech recognition computers 102, 104 includes an automatic speech recognition module and a local database containing acoustic models for local users. For speech recognition, each local acoustic model is the direct result of training by a specific local user. Further, a global database is maintained on at least one speech recognition server 106. The global database may include multiple acoustic models for users of connected computers, as well as individual user speech data. Further, as individual user features or characteristics are collected from local user databases in computers 102, 104, the collected features are aggregated in the global databases on servers 106. Interface devices 108, 110 may avail themselves of excess capacity on servers 106, storing local databases on a server 106 and using acoustic models from a server's global database for speech recognition.
The preferred embodiment of the present invention includes a dialog management unit for conducting a conversation with an individual user. An audio capture module coupled to the dialog management unit captures a speech waveform from utterances spoken by the individual user during the conversation. An acoustic front end coupled to the audio capture module is configured to receive and digitize the speech waveform so as to provide a digital waveform, and to extract, from the digital waveform, at least one acoustic feature. The acoustic front end and audio capture module may be, for example, a microphone connected to an analog-to-digital converter located on a sound board in a personal computer or a telephone connected to an automatic interactive voice response (IVR) system. The dialog management unit can include a telephone IVR system that may be, for example, the same automatic IVR system used to implement the audio capture module. Alternatively, the dialog management unit may simply be an acoustic interface to a human operator.
The preferred embodiment system includes at least one processing module coupled to the acoustic front end that analyzes the extracted acoustic features to determine user cluster attributes, i.e., to classify the user or speaker. Each processing module includes a speaker clusterer and classifier. Preferably, the processing module is implemented by the processor of the IVR system. Alternatively, dedicated hardware may be used for the processing module such as an application specific integrated circuit (ASIC) or a separate general purpose computer with appropriate software. The classifier can include a speaker clustering and classification module as well as a speaker classification data base. Cluster user attributes from the processing module are stored in a speaker cluster database stores. A speech adaptation module transmits data to other connected user speech systems. Thus, the present invention is an apparatus for collecting data associated with the voice of a user, which is then supplemented by previously collected data and used for speech recognition for the user.
So, by conducting a conversation with a voice system user, capturing the user's speech waveform and digitizing the speech waveform acoustic features may be extracted from the digitized speech waveform. The extracted features are passed to other connected systems and used to modify the speech recognition systems of similar users clustered in the same user cluster.
Speaker classification may be supervised or unsupervised. For supervised classification, the classes are decided beforehand based on externally provided information. Typically, such classification employs distinctions between male and female, adult versus child, native speakers versus different potential nonnative speakers, and the like. For unsupervised classification there is no advanced user labeling and classes are developed on the fly with the classification information being extracted from data using very little supervision, if any, and with sounds being clustered as classes develop.
Preferably, the processing module includes an accent identifier. The accent identifier includes an accent identification module and an accent data base. The accent identifier is employed for native language identification in a manner equivalent, essentially, to accent classification. Meta information about the identified native language of a speaker provides additional to definition each accent/native language model. A dialect can be determined from the user's accent.
According to the preferred embodiment, a continuous speech recognizor is trained by several speakers with different accents. For accent identification an accent vector is extracted from each individual user's speech and the accent vector is classified. The accent vector is associated with each of the training speakers. Accent vector dimensions represent the most likely component mixture associated with each state of each phoneme or phone. Then, the speakers are clustered based on the distance between corresponding accent vectors, and the clusters are identified by the accent of the member speakers.
For each cluster various types of speech recognition may be employed for speech recognition, which in combination with the particular speech recognition computer 102, 104, 106 determine the form of the individual acoustic models. Thus, individual model types may include what are known as acoustic prototypes, Hidden Markov Models (HMM) modeling words and phonemes or phones, acoustic ranks, acoustic decision trees. The preferred statistical analysis techniques include analysis of parameters such as weighted mixtures of decoding scores, thresholds that control decoding stacks, duration of phones or words, sizes of previously listed decoding alternatives, and/or the size of decoding trees.
Further, each acoustic model is directed to a different speech domain. It is important to distinguish between speech domains, because the particular speech domain deeply influences the resulting speech model. Accordingly, in addition to dictation, the speech domains may include telephone speech, speaker independent speech, gender related speech, age related speech, broadcasting speech, speech partially obscured by noise, speech with music, discrete and continuous speech. Further, speech may be the result of different user production activities such as dictation or conversation that is supplemented by error correction and may be partially obscured by noise or music or by some other type of sound generation. As used herein, user production activities refers to, generally, speech or audio related activities that are intended to produce specific computer related responses.
So, for the networked speech recognition system of the preferred embodiment, as users issue commands, dictate letters, etc., a corresponding local acoustic model on each corresponding user's system, e.g., PDA 104, recognizes the particular user's speech. If the user corrects the results of the recognition, the corrections are stored locally in a local database and used to adjust and refine the local acoustic model. As correction or modifications are made to the local model, the corrections and modifications are passed across the network 100 to the global database on one of the connected recognition servers 106, which in turn distributes the corrections across the network to computers of other clustered similar users.
The preferred processing module also includes a speaker recognizor, which may be the same as the speech recognizor above. The speaker recognizor is a speaker recognition module that includes a speaker prototype model, a language model and a grammar database. Preferably, the speaker recognizor transcribes queries from the user. In one preferred embodiment, the speaker recognizor is a speaker-independent large-vocabulary continuous-speech recognition system. In a second preferred embodiment, the speaker recognizor is a class-dependent large-vocabulary continuous-speech recognition system. Such speech recognition systems are well known in the art. The output of the normal speech recognizor is complete sentences. However, finer granularity also is selectable, e.g., time alignment of the recognized words.
As described hereinabove, the acoustic front end extracts acoustic features that are supplied to the speaker clusterer and classifier, the speaker recognizor and the accent identifier. Preferably, the acoustic front end is an eight-dimension-plus-energy front end, such as are well known in the art. For the preferred front end, the speech spectrum is divided into cepstral coefficients using a bank of MEL filters providing what is referred to as MEL cepstra. Thus, for example, MEL cepstra is computed over 25 ms frames with a 10 ms overlap, in combination with deriving first and second finite derivatives, typically referred to as the delta and delta-delta parameters of the speech. Other types of optional acoustic features that may be extracted by the acoustic front end include a running average pitch, a running pitch variance, pitch jitter, running energy variance, speech rate, shimmer, fundamental frequency, and variation in fundamental frequency. Pitch jitter is the number of sign changes of the first derivative of pitch. Shimmer is energy jitter.
These optional acoustic features may be supplied from the acoustic front end to the classifier. The aforementioned acoustic features, including the MEL cepstra are, essentially, raw, unprocessed features.
User queries are transcribed by an IVR, for example, and speech features are first processed by a text-independent speaker classification system according to the preferred speaker clusterer and classifier 120 of FIG. 2 which shows a flow chart of the preferred embodiment speech recognition process 120 operating over the network 100 of FIG. 1. This permits classification of the speakers based on acoustic similarities of their voices. Systems and methods of classifying users according to voice similarities is taught in U.S. patent application Ser. No. 08/787,031, filed Jan. 28, 1997 entitled “Speaker Recognition Using Thresholded Speaker Class Model Selection or Model Adaptation” to Ittycheriah, et al. now issued as U.S. Pat. No. 5,895,447, U.S. patent application Ser. No. 08/788,471, filed Jan. 28, 1997 entitled “Text Independent Speaker Recognition for Transparent Command Ambiguity Resolution and Continuous Access Control” and U.S. patent application Ser. No. 08/787,029, filed Jan. 28, 1997 entitled “Speaker Model Prefetching” both to Stephane Maes, all three assigned to the assignee of the present invention and incorporated herein by reference.
Different user acoustic models are clustered into classes according to acoustic similarities of the users, thereby clustering the speakers based on vocal and verbal similarities. First, in step 122, acoustic profile data for individual users previously accumulated and stored in the local databases are passed over the network 100 to the server 106. The user acoustic data are compared in step 124 in the server 106. In step 126, based on that comparison, users are clustered into classes of similar users according to acoustic voice similarities. Then, in step 128, different acoustic models (i.e., different domains) are compared in sets associated with similar users, to derive cluster update data. Finally, in step 130, acoustic model components for similar users are modified relative to user production activities. As each individual acoustic model is changed, similar acoustic models from different user sets located elsewhere on the network also are modified in the server 106. At appropriate times, such as upon user request, modified acoustic models are transmitted from the server 106 to other sites on the network 100. So, acoustic model components, including data about users and information about user activities, are thereby synchronized in all similar acoustic models across the network.
FIG. 3 is a flowchart showing how one or more acoustic models are modified in step 130. The input changes to the model may be supervised input 134 or unsupervised input 132, the output of automatic speech recognition 136 or the result of user production activities 138 as described in detail hereinbelow with reference to FIG. 4. Further, the result of the user production activities 138 may be additional speech data (i.e., data collected from other speech related tasks such as speaker identification, speech recording, etc.) 140 or acoustic training data 142.
Acoustic training data 142 is generated at each initial use by a new user. Acoustic training data 142 includes, for example, acoustic prototypes, or Hidden Markov Models. Alternately, acoustic training data 142 may be employed for growing acoustic decision trees, each decision tree being based on the user's speech training data. Furthermore, acoustic training 142 may include estimating parameters that control the decoding process, estimating parameters that control signal processing and compiling a code book of user speech data. Parameters that control the decoding process may include, but are not limited to, weights of decoding score mixtures, decision thresholds that control decoding stacks, phone or word durations, decoding alternative list sizes, decoding tree sizes.
After receiving all acoustic input data, user acoustic model components are modified in step 144. Then, in step 146, acoustic prototypes are adapted for any additional or subsequent speech data produced by a user. In step 148, HMM parameters are adapted incrementally to additional user speech data. Finally, in step 150, new words are added to the acoustic vocabulary and new queries are added to acoustic decision trees. Additionally, adding new words in step 150 may entail modifying acoustic ranks, as well as adapting relative weights of language models and acoustic models.
FIG. 4 is an illustration of user actions and user production activities. User production activities may include activities such as dictation 160, conversation 162, error correction 164, generation of sounds 166 including noise and music. So, dictation 160, conversation 164 and background audio 166 are provided to automatic speech recognition module 168. The automatic speech recognition module 168 generates either text 170 or passes the recognition results to dialog module 172. Error correction 164 operates on the text 170, correcting any recognition errors, providing a supervised adaptation 174 of the input. The dialog module 172 generates system commands 176 and queries 178 in response to recognition results passed to it.
FIG. 5 is a flow diagram of the user clustering step 126 in FIG. 1. First, in step 180, the user's speaker characteristics, including but not limited to the user's educational level, age, gender, family relationship and nationality are gathered and provided as a user profile in step 182. Network data for all users including user profiles are compared to identify similar users in step 184. Independently, in step 186 an acoustic front end produces acoustic features, e.g., as the result of training. In step 188, corresponding acoustic features are identified in the speaker's voice. As noted above acoustic features may include, for example, accent, vocal tract characteristics, voice source characteristics, fundamental frequency, running average pitch, running pitch variance, pitch jitter, running energy variance, speech rate, shimmer, fundamental frequency, variation in fundamental frequency and MEL cepstra. Then, in step 192, acoustic features collected from various users are compared. Acoustic models from the same domain but from different sets or systems are compared in step 192. Common features are identified in step 194 and passed to step 184 to identify similar users. Similar users identified in step 184 are users that have one or more common characteristics or, one or more common acoustic features. In step 196, user clusters are identified to cluster users with one or several common features, with several similar acoustic components or with similar profile characteristics, thereby classifying such users in the same classes. Additionally, thereafter, user characteristics are recorded, collected and used for further user classification.
FIG. 6 is a flow chart of the acoustic component comparison step 192 of FIG. 5. In step 200 acoustic vocabularies and features are provided and represented as vectors in step 202. In step 204, the distance, preferably the Euclidean distance between vectors is calculated. Alternately the Kulback distance may be calculated. In step 206, the computed distances are compared against threshold values to identify similar models, similar models being defined as having calculated values that fall below the threshold values. Acoustic user vocabularies, acoustic features, acoustic user components, acoustic prototypes, Hidden Markov Models for words and phones and accent vectors are compared to determine similarities. Also, acoustic vocabularies of similar users may be analyzed to update the user acoustic vocabulary.
FIG. 7 is a flowchart illustrating supervised and unsupervised speech adaptation according to the preferred embodiment of the present invention, further illustrating aspects of the invention not shown in FIG. 4. Automatic speech recognition module 210 receives speech and, depending on the content of the user's speech, provides commands 212, queries 214 or uncorrected decoded textual data 216. Commands 212 and queries 214 are passed directly to one or more applications 218 and/or to the operating system. Commands 212 may direct an application operation, e.g., “open file . . . , ” “close,” “indent,” or, when passed to the operating system, may direct window navigation. Queries 214 are passed to appropriate applications 218, e.g., queries 214 are passed to a database manager for database searching.
Further, Commands 212 and queries 214 that are passed to applications 218 elicit a textual output. The textual output is passed to a supervisor 222 for approval. The text may be, for example, used for transmission as e-mail; a decoded document for storage after some period of time that signifies supervisor approval; a decoded document that was corrected; or, a newly decoded document. For supervised applications, uncorrected decoded textual data 216 is corrected in step 224 and passed to the supervisor 222 as corrected text 226. For unsupervised recognition, the decoded textual data 216 is provided directly for unsupervised adaptation 228. Accordingly, the decoded textual data 216 is either uncorrected textual data, unexecuted commands or unissued queries.
While the invention has been described in terms of preferred embodiments, those skilled in the art will recognize that the invention can be practiced with modification within the spirit and scope of the appended claims.

Claims (26)

We claim:
1. A speech recognition system for recognizing speech input from computer users connected together over a network of computers, a plurality of said computers each including at least one acoustic model trained for a particular user, said system comprising:
means for comparing acoustic models of one or more computer users, each of said computer users using one of a plurality of computers;
means for clustering users on a network of said plurality of computers into clusters of similar users responsive to said comparison of acoustic models;
means for modifying each of said acoustic models responsive to user production activities;
means for comparing identified similar acoustic models and, responsive to modification of one or more of said acoustic models, modifying one or more compared said identified similar acoustic models; and
means for transmitting acoustic model data over said network to other computers connected to said network.
2. A speech recognition system as in claim 1, wherein the means for comparing acoustic models further comprises:
means for identifying an acoustic model domain, similar acoustic models being clustered according to said identified domain.
3. A speech recognition system as in claim 2, wherein the means for identifying said acoustic model domain comprises means for identifying a domain selected from the group of domains consisting of a telephone speech domain, a speaker independent speech domain, a gender related speech domain, an age related speech domain, a broadcasting speech domain, a noise mixed with speech domain, a music mixed with speech domain, a discrete speech domain and a continuous speech domain.
4. A speech recognition system as in claim 2, further comprising:
means for converting speech input from a user into an acoustic model.
5. A speech recognition system as in claim 4, wherein the means for converting speech into an acoustic model being selected from the group consisting of:
means for converting speech into an acoustic prototype;
means for converting speech into a Hidden Markov Model (HMM) for words;
means for converting speech into a HMM for phones;
means for converting speech into an acoustic rank;
means for converting speech into an acoustic decision tree;
means for converting speech into a weighted mixture of decoding scores;
means for converting speech into a decoding stack threshold;
means for converting speech into a phone duration;
means for converting speech into a word duration;
means for converting speech into a decoding alternative list size; and
means for converting speech into a plurality of signal processing control parameters.
6. A speech recognition system as in claim 2, further comprising means for receiving user production activities, said means for receiving user production activities being capable of receiving activity selected from the group consisting of dictation, conversation, error correction, sound generation, noise generation and music generation.
7. A speech recognition system as in claim 6, further comprising means for identifying and issuing commands, queries and text from said received user production activities.
8. A speech recognition system as in claim 7, further comprising:
means for converting said commands and queries into textual data; and
means for providing said text and said converted textual data to a supervisor.
9. A speech recognition system as in claim 2, further comprising:
means for maintaining a plurality of user profiles; and
means for extracting acoustic features.
10. A speech recognition system as in claim 9, wherein the means for maintaining a plurality of user profiles is a server.
11. A speech recognition system as in claim 9, wherein the means for extracting acoustic features comprises:
means for extracting acoustic features selected from the group of features consisting of accent, vocal tract characteristics, voice source characteristics, fundamental frequency, running average pitch, running pitch variance, pitch jitter, running energy variance, speech rate, shimmer, fundamental frequency, variation in fundamental frequency and MEL cepstra.
12. A speech recognition system as in claim 1, wherein the means for comparing acoustic models comprises means for measuring the distance between acoustic model components, acoustic models having components separated by less than a threshold being identified as similar.
13. A speech recognition system as in claim 2, wherein the plurality of computers comprises:
at least one server;
at least one personal computer; and
at least one embedded device.
14. A speech recognition system as in claim 13, wherein at least one embedded device includes at least one personal digital assistant.
15. A speech recognition method for recognizing speech from each of a plurality of computer users, said method comprising the steps of:
a) clustering computer users coupled together over a network of connected computers into classes of similar users, at least one acoustic model being maintained on a corresponding one of said connected computers for each of said computer users;
b) for each of said classes, identifying similar acoustic models being used by clustered users;
c) modifying one user acoustic model responsive to user production activities by a corresponding clustered user;
d) comparing and adapting all said identified similar acoustic models responsive to modification of said one user acoustic model; and
e) transmitting user data over said network, said transmitted user data including information about user activities and user acoustic model data.
16. A speech recognition method as in claim 15, wherein each said acoustic model is directed to one of a plurality of speech domains, said plurality of speech domains comprising:
a telephone speech domain;
a speaker independent speech domain;
a gender related speech domain;
an age related speech domain;
a broadcasting speech domain;
a speech mixed with noise domain;
a speech mixed with music domain;
a discrete speech domain; and
a continuous speech domain.
17. A speech recognition method as in claim 15, wherein the step (a) of clustering users comprises comparing acoustic profile data for connected said users.
18. A speech recognition method as in claim 17 wherein said comparison is supervised, said users being classed into a plurality of established classes. identifying users having common speaker domains.
19. A speech recognition method as in claim 17 wherein said acoustic profile data includes user sex, age and nationality.
20. A speech recognition method as in claim 16, wherein the step (d) of comparing user acoustic models, similar users are identified as users having models with features falling within a specified threshold of each other.
21. A computer program product for recognizing speech from each of a plurality of computer users, said computer users using computers coupled together over a network, said computer program product comprising a computer usable medium having computer readable program code thereon, said computer readable program code comprising:
computer readable program code means for clustering computer users coupled together over a network of connected computers into classes of similar users, at least one acoustic model being maintained on a corresponding one of said connected computers for each of said computer users;
computer readable program code means for identifying similar acoustic models being used by clustered users for each of said classes;
computer readable program code means for modifying one user acoustic model responsive to user production activities by a corresponding clustered user;
computer readable program code means for comparing and adapting all said identified similar acoustic models responsive to modification of said one user acoustic model; and
computer readable program code means for transmitting user data over said network, said transmitted user data including information about user activities and user acoustic model data.
22. A computer program product as in claim 21, wherein each said acoustic model is directed to one of a plurality of speech domains, said plurality of speech domains comprising:
a telephone speech domain;
a speaker independent speech domain;
a gender related speech domain;
an age related speech domain;
a broadcasting speech domain;
a speech mixed with noise domain;
a speech mixed with music domain;
a discrete speech domain; and
a continuous speech domain.
23. A computer program product as in claim 21, wherein the computer readable code means for clustering users comprises computer readable code means for comparing acoustic profile data for connected said users.
24. A computer program product as in claim 23 wherein said comparison is supervised, said users being classed into a plurality of established classes, identifying users having common speaker domains.
25. A computer program product as in claim 23, wherein said acoustic profile data includes user sex, age and nationality.
26. A computer program product as in claim 22, wherein the computer readable code means for comparing individual user acoustic models, compares similar users having models with features falling within a specified threshold of each other.
US09/437,646 1999-11-10 1999-11-10 Speaker model adaptation via network of similar users Expired - Lifetime US6442519B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US09/437,646 US6442519B1 (en) 1999-11-10 1999-11-10 Speaker model adaptation via network of similar users

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US09/437,646 US6442519B1 (en) 1999-11-10 1999-11-10 Speaker model adaptation via network of similar users

Publications (1)

Publication Number Publication Date
US6442519B1 true US6442519B1 (en) 2002-08-27

Family

ID=23737300

Family Applications (1)

Application Number Title Priority Date Filing Date
US09/437,646 Expired - Lifetime US6442519B1 (en) 1999-11-10 1999-11-10 Speaker model adaptation via network of similar users

Country Status (1)

Country Link
US (1) US6442519B1 (en)

Cited By (124)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020038212A1 (en) * 2000-09-25 2002-03-28 Prosodie Telephony system with subtitles and/or translation
US20020059068A1 (en) * 2000-10-13 2002-05-16 At&T Corporation Systems and methods for automatic speech recognition
US20020128820A1 (en) * 2001-03-07 2002-09-12 Silke Goronzy Method for recognizing speech using eigenpronunciations
US20020133343A1 (en) * 2000-10-12 2002-09-19 Keiko Morii Method for speech recognition, apparatus for the same, and voice controller
US20020138274A1 (en) * 2001-03-26 2002-09-26 Sharma Sangita R. Server based adaption of acoustic models for client-based speech systems
US20020169618A1 (en) * 2001-03-07 2002-11-14 Siemens Aktiengesellschaft Providing help information in a speech dialog system
US20020188443A1 (en) * 2001-05-11 2002-12-12 Gopi Reddy System, method and computer program product for comprehensive playback using a vocal player
US20030036903A1 (en) * 2001-08-16 2003-02-20 Sony Corporation Retraining and updating speech models for speech recognition
US20030050783A1 (en) * 2001-09-13 2003-03-13 Shinichi Yoshizawa Terminal device, server device and speech recognition method
US20030078828A1 (en) * 2001-04-17 2003-04-24 International Business Machines Corporation Method for the promotion of recognition software products
US20030088397A1 (en) * 2001-11-03 2003-05-08 Karas D. Matthew Time ordered indexing of audio data
US20030093263A1 (en) * 2001-11-13 2003-05-15 Zheng Chen Method and apparatus for adapting a class entity dictionary used with language models
US20030171931A1 (en) * 2002-03-11 2003-09-11 Chang Eric I-Chao System for creating user-dependent recognition models and for making those models accessible by a user
US20030191639A1 (en) * 2002-04-05 2003-10-09 Sam Mazza Dynamic and adaptive selection of vocabulary and acoustic models based on a call context for speech recognition
US20040010409A1 (en) * 2002-04-01 2004-01-15 Hirohide Ushida Voice recognition system, device, voice recognition method and voice recognition program
US20040117180A1 (en) * 2002-12-16 2004-06-17 Nitendra Rajput Speaker adaptation of vocabulary for speech recognition
US20040138893A1 (en) * 2003-01-13 2004-07-15 Ran Mochary Adaptation of symbols
US20040148165A1 (en) * 2001-06-06 2004-07-29 Peter Beyerlein Pattern processing system specific to a user group
US20040162726A1 (en) * 2003-02-13 2004-08-19 Chang Hisao M. Bio-phonetic multi-phrase speaker identity verification
US20040215451A1 (en) * 2003-04-25 2004-10-28 Macleod John B. Telephone call handling center where operators utilize synthesized voices generated or modified to exhibit or omit prescribed speech characteristics
US20040230424A1 (en) * 2003-05-15 2004-11-18 Microsoft Corporation Adaptation of compressed acoustic models
US6823306B2 (en) * 2000-11-30 2004-11-23 Telesector Resources Group, Inc. Methods and apparatus for generating, updating and distributing speech recognition models
US20050038655A1 (en) * 2003-08-13 2005-02-17 Ambroise Mutel Bubble splitting for compact acoustic modeling
US20050043957A1 (en) * 2003-08-21 2005-02-24 Xiaofan Lin Selective sampling for sound signal classification
US20050123202A1 (en) * 2003-12-04 2005-06-09 Samsung Electronics Co., Ltd. Face recognition apparatus and method using PCA learning per subgroup
US20050137866A1 (en) * 2003-12-23 2005-06-23 International Business Machines Corporation Interactive speech recognition model
US20050182626A1 (en) * 2004-02-18 2005-08-18 Samsung Electronics Co., Ltd. Speaker clustering and adaptation method based on the HMM model variation information and its apparatus for speech recognition
US20050192730A1 (en) * 2004-02-29 2005-09-01 Ibm Corporation Driver safety manager
US20050216273A1 (en) * 2000-11-30 2005-09-29 Telesector Resources Group, Inc. Methods and apparatus for performing speech recognition over a network and using speech recognition results
US20050255431A1 (en) * 2004-05-17 2005-11-17 Aurilab, Llc Interactive language learning system and method
US20050267760A1 (en) * 2000-09-22 2005-12-01 Meyer John D System and user interface for producing acoustic response predictions via a communications network
US20060004570A1 (en) * 2004-06-30 2006-01-05 Microsoft Corporation Transcribing speech data with dialog context and/or recognition alternative information
US20060020463A1 (en) * 2004-07-22 2006-01-26 International Business Machines Corporation Method and system for identifying and correcting accent-induced speech recognition difficulties
US20060053014A1 (en) * 2002-11-21 2006-03-09 Shinichi Yoshizawa Standard model creating device and standard model creating method
US20060074676A1 (en) * 2004-09-17 2006-04-06 Microsoft Corporation Quantitative model for formant dynamics and contextually assimilated reduction in fluent speech
US20060089834A1 (en) * 2004-10-22 2006-04-27 Microsoft Corporation Verb error recovery in speech recognition
US20060100862A1 (en) * 2004-11-05 2006-05-11 Microsoft Corporation Acoustic models with structured hidden dynamics with integration over many possible hidden trajectories
US20060229876A1 (en) * 2005-04-07 2006-10-12 International Business Machines Corporation Method, apparatus and computer program providing a multi-speaker database for concatenative text-to-speech synthesis
US20060229875A1 (en) * 2005-03-30 2006-10-12 Microsoft Corporation Speaker adaptive learning of resonance targets in a hidden trajectory model of speech coarticulation
US20060235684A1 (en) * 2005-04-14 2006-10-19 Sbc Knowledge Ventures, Lp Wireless device to access network-based voice-activated services using distributed speech recognition
US20060235698A1 (en) * 2005-04-13 2006-10-19 Cane David A Apparatus for controlling a home theater system by speech commands
US20070038459A1 (en) * 2005-08-09 2007-02-15 Nianjun Zhou Method and system for creation of voice training profiles with multiple methods with uniform server mechanism using heterogeneous devices
US20070061142A1 (en) * 2005-09-15 2007-03-15 Sony Computer Entertainment Inc. Audio, video, simulation, and user interface paradigms
US7206303B2 (en) 2001-11-03 2007-04-17 Autonomy Systems Limited Time ordered indexing of an information stream
US20070143107A1 (en) * 2005-12-19 2007-06-21 International Business Machines Corporation Remote tracing and debugging of automatic speech recognition servers by speech reconstruction from cepstra and pitch information
US20070192109A1 (en) * 2006-02-14 2007-08-16 Ivc Inc. Voice command interface device
US20070198263A1 (en) * 2006-02-21 2007-08-23 Sony Computer Entertainment Inc. Voice recognition with speaker adaptation and registration with pitch
US20070198261A1 (en) * 2006-02-21 2007-08-23 Sony Computer Entertainment Inc. Voice recognition with parallel gender and age normalization
US20080004876A1 (en) * 2006-06-30 2008-01-03 Chuang He Non-enrolled continuous dictation
US20080027706A1 (en) * 2006-07-27 2008-01-31 Microsoft Corporation Lightweight windowing method for screening harvested data for novelty
US20080103781A1 (en) * 2006-10-28 2008-05-01 General Motors Corporation Automatically adapting user guidance in automated speech recognition
US20080103771A1 (en) * 2004-11-08 2008-05-01 France Telecom Method for the Distributed Construction of a Voice Recognition Model, and Device, Server and Computer Programs Used to Implement Same
US20080133235A1 (en) * 2006-12-01 2008-06-05 Simoneau Laurent Method to train the language model of a speech recognition system to convert and index voicemails on a search engine
US7472062B2 (en) * 2002-01-04 2008-12-30 International Business Machines Corporation Efficient recursive clustering based on a splitting function derived from successive eigen-decompositions
US20090024390A1 (en) * 2007-05-04 2009-01-22 Nuance Communications, Inc. Multi-Class Constrained Maximum Likelihood Linear Regression
GB2451907A (en) * 2007-08-17 2009-02-18 Fluency Voice Technology Ltd Device for modifying and improving the behavior of speech recognition systems
US20090106028A1 (en) * 2007-10-18 2009-04-23 International Business Machines Corporation Automated tuning of speech recognition parameters
US7552098B1 (en) * 2005-12-30 2009-06-23 At&T Corporation Methods to distribute multi-class classification learning on several processors
US20090204390A1 (en) * 2006-06-29 2009-08-13 Nec Corporation Speech processing apparatus and program, and speech processing method
US7599861B2 (en) 2006-03-02 2009-10-06 Convergys Customer Management Group, Inc. System and method for closed loop decisionmaking in an automated care system
US20100070278A1 (en) * 2008-09-12 2010-03-18 Andreas Hagen Method for Creating a Speech Model
US20100125459A1 (en) * 2008-11-18 2010-05-20 Nuance Communications, Inc. Stochastic phoneme and accent generation using accent class
US20100138040A1 (en) * 2007-01-18 2010-06-03 Korea Institute Of Science And Technology Apparatus for detecting user and method for detecting user by the same
US20100169093A1 (en) * 2008-12-26 2010-07-01 Fujitsu Limited Information processing apparatus, method and recording medium for generating acoustic model
US20100211387A1 (en) * 2009-02-17 2010-08-19 Sony Computer Entertainment Inc. Speech processing with source location estimation using signals from two or more microphones
US20100211376A1 (en) * 2009-02-17 2010-08-19 Sony Computer Entertainment Inc. Multiple language voice recognition
US20100211391A1 (en) * 2009-02-17 2010-08-19 Sony Computer Entertainment Inc. Automatic computation streaming partition for voice recognition on multiple processors with limited memory
US7809663B1 (en) 2006-05-22 2010-10-05 Convergys Cmg Utah, Inc. System and method for supporting the utilization of machine language
US20100268538A1 (en) * 2009-04-20 2010-10-21 Samsung Electronics Co., Ltd. Electronic apparatus and voice recognition method for the same
US7835910B1 (en) * 2003-05-29 2010-11-16 At&T Intellectual Property Ii, L.P. Exploiting unlabeled utterances for spoken language understanding
US7920682B2 (en) * 2001-08-21 2011-04-05 Byrne William J Dynamic interactive voice interface
US20110087488A1 (en) * 2009-03-25 2011-04-14 Kabushiki Kaisha Toshiba Speech synthesis apparatus and method
US20110218804A1 (en) * 2010-03-02 2011-09-08 Kabushiki Kaisha Toshiba Speech processor, a speech processing method and a method of training a speech processor
US20110276325A1 (en) * 2010-05-05 2011-11-10 Cisco Technology, Inc. Training A Transcription System
US20110295603A1 (en) * 2010-04-28 2011-12-01 Meisel William S Speech recognition accuracy improvement through speaker categories
US20120109646A1 (en) * 2010-11-02 2012-05-03 Samsung Electronics Co., Ltd. Speaker adaptation method and apparatus
US20120130709A1 (en) * 2010-11-23 2012-05-24 At&T Intellectual Property I, L.P. System and method for building and evaluating automatic speech recognition via an application programmer interface
US8260619B1 (en) 2008-08-22 2012-09-04 Convergys Cmg Utah, Inc. Method and system for creating natural language understanding grammars
US8379830B1 (en) 2006-05-22 2013-02-19 Convergys Customer Management Delaware Llc System and method for automated customer service with contingent live interaction
US8380502B1 (en) 2001-02-07 2013-02-19 Google Inc. Voice interface for a search engine
US8401846B1 (en) 2000-11-30 2013-03-19 Google Inc. Performing speech recognition over a network and using speech recognition results
US20130090921A1 (en) * 2011-10-07 2013-04-11 Microsoft Corporation Pronunciation learning from user correction
US20130268270A1 (en) * 2012-04-05 2013-10-10 Nuance Communications, Inc. Forced/Predictable Adaptation for Speech Recognition
US8769009B2 (en) 2011-02-18 2014-07-01 International Business Machines Corporation Virtual communication techniques
US20140214420A1 (en) * 2013-01-25 2014-07-31 Microsoft Corporation Feature space transformation for personalization using generalized i-vector clustering
US8825533B2 (en) 2012-02-01 2014-09-02 International Business Machines Corporation Intelligent dialogue amongst competitive user applications
US20140358541A1 (en) * 2013-05-31 2014-12-04 Nuance Communications, Inc. Method and Apparatus for Automatic Speaker-Based Speech Clustering
US8954325B1 (en) * 2004-03-22 2015-02-10 Rockstar Consortium Us Lp Speech recognition in automated information services systems
US20150046163A1 (en) * 2010-10-27 2015-02-12 Microsoft Corporation Leveraging interaction context to improve recognition confidence scores
US20150067822A1 (en) * 2013-09-05 2015-03-05 Barclays Bank Plc Biometric Verification Using Predicted Signatures
US8983836B2 (en) * 2012-09-26 2015-03-17 International Business Machines Corporation Captioning using socially derived acoustic profiles
US9043208B2 (en) 2012-07-18 2015-05-26 International Business Machines Corporation System, method and program product for providing automatic speech recognition (ASR) in a shared resource environment
US9123338B1 (en) * 2012-06-01 2015-09-01 Google Inc. Background audio identification for speech disambiguation
US9208156B2 (en) 2011-12-06 2015-12-08 Honeywell International Inc. Acquiring statistical access models
US20150371191A1 (en) * 2014-06-20 2015-12-24 Hirevue, Inc. Model-driven evaluator bias detection
US9275638B2 (en) 2013-03-12 2016-03-01 Google Technology Holdings LLC Method and apparatus for training a voice recognition model database
US20160098992A1 (en) * 2014-10-01 2016-04-07 XBrain, Inc. Voice and Connection Platform
US9460716B1 (en) * 2012-09-11 2016-10-04 Google Inc. Using social networks to improve acoustic models
US9472186B1 (en) * 2014-01-28 2016-10-18 Nvoq Incorporated Automated training of a user audio profile using transcribed medical record recordings
US9478216B2 (en) 2009-12-08 2016-10-25 Nuance Communications, Inc. Guest speaker robust adapted speech recognition
TWI559216B (en) * 2015-04-17 2016-11-21 惠普發展公司有限責任合夥企業 Adjusting speaker settings
US20160372115A1 (en) * 2015-06-17 2016-12-22 Volkswagen Ag Speech recognition system and method for operating a speech recognition system with a mobile unit and an external server
US20160379630A1 (en) * 2015-06-25 2016-12-29 Intel Corporation Speech recognition services
US20170053645A1 (en) * 2015-08-22 2017-02-23 Toyota Motor Engineering & Manufacturing North America, Inc. Speech recognition system with abbreviated training
CN106663096A (en) * 2014-07-22 2017-05-10 纽昂斯通讯公司 Systems and methods for speech-based searching of content repositories
US20170154640A1 (en) * 2015-11-26 2017-06-01 Le Holdings (Beijing) Co., Ltd. Method and electronic device for voice recognition based on dynamic voice model selection
US9691377B2 (en) 2013-07-23 2017-06-27 Google Technology Holdings LLC Method and device for voice recognition training
US9747895B1 (en) * 2012-07-10 2017-08-29 Google Inc. Building language models for a user in a social network from linguistic information
US9786279B2 (en) 2012-09-10 2017-10-10 Google Inc. Answering questions using environmental context
US9786296B2 (en) 2013-07-08 2017-10-10 Qualcomm Incorporated Method and apparatus for assigning keyword model to voice operated function
US9786281B1 (en) * 2012-08-02 2017-10-10 Amazon Technologies, Inc. Household agent learning
US9997157B2 (en) 2014-05-16 2018-06-12 Microsoft Technology Licensing, Llc Knowledge source personalization to improve language models
WO2018208191A1 (en) * 2017-05-08 2018-11-15 Telefonaktiebolaget Lm Ericsson (Publ) Asr training and adaptation
US10163438B2 (en) 2013-07-31 2018-12-25 Google Technology Holdings LLC Method and apparatus for evaluating trigger phrase enrollment
US10720149B2 (en) * 2018-10-23 2020-07-21 Capital One Services, Llc Dynamic vocabulary customization in automated voice systems
US10785171B2 (en) 2019-02-07 2020-09-22 Capital One Services, Llc Chat bot utilizing metaphors to both relay and obtain information
CN111739517A (en) * 2020-07-01 2020-10-02 腾讯科技(深圳)有限公司 Speech recognition method, speech recognition device, computer equipment and medium
US20210272552A1 (en) * 2014-05-12 2021-09-02 Soundhound, Inc. Deriving acoustic features and linguistic features from received speech audio
US20220116493A1 (en) * 2019-03-21 2022-04-14 Capital One Services, Llc Methods and systems for automatic discovery of fraudulent calls using speaker recognition
US11341962B2 (en) 2010-05-13 2022-05-24 Poltorak Technologies Llc Electronic personal interactive device
US11340925B2 (en) 2017-05-18 2022-05-24 Peloton Interactive Inc. Action recipes for a crowdsourced digital assistant system
US11520610B2 (en) * 2017-05-18 2022-12-06 Peloton Interactive Inc. Crowdsourced on-boarding of digital assistant operations
US11682380B2 (en) 2017-05-18 2023-06-20 Peloton Interactive Inc. Systems and methods for crowdsourced actions and commands
US11862156B2 (en) 2017-05-18 2024-01-02 Peloton Interactive, Inc. Talk back from actions in applications

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5664058A (en) * 1993-05-12 1997-09-02 Nynex Science & Technology Method of training a speaker-dependent speech recognizer with automated supervision of training sufficiency
US5864807A (en) * 1997-02-25 1999-01-26 Motorola, Inc. Method and apparatus for training a speaker recognition system
US5895447A (en) * 1996-02-02 1999-04-20 International Business Machines Corporation Speech recognition using thresholded speaker class model selection or model adaptation
US5897616A (en) * 1997-06-11 1999-04-27 International Business Machines Corporation Apparatus and methods for speaker verification/identification/classification employing non-acoustic and/or acoustic models and databases
US5950157A (en) * 1997-02-28 1999-09-07 Sri International Method for establishing handset-dependent normalizing models for speaker recognition
US6088669A (en) * 1997-01-28 2000-07-11 International Business Machines, Corporation Speech recognition with attempted speaker recognition for speaker model prefetching or alternative speech modeling
US6141641A (en) * 1998-04-15 2000-10-31 Microsoft Corporation Dynamically configurable acoustic model for speech recognition system
US6163769A (en) * 1997-10-02 2000-12-19 Microsoft Corporation Text-to-speech using clustered context-dependent phoneme-based units
US6182037B1 (en) * 1997-05-06 2001-01-30 International Business Machines Corporation Speaker recognition over large population with fast and detailed matches
US6182038B1 (en) * 1997-12-01 2001-01-30 Motorola, Inc. Context dependent phoneme networks for encoding speech information
US6253179B1 (en) * 1999-01-29 2001-06-26 International Business Machines Corporation Method and apparatus for multi-environment speaker verification
US6327568B1 (en) * 1997-11-14 2001-12-04 U.S. Philips Corporation Distributed hardware sharing for speech processing
US6363348B1 (en) * 1997-10-20 2002-03-26 U.S. Philips Corporation User model-improvement-data-driven selection and update of user-oriented recognition model of a given type for word recognition at network server

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5664058A (en) * 1993-05-12 1997-09-02 Nynex Science & Technology Method of training a speaker-dependent speech recognizer with automated supervision of training sufficiency
US5895447A (en) * 1996-02-02 1999-04-20 International Business Machines Corporation Speech recognition using thresholded speaker class model selection or model adaptation
US6088669A (en) * 1997-01-28 2000-07-11 International Business Machines, Corporation Speech recognition with attempted speaker recognition for speaker model prefetching or alternative speech modeling
US5864807A (en) * 1997-02-25 1999-01-26 Motorola, Inc. Method and apparatus for training a speaker recognition system
US5950157A (en) * 1997-02-28 1999-09-07 Sri International Method for establishing handset-dependent normalizing models for speaker recognition
US6182037B1 (en) * 1997-05-06 2001-01-30 International Business Machines Corporation Speaker recognition over large population with fast and detailed matches
US5897616A (en) * 1997-06-11 1999-04-27 International Business Machines Corporation Apparatus and methods for speaker verification/identification/classification employing non-acoustic and/or acoustic models and databases
US6163769A (en) * 1997-10-02 2000-12-19 Microsoft Corporation Text-to-speech using clustered context-dependent phoneme-based units
US6363348B1 (en) * 1997-10-20 2002-03-26 U.S. Philips Corporation User model-improvement-data-driven selection and update of user-oriented recognition model of a given type for word recognition at network server
US6327568B1 (en) * 1997-11-14 2001-12-04 U.S. Philips Corporation Distributed hardware sharing for speech processing
US6182038B1 (en) * 1997-12-01 2001-01-30 Motorola, Inc. Context dependent phoneme networks for encoding speech information
US6141641A (en) * 1998-04-15 2000-10-31 Microsoft Corporation Dynamically configurable acoustic model for speech recognition system
US6253179B1 (en) * 1999-01-29 2001-06-26 International Business Machines Corporation Method and apparatus for multi-environment speaker verification

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
Chin-Hui Lee and J.L. Gauvain, Bayesian Adaptive Learning and Map Estimation of HMM, Automatic Speech and Speaker Recognition, 1996 Kluwer Academic Publishers, Boston, pp. 83-105.
D. Matrouf, M. Adda-Decker, L. Lamel, and J. Gauvain, Language Identification Incorporating Lexical Information, Proceedings of the 1998 International Conference on Spoken Language Processing, ICSLP '98, Sydney, Australia, Dec. 1998, pp. 181-184.
Frederick Jelinek, Statistical Methods for Speech Recognition, , The MIT Press, Cambridge, Jan. 1999, pp. 165-171.
Jerome R. Bellegarda, Context-Dependent Vector Clustering for Speech Recognition, Automatic Speech and Speaker Recognition, Kluwer Academic Publishers, Boston, pp. 133-153.
L.R. Bahl, P.V. de Souza, P.S. Gopalakrishnan, D. Nahamoo, M. Picheny, Decision Trees for Phonological Rules in Continuous Speech, Proceeding of the International Conference on Acoustics, Speech, and Signal Processing, Toronto, Canada, May 1991.
M.J.F. Gales and P.C. Woodland, Means and variance adaptation within the MLLR framework, Computer Speech and Language (1996) 10, 249-264.

Cited By (251)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7069219B2 (en) * 2000-09-22 2006-06-27 Meyer Sound Laboratories Incorporated System and user interface for producing acoustic response predictions via a communications network
US20050267760A1 (en) * 2000-09-22 2005-12-01 Meyer John D System and user interface for producing acoustic response predictions via a communications network
US20020038212A1 (en) * 2000-09-25 2002-03-28 Prosodie Telephony system with subtitles and/or translation
US20020133343A1 (en) * 2000-10-12 2002-09-19 Keiko Morii Method for speech recognition, apparatus for the same, and voice controller
US7003465B2 (en) * 2000-10-12 2006-02-21 Matsushita Electric Industrial Co., Ltd. Method for speech recognition, apparatus for the same, and voice controller
US20090063144A1 (en) * 2000-10-13 2009-03-05 At&T Corp. System and method for providing a compensated speech recognition model for speech recognition
US20020059068A1 (en) * 2000-10-13 2002-05-16 At&T Corporation Systems and methods for automatic speech recognition
US7996220B2 (en) 2000-10-13 2011-08-09 At&T Intellectual Property Ii, L.P. System and method for providing a compensated speech recognition model for speech recognition
US7451085B2 (en) * 2000-10-13 2008-11-11 At&T Intellectual Property Ii, L.P. System and method for providing a compensated speech recognition model for speech recognition
US20050216273A1 (en) * 2000-11-30 2005-09-29 Telesector Resources Group, Inc. Methods and apparatus for performing speech recognition over a network and using speech recognition results
US8731937B1 (en) 2000-11-30 2014-05-20 Google Inc. Updating speech recognition models for contacts
US8335687B1 (en) 2000-11-30 2012-12-18 Google Inc. Performing speech recognition over a network and using speech recognition results
US8401846B1 (en) 2000-11-30 2013-03-19 Google Inc. Performing speech recognition over a network and using speech recognition results
US8520810B1 (en) 2000-11-30 2013-08-27 Google Inc. Performing speech recognition over a network and using speech recognition results
US8447599B2 (en) * 2000-11-30 2013-05-21 Google Inc. Methods and apparatus for generating, updating and distributing speech recognition models
US8818809B2 (en) 2000-11-30 2014-08-26 Google Inc. Methods and apparatus for generating, updating and distributing speech recognition models
US20120101812A1 (en) * 2000-11-30 2012-04-26 Google Inc. Methods and apparatus for generating, updating and distributing speech recognition models
US8494848B2 (en) 2000-11-30 2013-07-23 Google Inc. Methods and apparatus for generating, updating and distributing speech recognition models
US9380155B1 (en) 2000-11-30 2016-06-28 Google Inc. Forming speech recognition over a network and using speech recognition results based on determining that a network connection exists
US9787830B1 (en) 2000-11-30 2017-10-10 Google Inc. Performing speech recognition over a network and using speech recognition results based on determining that a network connection exists
US9818399B1 (en) 2000-11-30 2017-11-14 Google Inc. Performing speech recognition over a network and using speech recognition results based on determining that a network connection exists
US6823306B2 (en) * 2000-11-30 2004-11-23 Telesector Resources Group, Inc. Methods and apparatus for generating, updating and distributing speech recognition models
US8682663B2 (en) 2000-11-30 2014-03-25 Google Inc. Performing speech recognition over a network and using speech recognition results based on determining that a network connection exists
US7302391B2 (en) 2000-11-30 2007-11-27 Telesector Resources Group, Inc. Methods and apparatus for performing speech recognition over a network and using speech recognition results
US20050049854A1 (en) * 2000-11-30 2005-03-03 Craig Reding Methods and apparatus for generating, updating and distributing speech recognition models
US8515752B1 (en) * 2001-02-07 2013-08-20 Google Inc. Voice interface for a search engine
US8768700B1 (en) 2001-02-07 2014-07-01 Google Inc. Voice search engine interface for scoring search hypotheses
US8380502B1 (en) 2001-02-07 2013-02-19 Google Inc. Voice interface for a search engine
US7113908B2 (en) * 2001-03-07 2006-09-26 Sony Deutschland Gmbh Method for recognizing speech using eigenpronunciations
US20020128820A1 (en) * 2001-03-07 2002-09-12 Silke Goronzy Method for recognizing speech using eigenpronunciations
US20020169618A1 (en) * 2001-03-07 2002-11-14 Siemens Aktiengesellschaft Providing help information in a speech dialog system
US20020138274A1 (en) * 2001-03-26 2002-09-26 Sharma Sangita R. Server based adaption of acoustic models for client-based speech systems
US7200565B2 (en) * 2001-04-17 2007-04-03 International Business Machines Corporation System and method for promoting the use of a selected software product having an adaptation module
US20030078828A1 (en) * 2001-04-17 2003-04-24 International Business Machines Corporation Method for the promotion of recognition software products
US20020188443A1 (en) * 2001-05-11 2002-12-12 Gopi Reddy System, method and computer program product for comprehensive playback using a vocal player
US9009043B2 (en) * 2001-06-06 2015-04-14 Nuance Communications, Inc. Pattern processing system specific to a user group
US9424838B2 (en) * 2001-06-06 2016-08-23 Nuance Communications, Inc. Pattern processing system specific to a user group
US20040148165A1 (en) * 2001-06-06 2004-07-29 Peter Beyerlein Pattern processing system specific to a user group
US20120310647A1 (en) * 2001-06-06 2012-12-06 Nuance Communications, Inc. Pattern processing system specific to a user group
US20150179164A1 (en) * 2001-06-06 2015-06-25 Nuance Communications, Inc. Pattern processing system specific to a user group
US6941264B2 (en) * 2001-08-16 2005-09-06 Sony Electronics Inc. Retraining and updating speech models for speech recognition
US20030036903A1 (en) * 2001-08-16 2003-02-20 Sony Corporation Retraining and updating speech models for speech recognition
US9729690B2 (en) * 2001-08-21 2017-08-08 Ben Franklin Patent Holding Llc Dynamic interactive voice interface
US7920682B2 (en) * 2001-08-21 2011-04-05 Byrne William J Dynamic interactive voice interface
US20110246203A1 (en) * 2001-08-21 2011-10-06 Ben Franklin Patent Holding Llc Dynamic Interactive Voice Interface
US20030050783A1 (en) * 2001-09-13 2003-03-13 Shinichi Yoshizawa Terminal device, server device and speech recognition method
US20070198259A1 (en) * 2001-11-03 2007-08-23 Karas D M Time ordered indexing of an information stream
US20030088397A1 (en) * 2001-11-03 2003-05-08 Karas D. Matthew Time ordered indexing of audio data
US8972840B2 (en) 2001-11-03 2015-03-03 Longsand Limited Time ordered indexing of an information stream
US7206303B2 (en) 2001-11-03 2007-04-17 Autonomy Systems Limited Time ordered indexing of an information stream
US7292979B2 (en) * 2001-11-03 2007-11-06 Autonomy Systems, Limited Time ordered indexing of audio data
US7124080B2 (en) * 2001-11-13 2006-10-17 Microsoft Corporation Method and apparatus for adapting a class entity dictionary used with language models
US20030093263A1 (en) * 2001-11-13 2003-05-15 Zheng Chen Method and apparatus for adapting a class entity dictionary used with language models
US7472062B2 (en) * 2002-01-04 2008-12-30 International Business Machines Corporation Efficient recursive clustering based on a splitting function derived from successive eigen-decompositions
US20030171931A1 (en) * 2002-03-11 2003-09-11 Chang Eric I-Chao System for creating user-dependent recognition models and for making those models accessible by a user
US20040010409A1 (en) * 2002-04-01 2004-01-15 Hirohide Ushida Voice recognition system, device, voice recognition method and voice recognition program
US20030191639A1 (en) * 2002-04-05 2003-10-09 Sam Mazza Dynamic and adaptive selection of vocabulary and acoustic models based on a call context for speech recognition
US20060053014A1 (en) * 2002-11-21 2006-03-09 Shinichi Yoshizawa Standard model creating device and standard model creating method
US7603276B2 (en) * 2002-11-21 2009-10-13 Panasonic Corporation Standard-model generation for speech recognition using a reference model
US20090271201A1 (en) * 2002-11-21 2009-10-29 Shinichi Yoshizawa Standard-model generation for speech recognition using a reference model
US7389228B2 (en) * 2002-12-16 2008-06-17 International Business Machines Corporation Speaker adaptation of vocabulary for speech recognition
US8731928B2 (en) 2002-12-16 2014-05-20 Nuance Communications, Inc. Speaker adaptation of vocabulary for speech recognition
US8046224B2 (en) 2002-12-16 2011-10-25 Nuance Communications, Inc. Speaker adaptation of vocabulary for speech recognition
US8417527B2 (en) 2002-12-16 2013-04-09 Nuance Communications, Inc. Speaker adaptation of vocabulary for speech recognition
US20040117180A1 (en) * 2002-12-16 2004-06-17 Nitendra Rajput Speaker adaptation of vocabulary for speech recognition
US20080215326A1 (en) * 2002-12-16 2008-09-04 International Business Machines Corporation Speaker adaptation of vocabulary for speech recognition
US20040138893A1 (en) * 2003-01-13 2004-07-15 Ran Mochary Adaptation of symbols
US7676366B2 (en) * 2003-01-13 2010-03-09 Art Advanced Recognition Technologies Inc. Adaptation of symbols
US9524719B2 (en) 2003-02-13 2016-12-20 At&T Intellectual Property I, L.P. Bio-phonetic multi-phrase speaker identity verification
US9236051B2 (en) 2003-02-13 2016-01-12 At&T Intellectual Property I, L.P. Bio-phonetic multi-phrase speaker identity verification
US20070198264A1 (en) * 2003-02-13 2007-08-23 Chang Hisao M Bio-phonetic multi-phrase speaker identity verification
US7567901B2 (en) 2003-02-13 2009-07-28 At&T Intellectual Property 1, L.P. Bio-phonetic multi-phrase speaker identity verification
US7222072B2 (en) 2003-02-13 2007-05-22 Sbc Properties, L.P. Bio-phonetic multi-phrase speaker identity verification
US20040162726A1 (en) * 2003-02-13 2004-08-19 Chang Hisao M. Bio-phonetic multi-phrase speaker identity verification
US7275032B2 (en) * 2003-04-25 2007-09-25 Bvoice Corporation Telephone call handling center where operators utilize synthesized voices generated or modified to exhibit or omit prescribed speech characteristics
US20040215451A1 (en) * 2003-04-25 2004-10-28 Macleod John B. Telephone call handling center where operators utilize synthesized voices generated or modified to exhibit or omit prescribed speech characteristics
US7499857B2 (en) * 2003-05-15 2009-03-03 Microsoft Corporation Adaptation of compressed acoustic models
US20040230424A1 (en) * 2003-05-15 2004-11-18 Microsoft Corporation Adaptation of compressed acoustic models
US7835910B1 (en) * 2003-05-29 2010-11-16 At&T Intellectual Property Ii, L.P. Exploiting unlabeled utterances for spoken language understanding
US7328154B2 (en) * 2003-08-13 2008-02-05 Matsushita Electrical Industrial Co., Ltd. Bubble splitting for compact acoustic modeling
US20050038655A1 (en) * 2003-08-13 2005-02-17 Ambroise Mutel Bubble splitting for compact acoustic modeling
US20050043957A1 (en) * 2003-08-21 2005-02-24 Xiaofan Lin Selective sampling for sound signal classification
US7340398B2 (en) * 2003-08-21 2008-03-04 Hewlett-Packard Development Company, L.P. Selective sampling for sound signal classification
US20050123202A1 (en) * 2003-12-04 2005-06-09 Samsung Electronics Co., Ltd. Face recognition apparatus and method using PCA learning per subgroup
US7734087B2 (en) * 2003-12-04 2010-06-08 Samsung Electronics Co., Ltd. Face recognition apparatus and method using PCA learning per subgroup
US8160876B2 (en) 2003-12-23 2012-04-17 Nuance Communications, Inc. Interactive speech recognition model
US8463608B2 (en) 2003-12-23 2013-06-11 Nuance Communications, Inc. Interactive speech recognition model
US20050137866A1 (en) * 2003-12-23 2005-06-23 International Business Machines Corporation Interactive speech recognition model
US20050182626A1 (en) * 2004-02-18 2005-08-18 Samsung Electronics Co., Ltd. Speaker clustering and adaptation method based on the HMM model variation information and its apparatus for speech recognition
US7590537B2 (en) * 2004-02-18 2009-09-15 Samsung Electronics Co., Ltd. Speaker clustering and adaptation method based on the HMM model variation information and its apparatus for speech recognition
US20050192730A1 (en) * 2004-02-29 2005-09-01 Ibm Corporation Driver safety manager
US7349782B2 (en) 2004-02-29 2008-03-25 International Business Machines Corporation Driver safety manager
US8954325B1 (en) * 2004-03-22 2015-02-10 Rockstar Consortium Us Lp Speech recognition in automated information services systems
US20050255431A1 (en) * 2004-05-17 2005-11-17 Aurilab, Llc Interactive language learning system and method
US20060004570A1 (en) * 2004-06-30 2006-01-05 Microsoft Corporation Transcribing speech data with dialog context and/or recognition alternative information
US8036893B2 (en) 2004-07-22 2011-10-11 Nuance Communications, Inc. Method and system for identifying and correcting accent-induced speech recognition difficulties
US8285546B2 (en) 2004-07-22 2012-10-09 Nuance Communications, Inc. Method and system for identifying and correcting accent-induced speech recognition difficulties
US20060020463A1 (en) * 2004-07-22 2006-01-26 International Business Machines Corporation Method and system for identifying and correcting accent-induced speech recognition difficulties
US7565292B2 (en) 2004-09-17 2009-07-21 Micriosoft Corporation Quantitative model for formant dynamics and contextually assimilated reduction in fluent speech
US20060074676A1 (en) * 2004-09-17 2006-04-06 Microsoft Corporation Quantitative model for formant dynamics and contextually assimilated reduction in fluent speech
US20060089834A1 (en) * 2004-10-22 2006-04-27 Microsoft Corporation Verb error recovery in speech recognition
US8725505B2 (en) * 2004-10-22 2014-05-13 Microsoft Corporation Verb error recovery in speech recognition
US20060100862A1 (en) * 2004-11-05 2006-05-11 Microsoft Corporation Acoustic models with structured hidden dynamics with integration over many possible hidden trajectories
US7565284B2 (en) 2004-11-05 2009-07-21 Microsoft Corporation Acoustic models with structured hidden dynamics with integration over many possible hidden trajectories
US20080103771A1 (en) * 2004-11-08 2008-05-01 France Telecom Method for the Distributed Construction of a Voice Recognition Model, and Device, Server and Computer Programs Used to Implement Same
US7519531B2 (en) * 2005-03-30 2009-04-14 Microsoft Corporation Speaker adaptive learning of resonance targets in a hidden trajectory model of speech coarticulation
US20060229875A1 (en) * 2005-03-30 2006-10-12 Microsoft Corporation Speaker adaptive learning of resonance targets in a hidden trajectory model of speech coarticulation
US20060229876A1 (en) * 2005-04-07 2006-10-12 International Business Machines Corporation Method, apparatus and computer program providing a multi-speaker database for concatenative text-to-speech synthesis
US7716052B2 (en) * 2005-04-07 2010-05-11 Nuance Communications, Inc. Method, apparatus and computer program providing a multi-speaker database for concatenative text-to-speech synthesis
US20060235698A1 (en) * 2005-04-13 2006-10-19 Cane David A Apparatus for controlling a home theater system by speech commands
US20060235684A1 (en) * 2005-04-14 2006-10-19 Sbc Knowledge Ventures, Lp Wireless device to access network-based voice-activated services using distributed speech recognition
US7440894B2 (en) * 2005-08-09 2008-10-21 International Business Machines Corporation Method and system for creation of voice training profiles with multiple methods with uniform server mechanism using heterogeneous devices
US8239198B2 (en) 2005-08-09 2012-08-07 Nuance Communications, Inc. Method and system for creation of voice training profiles with multiple methods with uniform server mechanism using heterogeneous devices
US20090043582A1 (en) * 2005-08-09 2009-02-12 International Business Machines Corporation Method and system for creation of voice training profiles with multiple methods with uniform server mechanism using heterogeneous devices
US20070038459A1 (en) * 2005-08-09 2007-02-15 Nianjun Zhou Method and system for creation of voice training profiles with multiple methods with uniform server mechanism using heterogeneous devices
US10376785B2 (en) 2005-09-15 2019-08-13 Sony Interactive Entertainment Inc. Audio, video, simulation, and user interface paradigms
US8825482B2 (en) * 2005-09-15 2014-09-02 Sony Computer Entertainment Inc. Audio, video, simulation, and user interface paradigms
US9405363B2 (en) 2005-09-15 2016-08-02 Sony Interactive Entertainment Inc. (Siei) Audio, video, simulation, and user interface paradigms
US20070061142A1 (en) * 2005-09-15 2007-03-15 Sony Computer Entertainment Inc. Audio, video, simulation, and user interface paradigms
US7783488B2 (en) * 2005-12-19 2010-08-24 Nuance Communications, Inc. Remote tracing and debugging of automatic speech recognition servers by speech reconstruction from cepstra and pitch information
US20070143107A1 (en) * 2005-12-19 2007-06-21 International Business Machines Corporation Remote tracing and debugging of automatic speech recognition servers by speech reconstruction from cepstra and pitch information
US7983999B1 (en) 2005-12-30 2011-07-19 At&T Intellectual Property Ii, L.P. Multi-class classification learning on several processors
US7552098B1 (en) * 2005-12-30 2009-06-23 At&T Corporation Methods to distribute multi-class classification learning on several processors
US20090222270A2 (en) * 2006-02-14 2009-09-03 Ivc Inc. Voice command interface device
US20070192109A1 (en) * 2006-02-14 2007-08-16 Ivc Inc. Voice command interface device
CN101390155B (en) * 2006-02-21 2012-08-15 索尼电脑娱乐公司 Voice recognition with speaker adaptation and registration with pitch
US20070198263A1 (en) * 2006-02-21 2007-08-23 Sony Computer Entertainment Inc. Voice recognition with speaker adaptation and registration with pitch
US8050922B2 (en) 2006-02-21 2011-11-01 Sony Computer Entertainment Inc. Voice recognition with dynamic filter bank adjustment based on speaker categorization
US20070198261A1 (en) * 2006-02-21 2007-08-23 Sony Computer Entertainment Inc. Voice recognition with parallel gender and age normalization
US20100324898A1 (en) * 2006-02-21 2010-12-23 Sony Computer Entertainment Inc. Voice recognition with dynamic filter bank adjustment based on speaker categorization
US8010358B2 (en) 2006-02-21 2011-08-30 Sony Computer Entertainment Inc. Voice recognition with parallel gender and age normalization
US7778831B2 (en) * 2006-02-21 2010-08-17 Sony Computer Entertainment Inc. Voice recognition with dynamic filter bank adjustment based on speaker categorization determined from runtime pitch
US8452668B1 (en) 2006-03-02 2013-05-28 Convergys Customer Management Delaware Llc System for closed loop decisionmaking in an automated care system
US7599861B2 (en) 2006-03-02 2009-10-06 Convergys Customer Management Group, Inc. System and method for closed loop decisionmaking in an automated care system
US7809663B1 (en) 2006-05-22 2010-10-05 Convergys Cmg Utah, Inc. System and method for supporting the utilization of machine language
US9549065B1 (en) 2006-05-22 2017-01-17 Convergys Customer Management Delaware Llc System and method for automated customer service with contingent live interaction
US8379830B1 (en) 2006-05-22 2013-02-19 Convergys Customer Management Delaware Llc System and method for automated customer service with contingent live interaction
US20090204390A1 (en) * 2006-06-29 2009-08-13 Nec Corporation Speech processing apparatus and program, and speech processing method
US8751226B2 (en) * 2006-06-29 2014-06-10 Nec Corporation Learning a verification model for speech recognition based on extracted recognition and language feature information
US20080004876A1 (en) * 2006-06-30 2008-01-03 Chuang He Non-enrolled continuous dictation
US20080027706A1 (en) * 2006-07-27 2008-01-31 Microsoft Corporation Lightweight windowing method for screening harvested data for novelty
US8069032B2 (en) 2006-07-27 2011-11-29 Microsoft Corporation Lightweight windowing method for screening harvested data for novelty
US20080103781A1 (en) * 2006-10-28 2008-05-01 General Motors Corporation Automatically adapting user guidance in automated speech recognition
US20080133235A1 (en) * 2006-12-01 2008-06-05 Simoneau Laurent Method to train the language model of a speech recognition system to convert and index voicemails on a search engine
US7415409B2 (en) 2006-12-01 2008-08-19 Coveo Solutions Inc. Method to train the language model of a speech recognition system to convert and index voicemails on a search engine
US8326457B2 (en) * 2007-01-18 2012-12-04 Korea Institute Of Science And Technology Apparatus for detecting user and method for detecting user by the same
US20100138040A1 (en) * 2007-01-18 2010-06-03 Korea Institute Of Science And Technology Apparatus for detecting user and method for detecting user by the same
US8386254B2 (en) 2007-05-04 2013-02-26 Nuance Communications, Inc. Multi-class constrained maximum likelihood linear regression
US20090024390A1 (en) * 2007-05-04 2009-01-22 Nuance Communications, Inc. Multi-Class Constrained Maximum Likelihood Linear Regression
GB2451907B (en) * 2007-08-17 2010-11-03 Fluency Voice Technology Ltd Device for modifying and improving the behaviour of speech recognition systems
GB2451907A (en) * 2007-08-17 2009-02-18 Fluency Voice Technology Ltd Device for modifying and improving the behavior of speech recognition systems
US20090086934A1 (en) * 2007-08-17 2009-04-02 Fluency Voice Limited Device for Modifying and Improving the Behaviour of Speech Recognition Systems
US8335690B1 (en) 2007-08-23 2012-12-18 Convergys Customer Management Delaware Llc Method and system for creating natural language understanding grammars
US9129599B2 (en) * 2007-10-18 2015-09-08 Nuance Communications, Inc. Automated tuning of speech recognition parameters
US20090106028A1 (en) * 2007-10-18 2009-04-23 International Business Machines Corporation Automated tuning of speech recognition parameters
US8260619B1 (en) 2008-08-22 2012-09-04 Convergys Cmg Utah, Inc. Method and system for creating natural language understanding grammars
US8645135B2 (en) * 2008-09-12 2014-02-04 Rosetta Stone, Ltd. Method for creating a speech model
US20100070278A1 (en) * 2008-09-12 2010-03-18 Andreas Hagen Method for Creating a Speech Model
US20100125459A1 (en) * 2008-11-18 2010-05-20 Nuance Communications, Inc. Stochastic phoneme and accent generation using accent class
US8290773B2 (en) * 2008-12-26 2012-10-16 Fujitsu Limited Information processing apparatus, method and recording medium for generating acoustic model
US20100169093A1 (en) * 2008-12-26 2010-07-01 Fujitsu Limited Information processing apparatus, method and recording medium for generating acoustic model
US20100211387A1 (en) * 2009-02-17 2010-08-19 Sony Computer Entertainment Inc. Speech processing with source location estimation using signals from two or more microphones
US20100211391A1 (en) * 2009-02-17 2010-08-19 Sony Computer Entertainment Inc. Automatic computation streaming partition for voice recognition on multiple processors with limited memory
US20100211376A1 (en) * 2009-02-17 2010-08-19 Sony Computer Entertainment Inc. Multiple language voice recognition
US8442833B2 (en) 2009-02-17 2013-05-14 Sony Computer Entertainment Inc. Speech processing with source location estimation using signals from two or more microphones
US8442829B2 (en) 2009-02-17 2013-05-14 Sony Computer Entertainment Inc. Automatic computation streaming partition for voice recognition on multiple processors with limited memory
US8788256B2 (en) 2009-02-17 2014-07-22 Sony Computer Entertainment Inc. Multiple language voice recognition
US9002711B2 (en) * 2009-03-25 2015-04-07 Kabushiki Kaisha Toshiba Speech synthesis apparatus and method
US20110087488A1 (en) * 2009-03-25 2011-04-14 Kabushiki Kaisha Toshiba Speech synthesis apparatus and method
US20100268538A1 (en) * 2009-04-20 2010-10-21 Samsung Electronics Co., Ltd. Electronic apparatus and voice recognition method for the same
US8965764B2 (en) * 2009-04-20 2015-02-24 Samsung Electronics Co., Ltd. Electronic apparatus and voice recognition method for the same
US10062376B2 (en) 2009-04-20 2018-08-28 Samsung Electronics Co., Ltd. Electronic apparatus and voice recognition method for the same
US9478216B2 (en) 2009-12-08 2016-10-25 Nuance Communications, Inc. Guest speaker robust adapted speech recognition
US20110218804A1 (en) * 2010-03-02 2011-09-08 Kabushiki Kaisha Toshiba Speech processor, a speech processing method and a method of training a speech processor
US9043213B2 (en) * 2010-03-02 2015-05-26 Kabushiki Kaisha Toshiba Speech recognition and synthesis utilizing context dependent acoustic models containing decision trees
US20110295603A1 (en) * 2010-04-28 2011-12-01 Meisel William S Speech recognition accuracy improvement through speaker categories
US9305553B2 (en) * 2010-04-28 2016-04-05 William S. Meisel Speech recognition accuracy improvement through speaker categories
US9009040B2 (en) * 2010-05-05 2015-04-14 Cisco Technology, Inc. Training a transcription system
US20110276325A1 (en) * 2010-05-05 2011-11-10 Cisco Technology, Inc. Training A Transcription System
US11341962B2 (en) 2010-05-13 2022-05-24 Poltorak Technologies Llc Electronic personal interactive device
US11367435B2 (en) 2010-05-13 2022-06-21 Poltorak Technologies Llc Electronic personal interactive device
US9542931B2 (en) * 2010-10-27 2017-01-10 Microsoft Technology Licensing, Llc Leveraging interaction context to improve recognition confidence scores
US20150046163A1 (en) * 2010-10-27 2015-02-12 Microsoft Corporation Leveraging interaction context to improve recognition confidence scores
US20120109646A1 (en) * 2010-11-02 2012-05-03 Samsung Electronics Co., Ltd. Speaker adaptation method and apparatus
US9484018B2 (en) * 2010-11-23 2016-11-01 At&T Intellectual Property I, L.P. System and method for building and evaluating automatic speech recognition via an application programmer interface
US20120130709A1 (en) * 2010-11-23 2012-05-24 At&T Intellectual Property I, L.P. System and method for building and evaluating automatic speech recognition via an application programmer interface
US8769009B2 (en) 2011-02-18 2014-07-01 International Business Machines Corporation Virtual communication techniques
US20130090921A1 (en) * 2011-10-07 2013-04-11 Microsoft Corporation Pronunciation learning from user correction
US9640175B2 (en) * 2011-10-07 2017-05-02 Microsoft Technology Licensing, Llc Pronunciation learning from user correction
US9208156B2 (en) 2011-12-06 2015-12-08 Honeywell International Inc. Acquiring statistical access models
US8825533B2 (en) 2012-02-01 2014-09-02 International Business Machines Corporation Intelligent dialogue amongst competitive user applications
US20130268270A1 (en) * 2012-04-05 2013-10-10 Nuance Communications, Inc. Forced/Predictable Adaptation for Speech Recognition
US8838448B2 (en) * 2012-04-05 2014-09-16 Nuance Communications, Inc. Forced/predictable adaptation for speech recognition
US9123338B1 (en) * 2012-06-01 2015-09-01 Google Inc. Background audio identification for speech disambiguation
US10224024B1 (en) 2012-06-01 2019-03-05 Google Llc Background audio identification for speech disambiguation
US9812123B1 (en) 2012-06-01 2017-11-07 Google Inc. Background audio identification for speech disambiguation
US9747895B1 (en) * 2012-07-10 2017-08-29 Google Inc. Building language models for a user in a social network from linguistic information
US9043208B2 (en) 2012-07-18 2015-05-26 International Business Machines Corporation System, method and program product for providing automatic speech recognition (ASR) in a shared resource environment
US9053708B2 (en) 2012-07-18 2015-06-09 International Business Machines Corporation System, method and program product for providing automatic speech recognition (ASR) in a shared resource environment
US9786281B1 (en) * 2012-08-02 2017-10-10 Amazon Technologies, Inc. Household agent learning
US9786279B2 (en) 2012-09-10 2017-10-10 Google Inc. Answering questions using environmental context
US9460716B1 (en) * 2012-09-11 2016-10-04 Google Inc. Using social networks to improve acoustic models
US8983836B2 (en) * 2012-09-26 2015-03-17 International Business Machines Corporation Captioning using socially derived acoustic profiles
US9208777B2 (en) * 2013-01-25 2015-12-08 Microsoft Technology Licensing, Llc Feature space transformation for personalization using generalized i-vector clustering
US20140214420A1 (en) * 2013-01-25 2014-07-31 Microsoft Corporation Feature space transformation for personalization using generalized i-vector clustering
US9275638B2 (en) 2013-03-12 2016-03-01 Google Technology Holdings LLC Method and apparatus for training a voice recognition model database
US20140358541A1 (en) * 2013-05-31 2014-12-04 Nuance Communications, Inc. Method and Apparatus for Automatic Speaker-Based Speech Clustering
US9368109B2 (en) * 2013-05-31 2016-06-14 Nuance Communications, Inc. Method and apparatus for automatic speaker-based speech clustering
US9786296B2 (en) 2013-07-08 2017-10-10 Qualcomm Incorporated Method and apparatus for assigning keyword model to voice operated function
US9966062B2 (en) 2013-07-23 2018-05-08 Google Technology Holdings LLC Method and device for voice recognition training
US9691377B2 (en) 2013-07-23 2017-06-27 Google Technology Holdings LLC Method and device for voice recognition training
US9875744B2 (en) 2013-07-23 2018-01-23 Google Technology Holdings LLC Method and device for voice recognition training
US10163438B2 (en) 2013-07-31 2018-12-25 Google Technology Holdings LLC Method and apparatus for evaluating trigger phrase enrollment
US10163439B2 (en) 2013-07-31 2018-12-25 Google Technology Holdings LLC Method and apparatus for evaluating trigger phrase enrollment
US10170105B2 (en) 2013-07-31 2019-01-01 Google Technology Holdings LLC Method and apparatus for evaluating trigger phrase enrollment
US10192548B2 (en) 2013-07-31 2019-01-29 Google Technology Holdings LLC Method and apparatus for evaluating trigger phrase enrollment
US20150067822A1 (en) * 2013-09-05 2015-03-05 Barclays Bank Plc Biometric Verification Using Predicted Signatures
US9830440B2 (en) * 2013-09-05 2017-11-28 Barclays Bank Plc Biometric verification using predicted signatures
US9472186B1 (en) * 2014-01-28 2016-10-18 Nvoq Incorporated Automated training of a user audio profile using transcribed medical record recordings
US20210272552A1 (en) * 2014-05-12 2021-09-02 Soundhound, Inc. Deriving acoustic features and linguistic features from received speech audio
US9997157B2 (en) 2014-05-16 2018-06-12 Microsoft Technology Licensing, Llc Knowledge source personalization to improve language models
US10685329B2 (en) 2014-06-20 2020-06-16 Hirevue, Inc. Model-driven evaluator bias detection
US9652745B2 (en) * 2014-06-20 2017-05-16 Hirevue, Inc. Model-driven evaluator bias detection
US20150371191A1 (en) * 2014-06-20 2015-12-24 Hirevue, Inc. Model-driven evaluator bias detection
CN106663096A (en) * 2014-07-22 2017-05-10 纽昂斯通讯公司 Systems and methods for speech-based searching of content repositories
US10789953B2 (en) 2014-10-01 2020-09-29 XBrain, Inc. Voice and connection platform
US20160098992A1 (en) * 2014-10-01 2016-04-07 XBrain, Inc. Voice and Connection Platform
US10235996B2 (en) * 2014-10-01 2019-03-19 XBrain, Inc. Voice and connection platform
US10547910B2 (en) 2015-04-17 2020-01-28 Hewlett-Packard Development Company, L.P. Adjusting speaker settings
TWI559216B (en) * 2015-04-17 2016-11-21 惠普發展公司有限責任合夥企業 Adjusting speaker settings
US20160372115A1 (en) * 2015-06-17 2016-12-22 Volkswagen Ag Speech recognition system and method for operating a speech recognition system with a mobile unit and an external server
US10170121B2 (en) * 2015-06-17 2019-01-01 Volkswagen Ag Speech recognition system and method for operating a speech recognition system with a mobile unit and an external server
US20160379630A1 (en) * 2015-06-25 2016-12-29 Intel Corporation Speech recognition services
US20170053645A1 (en) * 2015-08-22 2017-02-23 Toyota Motor Engineering & Manufacturing North America, Inc. Speech recognition system with abbreviated training
US10008199B2 (en) * 2015-08-22 2018-06-26 Toyota Motor Engineering & Manufacturing North America, Inc. Speech recognition system with abbreviated training
US20170154640A1 (en) * 2015-11-26 2017-06-01 Le Holdings (Beijing) Co., Ltd. Method and electronic device for voice recognition based on dynamic voice model selection
US11749286B2 (en) 2017-05-08 2023-09-05 Telefonaktiebolaget Lm Ericsson (Publ) ASR training and adaptation
US11610590B2 (en) 2017-05-08 2023-03-21 Telefonaktiebolaget Lm Ericsson (Publ) ASR training and adaptation
WO2018208191A1 (en) * 2017-05-08 2018-11-15 Telefonaktiebolaget Lm Ericsson (Publ) Asr training and adaptation
US10984801B2 (en) 2017-05-08 2021-04-20 Telefonaktiebolaget Lm Ericsson (Publ) ASR training and adaptation
US11340925B2 (en) 2017-05-18 2022-05-24 Peloton Interactive Inc. Action recipes for a crowdsourced digital assistant system
US11520610B2 (en) * 2017-05-18 2022-12-06 Peloton Interactive Inc. Crowdsourced on-boarding of digital assistant operations
US11682380B2 (en) 2017-05-18 2023-06-20 Peloton Interactive Inc. Systems and methods for crowdsourced actions and commands
US11862156B2 (en) 2017-05-18 2024-01-02 Peloton Interactive, Inc. Talk back from actions in applications
US11495212B2 (en) * 2018-10-23 2022-11-08 Capital One Services, Llc Dynamic vocabulary customization in automated voice systems
US20230022004A1 (en) * 2018-10-23 2023-01-26 Capital One Services, Llc Dynamic vocabulary customization in automated voice systems
US10720149B2 (en) * 2018-10-23 2020-07-21 Capital One Services, Llc Dynamic vocabulary customization in automated voice systems
US10785171B2 (en) 2019-02-07 2020-09-22 Capital One Services, Llc Chat bot utilizing metaphors to both relay and obtain information
US20220116493A1 (en) * 2019-03-21 2022-04-14 Capital One Services, Llc Methods and systems for automatic discovery of fraudulent calls using speaker recognition
CN111739517A (en) * 2020-07-01 2020-10-02 腾讯科技(深圳)有限公司 Speech recognition method, speech recognition device, computer equipment and medium
CN111739517B (en) * 2020-07-01 2024-01-30 腾讯科技(深圳)有限公司 Speech recognition method, device, computer equipment and medium

Similar Documents

Publication Publication Date Title
US6442519B1 (en) Speaker model adaptation via network of similar users
US11455995B2 (en) User recognition for speech processing systems
US11270685B2 (en) Speech based user recognition
JP6705008B2 (en) Speaker verification method and system
O’Shaughnessy Automatic speech recognition: History, methods and challenges
US7231019B2 (en) Automatic identification of telephone callers based on voice characteristics
US8280733B2 (en) Automatic speech recognition learning using categorization and selective incorporation of user-initiated corrections
EP1395803B1 (en) Background learning of speaker voices
US5995928A (en) Method and apparatus for continuous spelling speech recognition with early identification
JP3434838B2 (en) Word spotting method
US10170107B1 (en) Extendable label recognition of linguistic input
Gray et al. Child automatic speech recognition for US English: child interaction with living-room-electronic-devices.
WO1992000585A1 (en) Continuous speech processing system
US20090240499A1 (en) Large vocabulary quick learning speech recognition system
US7181395B1 (en) Methods and apparatus for automatic generation of multiple pronunciations from acoustic data
US20220068257A1 (en) Synthesized Data Augmentation Using Voice Conversion and Speech Recognition Models
Aggarwal et al. Integration of multiple acoustic and language models for improved Hindi speech recognition system
Chen et al. Automatic transcription of broadcast news
US7289958B2 (en) Automatic language independent triphone training using a phonetic table
Furui Robust methods in automatic speech recognition and understanding.
Chu et al. Automatic speech recognition and speech activity detection in the CHIL smart room
Wu et al. Cohorts based custom models for rapid speaker and dialect adaptation
KR20180057315A (en) System and method for classifying spontaneous speech
Miyazaki et al. Connectionist temporal classification-based sound event encoder for converting sound events into onomatopoeic representations
Salimbajevs Modelling latvian language for automatic speech recognition

Legal Events

Date Code Title Description
AS Assignment

Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LIBAL, VIT V.;SEDIVY, JAN;REEL/FRAME:010385/0632

Effective date: 19991019

Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KANEVSKY, DIMITRI;ZADROZNY, WLODEK W.;REEL/FRAME:010385/0651;SIGNING DATES FROM 19991012 TO 19991013

STCF Information on status: patent grant

Free format text: PATENTED CASE

CC Certificate of correction
FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

FPAY Fee payment

Year of fee payment: 4

AS Assignment

Owner name: NUANCE COMMUNICATIONS, INC., MASSACHUSETTS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:INTERNATIONAL BUSINESS MACHINES CORPORATION;REEL/FRAME:022354/0566

Effective date: 20081231

FPAY Fee payment

Year of fee payment: 8

FPAY Fee payment

Year of fee payment: 12