A system and method for a highly interactive style of speech-to-speech translation is provided. The interactive procedures enable a user to recognize, and if necessary correct, errors in both speech recognition and translation, thus providing robust translation output than would otherwise be possible. The interactive techniques for monitoring and correcting word ambiguity errors during automatic translation, search, or other natural language processing tasks depend upon the correlation of Meaning Cues and their alignment with, or mapping into, the word senses of third party lexical resources, such as those of a machine translation or search lexicon. This correlation and mapping can be carried out through the creation and use of a database of Meaning Cues, i.e., SELECT. Embodiments described above permit the intelligent building and application of this database, which can be viewed as an interlingua, or language-neutral set of meaning symbols, applicable for many purposes. Innovative...
|US6078878||Jul 31, 1997||Jun 20, 2000||Microsoft Corporation||Bootstrapping sense characterizations of occurrences of polysemous words|
|US6287125||Aug 14, 1998||Sep 11, 2001||Distributed interactive presentation and response system with translation feature|
|US6385568||Sep 10, 1999||May 7, 2002||Operator-assisted translation system and method for unconstrained source text|
|US6442522||Oct 12, 1999||Aug 27, 2002||International Business Machines Corporation||Bi-directional natural language system for interfacing with multiple back-end applications|
|US6446035||May 5, 1999||Sep 3, 2002||Xerox Corporation||Finding groups of people based on linguistically analyzable content of resources accessed|
|US6996520||Nov 14, 2003||Feb 7, 2006||Transclick, Inc.||Language translation system and method using specialized dictionaries|
|US6996800||Dec 4, 2001||Feb 7, 2006||International Business Machines Corporation||MVC (model-view-controller) based multi-modal authoring tool and development environment|
|US7028306||Dec 4, 2001||Apr 11, 2006||International Business Machines Corporation||Systems and methods for implementing modular DOM (Document Object Model)-based multi-modal browsers|
|US7184948||Jun 15, 2001||Feb 27, 2007||Sakhr Software Company||Method and system for theme-based word sense ambiguity reduction|
|US7286651||Feb 12, 2002||Oct 23, 2007||Sprint Spectrum L.P.||Method and system for multi-modal interaction|
|US7668716||May 5, 2006||Feb 23, 2010||Dictaphone Corporation||Incorporation of external knowledge in multimodal dialog systems|
|US7668718||Aug 12, 2005||Feb 23, 2010||Custom Speech USA, Inc.||Synchronized pattern recognition source data processed by manual or automatic means for creation of shared speaker-dependent speech user profile|
|US7848915||Aug 9, 2006||Dec 7, 2010||International Business Machines Corporation||Apparatus for providing feedback of translation quality using concept-based back translation|
|US8055495||Sep 6, 2007||Nov 8, 2011||Kabushiki Kaisha Toshiba||Apparatus and method for translating input speech sentences in accordance with information obtained from a pointing device|
|US8090570||Oct 26, 2007||Jan 3, 2012||Mobile Technologies, LLC||Simultaneous translation of open domain lectures and speeches|
|US8140341||Jan 19, 2007||Mar 20, 2012||International Business Machines Corporation||Method for the semi-automatic editing of timed and annotated data|
|US8209166||Mar 18, 2008||Jun 26, 2012||Kabushiki Kaisha Toshiba||Apparatus, method, and computer program product for machine translation|
|US8214197||Sep 11, 2007||Jul 3, 2012||Kabushiki Kaisha Toshiba||Apparatus, system, method, and computer program product for resolving ambiguities in translations|
|US8239207||Apr 15, 2009||Aug 7, 2012||Spoken Translation, Inc.||Speech-enabled language translation system and method enabling interactive user supervision of translation and speech recognition accuracy|
1. A system providing cross-linguistic communication, comprising:
- a client component capturing speech and tactile inputs, the client component providing a user interface configured to display and correct the inputs and meanings for the inputs, a translation of the input term into a different language than an input language, and a back translation of the translation, i.e., a retranslation from the target language back into the source language, intended to help the user to judge the translation quality; and
- a server component providing the meanings, the translation and the back translation to the client component based upon the input term, the server component including,
- an interaction manager requesting the translation of the inputs, the interaction manager accessing a database of Meaning Cues corresponding to the inputs, the corresponding Meaning Cues being displayed through the client component to facilitate word sense selection for the inputs.
2. The system of claim 1 further comprising:
- at least two lexical resources, the first lexical resource including a dictionary database containing a plurality of words, wherein each of the plurality of words is associated with a meaning, the second lexical resource containing a plurality of word meanings in a central repository, each word meaning mapped to a corresponding word sense of the database of the Meaning Cues.
3. The system of claim 1 further comprising:
- a memory database storing user profiles defining word sense preferences for each user profile.
4. The system of claim 1, wherein the server component includes a speech recognition module in communication with the interaction manager, the speech recognition module permitting use of voice commands captured through the client component.
5. The system of claim 1, wherein the server component includes a text-to-speech module providing an audio signal to the client component for pronunciation of the input term in the different, language and the input language.
6. The system of claim 2, wherein the second lexical resource is organized according to word senses and includes multiple Meaning Cues.
7. The system of claim 1, wherein the back translation reflects word sense selections for input terms.
8. The system of claim 1, wherein the Meaning Cues include one of synonyms, related words, definitions, example sentences, or graphic elements related to corresponding inputs.
9. A cross linguistic communication system, comprising:
- a client component including a multi-modal user interface; and
- a server component including an interaction manager in communication with the multi-modal user interface, the interaction manager providing semantic cross indexing or mapping between a term output from a translation engine accessible to the server component and a database comprised of Meaning Cues designed to clarify for users word sense options for the term, wherein the Meaning Cues are returned to the interaction manager for display through the multi-modal user interface to enable interactive user modification of the translation.
10. The system of claim 7, wherein Meaning Cue information is organized by word sense.
11. The system of claim 7, wherein the interaction manager
- receives the term for translation from the multi-modal user interface;
- accesses the translation engine to locate a translated term corresponding to the term;
- associates the term with Meaning Cues, the Meaning Cues including one of synonyms, related words, definitions, example sentences, or graphic elements; and
- transmits the Meaning Cues to the multi-modal user interface for display.
12. The system of claim 9, wherein the translation engine provides a back translation of the translation which reflects word sense selections for one of the input terms.
13. A computer-readable storage medium having computer-executable instructions stored thereon that, when executed, display a graphical user interface (GUI) enabling computer mediated communication, the GUI comprising:
- a first section of the GUI displaying input terms in a first language;
- a second section of the GUI displaying Meaning Cues for each of the input terms;
- a third section of the GUI displaying a translation of the input terms in a second language; and
- a fourth section of the GUI displaying a back translation in the first language, the back translation based upon the translation.
14. The computer-readable storage medium of claim 13, wherein selection of one of the Meaning Cues in the second section triggers a display of a fifth section, the fifth section enabling selection of a word sense for the input term.