|Publication number||USH2269 H1|
|Application number||US 12/622,627|
|Publication date||Jun 5, 2012|
|Priority date||Nov 20, 2009|
|Also published as||US20110125483|
|Publication number||12622627, 622627, US H2269 H1, US H2269H1, US-H1-H2269, USH2269 H1, USH2269H1|
|Inventors||Johnson Manuel-Devadoss (Johnson Smith)|
|Original Assignee||Manuel-Devadoss Johnson Smith Johnson|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (17), Classifications (10)|
|External Links: USPTO, USPTO Assignment, Espacenet|
The present invention relates generally to a speech translating method, and more particularly, to automatically translate speech from one language to a language natural to another which is understandable by the language areas of one or plurality of intended recipient brain.
U.S. patent application Ser. No. 12/543,054, filed Aug 18, 2009, assigned to the same assignee as the instant application, and which is herein incorporated by reference in its entirely.
Typically, communication is said to be successful between two people if someone speaks and opponent party can understand. In other words the intended recipient's brain language areas can comprehend the speech. The problem of not understanding the speech of others is the cause of language barriers. So, this invention discloses a method to solve the language barrier problem where it is capable of interpreting meaning of speech in one language to a language natural to another—basically to a language the recipient brain can comprehend.
Languages are mankind's principle tools for interacting expressing ideas, emotions, knowledge, memories and values. Languages are also primary vehicles of cultural expressions and intangible cultural heritage, essential to the identity of individuals and groups. Safeguarding endangered languages is a crucial task in maintaining cultural diversity worldwide. According to researchers more than 6,700 languages are spoken in 228 countries. For example, in India more than 250 languages are used for speech. People like to speak in their natural language and prefer to communicate with others in their natural language. This makes it difficult for people to travel to foreign states or countries as they need to learn the foreign language.
Most individuals living in the United States read, write, speak, and understand English. There are many individuals, however, for whom English is not their primary language. The 2000 census shows that 26 million individuals speak Spanish and almost 7 million individuals speak an Asian or Pacific Island language at home. If these individuals have a limited ability to read, write, speak, or understand English, they are limited English proficient, or “Limited English Proficiency.” In a 2001 Supplementary Survey by the U.S. Census Bureau, 33% of Spanish speakers and 22.4% of all Asian and Pacific Island language speakers aged 18-64 reported that they spoke English either “not well” or “not at all.”
In field of entertainment, if someone wants to watch a foreign movie/performance, they experience problems in clearly understanding the event. Obviously, lots of electronic translator equipments are available in the world, but it only supports popularly spoken languages.
Language barriers and misunderstandings can get in the way of effective communication and create complications in the workplace, including problems with safety. A recent Business Journal article on the rising number of foreign national workers in Charlotte-Mecklenburg's construction industry pointed out—those workers who speak little or no English are at much greater risk of having an accident on the job because of not having a full grasp of safety standards.
Approximately 22% of the Sheraton Corporation's workforce is Hispanic, primarily Mexicans. Language is the main barrier here. To help its employers deal with the language challenge, the company has bilingual employees to serve as translators and mentors. In addition, all printed material is provided in both the essential languages Spanish and English. Another example is Woonsocket Spinning Company—Woonsocket is one of the few remaining woolen mills in the United States. 70% of their employees are foreign-born. Overcoming language barriers is the greatest challenge for both workers and the employer. To help with this, the company hires interpreters or has other employees who speak the language help the non-English speaking employees, particularly during orientation and training. Studies like this suggest companies spend a lot of time and effort to overcome language barriers among employees.
Patients from under developing countries seeking medical care always need to be accompanied with human translators to explain their medical problems and also to understand physician's advice. According to a report, more language interpretation services are needed in Connecticut's hospitals, doctors' offices and other health-care facilities to provide adequate medical care to patients with limited English skills. For example, The Connecticut Health Foundation, a nonprofit group based in New Britain, found that use of language interpretation services in medical settings throughout the state is limited, resulting in problems such as misdiagnosis and patient misunderstandings about doctors' instructions. The report advocated that hospitals and other health-care providers work toward providing more face-to-face interpr
In the ever growing IT industry people from various nationalities collaborate in meetings and conferences. Due to language barrier they cannot communicate freely resulting in business people investing lot of time and money learning new languages.
Even in marketing, due to language as barrier quality retail and consumer product owners struggle to market their products on international market.
There are number of language translation systems available in the world designed and developed to translate an inputted language to another language. All these methods/systems require a device to capture the voice and deliver. Such systems are known in the prior patents as disclosed in U.S. Pat. No. 4,882,681 to Brotz et al for Remote Language Translating Device. This prior patent disposes the translation of conversation between the users by transmitting/receiving speech using external hardware device. But people would not prefer to carry or even remember to carry the hardware device all the time. Also the disadvantage of such system is that it can be used to convert only a certain number of languages which are pre-programmed on the device.
U.S. Pat. No. 6,161,082 to Goldberg et al for Network based language translation system performs a similar task. It disposes a network based language translation system—basically has a translation software installed on the network. It proves that software over network can do speech translation, but user still has to set their language preferences. More than 67% of world's population do not or have limited computer knowledge, so they cannot set their language preferences and operate high-tech gadgets. Another recent patent is U.S. Pat. No. US 2009/0157410 to Donohoe et al for speech translating system. U.S. Pat. Appl. No. US 2009/0157410 discloses a system for translating speech from one language to a language selected from a set of languages. Such a system disclosed in U.S. Pat. Appl. No. US 2009/0157410 can be applicable only for limited amount of users but more than 6,700 languages are being used by people to express their thoughts around the world.
Another patent is U.S. Pat. No. 4,641,264 to Nitta et al for a Method of Automatic Translation between Natural Languages—this discloses a system for the translation of entire sentences. Then again it also requires an input and output device to capture and deliver the speech. It is not capable to determine the recipients' understandable language. We have to manually set the targeted language or select from pre-defined languages (as target) in the device.
According to DiscoveryChannel.ca report, by using electrodes attached to a person's face and neck, the device detects the electrical signals sent to the person's facial muscles and tongue when specific words are mouthed. The software is able to decode the information into phonemes—the building blocks of words. Since there are only 45 different phonemes used in English, the system is able to predict what phonemes are most likely to appear next to each other. This helps the device translate phrases even if it hasn't heard them before. The system won't help make peace with any hostile aliens just yet, though. It only translates correctly with 62 percent efficiency when faced with a phrase for the first time.
Although there have been many advances in system and software for providing language translation to users interested in communicating in a language other than their own language, there has not been an apparatus or method that facilitate to identify intended recipients' natural language using brain language areas of one or plurality of intended recipients. Accordingly, the present inventor has developed a system that can identify the natural language of one or plurality of intended recipients by their brain language areas and uses the identified natural language for speech translation.
Therefore to overcome all the above language barriers, there is a need for a system to perform automatic translation of speech wherein when one speaks in a natural language others are able to comprehend in their own natural languages without interpreters, hand-held device and language translation books.
In view of the foregoing disadvantages inherent in the prior art, the general purpose of the present invention is to provide an “Automated Speech Translation System using Human Brain Language Areas Comprehension Capabilities” configured to include all the advantages of the prior art, and to overcome the drawbacks inherent therein.
Speech translation is basically converting from spoken words in one language to another language where the language area of recipient human brain can comprehend. Recipient(s) may not be able to comprehend the speech because of their brain language areas are not tuned to understand the spoken language.
The present invention discloses a method to identify the target language by using brain language areas of one or plurality of intended recipients. The language area of human brain is a large cortical area (in the left hemisphere in most people) containing all the centers associated with language.
The present invention disposes a process where humans are not going be aware a translation is happening in the background. They will be able to speak their own natural language but others surrounding them can automatically understand the speech in their own natural language. This system therefore bridges all communication gaps among people.
The main object of the present invention is to provide an “Automated Speech Translation System using Human Brain Language Areas Comprehension Capabilities” that is capable of providing a translation of speech in one language to a language natural to another which is understandable by the brain language areas of one or plurality of recipient. The present invention thereby replaces interpreters, hand-held device and language translation books.
This invention facilitates tourism. People are now free to travel to any corner of the world. They don't have to carry any hand-held devices. This invention facilitates people to enjoy foreign movie/performances without need of friends as human translators or sophisticated translation devices. Patients can be provided with the right care that they require. This invention also eliminates all miscommunications and reduces death totality in industries. Employers can hire people from any ethnicity as langua
This invention also facilitates businessmen from any country to expose their quality products worldwide within a less budget. Everyone can continue to effectively communicate in their own natural language in meetings and conferences while employers can save money on language translation books.
Still another object of the present invention is to provide an automated speech translation system that may enable a smooth communication between users.
All these put together with other aspects of the present invention, along with the various features that describe the present invention, especially those pointed out in the claims section form a part of the present invention. To gain more knowledge of the present invention understanding of the drawings attached and the detailed description is highly essential.
FlG. 2.a illustrates two people of the system speaking in their natural language using “Automated Speech Translation System using Human Brain Language Areas Comprehension Capabilities”;
FlG. 2.d illustrates spokesman of the system addressing a crowd in his natural language using “Automated Speech Translation System using Human Brain Language Areas Comprehension Capabilities”.
FlG. 3.b illustrates the electrodes placed in between language areas to record brain language areas activity for constructing “Human Brain Language Areas Knowledge Base” of present invention.
Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout.
Communication is said to be effective between two people, if one speaks and intended recipient can understand. In other words the intended recipients' brain language area can comprehend the words/sentence/speech. The present invention basically does that—interpreting meaning of word(s) in a language understandable by language areas of intended recipient brain.
The “Automated Speech Translation System using Human Brain Language Areas Comprehension Capabilities” of present invention has three main logical processing units—Intelligent Natural Language Program (INLP), Language Inference Engine and Speech Translation Module. The human ear can hear frequencies at ˜70 decibels. When we talk our thoughts are converted into voice signals and transmitted into the surrounding regions. The human speech contains the syntactic combination of lexicals and names that are drawn from very large vocabularies. Each spoken word is created out of the phonetic combination of a limited set of vowel and consonant speech sound units. These vocabularies, the syntax which structures them, and their set of speech sound units, differ creating the existence of many thousands of different types of mutually unintelligible human languages. This system employs a software broadcasting technique to broadcast the Intelligent Natural Language Program (INLP) over the air. The “Automated Speech Translation System using Human Brain Language Areas Comprehension Capabilities” of present invention makes use of electromagnetic radiation to broadcast the Language Area Acquisition Signal directed towards the intended recipient's head. The voice processing center of present invention receives the electromagnetic frequencies which contain a rapid analysis of the language area of intended recipient's brain. The rapid analysis of the brain language areas of one or plurality of intended recipients, are analyzed within seconds to provide an evaluation of the state of the cell's structure. Only information, not energy, is exchanged.
As shown in FlG. 1.a, the spoken dialog of man 102 travels through air in vocalized form 104. Spoken dialog of man 102 contains the syntactic combination of lexicals and names that are drawn from very large vocabularies. Each spoken dialog of man 102 is created out of the phonetic combination of a limited set of vowel and consonant speech sound units. These vocabularies, the syntax which structures them, and their set of speech sound units, differ creating the existence of many thousands of different types of mutually unintelligible human languages. An Intelligent Natural Language Program 118 of present invention, travels over air and looks for an acoustic waveform or voice pitches 104, 106, 108, 110, 112, 114, 116 in the air. The property of spoken voice of human being is determined by the rate of vibration of the vocal cords. The greater number of vibrations per second, the higher the pitch. The rate of vibration, in turn, is determined by the length and thickness of the vocal cords and by the tightening or relaxation of these cords.
As shown in
The analysis of the language areas of brain 126 of intended recipient 124 is then compared with “Human Brain Language Areas Knowledge Base” (shown in
For example, for building the sample for French language, a French speech is presented to a person for whom French is the natural language. During this experiment the electrodes (as shown in
The “Human Brain Language Areas Knowledge Base” thus built contains a massive store house of characteristics of “brain language areas activity signals” for over 6,700 natural languages spoken across the world. This massive repository of language characteristics is later used by the present invention to identify the natural language of the user. The identified natural language is fed into Speech Translation Module 180 to generate the corresponding words in particular natural language for spoken sentence 104 of man 102 (as shown in
As shown in
In human beings, it is the left hemisphere that usually contains the specialized language areas. While this holds true for 97% of right-handed people, about 19% of left-handed people have their language areas in the right hemisphere and as many as 68% of them have some language abilities in both the left and the right hemisphere. Both the two hemispheres are thought to contribute to the processing and understanding of language: the left hemisphere processes the linguistic of prosody, while the right hemisphere processes the emotions conveyed by prosody.
There are four distinct cortical language-related areas in the left hemisphere. These are: (1) a lateral and ventral temporal lobe region that includes superior temporal sulcus(STS) 316, middle temporal gyrus (MTG), parts of the inferior temporal gyrus (ITG) and fusiform and parahippocampal gyri; (2) a prefrontal region that included much of the inferior and superior frontal gyri, rostral and caudal aspects of the middle frontal gyrus, and a portion of the anterior cingulate; (3) angular gyrus; and (4) a perisplenial region including posterior cingulate, ventromedial precuneus, and cingulate isthmus. These regions were clearly distinct from auditory, premotor, supplementary motor area (SMA), and supramarginal gyrus areas that had been bilaterally activated by the tone task. The other large region activated by the semantic task is the right posterior cerebellum.
The first language area within the left hemisphere is called Broca's area 308. The Broca's area 308 doesn't just handle getting language out in a motor sense it is more generally involved in the ability to deal with grammar itself, at least the more complex aspects of grammar. The second language area is called Wernicke's area 310.
By analyzing data from numerous brain-imaging experiments, there are three distinguished subareas within Wernicke's area 310. The first sub-area responds to spoken words (including the individual's own) and other sounds. The second sub-area responds only to words spoken by someone else but is also activated when the individual recalls a list of words. The third sub-area is more closely associated with producing speech than with perceiving it. All of these findings are still compatible, however, the general role of Wernicke's area 310, relates to the representation of phonetic sequences, regardless of whether the individual hears them, generates them, or recalls them from memory.
Simultaneously, a rapid analysis of brain language areas activity of man 204 is collected by directing language area acquisition signal towards man's 204 head. The rapid analysis of man 204 brain includes the language area comprehensive information like Language Comprehension, Semantic Processing, Language Recognition, and Language Interpretation from brain language areas of man 204 and this collected information is sent to Voice Processing Center.
As shown in
The accurate translation of input speech is done by sophisticated parser 182, Phrase/Word Translator 186 and generation module 188. The speech translation module 180 comprises the Parser 182, Information Extractor 184, Phrase/Word Translator 186 and Generation Module 188. The parser 182 performs the process of prediction including complete semantic interpretations, constraint checks, and ambiguity resolution and discourse interpretations. The parser 182 handles multiple hypotheses in parallel rather than a single word sequence.
As shown in
This system performs real-time translations, which is far better performance than text-based machine translation systems. Unlike traditional methods of machine translation in which a generation module 188 process is invoked after parsing is completed; this system concurrently executes the generation process during parsing. It employs a parallel incremental generation scheme, where the generation process and the parsing process run almost concurrently. This enables the system to generate a part of the vocal expression of woman 202 during the parsing of the rest of the vocal expression of woman 202. Thus this system stimulates a live feeling—where one speaks and instantaneously the intended recipients can comprehend the speech in their natural languages.
The “Automated Speech Translation System using Human Brain Language Areas Comprehension Capabilities” of present invention handles the bi-directional conversations. This system provides the bidirectional translation with an ability to understand interaction at the discourse knowledge level, predict possible next vocal expression, understand what particular pronouns refer to, and also provides high-level constraints for the generation of contextually appropriate sentences involving various context-dependent phenomena.
F1G. 2.b illustrates the conversation between friends who are all foreign-language speaking people. Vietnamese speaking person is saying “This food is delicious” in his natural language such as shown in 216, this sentence is comprehended as shown in 218 by the Catalan speaking person, as shown in 220 by Finnish speaking person, and as shown in 222 by Hebrew speaking person and also as shown in 224 by English speaking person. The Finnish speaking person acknowledges back to them in his natural as shown in 226. Others comprehend the Finnish sentence as shown in 228, as shown in 230, as shown in 232 respectively using present invention of “Automated Speech Translation System using Human Brain Language Areas Comprehension Capabilities”.
FlG. 2.d illustrates the spokesman 250 is giving a speech in his natural language Spanish as shown in 252 to a crowd. There are Slovenian, Korean, Hindi, Hungarian, and Portuguese speaking people in the crowd. So, the spokesman's Spanish speech is automatically comprehended by Slovenian speaking person as shown in 254, by Korean speaking person as shown in 256, by Hindi speaking person as shown in 258, by Hungarian speaking person as shown in 260, and by Portuguese speaking person, as shown in 262, using present invention of “Automated Speech Translation System using Human Brain Language Areas Comprehension Capabilities”.
As described above, the present invention discloses a system for translating a speech in one language to a language native to the intended recipient(s). Accordingly, the present invention discloses a system of comprehending natural languages without the use of any handheld translators. This invention employs a system where there will no longer be a need to learn new language. Effective communication is now feasible between people speaking different languages. This system explores the capabilities of the human brain and utilizes the language information of the brain and performs the automatic translation in the background. It should be noted that with all the reading of language area of the human brain—the human brain will not be affected or caused any harm during this process.
The foregoing descriptions of specific embodiments of the present invention have been presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the present invention to the precise forms disclosed, and obviously many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the present invention and its practical application. Although the present invention has been described with reference to particular embodiments, it will be apparent to those skilled in the art that variations and modifications can be substituted without departing from the principles and spirit of the invention.
1. “How the brain learns to read” By David A. Sousa
2. “Natural Language Generation in Artificial Intelligence and Computational Linguistics” By Céile L Paris, William R. Swartout, William C. Mann
3. “Artificial intelligence methods and applications” By Nikolaos G. Bourbakis
4. T. Morimoto et al., “Spoken Language Translation,” Proc. Info Japan, Tokyo, 1990.
5. K. Kita, T. Kawabata, and H. Saito, “HMM Continuous Speech Recognition using Predictive LR Parsing,” Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, 1989.
6. “Natural language processing technologies in artificial intelligence” By Klaus K. Obermeier
7. “Advances in artificial intelligence: natural language and knowledge-based” By Martin Charles Golumbic
8. “Pediatric otolaryngology”, Volume 1 of fourth edition By Charles D. Bluestone, Cuneyt M. Alper, Sylvan E. Stool, Ellis M. Arjmand, Margaretha L. Casselbrant, Joseph E. Doh
9. Artificial Neural Net Based Signal Processing for Interaction with Peripheral Nervous System. In: Proceedings of the 1st International IEEE EMBS Conference on Neural Engineering. pp. 134-137. Mar. 20-22, 2003.
11. U.S. Pat. No. 6,356,865, issued to Franz et al., entitled “Method and system for performing spoken language translation”
12. U.S. Pat. No. 7,392,079, issued to Donoghue et al., entitled “Neurological Signal decoding”
13. U.S. Pat. No. 7,574,357, issued to Jorgensen et al., entitled “Applications of sub-audible speech recognition based upon electromyographic signals”
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US3335225 *||Feb 20, 1964||Aug 8, 1967||Melpar Inc||Formant period tracker|
|US4641264 *||Sep 7, 1982||Feb 3, 1987||Hitachi, Ltd.||Method for automatic translation between natural languages|
|US4882681 *||Sep 2, 1987||Nov 21, 1989||Brotz Gregory R||Remote language translating device|
|US6161082 *||Nov 18, 1997||Dec 12, 2000||At&T Corp||Network based language translation system|
|US6385195 *||Jul 21, 1997||May 7, 2002||Telefonaktiebolaget L M Ericsson (Publ)||Enhanced interworking function for interfacing digital cellular voice and fax protocols and internet protocols|
|US7574357 *||Jun 24, 2005||Aug 11, 2009||The United States Of America As Represented By The Admimnistrator Of The National Aeronautics And Space Administration (Nasa)||Applications of sub-audible speech recognition based upon electromyographic signals|
|US7689426 *||Aug 27, 2007||Mar 30, 2010||Avaya Inc.||Method and apparatus for distributed interactive voice processing|
|US8041023 *||Oct 18, 2011||Aspect Software, Inc.||System and method of using a phone to access information in a call center|
|US20020035474 *||Mar 26, 2001||Mar 21, 2002||Ahmet Alpdemir||Voice-interactive marketplace providing time and money saving benefits and real-time promotion publishing and feedback|
|US20030212558 *||May 7, 2002||Nov 13, 2003||Matula Valentine C.||Method and apparatus for distributed interactive voice processing|
|US20040186743 *||Jan 27, 2004||Sep 23, 2004||Angel Cordero||System, method and software for individuals to experience an interview simulation and to develop career and interview skills|
|US20070291924 *||Aug 27, 2007||Dec 20, 2007||Avaya Technology Corp||Method and Apparatus for Distributed Interactive Voice Processing|
|US20090022284 *||Oct 1, 2008||Jan 22, 2009||Avaya Inc.||Method and Apparatus for Distributed Interactive Voice Processing|
|US20090157410 *||Dec 13, 2007||Jun 18, 2009||James Donohoe||Speech Translating System|
|US20100082325 *||Nov 16, 2009||Apr 1, 2010||Manuel-Devadoss Johnson Smith||Automated phonetic language translation system using Human Brain Interface|
|US20110046941 *||Aug 18, 2009||Feb 24, 2011||Manuel-Devados Johnson Smith Johnson||Advanced Natural Language Translation System|
|US20110125483 *||May 26, 2011||Manuel-Devadoss Johnson Smith Johnson||Automated Speech Translation System using Human Brain Language Areas Comprehension Capabilities|
|U.S. Classification||704/2, 704/E15.001, 704/9|
|International Classification||G06F17/27, G06F17/28|
|Cooperative Classification||G10L25/90, G06F17/289, G10L15/26|
|European Classification||G10L15/26A, G06F17/28U|