US20030125952A1 - Voice and text transmission system - Google Patents

Voice and text transmission system Download PDF

Info

Publication number
US20030125952A1
US20030125952A1 US10/336,950 US33695003A US2003125952A1 US 20030125952 A1 US20030125952 A1 US 20030125952A1 US 33695003 A US33695003 A US 33695003A US 2003125952 A1 US2003125952 A1 US 2003125952A1
Authority
US
United States
Prior art keywords
user
text
packet
relay
format
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US10/336,950
Inventor
Robert Engelke
Kevin Colwell
Troy Vitex
Jeffrey Havens
Kurt Grittner
Dean Hofstetter
Mathew McCulley
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to US10/336,950 priority Critical patent/US20030125952A1/en
Publication of US20030125952A1 publication Critical patent/US20030125952A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/66Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M3/00Automatic or semi-automatic exchanges
    • H04M3/42Systems providing special services or facilities to subscribers
    • H04M3/42391Systems providing special services or facilities to subscribers where the subscribers are hearing-impaired persons, e.g. telephone devices for the deaf
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M7/00Arrangements for interconnection between switching centres
    • H04M7/006Networks other than PSTN/ISDN providing telephone service, e.g. Voice over Internet Protocol (VoIP), including next generation networks with a packet-switched transport layer
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M2201/00Electronic components, circuits, software, systems or apparatus used in telephone systems
    • H04M2201/40Electronic components, circuits, software, systems or apparatus used in telephone systems using speech recognition

Definitions

  • the present invention relates to the general field of telephone communications.
  • the invention relates to systems to assist telephone communications by those persons who are deaf, hard of hearing, or otherwise have impaired hearing capability.
  • TDD telecommunication device for the deaf
  • TT text telephone
  • TTY teletype
  • Current TDDs are electronic devices consisting of a keyboard and a display as well as a specific type of modem, to acoustically or directly couple to the telephone line. Modem TDDs permit the user to type characters into their keyboard, with the character strings then encoded and transmitted over the telephone line to be displayed on the display of a communicating or remote TDD device.
  • Baudot evolved historically at a time when many telecommunication devices for the deaf were based on mechanical or electromechanical devices rather than the current technology based on digital electronic components. Accordingly, the Baudot protocol was constructed for a set of constraints which are no longer relevant to present date devices.
  • the original Baudot protocol was a unidirectional or simplex system of communication conducted at 45.5 Baud.
  • the conventional Baudot character set was a character set consisting of 5 bit characters and the system encodes the bits of those characters in a two-tonal system based on carrier tones of 1400 and 1800 Hertz.
  • a relay refers to a system of voice to TDD communication in which an operator, referred to as a “call assistant,” serves as a human intermediary between a hearing user and a deaf person.
  • the call assistant wears a headset that communicates by voice with the hearing user and also has access to a TDD device which can communicate to the deaf user using a TDD appropriate protocol.
  • the call assistant types at a TDD keyboard the words which are voiced to her by the hearing user and then voices to the hearing user the words that the call assistant sees upon the display of his or her TDD.
  • the call assistant serves, in essence, as an interpreting intermediary between the deaf person and the hearing person to translate from voice to digital electronic forms of communication.
  • 6,075,842 methods and devices for providing text enhanced telephony are described in which a text stream is provided along with voice in telephone communications with hard of hearing users.
  • the text stream is used to provide the assisted user with a visual representation of the text of what is said by the other person in a communication session, so as to gently assist a person with some hearing deficiency in using the telephone.
  • the present invention is summarized in a method for transmitting voice and text of words over a telephonic connection between a hearing user and an assisted user through a relay, the method including the steps of digitizing the voice of the hearing user; creating a digital text at the relay corresponding to the words spoken by the hearing user; combining the digitized voice and the text into combined digital data packets, each packet including a format character indicating the type of format for that packet, at least some of the digital data packets combining at least one byte of digitized voice data with at least one byte of text representing a character in the text of the words spoken by the hearing user; and transmitting the combined packets to the station of the assisted user over a telephone connection so that the station can reconstitute both voice and text from the digital data packets for the assisted user.
  • the present invention is also summarized in a communication system using that method to communicate voice and text of the words spoken by the voice to a station used by an assisted user.
  • the present invention is intended to create a flexible communication protocol, using minimal overhead, which is capable of sending voice and the text for the words spoken by that voice, in digital form over common telephonic communication linkages.
  • FIG. 1 is a schematic diagram of a system using the method and protocol of the present invention, showing particular details of the exemplary relay used in the system.
  • FIG. 2 is a schematic diagram of a captioned telephone for use in the system of FIG. 1.
  • the communication protocol of the present invention is intended to facilitate voice and text communications between hearing persons and assisted persons by a relay that intermediates the call. Since the full implementation of the protocol is most appropriately done by a relatively sophisticated relay, the construction of such a relay and some details about the device at the assisted users end will be described first.
  • FIG. 1 Shown in FIG. 1 is a schematic view of the relay which can intermediate such a call.
  • the telephone 12 connects through a telephonic connection 14 to a DAA and hybrid circuit 16 at the relay.
  • the hearing user's telephone 12 can be any of the devices generally thought of by laypersons as a telephone, including but not limited to land line telephones, cellular telephones, PCS devices and audio links over the internet.
  • what is received at the DAA and hybrid circuit 16 of the relay is a voice signal.
  • the voice from the DAA and hybrid circuit 15 is connected as the input to a codec 18 . Codecs have become industry standard devices that convert an analog signal, such as analog voice, into digital data.
  • codec integrated circuits such as those from Texas Instruments and Analog Devices, can convert both analog voice signals into digital data and do the reverse, i.e. reconstruct an analog voice signal from digital data representing voice.
  • the output digital signal from the codec 18 is then connected to a circuit or software designed to cancel echo on the telephone line, as indicated at 20 .
  • a vocoder is a type of digital signal processing chip or an algorithm implemented by a processor specifically designed to transform digital data carrying voice to compress the data for transmission.
  • GSM Global System for Mobile communications
  • G.729 G. 729
  • the vocoder is indicated at 22 , and is capable of both encoding, or compressing, and decoding, or de-compressing, the digital data stream carrying voice.
  • the UVT formatter 24 is actually implemented electronically by a specially programmed microprocessor or digital signal processor.
  • the UVT formatter 24 combines the digital data stream representing voice, from the vocoder 22 with a digital data stream carrying text from a call assistant computer, indicated at 26 .
  • the call assistant computer 26 is a general purpose digital computer preferably equipped with a speech recognition software package.
  • the cell assistant wears a headset 28 connected to transmit the voice of the hearing user to the ear of the call assistant.
  • the call assistant repeats, or “re-voices,” the words spoken by the hearing user into a microphone of the headset 28 that is connected to the computer 26 .
  • the speech recognition engine in the call assistant computer 26 recognizes the voice of the call assistant and translates that voice into a text stream.
  • the output of the call assistant computer 26 is thus a digital data stream carrying text which is provided to the UVT formatter 24 as well.
  • the UVT formatter is constructed to combine the digital data stream carrying voice with the digital data stream carrying text, using the UVT format described below.
  • the UVT formatter is also capable of doing the reverse, that is, separating the digital data stream carrying voice from the digital data stream carrying text, using information contained in the UVT protocol to make that separation.
  • the output of the UVT formatter 24 is connected to a modem 30 , in this case illustrated as an industry standard V.32bis format modem.
  • the output of the modem 30 is connected through another codec 32 to a hybrid circuit 34 and DAA at the output of the relay.
  • the DAA and hybrid circuit 34 is connected, by any form of telephonic connection 36 to an assisted user station 38 .
  • the assisted user station 38 can be a personal interpreter of the general type as shown in U.S. Pat. No. 5,974,116, or a captioned telephone of the general type as shown in U.S. Pat. No. 6,075,842, or any other device intended to assist the assisted user in the communication session by providing text to the assisted user to help that user understand the words spoken by the hearing user.
  • FIG. 2 Shown in FIG. 2 is a captioned telephone device 100 with that capability.
  • the interior components of the captioned telephone device 100 are illustrated in block diagrams indicating the digital logic components from which the device may be constructed. It is preferred, however, that the components within the dotted lines in FIG. 2, labeled as DSP software, actually be implemented in the form of a software routine operating a digital signal processing integrated circuit to perform the functions of the illustrated blocks.
  • the telephonic connection to the relay is indicated at 102 .
  • the input telephonic signal connects to a DAA and hybrid 104 and then to a codec 106 to digitize input signals.
  • modem 108 Following the codec 106 is modem 108 , the output of which connects to a UVT formatter 110 .
  • the UVT formatter 110 operates to separate the digitized voice signals from the digital text signals.
  • the digitized text signals are transferred from the UVT formatter to a visually readable display 112 on which the text can be displayed for the assisted user.
  • the digitized voice signal is transferred from the UVT formatter 110 to a vocoder 114 , compatible in format to the vocoder used in the relay of FIG.
  • the output of the vocoder 114 is connected through an acoustic echo control 116 to another codec 118 which reconstructs the analog voice signal for delivery to the handset 120 of the assisted user.
  • the assisted station decompressed the digital data stream and separates the digital text data from the data representing digitized voice.
  • the text is displayed on the display 112 and the voice is reconstituted into analog and played on the speaker in the handset 120 .
  • the assisted user thus receives both the voice of the hearing user and is provided a text display of the words spoken by the hearing user.
  • the hearing user at telephone 12 and that person's voice is converted into digital form in the relay, that digital form being a direct digitalization of the hearing person's voice.
  • the call assistant computer creates a text data stream, also in digital form, of the text of the words spoken by the hearing person.
  • the UVT protocol implemented by the UVT formatter 24 in the system of FIG. 1 provides a methodology to efficiently combine the two digital data streams, one for voice one for text.
  • the two digital data streams are combined in a method that is convenient to create and transmit and convenient as well to separate at the receiving end.
  • the problem in combining the two digital data streams is that the digitized voice tends to require much more data to transit than the text data stream. It is inconvenient, however, to interrupt the transmission of voice to transmit the needed text data. It is also important that the total data transmitted not exceed the carrying capacity of any of the forms of telephonic connections supported by the network.
  • the solution described here is to continually send formatted information packets carrying the digitized voice to the remote station and then, in addition and as needed, some of the data packets are flagged to carry a portion of the digital text message data stream in the same packet. Since the data transmission requirements for the text message are so small in comparison to that of the digitized voice, single bytes of text message are combined with multi-byte portions of digitized voice data in these specialized data packets.
  • the packets carry digitized voice and at the same time carry text data for the words contained in the speech of the voice.
  • digitized voice and digitized text for the words spoken by that voice are carried in common digital data packets.
  • the text and voice may be delivered simultaneously or near simultaneously.
  • the text can be delivered to the assisted user as the text stream is created by the computer of the call assistant, in which case the text stream may lag the corresponding voice signal by a brief delay. In that event, the text character may not travel in the same packet as the voice for the word of which the character is a part.
  • This format uses a single format of data packet, sent as a digital data packet, but the packets do not all carry the same type of content. Some packets carry only digitized voice, while other packets carry both digitized voice and digital text data.
  • the packet header is used to indicate the type of packet, and as long as the designation of the packet header remains constant, later revisions of the format permit other later packet specifications to be defined. Other packet types include software upgrade data, user preferences for system set-up, settings for parameters of devices or for configurations, and error code information.
  • special packets are transmitted between the communicating devices.
  • Such special packets can be used to identify the version number of the protocol used by the communicating devices, information on the type of connection and speed, information of the type of data in following data packets, error correcting formats or codes, device status such a processor of memory resources available, or information on the downloading of software upgrades to devices.
  • Each call to the relay service begins with an exchange of special packets that allow the captioned telephone or personal interpreter to provide the relay service with the information necessary for the relay to automatically set-up the desired type of service and, for captioned telephone calls, to complete the connection to the hearing person.
  • the relay service sends a special packet called a Request for Call Set-up Information packet.
  • the captioned telephone or personal interpreter device responds with a special packet in return, here called the Call Set-up Information Block packet.
  • the Call Set-up Information Block packet includes the service type requested by the user (e.g.
  • captioned telephone or personal interpreting a user identification number
  • a user password in some cases
  • a captioned telephone call the telephone number of the hearing party.
  • the user had to type in information of this type, in response to inquiries from the call assistant, to complete the call set-up. This protocol permits the call to be set up automatically in a fraction of the time previously required.
  • Another type of special packet is used to transmit DTMF control signal to the relay equipment.
  • a captioned telephone user may dial, using the relay service, a telephone number that connects to an automated attendant or voice response unit.
  • These automated attendant type devices prompt the caller to enter their choice by pressing the number buttons on the their touch-tone telephone, i.e. “press 1 for sales.”
  • Such devices and voice mail systems are commonly encountered in telephone usage today.
  • a TDD user In current relays, a TDD user must type instructions to the relay call assistant to convey their intention, e.g. “press 1,” and the call assistant then manually presses a key to produce the DTMF signal on the second line.
  • the DTMF special packet in the present format provides the captioned telephone user with the functionality of a traditional touch tone telephone in the digital captioned telephone environment.
  • the captioned telephone connects to the relay service using a data communications protocol (e.g. V.32bis) which does not permit the captioned telephone to emit the DTMF tones directly onto the telephone line.
  • the captioned telephone could generate the DTMF tone signal and transmit that signal to the vocoder.
  • Such a signal would be carried as digital data to the relay equipment where it would be converted back to tones.
  • most vocoders cannot produce a precise enough set of frequencies or loud enough signals to meet network DTMF standards.
  • the present protocol permits the transmission of the special DTMF packet that instructs the relay to produce the DTMF signal directly on the telephone line to the other party.
  • the captioned telephone user simply presses the numerical dial of his or her telephone, and the captioned telephone then generates a command packet to the relay instructing it to produce the correct DTMF tones on the second telephone line automatically without interaction with the call assistant.
  • FIGS. 1 and 2 are illustrated with telephonic connections between the hearing person and the relay and between the relay and the assisted user. It is specifically contemplated that the manner of actual telephonic connection between these parties, and the number of telephone lines or telephone line equivalents that are used is not important.
  • the term telephonic connection, as used here, is intended to apply to actual dedicated connections through the telephone system, such as land-lines or analog cellular connections. The term is also intended to encompass other types of connections that can serve as telephonic connections in the lay sense of the term, such as digital cellular telephone service, PCS service and communication over the internet using IP protocol.
  • telephone line here is intended to encompass both traditional twisted pair physical telephone lines as well as any type of channels or software sockets that provide an equivalent connection between users of the telephone system.
  • protocol may be used using two or more telephone lines or over other higher bandwidth forms of interconnection. Examples of such higher bandwidth connections include ISDN or DSL telephone connections, or other standards that provide a bandwidth more than the equivalent of one telephone line.
  • the present or future internet system such as the present IP format, that permits communication sessions analogous to present telephone system sessions through computer to computer linkages.
  • One specific multiple line arrangement is specifically contemplated whereby the assisted user receives a telephone call from a hearing user over a telephone line, and then that assisted user conferences to the relay to obtain text assistance for the call.
  • the assisted user would communicate with the relay in the UVT format described here.
  • the connection to the relay could be by a second telephone line which carries voice to the relay and transcribed text back to the assisted user, using the UVT format.
  • the connection to the relay could also be accomplished by conferencing in the relay on the line the call was received on and then arranging a separate telephonic connection between the assistance device of the assisted user and the relay to transmit text.
  • UVT protocol for such services.
  • a relay is set up to communicate in UVT protocol, in order to support single line calls, and assuming only that the set-up options of the protocol for the relay are defined to support multiple line calling arrangements, it may be convenient to use this same UVT protocol for the multiple line calls.
  • text only packets can be defined and sent using the same basic format as described here for text and voice packets.
  • the specification of the UVT data packets begins with the basic packet header.
  • the standard packet begins with two special characters, each of one byte (eight bits).
  • the first byte is a specially designated packet initiation signal, indicated here as 0xda, which is the 8 bits hexadecimal character DA. This first byte just indicates the start of a packet.
  • the second special character is a single byte indicating the type or format of data packet. This is the data packet format type and is indicated in the following description using the nomenclature 0x09, which indicates a type 9 (hexadecimal format) packet format.
  • the packet start is the special character, hexadecimal value DA.
  • the second byte is an indication of the packet type, in this case type 9. That packet type is defined as a single byte of text data, followed by two frames each of 80 bits (10 bytes) of digitized speech data.
  • the packet type also defines the type of compression on the speech data, in this case G.729A digitized speech standard.
  • the designation 0xNN indicates any 8 bit value.
  • a type 19 UVT data format is defined as follows: Start packet Packet format Speech format Speech frame 0xda 0x19 0xNN 0xNN . . . 8 bits 8 bits 80 bits 80 bits
  • a type 19 data packet is defined like data packet nine, except that a text character is not included. Thus this data packet is used when no text data needs to be sent, so includes only speech in format G.729A.
  • This UVT protocol is thus able to mix the transmission of both text and voice data. While there is a connection between the parties, packets of voice (or sounds) are continuously transmitted. As the hearing party speaks, the words spoken by the hearing person are transcribed into text at the relay and the relay combines voice data and text data for transmission to the assisted user.
  • This protocol requires very little overhead, as little as two bytes per packet, one start character and one to indicate the packet type. This is preferable to the alternative, sending text and voice in separate packets, since that would require the additional overhead associated with sending another packet (start of packet and packet type) for the text characters.
  • This format permits transmission of voice and text at a steady transmission rate of 9,600 bits per second. This allows the protocol to be used over analog cellular telephone systems that currently support only 9600 baud. At this relatively slow speed, there is simply not enough time to send a speech packet followed by a text data packet.
  • a variable length text only packet This may be used to transmit prompts to the user's device during call set-up or at other times when it is not necessary to carry voice data. Examples of when such a packet might be used include indicating to the assisted user that the captioning device is on-line or indicating when the outbound telephone call has been completed.
  • This type of packet is referred to here as a type ID (again hexadecimal notation) packet, which is specified as follows: Start of Packet packet format Sequence number Length Characters CRC 0xda 0x1d 0xNN 0xNN 0xNN 0xNN 8 bits 8 bits 8 bits 8 bits 8 bits 8 bits 8 bits 8 bits 8 bits ⁇ length 8 bits
  • the sequence number is simply an ordering of the sequence of packets which together form a single message. Length refers to the number of characters in this packet.
  • the notation CRC refers a type of commonly used error-checking methodology (cyclic redundancy check) that may be used to conveniently perform error-checking in this packet type.
  • command and control packet types are used to initiate the service and for other specialized functions. Examples of these are described next.
  • the packet transmitted by the relay service to the calling device, to request the transmission of call set-up information is referred to here as a type 20 packet.
  • the format of a type 20 packet is as follows. Start of packet Packet format 0xda 0x20 8 bits 8 bits
  • a type 21 packet is intended to provide call set-up information to the relay service to specify the type of service and service options to be used on the call. These options can be implemented in the calling device as pre-selected parameters that are automatically transferred to the relay when a defined type of call (e.g. a captioned telephone call) is initiated.
  • a defined type of call e.g. a captioned telephone call
  • the call set-up information is provided in 8 bit ASCII characters. Some of the fields are of variable length and therefore are separated by a special character “;” or 0x3b which is intended only to serve as an indicator of the end of a variable length field.
  • a “CR” (0x0d) character indicates the end of all of the variable length data fields.
  • a CRC is again used to confirm the accuracy of the data within the information block by permitting an error check to be performed.
  • a UVT command packet for DTMF information includes the identification of the DTMF digit tone to be produced, the duration of the tone, and a CRC.
  • the DTMF digit information indicates to the relay which DTMF digit signal to produce, the duration indicates how long the relay equipment should produce the signal and the CRC again is for error checking.
  • the captioned telephone device will continue to send such UVT DTMF command packets to the relay, and the relay will continue to impress DTMF tones on the telephone line to the other party.
  • the user can control the length of time that the DTMF tone is sent.
  • the captioned telephone will normally select a duration for each DTMF command packet that is longer than twice the interval between transmission of DTMF command packets to the relay, so that the DTMF tone continues from the relay even if a single packet is missed or corrupted in some way.
  • the format for packet type 1C (again hexadecimal notation) is as follows: Start of packet Packet format DTMF digit DTMF duration CRC 0xda 0x1c 0xNN 0xNN 0xnn 8 bits 8 bits 8 bits 8 bits 8 bits 80 bits
  • each packet be in the same format. Since the identification of packet type travels with the packet, packets that carry only voice can be interspersed with packets carrying voice data and text without disruption or difficulty. This also permits housekeeping packets, about machine settings or protocols, to be transmitted at the beginning of the communication session, or during lulls, without creating confusion.

Abstract

A communication system and format is described for use in assisted telephonic communications, intended to help users who are hearing impaired use the telephone system. A relay connects a hearing user with the assisted user. The relay creates a text message stream containing the words spoken by the hearing user. The relay then combines the digital characters of the text message with packets of digitized voice spoken by the hearing user and sends the combined digital data packets to the station of the assisted user. The station of the assisted user is capable of separating the voice from the text and displaying the text for reading by the assisted user.

Description

    CROSS-REFERENCE TO RELATED APPLICATIONS
  • None. [0001]
  • STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT
  • None. [0002]
  • BACKGROUND OF THE INVENTION
  • The present invention relates to the general field of telephone communications. In more particular, the invention relates to systems to assist telephone communications by those persons who are deaf, hard of hearing, or otherwise have impaired hearing capability. [0003]
  • Most modem human communications in both social and business environments takes place through sound communications. Yet within modem society there are many persons who have attenuated hearing capability. To assist those persons in making use of our telephonic communication system built for the hearing majority, there has been developed a system of telephone communication which has been principally used by the deaf community. That system makes use of a category of device known variously as a telecommunication device for the deaf (TDD), text telephone (TT) or teletype (TTY). Current TDDs are electronic devices consisting of a keyboard and a display as well as a specific type of modem, to acoustically or directly couple to the telephone line. Modem TDDs permit the user to type characters into their keyboard, with the character strings then encoded and transmitted over the telephone line to be displayed on the display of a communicating or remote TDD device. [0004]
  • Most TDD communication is conducted in an idiosyncratic code specific to the community of TDD users. This code, known as Baudot, evolved historically at a time when many telecommunication devices for the deaf were based on mechanical or electromechanical devices rather than the current technology based on digital electronic components. Accordingly, the Baudot protocol was constructed for a set of constraints which are no longer relevant to present date devices. The original Baudot protocol was a unidirectional or simplex system of communication conducted at 45.5 Baud. The conventional Baudot character set was a character set consisting of 5 bit characters and the system encodes the bits of those characters in a two-tonal system based on carrier tones of 1400 and 1800 Hertz. [0005]
  • The system of TDD communications is widely used and in fact has become indispensable to the deaf community throughout the industrialized world. Deaf persons extensively communicate with their neighbors and with other deaf and hearing people remotely, using the TDD system. In addition, systems have been developed to facilitate the exchange of communication between the deaf community and hearing users who do not have access to or utilize a TDD device. In the United States, telephone companies have set up a service referred to as a “relay.” A relay, as the term is used herein, refers to a system of voice to TDD communication in which an operator, referred to as a “call assistant,” serves as a human intermediary between a hearing user and a deaf person. Normally the call assistant wears a headset that communicates by voice with the hearing user and also has access to a TDD device which can communicate to the deaf user using a TDD appropriate protocol. In normal relay operations in the prior art, the call assistant types at a TDD keyboard the words which are voiced to her by the hearing user and then voices to the hearing user the words that the call assistant sees upon the display of his or her TDD. The call assistant serves, in essence, as an interpreting intermediary between the deaf person and the hearing person to translate from voice to digital electronic forms of communication. [0006]
  • To facilitate and modernize the systems available for providing telecommunication services for the deaf, efforts have been made to both update the techniques for providing assistance to the hearing impaired as well as providing services to users who are modestly hearing impaired but not deaf. In U.S. Pat. No. 5,909,482, a relay is described which uses a re-voicing technique and a speech recognition engine to greatly improved the speed of services provided by a relay. This patent also discloses a small portable device, called a personal interpreter, which make possible providing location independent and instantaneously available interpreting services to the deaf. In U.S. Pat. No. 6,075,842, methods and devices for providing text enhanced telephony are described in which a text stream is provided along with voice in telephone communications with hard of hearing users. The text stream is used to provide the assisted user with a visual representation of the text of what is said by the other person in a communication session, so as to gently assist a person with some hearing deficiency in using the telephone. The full specification of U.S. Pat. Nos. 5,909,482 and 6,075,842, as well of that of each other patent referred to in this document, is incorporated herein by reference. [0007]
  • BRIEF SUMMARY OF THE INVENTION
  • The present invention is summarized in a method for transmitting voice and text of words over a telephonic connection between a hearing user and an assisted user through a relay, the method including the steps of digitizing the voice of the hearing user; creating a digital text at the relay corresponding to the words spoken by the hearing user; combining the digitized voice and the text into combined digital data packets, each packet including a format character indicating the type of format for that packet, at least some of the digital data packets combining at least one byte of digitized voice data with at least one byte of text representing a character in the text of the words spoken by the hearing user; and transmitting the combined packets to the station of the assisted user over a telephone connection so that the station can reconstitute both voice and text from the digital data packets for the assisted user. [0008]
  • The present invention is also summarized in a communication system using that method to communicate voice and text of the words spoken by the voice to a station used by an assisted user. [0009]
  • The present invention is intended to create a flexible communication protocol, using minimal overhead, which is capable of sending voice and the text for the words spoken by that voice, in digital form over common telephonic communication linkages. [0010]
  • Other objects, advantages and features of the present invention will become apparent from the following specification when taken in conjunction with the accompanying drawings.[0011]
  • BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS
  • FIG. 1 is a schematic diagram of a system using the method and protocol of the present invention, showing particular details of the exemplary relay used in the system. [0012]
  • FIG. 2 is a schematic diagram of a captioned telephone for use in the system of FIG. 1.[0013]
  • DETAILED DESCRIPTION OF THE INVENTION
  • The communication protocol of the present invention is intended to facilitate voice and text communications between hearing persons and assisted persons by a relay that intermediates the call. Since the full implementation of the protocol is most appropriately done by a relatively sophisticated relay, the construction of such a relay and some details about the device at the assisted users end will be described first. [0014]
  • Shown in FIG. 1 is a schematic view of the relay which can intermediate such a call. At [0015] 12 is indicated the normal telephone of the hearing user. The telephone 12 connects through a telephonic connection 14 to a DAA and hybrid circuit 16 at the relay. It is envisioned that the hearing user's telephone 12 can be any of the devices generally thought of by laypersons as a telephone, including but not limited to land line telephones, cellular telephones, PCS devices and audio links over the internet. At any event, what is received at the DAA and hybrid circuit 16 of the relay is a voice signal. The voice from the DAA and hybrid circuit 15 is connected as the input to a codec 18. Codecs have become industry standard devices that convert an analog signal, such as analog voice, into digital data. Commercially available codec integrated circuits, such as those from Texas Instruments and Analog Devices, can convert both analog voice signals into digital data and do the reverse, i.e. reconstruct an analog voice signal from digital data representing voice. The output digital signal from the codec 18 is then connected to a circuit or software designed to cancel echo on the telephone line, as indicated at 20.
  • Another type of communication circuit in common use today is referred to as a voice coding and decoding circuit or “vocoder.” A vocoder is a type of digital signal processing chip or an algorithm implemented by a processor specifically designed to transform digital data carrying voice to compress the data for transmission. There are several common standards for vocoders so that telephones from different manufacturers using digital transmission formats can communicate with each other. Such formats include GSM, G.729, and G. 723. One preferred format for the relay of the present invention is G. 729, which encodes speech into 8000 bits per second with an audio quality comparable to a long distance telephone line. This format, and the 8000 bit per second data rate, enables the communication protocol of the present invention to be used even over cellular telephone connections. The vocoder is indicated at [0016] 22, and is capable of both encoding, or compressing, and decoding, or de-compressing, the digital data stream carrying voice.
  • Indicated at [0017] 24 is the UVT formatter, which is actually implemented electronically by a specially programmed microprocessor or digital signal processor. The UVT formatter 24 combines the digital data stream representing voice, from the vocoder 22 with a digital data stream carrying text from a call assistant computer, indicated at 26. The call assistant computer 26 is a general purpose digital computer preferably equipped with a speech recognition software package. The cell assistant wears a headset 28 connected to transmit the voice of the hearing user to the ear of the call assistant. The call assistant repeats, or “re-voices,” the words spoken by the hearing user into a microphone of the headset 28 that is connected to the computer 26. The speech recognition engine in the call assistant computer 26 recognizes the voice of the call assistant and translates that voice into a text stream. The output of the call assistant computer 26 is thus a digital data stream carrying text which is provided to the UVT formatter 24 as well. The UVT formatter is constructed to combine the digital data stream carrying voice with the digital data stream carrying text, using the UVT format described below. The UVT formatter is also capable of doing the reverse, that is, separating the digital data stream carrying voice from the digital data stream carrying text, using information contained in the UVT protocol to make that separation.
  • The output of the [0018] UVT formatter 24 is connected to a modem 30, in this case illustrated as an industry standard V.32bis format modem. The output of the modem 30 is connected through another codec 32 to a hybrid circuit 34 and DAA at the output of the relay. The DAA and hybrid circuit 34 is connected, by any form of telephonic connection 36 to an assisted user station 38. The assisted user station 38 can be a personal interpreter of the general type as shown in U.S. Pat. No. 5,974,116, or a captioned telephone of the general type as shown in U.S. Pat. No. 6,075,842, or any other device intended to assist the assisted user in the communication session by providing text to the assisted user to help that user understand the words spoken by the hearing user.
  • Whether the protocol of the present invention, here sometimes referred to as “UVT,” is used with a personal interpreter or a captioned telephone, the assisted user station needs to be able to receive and process the type of packets sent by the relay. Shown in FIG. 2 is a captioned [0019] telephone device 100 with that capability. In FIG. 2 the interior components of the captioned telephone device 100 are illustrated in block diagrams indicating the digital logic components from which the device may be constructed. It is preferred, however, that the components within the dotted lines in FIG. 2, labeled as DSP software, actually be implemented in the form of a software routine operating a digital signal processing integrated circuit to perform the functions of the illustrated blocks. In the captioned telephone 100, the telephonic connection to the relay is indicated at 102. The input telephonic signal connects to a DAA and hybrid 104 and then to a codec 106 to digitize input signals. Following the codec 106 is modem 108, the output of which connects to a UVT formatter 110. The UVT formatter 110 operates to separate the digitized voice signals from the digital text signals. The digitized text signals are transferred from the UVT formatter to a visually readable display 112 on which the text can be displayed for the assisted user. The digitized voice signal is transferred from the UVT formatter 110 to a vocoder 114, compatible in format to the vocoder used in the relay of FIG. 1, in this case using format G.729A. The output of the vocoder 114 is connected through an acoustic echo control 116 to another codec 118 which reconstructs the analog voice signal for delivery to the handset 120 of the assisted user. In summary, the assisted station decompressed the digital data stream and separates the digital text data from the data representing digitized voice. The text is displayed on the display 112 and the voice is reconstituted into analog and played on the speaker in the handset 120. The assisted user thus receives both the voice of the hearing user and is provided a text display of the words spoken by the hearing user.
  • In the basic operation of the system illustrated in FIG. 1, the hearing user at [0020] telephone 12, and that person's voice is converted into digital form in the relay, that digital form being a direct digitalization of the hearing person's voice. At the same time, or at very nearly the same time, the call assistant computer creates a text data stream, also in digital form, of the text of the words spoken by the hearing person. The UVT protocol, implemented by the UVT formatter 24 in the system of FIG. 1 provides a methodology to efficiently combine the two digital data streams, one for voice one for text. The two digital data streams are combined in a method that is convenient to create and transmit and convenient as well to separate at the receiving end. The problem in combining the two digital data streams is that the digitized voice tends to require much more data to transit than the text data stream. It is inconvenient, however, to interrupt the transmission of voice to transmit the needed text data. It is also important that the total data transmitted not exceed the carrying capacity of any of the forms of telephonic connections supported by the network. The solution described here is to continually send formatted information packets carrying the digitized voice to the remote station and then, in addition and as needed, some of the data packets are flagged to carry a portion of the digital text message data stream in the same packet. Since the data transmission requirements for the text message are so small in comparison to that of the digitized voice, single bytes of text message are combined with multi-byte portions of digitized voice data in these specialized data packets. It is a unique attribute of the data packets described here that the packets carry digitized voice and at the same time carry text data for the words contained in the speech of the voice. In other words, digitized voice and digitized text for the words spoken by that voice are carried in common digital data packets. The text and voice may be delivered simultaneously or near simultaneously. The text can be delivered to the assisted user as the text stream is created by the computer of the call assistant, in which case the text stream may lag the corresponding voice signal by a brief delay. In that event, the text character may not travel in the same packet as the voice for the word of which the character is a part. As an alternative, it is possible to slightly delay the transmission of the voice of the hearing user through the relay so that the text and corresponding voice are transmitted to the assisted user at approximately the same time.
  • To accomplish these unique objectives, a new protocol for voice and text transmission has been designed. This format uses a single format of data packet, sent as a digital data packet, but the packets do not all carry the same type of content. Some packets carry only digitized voice, while other packets carry both digitized voice and digital text data. The packet header is used to indicate the type of packet, and as long as the designation of the packet header remains constant, later revisions of the format permit other later packet specifications to be defined. Other packet types include software upgrade data, user preferences for system set-up, settings for parameters of devices or for configurations, and error code information. Typically, at the initiation of the communication session, special packets are transmitted between the communicating devices. Such special packets can be used to identify the version number of the protocol used by the communicating devices, information on the type of connection and speed, information of the type of data in following data packets, error correcting formats or codes, device status such a processor of memory resources available, or information on the downloading of software upgrades to devices. [0021]
  • Each call to the relay service begins with an exchange of special packets that allow the captioned telephone or personal interpreter to provide the relay service with the information necessary for the relay to automatically set-up the desired type of service and, for captioned telephone calls, to complete the connection to the hearing person. When the data communication connection (e.g. V.32bis) is established, the relay service sends a special packet called a Request for Call Set-up Information packet. The captioned telephone or personal interpreter device responds with a special packet in return, here called the Call Set-up Information Block packet. The Call Set-up Information Block packet includes the service type requested by the user (e.g. captioned telephone or personal interpreting), a user identification number, a user password in some cases, and, for a captioned telephone call, the telephone number of the hearing party. In relays as operated in the prior art, the user had to type in information of this type, in response to inquiries from the call assistant, to complete the call set-up. This protocol permits the call to be set up automatically in a fraction of the time previously required. [0022]
  • Another type of special packet is used to transmit DTMF control signal to the relay equipment. A captioned telephone user may dial, using the relay service, a telephone number that connects to an automated attendant or voice response unit. These automated attendant type devices prompt the caller to enter their choice by pressing the number buttons on the their touch-tone telephone, i.e. “press 1 for sales.” Such devices and voice mail systems are commonly encountered in telephone usage today. In current relays, a TDD user must type instructions to the relay call assistant to convey their intention, e.g. “press 1,” and the call assistant then manually presses a key to produce the DTMF signal on the second line. The delays involved when the TDD user types to the call assistant and the call assistant manually selects the digit often exceeds the time permitted by the automated systems for the user to make a selection. In this event, the traditional relay call assistant must dial back the telephone number and wait for the system to reach the point to enter the user's choice. The problem can result in multiple calls and long time delays for prior art relay users. [0023]
  • The DTMF special packet in the present format provides the captioned telephone user with the functionality of a traditional touch tone telephone in the digital captioned telephone environment. The captioned telephone connects to the relay service using a data communications protocol (e.g. V.32bis) which does not permit the captioned telephone to emit the DTMF tones directly onto the telephone line. The captioned telephone could generate the DTMF tone signal and transmit that signal to the vocoder. Such a signal would be carried as digital data to the relay equipment where it would be converted back to tones. However, most vocoders cannot produce a precise enough set of frequencies or loud enough signals to meet network DTMF standards. To avoid that limitation, the present protocol permits the transmission of the special DTMF packet that instructs the relay to produce the DTMF signal directly on the telephone line to the other party. To use this feature, the captioned telephone user simply presses the numerical dial of his or her telephone, and the captioned telephone then generates a command packet to the relay instructing it to produce the correct DTMF tones on the second telephone line automatically without interaction with the call assistant. [0024]
  • The embodiments of FIGS. 1 and 2 are illustrated with telephonic connections between the hearing person and the relay and between the relay and the assisted user. It is specifically contemplated that the manner of actual telephonic connection between these parties, and the number of telephone lines or telephone line equivalents that are used is not important. The term telephonic connection, as used here, is intended to apply to actual dedicated connections through the telephone system, such as land-lines or analog cellular connections. The term is also intended to encompass other types of connections that can serve as telephonic connections in the lay sense of the term, such as digital cellular telephone service, PCS service and communication over the internet using IP protocol. The term telephone line here is intended to encompass both traditional twisted pair physical telephone lines as well as any type of channels or software sockets that provide an equivalent connection between users of the telephone system. So while the use of the protocol described here is particularly intended to make possible transmission of text and voice using a minimal number of telephone lines, it is envisioned that the protocol may be used using two or more telephone lines or over other higher bandwidth forms of interconnection. Examples of such higher bandwidth connections include ISDN or DSL telephone connections, or other standards that provide a bandwidth more than the equivalent of one telephone line. Another example is the use of the present or future internet system, such as the present IP format, that permits communication sessions analogous to present telephone system sessions through computer to computer linkages. [0025]
  • One specific multiple line arrangement is specifically contemplated whereby the assisted user receives a telephone call from a hearing user over a telephone line, and then that assisted user conferences to the relay to obtain text assistance for the call. The assisted user would communicate with the relay in the UVT format described here. The connection to the relay could be by a second telephone line which carries voice to the relay and transcribed text back to the assisted user, using the UVT format. The connection to the relay could also be accomplished by conferencing in the relay on the line the call was received on and then arranging a separate telephonic connection between the assistance device of the assisted user and the relay to transmit text. While these sorts of connections might not make use of all of the capabilities of the UVT format, since they may not require voice and text on the same telephone line (or in the same packets), it may still be useful to employ the UVT protocol for such services. Once a relay is set up to communicate in UVT protocol, in order to support single line calls, and assuming only that the set-up options of the protocol for the relay are defined to support multiple line calling arrangements, it may be convenient to use this same UVT protocol for the multiple line calls. In that event, text only packets can be defined and sent using the same basic format as described here for text and voice packets. [0026]
  • The specification of the UVT data packets begins with the basic packet header. The standard packet begins with two special characters, each of one byte (eight bits). The first byte is a specially designated packet initiation signal, indicated here as 0xda, which is the 8 bits hexadecimal character DA. This first byte just indicates the start of a packet. The second special character is a single byte indicating the type or format of data packet. This is the data packet format type and is indicated in the following description using the nomenclature 0x09, which indicates a type 9 (hexadecimal format) packet format. [0027]
  • Thus the structure of a type 9 data packet is as follows: [0028]
    Speech Speech
    Start of packet Packet format Text character frame 1 frame 2
    0xda 0x09 0nNN 0xNN . . . 0xNN . . .
    8 bits 8 bits 8 bits 80 bits 80 bits
  • In this representation, the packet start is the special character, hexadecimal value DA. The second byte is an indication of the packet type, in this case type 9. That packet type is defined as a single byte of text data, followed by two frames each of 80 bits (10 bytes) of digitized speech data. The packet type also defines the type of compression on the speech data, in this case G.729A digitized speech standard. The designation 0xNN indicates any 8 bit value. [0029]
  • A type [0030] 19 UVT data format is defined as follows:
    Start packet Packet format Speech format Speech frame
    0xda 0x19 0xNN 0xNN . . .
    8 bits 8 bits 80 bits 80 bits
  • A type 19 data packet is defined like data packet nine, except that a text character is not included. Thus this data packet is used when no text data needs to be sent, so includes only speech in format G.729A. [0031]
  • This UVT protocol is thus able to mix the transmission of both text and voice data. While there is a connection between the parties, packets of voice (or sounds) are continuously transmitted. As the hearing party speaks, the words spoken by the hearing person are transcribed into text at the relay and the relay combines voice data and text data for transmission to the assisted user. This protocol requires very little overhead, as little as two bytes per packet, one start character and one to indicate the packet type. This is preferable to the alternative, sending text and voice in separate packets, since that would require the additional overhead associated with sending another packet (start of packet and packet type) for the text characters. This format permits transmission of voice and text at a steady transmission rate of 9,600 bits per second. This allows the protocol to be used over analog cellular telephone systems that currently support only 9600 baud. At this relatively slow speed, there is simply not enough time to send a speech packet followed by a text data packet. [0032]
  • The fact that text and speech packets are combined means that loss of a packet is not a significant problem. Since each packet only contains a time period of 0.010 to 0.020 seconds, and the omission of the speech from such a time period would not usually be noticed by the hearing users. Since loss of a packet means loss of only a single text byte, only one character is lost from the text data transmitted to the assisted user. Assisted users are accustomed to correcting mentally for informal and erroneous spellings, and so in most instances the loss of a single character should not cause serious disruption of the conversation. If the text was sent in separate packets, the loss of a packet could lose a significant amount of text information. [0033]
  • Larger blocks of text can be sent in a variable length text only packet. This may be used to transmit prompts to the user's device during call set-up or at other times when it is not necessary to carry voice data. Examples of when such a packet might be used include indicating to the assisted user that the captioning device is on-line or indicating when the outbound telephone call has been completed. This type of packet is referred to here as a type ID (again hexadecimal notation) packet, which is specified as follows: [0034]
    Start of Packet
    packet format Sequence number Length Characters CRC
    0xda 0x1d 0xNN 0xNN 0xNN 0xNN
    8 bits 8 bits 8 bits 8 bits 8 bits × length 8 bits
  • The sequence number is simply an ordering of the sequence of packets which together form a single message. Length refers to the number of characters in this packet. The notation CRC refers a type of commonly used error-checking methodology (cyclic redundancy check) that may be used to conveniently perform error-checking in this packet type. [0035]
  • Other types of command and control packet types are used to initiate the service and for other specialized functions. Examples of these are described next. [0036]
  • The packet transmitted by the relay service to the calling device, to request the transmission of call set-up information, is referred to here as a [0037] type 20 packet. The format of a type 20 packet is as follows.
    Start of packet Packet format
    0xda 0x20
    8 bits 8 bits
  • When the relay service sends the [0038] type 20 packet to the calling device, the calling device should respond with a type 21 packet. A type 21 packet is intended to provide call set-up information to the relay service to specify the type of service and service options to be used on the call. These options can be implemented in the calling device as pre-selected parameters that are automatically transferred to the relay when a defined type of call (e.g. a captioned telephone call) is initiated. The format of the type 21 packet, or call set-up information block packet, is as follows:
    Start Dial
    of Packet through Service ID
    packet format number ; type ; number ; Password CR CRC
    Oxda 0x21 DT = 1608 0x3b S = 2 0x3b U = 1234 0x3b P = x456 0x0d 0xNN
    2385400 5678
    8 bits 8 bits Variable 8 bits 8 bits 8 bits Variable 8 bits Variable 8 bits 8 bits
  • In this format, the call set-up information is provided in 8 bit ASCII characters. Some of the fields are of variable length and therefore are separated by a special character “;” or 0x3b which is intended only to serve as an indicator of the end of a variable length field. A “CR” (0x0d) character indicates the end of all of the variable length data fields. Each data filed begins with a filed type indicator, such as the indicator “DT=” indicates that the characters following make up a dial through telephone number. This allows the fields to be sent in any order and permits unused fields to be omitted altogether. It is then also possible to define new filed types at a later time by selecting a new filed type indicator and separating the new filed from others by the “;” indication. A CRC is again used to confirm the accuracy of the data within the information block by permitting an error check to be performed. [0039]
  • A UVT command packet for DTMF information includes the identification of the DTMF digit tone to be produced, the duration of the tone, and a CRC. The DTMF digit information indicates to the relay which DTMF digit signal to produce, the duration indicates how long the relay equipment should produce the signal and the CRC again is for error checking. When the captioned telephone user presses a number button on his or her device during a captioned telephone call, the captioned telephone device sends a UVT DTMF command packet to the relay. The relay then generates the DTMF tone on the telephone line to the other party. This tone will sound to all users like the normal DTMF tones produced by a conventional telephone. As long as the user holds down the button, the captioned telephone device will continue to send such UVT DTMF command packets to the relay, and the relay will continue to impress DTMF tones on the telephone line to the other party. Thus, as in traditional telephone systems, the user can control the length of time that the DTMF tone is sent. The captioned telephone will normally select a duration for each DTMF command packet that is longer than twice the interval between transmission of DTMF command packets to the relay, so that the DTMF tone continues from the relay even if a single packet is missed or corrupted in some way. The format for packet type 1C (again hexadecimal notation) is as follows: [0040]
    Start of packet Packet format DTMF digit DTMF duration CRC
    0xda 0x1c 0xNN 0xNN 0xnn
    8 bits 8 bits 8 bits 8 bits 80 bits
  • Note that is not required that each packet be in the same format. Since the identification of packet type travels with the packet, packets that carry only voice can be interspersed with packets carrying voice data and text without disruption or difficulty. This also permits housekeeping packets, about machine settings or protocols, to be transmitted at the beginning of the communication session, or during lulls, without creating confusion. [0041]
  • It is to be understood that the present invention is not limited to the embodiment described above, but embraces all such modified forms thereof as come within the scope of the following claims. [0042]

Claims (21)

We claim:
1. A method for transmitting voice and text of words over a telephonic connection between a hearing user and an assisted user through a relay, the method comprising the steps of
digitizing the voice of the hearing user;
creating a digital text at the relay corresponding to the words spoken by the hearing user;
combining the digitized voice and the digital text into combined digital data packets, at least some of the digital data packets combining at least one byte of digitized voice data with at least one byte of text representing a character in the text of the words spoken by the hearing user; and
transmitting the combined packets to the station of the assisted user over a telephone connection so that the station can reconstitute both voice and text from the digital data packets for the assisted user.
2. A method as claimed in claim 1 wherein for each digital data packet, the first byte is a hexadecimal DA.
3. A method as claimed in claim 1 wherein each data packet includes within it a format character indicating the format of that packet.
4. A method as claimed in claim 3 wherein for each digital data packet, the second byte is the format character indicating the format of that digital data packet.
5. A method as claimed in claim 3 wherein some of the data packets contains only digitized voice and some of the data packets combine digitized voice with a text character, the nature of the data in each packet indicated by the format character.
6. A method as claimed in claim 3 wherein at least one data packet is defined to carry call set-up information from the assisted user to the relay.
7. A method as claimed in claim 3 wherein at least one format of data packet provides for the transmission of a DTMF command from the assisted user to the relay to command the relay to transmit a DTMF tone on the telephone connection to the hearing user.
8. A system for assisting a user in telephonic communications with a hearing user, the system adapted to communicate with an assisted user station capable of displaying text for the assisted user, the system comprising
a relay capable of converting spoken voice received over the first telephonic connection into text;
a first telephonic connection between the hearing user and a relay;
a second telephonic connection between the relay and the station of the assisted user;
the relay programmed to created a series of digital data packets for transmission to the station of the assisted user over the second telephonic connection, each of the digital packets including digitized voice of the hearing users and at least some of the digital data packets including text of the words spoken by the hearing user.
9. A system as claimed in claim 8 wherein for each digital data packet, the first byte is a hexadecimal DA.
10. A system as claimed in claim 8 wherein each packet includes a format character indicating the format of that particular packet so that packets of varying format can be transmitted in a single communication session.
11. A system as claimed in claim 10 wherein for each digital data packet, the second byte is the format character indicating the format of that digital data packet.
12. A system as claimed in claim 10 wherein some of the data packets contains only digitized voice and some of the data packets combine digitized voice with a text character, the nature of the data in each packet indicated by the format character.
13. A method as claimed in claim 10 wherein at least one data packet is defined to carry call set-up information from the assisted user to the relay, the content of that packet being indicated by the format character in the packet.
14. A method as claimed in claim 10 wherein at least one format of data packet provides for the transmission of a DTMF command from the assisted user to the relay to command the relay to transmit a DTMF tone on the telephone connection to the hearing user.
15. A system as claimed in claim 8 wherein the station of the assisted user is a captioned telephone, operating as an analog telephone but also capable of displaying the text of the words spoken by the hearing user for the benefit of the assisted user.
16. A system as claimed in claim 8 wherein the station of the assisted user is a portable personal interpreted device capable of providing the assisted user with a visual display of the text created at the relay from the words spoken by the hearing user.
17. A method capable of transmitting both voice and text of words over a telephonic connection to facilitate a conversation between two users, the method comprising the steps of
digitizing the voice of a first of the users;
generating a digital text corresponding to the words spoken by the first of the users;
combining the digitized voice and the digital text into digital data packets, at least some of the digital data packets having at least one byte of digitized voice data and at least some of the digital data packets having at least one byte of text representing a character in the text of the words spoken by the first user, each packet including a format character indicating the format of that packet so that packets of differing format can be sent in a single communication session; and
transmitting the combined packets to the station of the second user over a telephone connection so that the station of the second user can supply text to the second user of the words spoken by the first user.
18. A method as claimed in claim 17 wherein for each digital data packet, the first byte is a hexadecimal DA.
19. A method as claimed in claim 17 wherein for each digital data packet, the second byte is the format character indicating the format of that digital data packet.
20. A system as claimed in claim 17 wherein some of the data packets contains only digitized voice and some of the data packets combine digitized voice with a text character, the nature of the data in each packet indicated by the format character.
21. A method as claimed in claim 17 wherein at least one data packet is defined to carry call set-up information from the assisted user to the relay, the content of that packet being indicated by the format character in the packet.
US10/336,950 2001-06-07 2003-01-03 Voice and text transmission system Abandoned US20030125952A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US10/336,950 US20030125952A1 (en) 2001-06-07 2003-01-03 Voice and text transmission system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US09/876,340 US6504910B1 (en) 2001-06-07 2001-06-07 Voice and text transmission system
US10/336,950 US20030125952A1 (en) 2001-06-07 2003-01-03 Voice and text transmission system

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US09/876,340 Continuation US6504910B1 (en) 2001-06-07 2001-06-07 Voice and text transmission system

Publications (1)

Publication Number Publication Date
US20030125952A1 true US20030125952A1 (en) 2003-07-03

Family

ID=25367483

Family Applications (2)

Application Number Title Priority Date Filing Date
US09/876,340 Expired - Lifetime US6504910B1 (en) 2001-06-07 2001-06-07 Voice and text transmission system
US10/336,950 Abandoned US20030125952A1 (en) 2001-06-07 2003-01-03 Voice and text transmission system

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US09/876,340 Expired - Lifetime US6504910B1 (en) 2001-06-07 2001-06-07 Voice and text transmission system

Country Status (4)

Country Link
US (2) US6504910B1 (en)
CA (1) CA2419150C (en)
GB (1) GB2382497B (en)
WO (1) WO2002100084A1 (en)

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030163310A1 (en) * 2002-01-22 2003-08-28 Caldwell Charles David Method and device for providing speech-to-text encoding and telephony service
US20050153718A1 (en) * 2004-01-14 2005-07-14 International Business Machines Corporation Apparatus, system and method of delivering a text message to a landline telephone
US20070165598A1 (en) * 2003-09-27 2007-07-19 Hynonen Olli M Intelligent multimedia calls
US20080319745A1 (en) * 2002-01-22 2008-12-25 At&T Corp. Method and device for providing speech-to-text encoding and telephony service
US20090198497A1 (en) * 2008-02-04 2009-08-06 Samsung Electronics Co., Ltd. Method and apparatus for speech synthesis of text message
US7656861B2 (en) 2004-07-09 2010-02-02 Cisco Technology, Inc. Method and apparatus for interleaving text and media in a real-time transport session
US20100088613A1 (en) * 2008-10-03 2010-04-08 Lisa Seacat Deluca Voice response unit proxy utilizing dynamic web interaction
US7792143B1 (en) * 2005-03-25 2010-09-07 Cisco Technology, Inc. Method and apparatus for interworking dissimilar text phone protocols over a packet switched network
US20110053587A1 (en) * 2009-09-03 2011-03-03 John Turk Changing parameters in a telecommunications system
US9002937B2 (en) 2011-09-28 2015-04-07 Elwha Llc Multi-party multi-modality communication
US20150100315A1 (en) * 2013-08-29 2015-04-09 Vonage Network, Llc Methods and apparatus for conducting internet protocol telephony communications
US9230546B2 (en) 2011-11-03 2016-01-05 International Business Machines Corporation Voice content transcription during collaboration sessions
US9324324B2 (en) 2014-05-22 2016-04-26 Nedelco, Inc. Adaptive telephone relay service systems
US9477943B2 (en) 2011-09-28 2016-10-25 Elwha Llc Multi-modality communication
US9503550B2 (en) 2011-09-28 2016-11-22 Elwha Llc Multi-modality communication modification
US9699632B2 (en) 2011-09-28 2017-07-04 Elwha Llc Multi-modality communication with interceptive conversion
US9762524B2 (en) 2011-09-28 2017-09-12 Elwha Llc Multi-modality communication participation
US9788349B2 (en) 2011-09-28 2017-10-10 Elwha Llc Multi-modality communication auto-activation
US9906927B2 (en) 2011-09-28 2018-02-27 Elwha Llc Multi-modality communication initiation
US10263665B2 (en) * 2012-12-26 2019-04-16 Icom Incorporated Communication system

Families Citing this family (72)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6594346B2 (en) * 1997-09-08 2003-07-15 Ultratec, Inc. Relay for personal interpreter
US6603835B2 (en) 1997-09-08 2003-08-05 Ultratec, Inc. System for text assisted telephony
US6181694B1 (en) 1998-04-03 2001-01-30 Vertical Networks, Inc. Systems and methods for multiple mode voice and data communciations using intelligently bridged TDM and packet buses
US6389009B1 (en) 2000-12-28 2002-05-14 Vertical Networks, Inc. Systems and methods for multiple mode voice and data communications using intelligently bridged TDM and packet buses
US6775360B2 (en) * 2000-12-28 2004-08-10 Intel Corporation Method and system for providing textual content along with voice messages
US20030018793A1 (en) * 2001-07-19 2003-01-23 Oscar Mora Reliable transport layer protocol in low performance 8-bit microcontrollers
US7881441B2 (en) * 2005-06-29 2011-02-01 Ultratec, Inc. Device independent text captioned telephone service
US8416925B2 (en) * 2005-06-29 2013-04-09 Ultratec, Inc. Device independent text captioned telephone service
US7233655B2 (en) * 2001-10-03 2007-06-19 Accenture Global Services Gmbh Multi-modal callback
US7472091B2 (en) * 2001-10-03 2008-12-30 Accenture Global Services Gmbh Virtual customer database
US7640006B2 (en) * 2001-10-03 2009-12-29 Accenture Global Services Gmbh Directory assistance with multi-modal messaging
US7441016B2 (en) * 2001-10-03 2008-10-21 Accenture Global Services Gmbh Service authorizer
US7254384B2 (en) * 2001-10-03 2007-08-07 Accenture Global Services Gmbh Multi-modal messaging
US7286651B1 (en) * 2002-02-12 2007-10-23 Sprint Spectrum L.P. Method and system for multi-modal interaction
US7218629B2 (en) * 2002-07-01 2007-05-15 Lonverged Data Solutions Llc Methods for initiating telephone communications using a telephone number extracted from user-highlighted content on a computer
US7379431B2 (en) * 2002-07-10 2008-05-27 Avaya Technology Corp. Error correction method and apparatus for TTY on VoIP transmissions
US7535922B1 (en) * 2002-09-26 2009-05-19 At&T Intellectual Property I, L.P. Devices, systems and methods for delivering text messages
US7376127B2 (en) * 2003-05-12 2008-05-20 Avaya Technology Corp. Methods for reconstructing missing packets in TTY over voice over IP transmission
US20050086699A1 (en) * 2003-10-16 2005-04-21 Hamilton Relay, Inc. Video relay system and method
US7406414B2 (en) * 2003-12-15 2008-07-29 International Business Machines Corporation Providing translations encoded within embedded digital information
CA2556933C (en) * 2004-02-18 2013-10-22 Ultratec, Inc. Captioned telephone service
US8515024B2 (en) 2010-01-13 2013-08-20 Ultratec, Inc. Captioned telephone service
US7245705B2 (en) * 2004-02-26 2007-07-17 Hamilton Relay, Inc. Internet protocol (IP) relay system and method
US20050226398A1 (en) * 2004-04-09 2005-10-13 Bojeun Mark C Closed Captioned Telephone and Computer System
US20060158357A1 (en) * 2005-01-19 2006-07-20 Visteon Global Technologies, Inc. Text compression method for multi-level display
KR20080004553A (en) * 2005-03-31 2008-01-09 프랑스 텔레콤 Method of copying voice messages in the form of text messages in a packet communication network
US11258900B2 (en) 2005-06-29 2022-02-22 Ultratec, Inc. Device independent text captioned telephone service
US20070048076A1 (en) * 2005-08-31 2007-03-01 To Chun Y Fastening system for a ring binder mechanism
US7620158B2 (en) 2005-09-02 2009-11-17 Hamilton Relay, Inc. Video relay system and method
KR100747874B1 (en) * 2006-02-13 2007-08-08 주식회사 팬택앤큐리텔 A method for transmitting a message by encoding the message into dtmf signals in communication terminal
US20080144134A1 (en) * 2006-10-31 2008-06-19 Mohamed Nooman Ahmed Supplemental sensory input/output for accessibility
US8000969B2 (en) 2006-12-19 2011-08-16 Nuance Communications, Inc. Inferring switching conditions for switching between modalities in a speech application environment extended for interactive text exchanges
US8526581B2 (en) * 2008-07-03 2013-09-03 C21 Patents, Llc Internet protocol text relay for hearing impaired users
US9124716B1 (en) 2008-07-03 2015-09-01 C21 Patents, Llc Internet protocol text relay for hearing impaired users
US8379801B2 (en) 2009-11-24 2013-02-19 Sorenson Communications, Inc. Methods and systems related to text caption error correction
KR20130057338A (en) * 2011-11-23 2013-05-31 김용진 Method and apparatus for providing voice value added service
US9344562B2 (en) 2012-11-30 2016-05-17 At&T Intellectual Property I, Lp Apparatus and method for managing interactive television and voice communication services
US9208784B2 (en) 2013-01-08 2015-12-08 C21 Patents, Llc Methododolgy for live text broadcasting
US9406296B2 (en) 2013-04-19 2016-08-02 C21 Patents, Llc Two way automatic universal transcription telephone
US8914284B1 (en) * 2013-08-29 2014-12-16 Vonage Networks, LLC Methods and apparatus for conducting internet protocol telephony communication
US9191789B2 (en) 2013-10-02 2015-11-17 Captioncall, Llc Systems and methods for using a caption device with a mobile device
US9191493B2 (en) 2013-12-09 2015-11-17 Captioncall, Llc Methods and devices for updating an adaptive filter for echo cancellation
US10878721B2 (en) 2014-02-28 2020-12-29 Ultratec, Inc. Semiautomated relay method and apparatus
US10748523B2 (en) 2014-02-28 2020-08-18 Ultratec, Inc. Semiautomated relay method and apparatus
US10389876B2 (en) 2014-02-28 2019-08-20 Ultratec, Inc. Semiautomated relay method and apparatus
US20180270350A1 (en) 2014-02-28 2018-09-20 Ultratec, Inc. Semiautomated relay method and apparatus
US20180034961A1 (en) 2014-02-28 2018-02-01 Ultratec, Inc. Semiautomated Relay Method and Apparatus
US9893938B1 (en) 2014-10-31 2018-02-13 Sorenson Ip Holdings, Llc Apparatuses and methods for distributed configuration management of communication devices
US9565231B1 (en) 2014-11-11 2017-02-07 Sprint Spectrum L.P. System and methods for providing multiple voice over IP service modes to a wireless device in a wireless network
US10187894B1 (en) 2014-11-12 2019-01-22 Sprint Spectrum L.P. Systems and methods for improving voice over IP capacity in a wireless network
US9247052B1 (en) 2014-11-14 2016-01-26 Captioncall, Llc Apparatuses and methods for routing digital voice data in a communication system for hearing-impaired users
US9946842B1 (en) 2015-03-02 2018-04-17 Sorenson Ip Holdings, Llc Methods and systems for routing patient information to a communication service provider and to a communication device for hearing-impaired users
US9197745B1 (en) 2015-03-25 2015-11-24 Captioncall, Llc Communication device and related methods for automatically connecting to a captioning communication service to receive text captions following an interruption during a call
US9258415B1 (en) 2015-03-30 2016-02-09 Captioncall, Llc Communication device and related methods for offline data record modification and synchronization with a backend server associated with a relay service for hearing-impaired users
US9191494B1 (en) 2015-04-06 2015-11-17 Captioncall, Llc Device, system, and method for performing echo cancellation in different modes of a communication device
US9479650B1 (en) 2015-05-04 2016-10-25 Captioncall, Llc Methods and devices for updating filter coefficients during echo cancellation
US9686404B1 (en) 2015-08-05 2017-06-20 Sorenson Ip Holdings, Llc Methods and devices for automatically connecting to a communication service through a password protected network connection
US9380150B1 (en) 2015-09-16 2016-06-28 Captioncall, Llc Methods and devices for automatic volume control of a far-end voice signal provided to a captioning communication service
US9525830B1 (en) 2015-11-12 2016-12-20 Captioncall Llc Captioning communication systems
US9374536B1 (en) 2015-11-12 2016-06-21 Captioncall, Llc Video captioning communication system, devices and related methods for captioning during a real-time video communication session
US9614973B1 (en) 2016-06-16 2017-04-04 Sorenson Ip Holdings, Llc Voice over internet protocol credentials
US9503568B1 (en) 2016-06-30 2016-11-22 Captioncall, Llc Detecting dial tone on a telephone line
US9497315B1 (en) 2016-07-27 2016-11-15 Captioncall, Llc Transcribing audio communication sessions
US9679578B1 (en) 2016-08-31 2017-06-13 Sorenson Ip Holdings, Llc Signal clipping compensation
US9628596B1 (en) 2016-09-09 2017-04-18 Sorenson Ip Holdings, Llc Electronic device including a directional microphone
US9674351B1 (en) * 2016-10-06 2017-06-06 Sorenson Ip Holdings, Llc Remote voice recognition
US10833920B1 (en) 2019-12-12 2020-11-10 Sorenson Ip Holdings, Llc Network device maintenance
US11276392B2 (en) 2019-12-12 2022-03-15 Sorenson Ip Holdings, Llc Communication of transcriptions
US11539900B2 (en) 2020-02-21 2022-12-27 Ultratec, Inc. Caption modification and augmentation systems and methods for use by hearing assisted user
US11321047B2 (en) 2020-06-11 2022-05-03 Sorenson Ip Holdings, Llc Volume adjustments
US11659078B2 (en) * 2020-10-19 2023-05-23 Sorenson Ip Holdings, Llc Presentation of communications
US11368585B1 (en) 2021-03-23 2022-06-21 International Business Machines Corporation Secured switch for three-way communications

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6075842A (en) * 1988-10-11 2000-06-13 Ultratec, Inc. Text enhanced telephony
US5724405A (en) * 1988-10-11 1998-03-03 Ultratec, Inc. Text enhanced telephony

Cited By (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9361888B2 (en) 2002-01-22 2016-06-07 At&T Intellectual Property Ii, L.P. Method and device for providing speech-to-text encoding and telephony service
US8265931B2 (en) 2002-01-22 2012-09-11 At&T Intellectual Property Ii, L.P. Method and device for providing speech-to-text encoding and telephony service
US8489397B2 (en) 2002-01-22 2013-07-16 At&T Intellectual Property Ii, L.P. Method and device for providing speech-to-text encoding and telephony service
US20060106602A1 (en) * 2002-01-22 2006-05-18 At&T Corp. Method and Device for Providing Speech-to-Text Encoding and Telephony Service
US7027986B2 (en) * 2002-01-22 2006-04-11 At&T Corp. Method and device for providing speech-to-text encoding and telephony service
US7433818B2 (en) * 2002-01-22 2008-10-07 At&T Corp. Subscriber terminal for providing speech-text encoding and telephony service
US20080319745A1 (en) * 2002-01-22 2008-12-25 At&T Corp. Method and device for providing speech-to-text encoding and telephony service
US20030163310A1 (en) * 2002-01-22 2003-08-28 Caldwell Charles David Method and device for providing speech-to-text encoding and telephony service
US8139560B2 (en) * 2003-09-27 2012-03-20 Telefonaktiebolaget L M Ericsson (Publ) Intelligent multimedia calls
US20070165598A1 (en) * 2003-09-27 2007-07-19 Hynonen Olli M Intelligent multimedia calls
US20050153718A1 (en) * 2004-01-14 2005-07-14 International Business Machines Corporation Apparatus, system and method of delivering a text message to a landline telephone
US7656861B2 (en) 2004-07-09 2010-02-02 Cisco Technology, Inc. Method and apparatus for interleaving text and media in a real-time transport session
US7792143B1 (en) * 2005-03-25 2010-09-07 Cisco Technology, Inc. Method and apparatus for interworking dissimilar text phone protocols over a packet switched network
US20090198497A1 (en) * 2008-02-04 2009-08-06 Samsung Electronics Co., Ltd. Method and apparatus for speech synthesis of text message
US20100088613A1 (en) * 2008-10-03 2010-04-08 Lisa Seacat Deluca Voice response unit proxy utilizing dynamic web interaction
US9003300B2 (en) * 2008-10-03 2015-04-07 International Business Machines Corporation Voice response unit proxy utilizing dynamic web interaction
US20110053587A1 (en) * 2009-09-03 2011-03-03 John Turk Changing parameters in a telecommunications system
US8626075B2 (en) * 2009-09-03 2014-01-07 Vodafone Ip Licensing Limited Changing parameters in a telecommunications system
US9477943B2 (en) 2011-09-28 2016-10-25 Elwha Llc Multi-modality communication
US9002937B2 (en) 2011-09-28 2015-04-07 Elwha Llc Multi-party multi-modality communication
US9503550B2 (en) 2011-09-28 2016-11-22 Elwha Llc Multi-modality communication modification
US9699632B2 (en) 2011-09-28 2017-07-04 Elwha Llc Multi-modality communication with interceptive conversion
US9762524B2 (en) 2011-09-28 2017-09-12 Elwha Llc Multi-modality communication participation
US9788349B2 (en) 2011-09-28 2017-10-10 Elwha Llc Multi-modality communication auto-activation
US9794209B2 (en) 2011-09-28 2017-10-17 Elwha Llc User interface for multi-modality communication
US9906927B2 (en) 2011-09-28 2018-02-27 Elwha Llc Multi-modality communication initiation
US9230546B2 (en) 2011-11-03 2016-01-05 International Business Machines Corporation Voice content transcription during collaboration sessions
US10263665B2 (en) * 2012-12-26 2019-04-16 Icom Incorporated Communication system
US20150100315A1 (en) * 2013-08-29 2015-04-09 Vonage Network, Llc Methods and apparatus for conducting internet protocol telephony communications
US9767802B2 (en) * 2013-08-29 2017-09-19 Vonage Business Inc. Methods and apparatus for conducting internet protocol telephony communications
US9324324B2 (en) 2014-05-22 2016-04-26 Nedelco, Inc. Adaptive telephone relay service systems

Also Published As

Publication number Publication date
GB2382497B (en) 2004-04-28
WO2002100084A1 (en) 2002-12-12
GB0305451D0 (en) 2003-04-16
CA2419150A1 (en) 2002-12-12
CA2419150C (en) 2011-04-12
US6504910B1 (en) 2003-01-07
GB2382497A (en) 2003-05-28

Similar Documents

Publication Publication Date Title
US6504910B1 (en) Voice and text transmission system
US8917822B2 (en) System for text assisted telephony
US8265931B2 (en) Method and device for providing speech-to-text encoding and telephony service
US7433818B2 (en) Subscriber terminal for providing speech-text encoding and telephony service
US6510206B2 (en) Relay for personal interpreter
US6263202B1 (en) Communication system and wireless communication terminal device used therein
JP3237566B2 (en) Call method, voice transmitting device and voice receiving device
US20020064256A1 (en) Text enhanced telephony
JPH10229448A (en) Text/voice mutual conversion system
AU2002313798A1 (en) System for text assisted telephony

Legal Events

Date Code Title Description
STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION