|Publication number||US7881234 B2|
|Application number||US 11/550,832|
|Publication date||Feb 1, 2011|
|Filing date||Oct 19, 2006|
|Priority date||Oct 19, 2006|
|Also published as||US20080095348|
|Publication number||11550832, 550832, US 7881234 B2, US 7881234B2, US-B2-7881234, US7881234 B2, US7881234B2|
|Inventors||Michael Negley Abernethy, Jr., Travis M. Grigsby, Justin K Hong|
|Original Assignee||International Business Machines Corporation|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (11), Referenced by (7), Classifications (5), Legal Events (4)|
|External Links: USPTO, USPTO Assignment, Espacenet|
The present invention relates to computer controlled implementations for tracking of telephone and like audio conversations and conferences between a plurality of participants, and particularly to preventing interrupted conversation threads from being overlooked.
With the globalization of business, industry and trade wherein transactions and activities within these fields have been changing from localized organizations to diverse transactions over the face of the world, the telecommunications industries have been expanding rapidly. This was, of course, accelerated by the rapid expansion of the World Wide Web (Web) that gave rise to Internet Protocol (IP) telecommunications wherein voice and other audio telecommunications are transmitted over the Internet. In addition, restrictions on travel, as well as attempts at energy conservation, have made teleconferencing more attractive.
With this expansion of telephone channels, conferences and conversations throughout the world, involving a plurality of participants, have become part of the daily routine in most business, educational and governmental institutions. However, in view of language, cultural and time differences, participants frequently find such conferences and conversations difficult to monitor and control as to achieve the purposes of the participants. As a result, the telecommunications industry is seeking implementations for making telephone conversations and conferences easier on the participants.
The present invention provides an implementation for the handling of interrupts in the conversation of a participant resulting from overlapping conversation streams, e.g. two participants speaking at once or one participant starting to speak before the other has completed his conversation. The invention recognizes that in conversations, and particularly in conversations involving a number of participants as in a conference telephone situation, or where the participants have some language differences, it is quite easy for a participant to forget a fragment of his statement that has been overridden as a result of an interrupt. The present invention is directed to implementations for prompting the participant so that he does not forget or overlook the overridden speech fragment.
Accordingly, the present invention provides an implementation for detecting interruptions in audio conversations between a plurality of participants comprising continuously recording the most recent predetermined time segment of the conversation stream of one participant in the conversation, monitoring the conversation stream of the one participant for interruptions in the conversation stream and storing the most recent segment prior to the interruption in response to a detection of an interruption. The segment is represented by a conversation marker that is displayed. The user may then select the conversation marker to playback this most recent segment so that he may decide whether he wishes to pursue the interrupted conversation fragment.
The invention may be effectively implemented in telephones having built-in displays, such as mobile wireless, e.g. cellular telephones. The invention is particularly effective in telephones with associated laptop or desktop displays that function via the Internet using IP telecommunication protocols.
Also, playing back the most recent segment may be to other participants in the teleconference in addition to the one participant whose conversation is being monitored.
The fundamentals of the present invention are also applicable to conferences other than telephone conferences. It is now becoming customary to equip conferees in an auditorium or conference room with a computer controlled display speech recognition system. The present invention may be used to monitor the audio in such an environment.
The present invention will be better understood and its numerous objects and advantages will become more apparent to those skilled in the art by reference to the following drawings, in conjunction with the accompanying specification, in which:
In the conversation stream shown in balloons 16 and 17, and more clearly in time line 30, conversation statement 7 by Tom: “Look, I really . . . .” is interrupted by statement 8 by Max: “Did you hear about . . . ”. The recorded seven second segment associated with statement 7 is stored as a conversation marker on Tom's computer 15. With respect to item 11 shown in each balloon 16 and 17 and indicated in the sequential time line 30 as overlapping items 11 a and 11 b, the result is an interrupt in both conversations, Tom's and Max's. Therefore, item 11 a will result in the storage of a conversation marker in Tom's display computer 15 while item 11 b will result in the storage of a conversation marker in Max's display computer 14.
The display on respective computers 14 and 15 with appropriate conversation markers is shown in
While the playback of a stored segment resulting from an interrupt is usually played back by and for the benefit of the participant whose conversation stream has been interrupted, there may be circumstances wherein it is advantageous to provide the playback to other participants in the conference. The present invention may enable the participant selecting the playback to also provide his stored playback segment to other participants through conventional IP telephone transmissions.
Also, the participant, using conventional IP telephone technology, wherein his audio conversation stream is shown on his associated computer display during his telephone conversation, may also be set up to have his audio playback shown on the associated display.
While this invention has been illustrated with respect to
Since this embodiment is being presented using IP telecommunications, a general description of this technology is appropriate. In IP telecommunications, voice and other audio telecommunications are transmitted over the Internet or Web and are broken down into voice packets that are digitized and transmitted over the Internet using IP telecommunication protocols. The text, IP Telephony Demystified, Ken Camp, published 2003, McGraw-Hill, New York, N.Y., describes such transmission of voice packets, particularly in Chapter three, pp. 54-69. This has become the basic Voice Over IP (VOIP) which has become the primary technology used to transmit voice and audio content of the Internet or Web (these terms are used interchangeably herein). This VOIP technology is described in greater detail (page 870) in Newton's Telecom Dictionary, Harry Newton, 2003, CMP Books, San Francisco, Calif., as well as in the above-referenced text IP Telephony Demystified at pp. 222-223.
With the embodiment of
While the embodiment of the present invention has been described with IP telephone communications, the invention may be implemented with conventional PSTN (Public Switched Telephone Network) telephones. In a PSTN environment, the conversation of the speaker participant could conveniently be input through an appropriate microphone, processed via the sound card of an associated personal computer and stored on the disk drive of the computer. In order to display the conversation markers or any other input speech, standard speech recognition techniques may be used, such as the conventional techniques described in U.S. Pat. No. 6,937,984 (filed Dec. 18, 1998) may be applied. The functions of the present invention could also be applied in direct (non-telephone) conferences in which the participants use ancillary display computers to track the progress of the live conferences. In such live conferences, the monitoring of interruptions and the storing of appropriate conversation markers would be substantially similar to the implementations used for tracking the conventional PSTN telephone conversations described above.
Now, with reference to
For each of a plurality of participants in a telephone conference or conversation, provision is made at each participant's associated computer for continuously recording, as the telephone conversation progresses the last conversation segment, which may be of a predetermined effective time length, e.g. seven seconds. The length of the recorded segment should be sufficient when played back to alert or remind the participant of an interrupted conversation segment that the participant may wish to subsequently recall or continue, step 61. Provision is made for monitoring the conversation stream of a participant for conversation interruptions, step 62. The monitoring seeks triggers that are indicative of a conversation interruption. One of such triggers may be voice overlap. When two or more participants are speaking at the same time, there is likely to be a conversation lapse, i.e. all those speaking are likely to drown each other out, and each will require a conversation marker and the selective ability to playback the interrupted speech. The implementation may also determine the need for an interruption marker based upon the voice inflection of a participant. In English speech, a rise in the voice of the speaker typically indicates the end of a sentence. Thus, if the voice of another participant starts before such a rise in the voice of an original speaker, this may indicate that the original speaker has been interrupted before finishing his sentence. Also, there is a tendency for a speaker interrupting another to raise his voice. Raising the voice can, thus, serve as an interrupt trigger.
Two participants speaking simultaneously may also be determined through the conventional speech recognition techniques referenced hereinabove. Such speech recognition functions could distinguish the speech contents of each participant, and from such contents, determine unfinished words or sentence fragments.
In another embodiment, the user participant is enabled to manually create an interrupt trigger through a suitable input when he finds that the storage of the segment and the conversation marker would be appropriate because of some lack of clarity in the conversation exchange.
With reference again to
Now that the basic program set up has been described, there will be described with respect to
In the case where the telephone conversation is exited, the stored conversation markers and the recorded segments that these markers represent may be saved and used during any follow-up telephone calls.
One of the implementations of the present invention may be in application program 40 made up of programming steps or instructions resident in RAM 24,
One skilled in the art should appreciate that the processes controlling the present invention are capable of being distributed in the form of computer readable media of a variety of forms.
Although certain preferred embodiments have been shown and described, it will be understood that many changes and modifications may be made therein without departing from the scope and intent of the appended claims.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US5737405 *||Jul 25, 1995||Apr 7, 1998||Rockwell International Corporation||Apparatus and method for detecting conversation interruptions in a telephonic switch|
|US5867144||Jun 5, 1995||Feb 2, 1999||Microsoft Corporation||Method and system for the direct manipulation of information, including non-default drag and drop operation|
|US6134377 *||Dec 23, 1997||Oct 17, 2000||Hitachi, Ltd.||Disk regenerative apparatus|
|US20020196284||Jun 17, 2002||Dec 26, 2002||Berquist David T.||Software notes|
|US20040201630||Apr 26, 2004||Oct 14, 2004||Apple Computer, Inc.||Creation and manipulation of internet location objects in a graphical user interface environment|
|US20050041786 *||Mar 8, 2004||Feb 24, 2005||Craig John N.||Method for non-real time group interaction using a voice messaging system|
|US20050066287||Sep 10, 2004||Mar 24, 2005||Tattrie Scott I.||User-friendly data binding, such as drag-and-drop data binding in a workflow application|
|US20060036688 *||Aug 12, 2004||Feb 16, 2006||International Business Machines Corporation||Method and system for managing interrupts in an instant messaging application|
|US20060090137||Oct 26, 2004||Apr 27, 2006||International Business Machines Corporation||Chat user interface for threaded text chat systems|
|US20070086365 *||Oct 13, 2005||Apr 19, 2007||Yen-Fu Chen||System for selective teleconference interruption|
|WO1995024114A2||Feb 24, 1995||Sep 14, 1995||Scovell, Peter, George||Improvements in project management computer programs|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US8165874 *||Mar 6, 2009||Apr 24, 2012||International Business Machines Corporation||System, method, and program product for processing speech ratio difference data variations in a conversation between two persons|
|US8234289||Dec 17, 2007||Jul 31, 2012||International Business Machines Corporation||Restoration of conversation stub for recognized experts|
|US9703841||Oct 28, 2016||Jul 11, 2017||International Business Machines Corporation||Context-based notifications in multi-application based systems|
|US9722965||Jan 29, 2015||Aug 1, 2017||International Business Machines Corporation||Smartphone indicator for conversation nonproductivity|
|US20090157726 *||Dec 17, 2007||Jun 18, 2009||Abernethy Jr Michael Negley||Restoration of conversation stub for recognized experts|
|US20090228268 *||Mar 6, 2009||Sep 10, 2009||Gakuto Kurata||System, method, and program product for processing voice data in a conversation between two persons|
|US20090313010 *||Jun 11, 2008||Dec 17, 2009||International Business Machines Corporation||Automatic playback of a speech segment for media devices capable of pausing a media stream in response to environmental cues|
|U.S. Classification||370/260, 709/201|
|Oct 19, 2006||AS||Assignment|
Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ABERNETHY, JR., MICHAEL N;GRIGSBY, TRAVIS M;HONG, JUSTINK;REEL/FRAME:018410/0301
Effective date: 20061011
|Sep 12, 2014||REMI||Maintenance fee reminder mailed|
|Feb 1, 2015||LAPS||Lapse for failure to pay maintenance fees|
|Mar 24, 2015||FP||Expired due to failure to pay maintenance fee|
Effective date: 20150201