The present invention is embodied in a digital communication system where multiple media data sources are time multiplexed into a packetized data stream, each packet having an assigned priority and the packetized data stream transmitted in substantially the order of assigned priority. At both the transmit side, and the receive side, audio packets are given priority processing over video packets, which in turn have priority over text/graphics data packets. Continuous real time audio playback is maintained at the receiver by delaying the playback of received audio in a first in/first out (FIFO) buffer providing a delay at least equal to the predicted average packet delay for the communication system. Optionally, the average system delay is continuously monitored, and the audio playback delay time is adjusted accordingly. Audio playback is slowed or accelerated in order to shrink or grow the difference in time between the sender and receiver. In another aspect of the invention, a... |
Citations|
| US4516156 | Mar 15, 1982 | May 7, 1985 | Satellite Business Systems | Teleconferencing method and system | | US4707831 | Oct 23, 1985 | Nov 17, 1987 | STC plc | Packet switching system | | US4763317 | Dec 13, 1985 | Aug 9, 1988 | American Telephone and Telegraph Company, AT&T Bell Laboratories | Digital communication network architecture for providing universal information services | | US4847829 | Nov 25, 1987 | Jul 11, 1989 | Datapoint Corporation | Video conferencing network | | US4935953 | Apr 27, 1989 | Jun 19, 1990 | International Business Machines Corporation | Cyclic video region transmission for videoconferencing systems | | US4942470 | Jul 5, 1989 | Jul 17, 1990 | NEC Corporation | Real time processor for video signals | | US4942569 | Feb 24, 1989 | Jul 17, 1990 | Kabushiki Kaisha Toshiba | Congestion control method for packet switching apparatus | | US4945410 | Oct 24, 1988 | Jul 31, 1990 | Professional Satellite Imaging, Inc. | Satellite communications system for medical related images | | US4949169 | Oct 27, 1989 | Aug 14, 1990 | International Business Machines Corporation | Audio-video data interface for a high speed communication link in a video-graphics display window environment | | US4991169 | Aug 2, 1988 | Feb 5, 1991 | International Business Machines Corporation | Real-time digital signal processing relative to multiple digital communication channels | | US4995071 | May 30, 1989 | Feb 19, 1991 | Telenorma Telefonbau und Normalzeit GmbH | Video conference installation | | US5014267 | Apr 6, 1989 | May 7, 1991 | Datapoint Corporation | Video conferencing network | | US5155594 | May 10, 1991 | Oct 13, 1992 | PictureTel Corporation | Hierarchical encoding method and apparatus employing background references for efficiently communicating image sequences | | US5177604 | Jun 3, 1988 | Jan 5, 1993 | Radio Telcom & Technology, Inc. | Interactive television and data transmission system | | US5210836 | Oct 13, 1989 | May 11, 1993 | Texas Instruments Incorporated | Instruction generator architecture for a video signal processor controller | | US5231492 | Nov 16, 1990 | Jul 27, 1993 | Fujitsu Limited | Video and audio multiplex transmission system | | US5309433 | Jun 18, 1992 | May 3, 1994 | International Business Machines Corp. | Methods and apparatus for routing packets in packet transmission networks | | US5537548 | Jun 14, 1994 | Jul 16, 1996 | International Business Machines Corporation | Method of computer conferencing by intercepting commands issued by application programs and redirecting to all stations for execution | | US5594859 | Feb 21, 1995 | Jan 14, 1997 | Digital Equipment Corporation | Graphical user interface for video teleconferencing |
Referenced by|
| US6374316 | Mar 18, 2000 | Apr 16, 2002 | Sony Corporation Sony Electronics, Inc. | Method and system for circumscribing a topology to form ring structures | | US6502158 | Mar 18, 2000 | Dec 31, 2002 | Sony Corporation Sony Electronics, Inc. | Method and system for address spaces | | US6539450 | Mar 18, 2000 | Mar 25, 2003 | Sony Corporation Sony Electronics Inc. | Method and system for adjusting isochronous bandwidths on a bus | | US6624827 | Aug 4, 2000 | Sep 23, 2003 | Dae-Joon Hwang | Apparatus and method for locking or prohibiting access to designated object displayed on shared electronic whiteboard | | US6631415 | Mar 18, 2000 | Oct 7, 2003 | Sony Corporation Sony Electronics Inc. | Method and system for providing a communication connection using stream identifiers | | US6647446 | Mar 18, 2000 | Nov 11, 2003 | Sony Corporation Sony Electronics Inc. | Method and system for using a new bus identifier resulting from a bus topology change | | US6714986 | Nov 25, 2002 | Mar 30, 2004 | Matsushita Electric Industrial, Co., Ltd. | File system for handling real-time and non-real-time data | | US6728358 | Jan 25, 2001 | Apr 27, 2004 | Paltalk Holdings, Inc. | Efficient buffer allocation for current and predicted active speakers in voice conferencing systems | | US6728821 | Nov 27, 2000 | Apr 27, 2004 | Sony Corporation Sony Electronics Inc. | Method and system for adjusting isochronous bandwidths on a bus | | US6738357 | Jun 19, 2000 | May 18, 2004 | BTG International Inc. Micron Technology, Inc. | Method and apparatus for multiple media digital communication system | | US6738380 | Apr 26, 1999 | May 18, 2004 | Sony Corporation | Apparatus, method and transmitting medium for transmitting and receiving fragmented coded data | | US6757773 | Jun 30, 2000 | Jun 29, 2004 | Sony Corporation Sony Electronics Inc. | System and method for determining support capability of a device coupled to a bus system | | US6792470 | Mar 2, 2001 | Sep 14, 2004 | Matsushita Electric Industrial, Co., Ltd. | Method and apparatus for communicating with data frames having priority levels | | US6798829 | Dec 7, 1999 | Sep 28, 2004 | Sony Corporation | Transmitting apparatus and transmitting method, receiving apparatus and receiving method, and distribution medium | | US6810452 | Mar 18, 2000 | Oct 26, 2004 | Sony Corporation Sony Electronics Inc. | Method and system for quarantine during bus topology configuration | | US6907570 | Mar 29, 2001 | Jun 14, 2005 | International Business Machines Corporation | Video and multimedia browsing while switching between views | | US6981263 | Jun 29, 2001 | Dec 27, 2005 | BellSouth Intellectual Property Corp. | Methods and systems for converged service creation and execution environment applications | | US7046780 | Apr 26, 2004 | May 16, 2006 | | Efficient buffer allocation for current and predicted active speakers in voice conferencing systems | | US7050425 | Jun 22, 2004 | May 23, 2006 | BTG International Inc. Micron Technology, Inc. | Apparatus for multiple media digital communication | | US7075924 | Mar 19, 2004 | Jul 11, 2006 | BTG International Inc. Micron Technology, Inc. | Methods for multiple media digital communication | | US7103157 | Sep 17, 2002 | Sep 5, 2006 | International Business Machines Corporation | Audio quality when streaming audio to non-streaming telephony devices | | US7103644 | Jun 29, 2001 | Sep 5, 2006 | BellSouth Intellectual Property Corp. | Systems for an integrated data network voice-oriented service and non-voice-oriented service converged creation and execution environment | | US7295247 | Jun 28, 2001 | Nov 13, 2007 | Telefonaktiebolaget LM Ericsson (publ) | Synchronisation of audio and video signals | | US7366405 | Jul 9, 2004 | Apr 29, 2008 | Matsushita Electric Industrial Co., Ltd. | Recording medium, recording method, reproduction apparatus and method, and computer-readable program | | US7386111 | Feb 10, 2004 | Jun 10, 2008 | Vonage Network Inc. | Method and apparatus for placing a long distance call based on a virtual phone number | | US7415192 | Jul 9, 2004 | Aug 19, 2008 | Matsushita Electric Industrial Co., Ltd. | Recording medium, recording method, reproduction apparatus and method, and computer-readable program | | US7417981 | Oct 15, 2003 | Aug 26, 2008 | Vonage Holdings Corp. | Method and apparatus for enhanced Internet Telephony | | US7453990 | Jun 22, 2004 | Nov 18, 2008 | Vonage Holdings Corp. | Emergency call completion for VoIP based on location of call originator | | US7483523 | Feb 25, 2004 | Jan 27, 2009 | Alcatel | Time delay compensation device for the management of multimedia communications via an asynchronous network and a telephone network | | US7519278 | Jun 17, 2005 | Apr 14, 2009 | Panasonic Corporation | Playback apparatus, program, playback method | | US7580384 | Jan 16, 2004 | Aug 25, 2009 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US7583888 | Aug 20, 2007 | Sep 1, 2009 | Panasonic Corporation | Reproduction device, program, and reproduction method | | US7586861 | Feb 20, 2004 | Sep 8, 2009 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US7586907 | Nov 28, 2006 | Sep 8, 2009 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US7609948 | Aug 20, 2007 | Oct 27, 2009 | Panasonic Corporation | Reproduction device, reproduction method and program | | US7633934 | Jan 16, 2004 | Dec 15, 2009 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US7646743 | Feb 20, 2004 | Jan 12, 2010 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US7680262 | May 20, 2008 | Mar 16, 2010 | Vonage Network LLC | Method and apparatus for placing a long distance call based on a virtual phone number | | US7688811 | Apr 8, 2004 | Mar 30, 2010 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US7693176 | Feb 23, 2007 | Apr 6, 2010 | Vonage Network LLC | Method and system for bidirectional data transfer | | US7697467 | Jan 16, 2004 | Apr 13, 2010 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US7715375 | Mar 16, 2004 | May 11, 2010 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US7760703 | Feb 20, 2004 | Jul 20, 2010 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US7768951 | Sep 8, 2009 | Aug 3, 2010 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US7792519 | Aug 22, 2003 | Sep 7, 2010 | Nokia Corporation | Conferencing system | | US7848316 | Dec 15, 2009 | Dec 7, 2010 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US7853125 | May 27, 2008 | Dec 14, 2010 | Panasonic Corporation | Reproduction device, program, and reproduction method | | US7856169 | May 27, 2008 | Dec 21, 2010 | Panasonic Corporation | Reproduction device, program, and reproduction method | | US7877002 | Jun 17, 2005 | Jan 25, 2011 | Panasonic Corporation | Reproduction device, program, and reproduction method | | US7899007 | Oct 14, 2008 | Mar 1, 2011 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US7912016 | Jul 18, 2005 | Mar 22, 2011 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US7912043 | May 11, 2010 | Mar 22, 2011 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US7916706 | Mar 13, 2008 | Mar 29, 2011 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US7920553 | Feb 20, 2004 | Apr 5, 2011 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US7921445 | Jun 6, 2002 | Apr 5, 2011 | International Business Machines Corporation | Audio/video speedup system and method in a server-client streaming architecture | | US7924822 | Feb 14, 2008 | Apr 12, 2011 | Vonage Network LLC | Method and apparatus for enhanced internet telephony | | US7933252 | Feb 20, 2004 | Apr 26, 2011 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US7936713 | Jan 12, 2010 | May 3, 2011 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US7949225 | Jun 17, 2005 | May 24, 2011 | Panasonic Corporation | Reproduction device, program, and reproduction method | | US8018907 | May 5, 2004 | Sep 13, 2011 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US8027566 | May 27, 2008 | Sep 27, 2011 | Panasonic Corporation | Recording medium, reproduction device, recording method, and reproduction method | | US8116301 | Jul 10, 2006 | Feb 14, 2012 | RPX Corporation | Method and apparatus for multiple media digital communication system | | US8121117 | Sep 12, 2008 | Feb 21, 2012 | F5 Networks, Inc. | Application layer network traffic prioritization | | US8121463 | Sep 18, 2007 | Feb 21, 2012 | Panasonic Corporation | Recording medium, recording method, reproduction apparatus and method, and computer-readable program | | US8126316 | Sep 18, 2007 | Feb 28, 2012 | Panasonic Corporation | Recording medium, recording method, reproduction apparatus and method, and computer readable program | | US8126317 | Sep 18, 2007 | Feb 28, 2012 | Panasonic Corporation | Recording medium, recording method, reproduction apparatus and method, and computer readable program | | US8139749 | Oct 29, 2008 | Mar 20, 2012 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US8139915 | Jul 11, 2008 | Mar 20, 2012 | Panasonic Corporation | Recording medium, recording method, reproduction apparatus and method, and computer-readable program | | US8149825 | Jan 16, 2004 | Apr 3, 2012 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US8194595 | Aug 25, 2009 | Jun 5, 2012 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US8213594 | Jun 10, 2011 | Jul 3, 2012 | Vonage Network LLC | Method and apparatus for placing a long distance call based on a virtual phone number | | US8228879 | Jun 18, 2008 | Jul 24, 2012 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communications among wireless terminals and telephones | | US8233779 | Sep 18, 2007 | Jul 31, 2012 | Panasonic Corporation | Recording medium, recording method, reproduction apparatus and method, and computer-readable program | | US8238264 | Feb 20, 2004 | Aug 7, 2012 | Broadcom Corporation | Hierarchical data collection network supporting packetized voice communication among wireless terminals and telephones | | USRE40704 | May 28, 2004 | Apr 28, 2009 | Apple Inc. | System for terminating multicast channel and data broadcast when at least two second endpoints do not transmit positive acknowledgement message to first endpoint | | USRE42442 | Dec 7, 2001 | Jun 7, 2011 | Apple Inc. | System for terminating multicast channel and data broadcast when at least two second endpoints do not transmit positive acknowledgement message to first endpoint |
Claims1. A method of communicating media data packets from a first processing machine to at least one remote processing machine connected to said first processing machine over a communication system, said method comprising the steps of: - enabling at said first processing machine a connection processing stream for said remote processing machine with which said first processing machine communicates over said system during a connection;
- enabling at said first processing machine at least one media processing stream associated with said connection processing stream;
- establishing a relationship in said first processing machine which associates a media processing stream located in said remote processing machine with said connection processing stream in said first processing machine;
- processing with said media processing stream in said first machine at least one type of media data to produce media data packets;
- determining a communication path for sending said media data packets to said media processing stream located at said remote processing machine and through said connection stream at said first process machine based on said established relationship;
- transmitting said media data packets to said media processing stream in said remote processing machine through said communication path;
- receiving an acknowledgment message from said remote processing machine containing information representing when at least one media data packet from said first processing machine was processed at said remote processing machine;
- using the received acknowledgment message to determine when a remote processing machine received and processed a media data packet sent to it from said first processing machine.
2. A method as in claim 1 wherein the media data packets transmitted and received by said first processing machine are multimedia data packets. 3. A method as in claim 2 wherein said multimedia data packets include audio and video data packets. 4. A method as in claim 3 wherein said multimedia data packets further include text/graphics data packets. 5. A method of communicating media data packets from a first processing machine to at least one remote processing machine over a communication system, said method comprising: - enabling at said first processing machine a connection processing stream for said remote processing machine with which said first processing machine communicates over said system during a connection;
- enabling at said first processing machine at least one media processing stream associated with said connection processing stream;
- establishing a relationship in said first processing machine which associates a media processing stream located in said remote processing machine with said connection processing stream in said first processing machine;
- processing with said media processing stream in said first processing machine at least one type of media data to produce media data packets;
- determining a communication path for sending said media data packets to said media processing stream located at said one remote processing machine and through said connection stream at said first processing machine based on said established relationship;
- transmitting said media data packets to said media processing stream in said remote processing machine through said communication path;
- storing in said first processing machine a first time relating to a first event associated with the transmission of at least one of said transmitted media data packets,
- recording in said remote processing machine a second time relating to a second event associated with the transmission of said at least one of said transmitted media data packets,
- comparing said first time to said second time to determine end-to-end system delay; and
- receiving an acknowledgment message from said remote processing machine containing information related to said end-to-end system delay.
6. A method in accordance with claim 5, wherein said media data is audio data and said media data packets are audio data packets. 7. A method in accordance with claim 5, wherein plural types of media data packets are transmitted by said first processing machine, including audio data packets and video data packets, said method further comprising: - providing an association between a source of audio data and a source of video data and respective audio and video media processing streams which respectively provide audio and video data packets; and
- transmitting each of said audio and video data packets through said connection processing stream associated with said audio and video media processing streams to audio and video data packet processing streams in said remote processing machine in accordance with a relationship established in said first processing machine which associates the audio or video data packet processing stream in said remote machine with said connection processing stream.
8. A method in accordance with claim 7, wherein said plural types of media data packets further includes text/graphics data packets, said method further comprising: - providing an association between a source of text/graphics data and a respective text/graphics media processing stream which provides text/graphics data packets; and
- transmitting said text/graphics data packets through said associated connection processing stream in said first processing machine to a text/graphics media processing stream in said remote processing machine in accordance with a relationship established in said first processing machine which associates the audio and video data packet processing streams in said remote machine with said connection processing stream.
9. A method in accordance with claim 5, wherein said first processing machine receives media data packets from said remote processing machine, said method further comprising: - receiving at said connection processing stream in said first processing machine media data packets directed thereto from said remote processing machine; and
- directing received media data packets from said connection processing stream to a receiving media data processing stream for processing the same.
10. A method as in claim 9, further comprising the step of: - delaying received media packets, said delay being set in accordance with an end to end system delay.
11. A method as in claim 10, wherein setting said delay further comprises: - increasing and decreasing said delay to be substantially equal to said determined end to end system delay,
- wherein the delay of said received media packets increases and decreases substantially in accordance with said end to end system delay.
12. A method as in claim 11, wherein said received media data packets are audio data packets. 13. A method as in claim 11, wherein said received media data packets are video data packets. 14. A method as in claim 11, wherein said received media data packets are text/graphics data packets. 15. A method as in claim 5, wherein said first event comprises storing said at least one media data packet in a buffer associated with said first processing machine, and said second event comprises processing at least a portion of said at least one media data packet in said remote processing machine. 16. A method as in claim 5, wherein said first event comprises storing said at least one media data packet in a buffer associated with said first processing machine, and said second event comprises storing said at least one media data packet in a buffer associated with said remote processing machine. 17. A method as in claim 5, wherein said act of comparing comprises calculating the difference between said first time and said second time. 18. A method of communicating packets associated with media data from a first processing machine to at least one remote processing machine over a communication system, said method comprising: - enabling at said first processing machine a connection processing stream for said remote processing machine with which said first processing machine communicates over said system during a connection;
- enabling at said first processing machine at least one media processing stream associated with said connection processing stream;
- establishing a relationship in said first processing machine which associates a media processing stream located in said remote processing machine with said connection processing stream in said first processing machine;
- processing with said media processing stream in said first processing machine at least one type of media data to produce packets associated with media data;
- determining a communication path for sending said packets associated with media data to said media processing stream located at said one remote processing machine and through said connection stream at said first processing machine based on said established relationship;
- transmitting said packets associated with media data to said media processing stream in said remote processing machine through said communication path;
- recording in said first processing machine a time of transmission for at least one of said packets associated with media data;
- receiving an acknowledgment message from said remote processing machine containing information representing a receiver event time related to when said at least one packet associated with media data sent from said first processing machine was processed at said remote processing machine; and
- determining at said first processing machine a system path delay based on said recorded time of transmission of said at least one packet associated with data media and said information in said acknowledgment message.
19. A method as in claim 18, wherein said media data is audio data and said packets associated with media data are audio data packets. 20. A method in accordance with claim 19, wherein plural types of packets associated with media data are transmitted by said first processing machine, including audio data packets and video data packets, said method further comprising: - providing an association between a source of audio data and a source of video data and respective audio and video media processing streams which respectively provide audio and video data packets; and
- transmitting each of said audio and video data packets through said connection processing stream associated with said audio and video media processing streams to audio and video data packet processing streams in said remote processing machine in accordance with a relationship established in said first processing machine which associates the audio or video data packet processing stream in said remote machine with said connection processing stream.
21. A method in accordance with claim 20, wherein said plural types of packets associated with media data further includes text/graphics data packets, said method further comprising: - providing an association between a source of text/graphics data and a respective text/graphics media processing stream which provides text/graphics data packets; and
- transmitting said text/graphics data packets through said associated connection processing stream in said first processing machine to a text/graphics media processing stream in said remote processing machine in accordance with a relationship established in said first processing machine which associates the audio and video data packet processing streams in said remote machine with said connection processing stream.
22. A method in accordance with claim 21, wherein said first processing machine receives packets associated with media data from said remote processing machine, said method further comprising: - receiving at said connection processing stream in said first processing machine packets associated with media data directed thereto from said remote processing machine; and
- directing received packets associated with media data from said connection processing stream to a receiving media data processing stream for processing the same.
23. A method as in claim 22, further comprising the step of: - delaying received media packets, said delay being set in accordance with an end to end system delay.
24. A method as in claim 23, wherein setting said delay further comprises: - increasing and decreasing said delay to be substantially equal to said determined end to end system delay,
- wherein the delay of said received media packets increases and decreases substantially in accordance with said end to end system delay.
25. A method as in claim 24, wherein said received packets associated with media data are audio data packets. 26. A method as in claim 24, wherein said received packets associated with media data are video data packets. 27. A method as in claim 24, wherein said received packets associated with media data are text/graphics data packets. |