|Publication number||US7546355 B2|
|Application number||US 10/760,137|
|Publication date||Jun 9, 2009|
|Filing date||Jan 16, 2004|
|Priority date||Jan 16, 2004|
|Also published as||CA2553458A1, CA2553458C, DE602005027655D1, EP1704490A2, EP1704490A4, EP1704490B1, US20050198097, WO2005069862A2, WO2005069862A3|
|Publication number||10760137, 760137, US 7546355 B2, US 7546355B2, US-B2-7546355, US7546355 B2, US7546355B2|
|Original Assignee||Bloomberg Finance L.P.|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (15), Non-Patent Citations (5), Referenced by (34), Classifications (11), Legal Events (6)|
|External Links: USPTO, USPTO Assignment, Espacenet|
The invention relates to improvements in data transmission networks, and, more particularly, to network arrangements that reduce the amount of bandwidth required to transmit information to multiple network users.
Many different types of information may be transmitted from one point on a computer network to another. Such information is sometimes referred to as “content” which may include various forms or combinations of electronic information such as text, audio, and/or visual information (often referred to as multimedia content). One form of content transmission across a network is through a technique known as “streaming.” Using this streaming technique, a person at a remote computer terminal may request content over the Internet or other network which is subsequently “streamed” to the content requester. Employing one of several widely available media players such as Windows Media Player™, or RealOne Player™, the user may quickly and easily observe the streamed content at his or her computer terminal.
When a user requests a content stream directly from an information source, the response to the request is often handled by a server or other remote computer that sends a content stream directly to the requester. This method of sending an information stream from one point to another, in a more or less direct fashion, is called a “unicast” transmission in which one dedicated data stream is typically sent per request. Another way in which the server may respond to the request is by transmitting an information stream throughout the communication network without regard to whether a user downstream is interested the information being streamed. This method of sending information over a network is sometimes referred to as a “broadcast” transmission.
An alternative approach to unicast and broadcast transmission is called “multicasting.” Using this approach, a single stream of information is sent from a content server which is subsequently “divided” into additional identical content streams and provided to users.
Multicasting, unicast, and broadcast transmission all involve bandwidth issues that are of particular concern to network service providers.
Generally speaking, the invention provides for providing requested content, such as multimedia content, in a format suitable for transmission over existing unicast communication links and then subsequently changing the content to a multicast format for local distribution. This allows an application provider to transmit a single unicast stream of content to, e.g., a client-side media server in a local area network (LAN), and then distribute it to multiple interested users within the LAN using a multicasting transmission format. This significantly reduces bandwidth requirements by providing a single unicast transmission to a LAN rather than supplying each user within the LAN with a dedicated content stream.
Embodiments of the invention provide networks and methods that reduce the bandwidth required to transmit content while preserving a minimum amount of bandwidth in an upstream tail circuit for mission critical communication with a network server or other remote computer. This result is accomplished without relying on specialized multicasting hardware in the point-to-point transmission path.
Fault detection software deployed within the disclosed system may monitor network transmission paths and the status and activity level of client-side servers that locally distribute content to ensure a certain minimum quality of service. If a network connection or client-side server is overburdened or malfunctioning, the detection software may recognize this problem and take an appropriate remedial action such as redistributing the workload among client-side servers or rerouting the transmitted content across alternate, available or properly functioning transmission paths.
A method according to one embodiment of the invention provides electronic content to users from at least one remote electronic content source. The method comprises providing a unicast transmission of content, e.g., streaming media, from the at least one content source, receiving the content at a client-side server at a location serving a plurality of users; processing the received content in the client-side server such that the content may be provided to more than one of the plurality of users served by the location; and distributing the received and processed content to each of the plurality of users which has provided a request for the content.
In an embodiment of the invention, the received and processed content is distributed to a multicast group including each of the plurality of users which has provided a request for content by subscribing to the multicast group.
The client-side server is preferably monitored for distribution of content to the plurality of users. When the client-side server is not distributing the content to any of the plurality of users, transmission is terminated of the content from the at least one content source to the client-side server.
In an embodiment of the invention, additional content is transmitted from the at least one content source to the client-side server in response to a request by a user of the plurality of users serviced at the location by the client-side server for content not currently being transmitted by the at least one content source to the client-side server. Preferably, the additional content is transmitted in another unicast transmission. The additional unicast may be posted to an additional multicast group so that other users may access the additional content using the same additional unicast transmission from the at least one content source. The additional unicast may be provided by converting multicast format content into unicast format content for transmission to the client-side server.
Users may be running one or more applications in addition to receiving the content. The invention may also provide for monitoring transmitted content and limiting transmitted content to maintain an amount of bandwidth suitable for servicing at least one other application being run by each of a plurality of users other than content being received by such user.
The above and other objects and advantages of the present invention will be apparent upon consideration of the following detailed description, taken in conjunction with the accompanying drawings, in which like reference characters refer to like parts throughout, and in which:
Furthermore, although generally represented as Internet 25, this communication path may be any other suitable network interconnection desired such as an Intranet, a wireless interconnection, LAN, WAN, or other interconnection such as a secure hardwired interconnection. Internet 25, content source 20, and client-side server 30 are shown only for the purpose of illustration and not limitation. Moreover, it will be understood that although the present invention is well suited for the distribution of streaming media content, other suitable content distribution schemes may be used if desired.
Generally speaking, network 10 of the present invention operates as follows. A user (not shown) at terminal 50 selects a certain piece of electronic content stored on server 20. This request is routed to client-side server 30 over communication path 1 which forwards it to content source 20 via communication link 2. Communications paths 1 and 2 may be any suitable data communication paths known in the art such as an Ethernet link. Content source 20 processes the request and transmits the requested content to client-side server 30 for local distribution. The content is preferably transmitted to client-side server 30 over a single channel in standard unicast fashion over communication link 3. Once the content is received by server 30, rather than being forwarded directly to terminal 50, it is forwarded to a multicast group 40 (typically a virtual group or entity implemented in software on server 30), to which the requester terminal 50 is a member. The requester at terminal 50 then receives the content from a local software proxy 80 associated with that terminal. Local proxy 80 may receive the multicast transmitted content and covert it back into a suitable unicast format by manipulating the multicast data packets and arranging the content information into a proper order (i.e., sequential). In some instances, this may involve “producing” additional packets to fill any in “gaps” or missing packets in the received data by interpolation or other known data restoration technique. This content may then be displayed on the terminal using a conventional media player 70 such as Windows Media Player™, or RealOne Player™, or any other suitable content displaying software. Generating missed data packets as described above tends to prevent some content players from “stalling” when they attempt to retrieve data packets missing from the content stream, allowing the players to continuously display content despite “missing” some data.
Another way in which content may be requested by a user is by joining or “subscribing” to multicasting group 40. In this case, a monitoring program notices that a subscriber has joined the multicast group and is requesting the content associated therewith (such associated content may be predefined or may be user selectable). In response to the subscription, the monitoring program invokes a pseudo-media player (similar to the media players described above) that requests a unicast transmission of the selected content from source 20 (discussed in more detail below in connection with
In some embodiments, content source 20 and client-side server 30 may communicate with one another such that source 20 is aware of the identity and/or number of users that are subscribing to a particular multicast group 40 or that are receiving the unicast transmitted content. This may be done for a number of network management reasons including assessing popularity of certain content, or determining whether unauthorized or an unusual number of users are accessing or requesting certain content. Server 20 may be configured to discontinue or modify the content broadcast if such or other conditions are detected.
It will be appreciated that although only one multicasting group 40 is shown in
One advantage of posting the content to multicast group 40 is that other users at the requestor's the location may also access the resident content without invoking another dedicated unicast content stream that is transmitted across network 20. This is because client-side server 30 preferably includes multicasting hardware that permits replication of the incoming content stream so additional content streams may be sent to other terminals 50 at the user's location (not shown). Thus, when another user requests a content stream, before a request to content source 20 is made, the requesting terminal first polls (or listens) client-side server 30 through communication link 4 (sometimes referred to as a “heartbeat”) to determine if the desired content is already being received by server 30. If this is the case, the new requester merely joins the multicast group and receives the desired content therefrom rather than invoking a new content stream from source 20. This arrangement allows the amount of transmission bandwidth required by network 10 to be drastically reduced by eliminating duplicative content streams. It is also advantageous because this system may be implemented on a conventional network suitable for unicast communication and does not rely on specialized, expensive multicasting hardware.
Several methods may be employed to produce and/or convert unicast content streams received from source 20 into the multicast format streams provided by client-side server 30 in accordance with the present invention. In one embodiment, source 20 may merely provide a unicast format data stream that is manipulated by server 30 upon arrival such that it is converted into a form suitable for multicast distribution. This may be accomplished using any of the methods described herein or by using any other technique known in the art. Alternatively, source 20 may provide a multicast format stream that is “placed into” a unicast format by adding or removing header bits or otherwise manipulating the content stream so it appears to be in a unicast format before transmission to server 30. Upon arrival at server 30, the header file or additional bits may be stripped away or otherwise manipulated, leaving the data stream in its original multicast format.
Transmitting content in a unicast format and then locally delivering the content using multicast distribution techniques, in accordance with one embodiment of the present invention, allows the present system to reliably provide content without using large amounts of bandwidth. The unicast transmission of content from source 20 to server 30 is a reliable method for transmitting data due to the error checking and re-fetching of lost or corrupted data that normally occurs using this type of transmission. Thus, the quality of data in the content stream at server 30 is generally high. The multicast distribution technique used to distribute content from server 30 to local proxy 80 is less reliable, but correctable by interpolation or other techniques. The overall result is a system that provides high quality content distribution while using a relatively small transmission bandwidth. Moreover, it will be understood that the content distribution system described herein may accept many forms of content, and distribute them using this system. The content may be though of as the “payload” portion of the data stream that is moved around as described herein and then reassembled (with possible error correction) by local proxy 80 for subsequent use by local terminals.
When client-side server 30 activated, the software routines that govern its operations are initialized. This initialization process enables listening socket 35 (typically a virtual software entity that monitors incoming data streams) and directs it to monitor local proxy 80 for content requests and to place any such requests in queue 34 for subsequent processing by worker threads 33 (also a virtual machine or software entity responsible for, among other things, carrying out content queries). When such requests are received, they are routed from worker threads 33 through link 2 to content source 20.
When source 20 responds to the content request, the responsive content is forwarded to local proxy 80 (through multicast group 40) and the request is removed from queue 34. During this period, multicast monitor 31 monitors group 40 to determine if additional users are joining or leaving the group (i.e., requesting the same content currently being received or no longer interested in the content). In the case where another user joins the group, and is requesting content not currently being received, multicast monitor 31 directs pseudo player 32 to retrieve a unicast content stream from content source 20 and forward the data packets to multicast group 40 for possible format conversion and for distribution to the proper user terminals 50. In the case where monitor 31 detects no one is present in the user group, it directs pseudo-player 32 to terminate the content stream.
Another benefit of the content distribution network shown in
With the systems and methods described herein, however, significant additional routing options are available that make this problem far less likely. For example, if a particular user location has multiple client-side servers 30, these servers may be used to absorb the workload of a malfunctioning, overworked, or disconnected server 30, by merely reassigning (in software) user terminals 50 and multicast groups 40 from malfunctioning or overburdened server(s) 30 to properly operating servers. This may be done automatically by fault detection software such that the end user does not notice a service interruption (e.g., by sensing or anticipating a heartbeat failure or overcapacity in particular server 30 and effecting the switchover process immediately to balance the workload or reconnect to source 20).
Moreover, because network 10 merely requires a connection to content source 20 capable of supporting a unicast connection, multiple data paths to a particular client-side server 30 may be used to make a connection without being constrained by the need for specialized multicasting hardware, thereby improving the likelihood that an acceptable connection can be obtained even if the normal channels of communication are interrupted. This significantly reduces the chance that a particular user will be completely isolated from content source 20.
Some of the steps involved in monitoring the operation of system 10 to detect and correct potential failover or load balancing problems in accordance with the present invention are shown in the flow chart of
In operation, the fault detection software may monitor content source 20, client-side server 30, and the network connection between the two (Internet 25) to determine if content is being transmitted at step 100. If content is being transmitted, the fault detection program may monitor the network connection and client-side server to determine the overall “health” of the system by comparing various operating parameters to certain predetermined values such as throughput, bandwidth capacity and overall utilization (at step 110). If, after this comparison, it is determined that problems or potential problems exist, the appropriate remedial action is taken at steps 130 and 160 (or at least attempted).
For example, at step 120, the fault detection software may determine that the network connection between content source 20 and client-side server 30 is either malfunctioning or interrupted. If this is case, at step 130, the fault detection software may reroute the entire transmission path (for a service interruption) or may merely reroute some of the content through an alternate path if the currently used path is bandwidth limited to ensure a minimum quality of service. At this point, the fault detection software may also monitor the amount the network connection to ensure that a minimum amount of upstream bandwidth is available to service mission critical applications.
At step 140 the fault detection software may determine that a client-side server 30 at a particular site is either malfunctioning, overburdened, or inoperative. If this is the case, at step 150, the fault detection software may assess the availability and current workload of other nearby or associated client-side servers to determine if these servers may be used to either balance the workload or fully replace malfunctioning or inoperative servers. If some servers 30 are found to be available, then the fault detection software may reassign certain multicast groups or user terminals 50 to those servers at step 170. Once the systems returns to an acceptable operating state, the fault detection software may return to monitoring step 110.
Although several steps in the fault detection and monitoring process have been described above, it will be understood that these steps are merely illustrative, and are not meant to be comprehensive or necessarily performed in the order shown.
Moreover, while the invention has been described and illustrated in connection with preferred embodiments, many variations and modifications as will be evident to those skilled in this art may be made without departing from the spirit and scope of the invention, and the invention is thus not to be limited to the precise details of methodology or construction set forth above as such variations and modification are intended to be included within the scope of the invention.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US4903201||Nov 3, 1983||Feb 20, 1990||World Energy Exchange Corporation||Automated futures trading exchange|
|US6160843 *||Jul 14, 1997||Dec 12, 2000||Cisco Technology, Inc.||Communication server apparatus providing XDSL services and method|
|US6189039||Apr 10, 1997||Feb 13, 2001||International Business Machines Corporation||Selective tunneling of streaming data|
|US6321212||Nov 24, 1999||Nov 20, 2001||Longitude, Inc.||Financial products having a demand-based, adjustable return, and trading exchange therefor|
|US6546375||Sep 21, 1999||Apr 8, 2003||Johns Hopkins University||Apparatus and method of pricing financial derivatives|
|US6691094||Sep 28, 1999||Feb 10, 2004||Lee N. Herschkorn||Bank loan trading system and method|
|US6901445 *||Jul 1, 2002||May 31, 2005||Fastforward Networks, Inc.||Proximity-based redirection system for robust and scalable service-node location in an internetwork|
|US20020013823 *||Nov 30, 2000||Jan 31, 2002||Eubanks Thomas Marshall||Multicast peering in multicast points of presence (MULTIPOPs) network - neutral multicast internet exchange|
|US20020067730 *||Dec 5, 2001||Jun 6, 2002||Starguide Digital Networks, Inc.||Method and apparatus for IP multicast content distribution system having national and regional demographically targeted advertisement insertion|
|US20020143951 *||Mar 30, 2001||Oct 3, 2002||Eyeball.Com Network Inc.||Method and system for multicast to unicast bridging|
|US20030026254 *||May 15, 2001||Feb 6, 2003||Sim Siew Yong||Method and apparatus for large payload distribution in a network|
|US20050010653 *||Aug 5, 2004||Jan 13, 2005||Fastforward Networks, Inc.||Content distribution system for operation over an internetwork including content peering arrangements|
|US20050129017 *||Dec 11, 2003||Jun 16, 2005||Alcatel||Multicast flow accounting|
|US20060242311 *||Jun 19, 2006||Oct 26, 2006||Khanh Mai||Virtual multicasting|
|WO2002078258A2||Mar 8, 2002||Oct 3, 2002||Desktop Tv, Inc.||Method for data broadcasting|
|1||Beichuan Zhang et al.: "Host multicast: a framework for delivering multicast to end users" Proceedings of IEEE Infocom 2002, 21st Annual Joint Conference of the IEEE Computer and Communications Societies, New York, NY, Jun. 23-27, 2002, vol. 3, 23, pp. 1366-1375.|
|2||Eriksson H: "MBONE : The Multicast Backbone," Communications of the Association for Computing Machinery, ACM, New York, NY, US, vol. 37, No. 8, Aug. 1, 1994, pp. 54, 55-60.|
|3||International Search Report completed Oct. 6, 2005 in counterpart international application No. PCT/US04/033597 (Form PCT/ISA/210, second sheet and extra sheet).|
|4||Parnes P. et al.: "Lightweight application multicast tunnelling using mTunnel," Computer Communications, Elsevier Science Pub. BV, Amsterdam, NL, vol. 21, No. 15.|
|5||Ross Finalyson: "The UDP Multicast Tunneling Protocol," Internet Engineering Task Force, Live.Com, Nov. 21, 2003.|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US8204055 *||Oct 6, 2004||Jun 19, 2012||Thomson Licensing||Multicast over unicast in a network|
|US8223653 *||Apr 5, 2004||Jul 17, 2012||Ixia||Broadcast capable file system|
|US8370514||Mar 10, 2008||Feb 5, 2013||DISH Digital L.L.C.||System and method of minimizing network bandwidth retrieved from an external network|
|US8402156||Oct 18, 2010||Mar 19, 2013||DISH Digital L.L.C.||Apparatus, system, and method for multi-bitrate content streaming|
|US8612624||Sep 14, 2012||Dec 17, 2013||DISH Digital L.L.C.||Apparatus, system, and method for multi-bitrate content streaming|
|US8626925 *||Dec 16, 2005||Jan 7, 2014||Panasonic Corporation||Systems and methods for providing a selective multicast proxy on a computer network|
|US8683066||Aug 6, 2007||Mar 25, 2014||DISH Digital L.L.C.||Apparatus, system, and method for multi-bitrate content streaming|
|US8726382||Aug 20, 2008||May 13, 2014||The Boeing Company||Methods and systems for automated detection and tracking of network attacks|
|US8762515 *||Aug 20, 2008||Jun 24, 2014||The Boeing Company||Methods and systems for collection, tracking, and display of near real time multicast data|
|US8804721||Aug 31, 2011||Aug 12, 2014||International Business Machines Corporation||Multi-stream communication|
|US8813220||Aug 20, 2008||Aug 19, 2014||The Boeing Company||Methods and systems for internet protocol (IP) packet header collection and storage|
|US8868687||May 17, 2011||Oct 21, 2014||Philip Drope||Multimedia network system with content importation, content exportation, and integrated content management|
|US8868772||Apr 28, 2005||Oct 21, 2014||Echostar Technologies L.L.C.||Apparatus, system, and method for adaptive-rate shifting of streaming content|
|US8880721||Feb 1, 2013||Nov 4, 2014||Echostar Technologies L.L.C.||System and method for minimizing network bandwidth retrieved from an external network|
|US8964764||Mar 16, 2012||Feb 24, 2015||Philip Drope||Multimedia network system with content importation, content exportation, and integrated content management|
|US9047289||May 22, 2012||Jun 2, 2015||Philip Drope||Multimedia network system with content importation, content exportation, and integrated content management|
|US9071668||Dec 13, 2013||Jun 30, 2015||Echostar Technologies L.L.C.||Apparatus, system, and method for multi-bitrate content streaming|
|US9219729||Dec 3, 2010||Dec 22, 2015||Philip Drope|
|US9300657||Sep 18, 2014||Mar 29, 2016||Philip Drope|
|US9344496||Nov 3, 2014||May 17, 2016||Echostar Technologies L.L.C.||System and method for minimizing network bandwidth retrieved from an external network|
|US9398321||May 29, 2015||Jul 19, 2016||Philip Drope|
|US9407564||Oct 16, 2014||Aug 2, 2016||Echostar Technologies L.L.C.||Apparatus, system, and method for adaptive-rate shifting of streaming content|
|US9510029||Feb 14, 2011||Nov 29, 2016||Echostar Advanced Technologies L.L.C.||Systems and methods to provide trick play during streaming playback|
|US9571551||May 21, 2015||Feb 14, 2017||Echostar Technologies L.L.C.||Apparatus, system, and method for multi-bitrate content streaming|
|US20050220103 *||Apr 5, 2004||Oct 6, 2005||Wild Aloysius A Iii||Broadcast capable file system|
|US20070002858 *||Oct 6, 2004||Jan 4, 2007||Guillaume Bichot||Multicast over unicast in a network|
|US20070143458 *||Dec 16, 2005||Jun 21, 2007||Thomas Milligan||Systems and methods for providing a selective multicast proxy on a computer network|
|US20080040500 *||Jul 17, 2007||Feb 14, 2008||Veodia, Inc.||Method and apparaatus for distributing a media stream|
|US20100050084 *||Aug 20, 2008||Feb 25, 2010||Stephen Knapp||Methods and systems for collection, tracking, and display of near real time multicast data|
|US20100050256 *||Aug 20, 2008||Feb 25, 2010||Stephen Knapp||Methods and systems for internet protocol (ip) packet header collection and storage|
|US20100050262 *||Aug 20, 2008||Feb 25, 2010||Stephen Knapp||Methods and systems for automated detection and tracking of network attacks|
|US20110035507 *||Oct 18, 2010||Feb 10, 2011||Brueck David F||Apparatus, system, and method for multi-bitrate content streaming|
|US20110113122 *||Dec 3, 2010||May 12, 2011||Philip Drope||Multimedia Network System with Content Importation, Content Exportation, and Integrated Content Management|
|US20110219397 *||May 17, 2011||Sep 8, 2011||Philip Drope||Multimedia Network System with Content Importation, Content Exportation, and Integrated Content Management|
|U.S. Classification||709/219, 709/227|
|Cooperative Classification||H04L65/4069, H04L65/4076, H04L12/1859, H04L12/1886|
|European Classification||H04L29/06M4S2, H04L29/06M4S, H04L12/18T, H04L12/18P|
|Aug 9, 2004||AS||Assignment|
Owner name: BLOOMBERG LP, NEW YORK
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:KALNITSKY, YURY;REEL/FRAME:015663/0770
Effective date: 20040729
|Nov 14, 2007||AS||Assignment|
Owner name: BLOOMBERG FINANCE L.P., NEW YORK
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:BLOOMBERG L.P.;REEL/FRAME:020105/0768
Effective date: 20071114
Owner name: BANK OF AMERICA, N.A. AS ADMINISTRATIVE AGENT, NOR
Free format text: SECURITY AGREEMENT;ASSIGNOR:BLOOMBERG FINANCE L.P.;REEL/FRAME:020112/0080
Effective date: 20071114
Owner name: BLOOMBERG FINANCE L.P.,NEW YORK
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:BLOOMBERG L.P.;REEL/FRAME:020105/0768
Effective date: 20071114
Owner name: BANK OF AMERICA, N.A. AS ADMINISTRATIVE AGENT,NORT
Free format text: SECURITY AGREEMENT;ASSIGNOR:BLOOMBERG FINANCE L.P.;REEL/FRAME:020112/0080
Effective date: 20071114
|Jul 21, 2009||CC||Certificate of correction|
|Nov 2, 2012||AS||Assignment|
Owner name: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT, NO
Free format text: SECURITY AGREEMENT;ASSIGNOR:BLOOMBERG FINANCE L.P.;REEL/FRAME:029228/0503
Effective date: 20121030
|Dec 10, 2012||FPAY||Fee payment|
Year of fee payment: 4
|Nov 17, 2016||FPAY||Fee payment|
Year of fee payment: 8