|Publication number||US20060090009 A1|
|Application number||US 10/970,938|
|Publication date||Apr 27, 2006|
|Filing date||Oct 22, 2004|
|Priority date||Oct 22, 2004|
|Also published as||US7797720|
|Publication number||10970938, 970938, US 2006/0090009 A1, US 2006/090009 A1, US 20060090009 A1, US 20060090009A1, US 2006090009 A1, US 2006090009A1, US-A1-20060090009, US-A1-2006090009, US2006/0090009A1, US2006/090009A1, US20060090009 A1, US20060090009A1, US2006090009 A1, US2006090009A1|
|Inventors||Palamalai Gopalakrishnan, Christopher Knowlton, Bret O'Rourke, Ravi Raman|
|Original Assignee||Microsoft Corporation|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (18), Referenced by (9), Classifications (15), Legal Events (3)|
|External Links: USPTO, USPTO Assignment, Espacenet|
Media streaming systems and methods utilizing trick modes in conjunction with video-on-demand are described herein.
Some media systems can allow users to request video-on-demand presentations, such as movies or video clip from a web-site. Many users expect to be able to control the video on demand much as they can control their DVD player. For example, some users want to be able to fast-forward or rewind at one or more speeds. Existing technologies require additional bandwidth to achieve such data delivery to the user. For example, for a fast-forward play request some existing technologies send the same data stream but at a faster bit rate.
Satisfying user demands for fast-forward and/or rewind capability while reducing network bandwidth requirements can be advantageous.
Advanced trick modes are described herein. In one implementation, a system includes an upstream component configured to stream media to a downstream component. The downstream component is configured to present media to a user. For instance the downstream component may present a movie on a screen for the user. The upstream component is configured to stream a first media stream to the downstream component for presentation at a first playback rate in response to user input. Upon request for a faster playback rate the upstream component is configured to switch from the first media stream to a rate-appropriate point in a second media stream configured for presentation at the faster playback rate. In at least some embodiments, the system can function without the downstream component recognizing a media stream switch.
The methods and systems described below relate to multimedia streaming techniques. In at least some embodiments, a user or client can select an item located on a network for presentation to the user. The item can be a movie, a film-clip from a web-site or any other multimedia data that can be presented to a user. For purposes of explanation, the term movie will be referred to below with the understanding that the examples and descriptions also apply to other streamable items. The terms “stream” and “streaming” are used herein as terms of art with the recognition that these streams may comprise individual data packets.
In at least some embodiments, the system can access multiple media files associated with the user's movie selection. One of the files can be a normal playback file which is configured for presentation at a normal playback rate from the beginning of the movie to the end of the movie. The normal playback rate meaning the rate that the movie would be played if the user placed a commercially produced DVD of the movie in a DVD player and pushed play on the DVD player.
In some implementations, other individual media files may also relate to the movie from beginning to end, but may contain different total amounts of data when compared to the normal playback file. Those media files which contain lesser amounts of data relative to the normal playback file can be time compressed to play at some relative fraction of the presentation time of the normal playback file.
In response to a user request to play the movie, a first media stream is streamed from a first file for presentation to the user. In but one example, the first file can comprise the normal playback file. The first media file can be streamed at a first average bit rate. In the event that the user subsequently makes a fast-forward or rewind request related to the selected movie, a second media stream can be substituted for the first media stream at an appropriate position corresponding to the last data of the first media stream.
In some implementations, the second media stream can be streamed from a second file which contains less data than the first media file. If the second media stream is streamed and presented at generally the same average bit rate as the first media stream, then the movie is presented at a faster rate than when the movie is presented from the first media stream. Stated another way, the second media file has less data than the normal playback media file and, as such, is time compressed when streamed and presented at the normal playback rate. For example, the second media stream may be obtained from a media file that has one fifth of the data as the normal playback file yet has a profile which is the same or similar to the normal playback file with the exception of the play duration. The second media file and its media stream may be designated according to its relative presentation rate compared to the normal playback rate—in this instance 5.0×. In some implementations, various adjustments may be made to the second media stream such that the client unit and the user are unaware of the existence of more than one media stream. Many of the described systems and methods can be utilized with various data formats and network configurations.
The following discussion is directed to systems and methods for streaming media for presentation to a user. While features of the described systems and methods can be implemented in any number of different computing environments, they are described in the context of the following exemplary implementations.
Topology 100 can allow a user engaging a client unit (designated below) to select a movie, for presentation. The user can engage the client unit to request presentation of the movie by the client unit. For example, the client may request a normal “play” or “playback” of a hypothetical movie “ABC”.
A user may subsequently request to fast-forward or rewind the presentation. As will be described below, fast forward and rewind rates can be measured as a multiple of the normal playback rate, e.g. 20×, 50×.
Topology 100 can comprise a network 102, sending unit 104, transmission means 106 and a client unit or receiving unit 108. Sending unit 104 can contain and/or be in communication with a trick mode parser 110, a configuration file 112, one or more media parsers 114A-114C and one or more multimedia content files 116A-116C.
Sending unit 104, such as a media server, is communicatively coupled to client unit 108 via transmission means 106. In some implementations sending unit 104 can comprise a general purpose media server configured for use on various types of network configurations. Alternatively or additionally the sending unit may be configured to handle multiple types of media formats such as Windows MediaŽ format, MPEG and AVI among others.
Client unit 108 can comprise any suitable device that is capable of presenting media to a user such as, without limitation, a personal computer (PC), cell-phone, set-top box, and the like. In some embodiments, client unit 108 further comprises a display screen and one or more speakers. In other embodiments client unit 108 is configured to be coupled to a display screen and/or speakers. Transmission means 106 can comprise any suitable transmission means such as Ethernet, broadband, dial-up and wireless.
Sending unit 104 can store sources such as files for transmission to client unit 108 and/or can access the files via network 102. The functionality of the sending unit can be realized by a single component or multiple components. In various embodiments, network 102 may be a limited access network or can comprise the internet, among other configurations.
Sending unit 104 can comprise and/or access trick mode parser 110 and configuration file 112, as well as one or more media parsers 114A-114 c and one or more media files 116A-116C. In some configurations, the trick mode parser functionality is achieved as a plug-n-play component or sub-component added to the sending unit.
In this instance configuration file 112, media parsers 114A-114C and media files 116A-116C contain data for use in connection with processing and presenting user selected movie ABC. Media files 116A-116C may be located at the sending unit 104 as illustrated in
<?tmi version=“1.0”?> <tmi> <media src=“file://C:\wmpub\WMRoot\ABC_1x.wmv” rate=“1”/> <media src=“file://C:\wmpub\WMRoot\ABC_5x.wmv” rate=“5”/> </tmi> <media src=“file://C:\wmpub\WMRoot\ABC_10x.wmv” rate=“10”/> </tmi>
Media parsers 114A-114C are associated with the media files and are configured to access the data contained therein. In this particular configuration, individual media parsers are associated with individual media files. For example, media parser 114A is associated with media file 116A. Other configurations may have a single media parser associated with multiple media files. For example, in relation to the configuration of
The media files contain data or media from movie ABC at various presentation rates expressed relative to a normal playback or 1.0× file. In this instance, media file 116A contains an ABC 1.0× file, media file 116B contains the ABC 5.0× file, and media file 116C contains an ABC 10.0× file. In this implementation each of the media files 116A-116C is configured to present movie ABC from beginning to end, but the media files contain different amounts of data as will be explained in more detail below.
While three media files having playback rates of 1.0×, 5.0× and 10.0× are described in this example, the skilled artisan should recognize that is but one embodiment. For example, other embodiments may have more or less media files e.g. from two to any number “n”. Further, the relative playback rates can be varied as desired. For example, one embodiment may comprise a 1.0×, 10.0×, 20.0× and 100.0× media files.
The sending unit 104 receives the client's request to view a presentation, in this example to view a presentation of movie ABC. The sending unit may receive the client request directly from the client unit 108. Alternatively, the sending unit may receive the client request from another network component that is not specifically illustrated. For example, the client request may be received by a web-server and passed to the sending unit where the sending unit comprises a media server.
Upon receipt of a client request to play the movie ABC, sending unit 104 will access movie ABC's configuration file 112. In this particular implementation, the sending unit's trick mode parser 110 accesses and interprets the configuration file 112 for movie ABC.
Responsive to the user's play request, sending unit 104 accesses the 1.0× ABC media file 116A and causes data to be streamed from the media file to client unit 108. In this particular implementation this functionality is achieved by the trick mode parser 110. In some embodiments, the trick mode parser 110 causes the data to be streamed from the media file by creating a media parser for the media file. The media parser then accesses and streams the media file's data. Furthermore, the trick mode parser may examine the other ABC media files 116B and 116C. If these media files do not yet have an associated media parser, the trick mode parser may create such media parsers.
In some configurations, trick mode parser 110 can be configured to work with multiple data formats. For example, media content files 116A-116C relating to movie ABC may, for example, be in Windows MediaŽ format (wmv/wma), while media content files relating to a movie DEF may be in JPEG format and media content files relating to movie GHI may be in MPEG II format.
With some content formats, the trick mode parser and/or the appropriate media parser may undertake additional steps consistent with accessing and streaming data from the media files. For example, for Windows MediaŽ format files the parsers may create a data container containing the file and from which the data is streamed.
The media files 116A-116C can be created utilizing various techniques. Often the media files are created in advance by a third party, such as a third party who manages content on network 100. Various programs are commercially available to access a source file and create the various associated media files. In just one such example, Adobe Premier ProŽ can be utilized to access an un-coded source file and create source files at different desired relative playback speeds. Each of the source files can then be encoded. In another example, a compressed normal playback source file, i.e. 1.0×, can be accessed by an encoder such as Windows MediaŽ encoder which then creates the other desired files from the 1.0× file.
In some implementations, each of the media files associated with movie ABC can be configured to be streamed along transmission means 106 at a predetermined average bit rate. In but one such example, media files 116A-116C can be configured to be streamed at 1 Mega bit per second. Alternatively or additionally, each of the media files may be configured to be received and buffered by sending unit 104 from network 102 at a predetermined bit rate. Such implementations can allow more efficient allocation of bandwidth capacities within the system. These media files will be discussed in more detail below in relation to
One technique for achieving the same or generally equivalent bit rate for each of the media files is to remove some of the data or frames from the movie. For example, in some implementations the faster playback files may have all non key frames removed. In the present instance, the 5.0× and 10.0× media files may be comprised exclusively of key or I frames which are stand-alone frames which do not rely on information from adjacent frames. The skilled artisan should recognize other suitable techniques for removing data from the 5.0× and 10.0× files.
Media Files and Data Streaming from the Media Files
Act 402 receives a first request to stream media for presentation at a first normal playback rate. In some implementations, the method can locate a first media file corresponding to the play request so that corresponding data from the media file can be streamed to the user. In one such example where the first request is to play a movie, such as movie ABC, the method identifies media files associated with movie ABC. The method may access a configuration file for movie ABC which identifies the corresponding media files and various properties such as relative playback rates and locations of the media files.
Act 404 streams a first media stream configured to the first normal playback rate. The media can be streamed from the first media file configured for streaming at a bit rate which allows presentation at the first normal playback rate.
Act 406 receives a second request to stream the media for presentation at a second greater playback rate. Such a request may comprise a user selection to fast-forward or rewind the movie at a given relative rate, i.e. 5.0×, 10.0×, 50.0×, 100.0× etc. The method may attempt to identify a second media file corresponding to the second request. So, for example, if the second request is to fast-forward at 5.0× the method may look for a media file configured for presentation at 5.0× the normal presentation rate. If a corresponding media file is identified, the method may then determine an appropriate position to begin a second media stream from the corresponding media file.
Act 408 performs an action responsive to receiving the second request. The action may comprise switching from streaming the first media stream to streaming the second media stream starting at a rate-appropriate point.
To determine the rate-appropriate point in the second media stream, the method can determine the last presentation time streamed in the first media stream. Alternatively, the client unit may send a desired seek time in the second play request. The method can then, such as by utilizing the reference table described in
The method can begin streaming data from the 5.0× media file from the 1.0 minute point or the next subsequent presentation time. In some embodiments the method may also modify the presentation times of the second media stream so that they chronologically correspond to those of the first media stream. In this particular example, the method can multiply the presentation times of the second media stream by the same multiple or multiplier utilized to describe the files presentation rate relative to the normal playback rate. So in this example the presentation times of the second file can be multiplied by 5.0× so that they chronologically correspond to those of the first media stream. In this example, the first media stream was stopped at 5 minutes and the appropriate position in the second stream was identified from the reference table as 1 minute so the 1 minute presentation time contained in the data of the second file is converted to 5 minutes prior to streaming the data in the second data stream. Each subsequent presentation time of the 5.0× media stream can be similarly altered or adjusted before streaming.
In some implementations this presentation time adjustment is made ‘on the fly’ as the data is accessed and streamed from the second media file. The skilled artisan should recognize other alternative implementations. For example, the presentation times of the faster playback rates may be adjusted upon formation of the respective media file such that no further adjustment is made when the media file is accessed and its data streamed.
Adjusting the presentation times of the second media stream can allow much of the system to remain unaware of the existence of multiple media files and/or streams. Among other advantages, this can allow the system to work with existing client units which are not configured to operate with multiple streams. In some implementations accessing the various media files and adjusting the second and subsequent media streams to correspond to the first stream can be managed as close to the files as possible to minimize the system components that are configured to handle the multiple streams. In other implementations, the method may have more of this process occurring farther downstream. For example, in some implementations the client unit may be configured to receive the various streams and may make the any adjustments prior to presenting the streams to the user.
In an event that a second media stream corresponding to the requested second greater playback rate is not available, then various exemplary methods may utilize different responses. For example, in one implementation the method may have media files corresponding to 1.0×, 10.0× and 50.0× and be streaming from the 1.0× file when the user requests to fast-forward at 3.0×. In some of these implementations, lacking a 3.0× file, the method may simply continue to stream the current stream and ignore the second client request.
In other implementations the method may attempt to satisfy the client request by sending the current stream at a higher bit rate. For instance, in relation to the example described above, the method may stream the 1.0× media file at 3.0 times the average bit rate to satisfy the client request. Some methods may have further capabilities to attempt to satisfy the user request. For example, in one implementation the method may have media files corresponding to 1.0×, 10.0× and 50.0× and be streaming from the 1.0× media file when the user requests to fast-forward at 20.0×. The system lacks a 20.0× media file so these implementations may switch to streaming appropriate data from the 10.0× media file and may make the adjustments to the presentation times as described above. The method may then stream this data at twice the average bit rate to achieve a 20.0× playback rate.
Other implementations may simply stream from the available media file having the closest value to the second client request. So in the above example where the method has media files corresponding to 1.0×, 10.0× and 50.0× and is streaming from the 1.0× media file when the user requests to fast-forward at 20.0× the method may instead stream from the 10.0× media file.
Other implementations may simply consider the second client request invalid and not satisfy the client's request at all.
Computing environment 500 includes a variety of computer readable media which can be any media that is accessible by computing device 502 and includes both volatile and non-volatile media, removable and non-removable media. The system memory 506 includes computer-readable media in the form of volatile memory, such as random access memory (RAM) 510, and/or non-volatile memory, such as read only memory (ROM) 512. A basic input/output system (BIOS) 514 maintains the basic routines that facilitate information transfer between components within computing device 502, such as during start-up, and is stored in ROM 512. RAM 510 typically contains data and/or program modules that are immediately accessible to and/or presently operated on by one or more of the processors 504.
Computing device 502 may include other removable/non-removable, volatile/non-volatile computer storage media. By way of example, a hard disk drive 516 reads from and writes to a non-removable, non-volatile magnetic media (not shown), a magnetic disk drive 518 reads from and writes to a removable, non-volatile magnetic disk 520 (e.g., a “floppy disk”), and an optical disk drive 522 reads from and/or writes to a removable, non-volatile optical disk 524 such as a CD-ROM, digital versatile disk (DVD), or any other type of optical media. In this example, the hard disk drive 516, magnetic disk drive 518, and optical disk drive 522 are each connected to the system bus 508 by one or more data media interfaces 526. The disk drives and associated computer readable media provide non-volatile storage of computer readable instructions, data structures, program modules, and other data for computing device 502.
Any number of program modules can be stored on the hard disk 516, magnetic disk 520, optical disk 524, ROM 512, and/or RAM 510, including by way of example, an operating system 526, one or more application programs 528, other program modules 530, and program data 532. Each of such operating system 526, application programs 528, other program modules 530, and program data 532 (or some combination thereof) may include an embodiment of the systems and methods described herein.
A user can interface with computing device 502 via any number of different input devices such as a keyboard 534 and pointing device 536 (e.g., a “mouse”). Other input devices 538 (not shown specifically) may include a microphone, joystick, game pad, controller, satellite dish, serial port, scanner, and/or the like. These and other input devices are connected to the processors 504 via input/output interfaces 540 that are coupled to the system bus 508, but may be connected by other interface and bus structures, such as a parallel port, game port, and/or a universal serial bus (USB).
A monitor 542 or other type of display device can be connected to the system bus 508 via an interface, such as a video adapter 544. In addition to the monitor 542, other output peripheral devices can include components such as speakers (not shown) and a printer 546 which can be connected to computing device 502 via the input/output interfaces 540.
Computing device 502 can operate in a networked environment using logical connections to one or more remote computers, such as a remote computing device 548. By way of example, the remote computing device 548 can be a personal computer, portable computer, a server, a router, a network computer, a peer device or other common network node, and the like. The remote computing device 548 is illustrated as a portable computer that can include many or all of the elements and features described herein relative to computing device 502.
Logical connections between computing device 502 and the remote computing device 548 are depicted as a local area network (LAN) 550 and a general wide area network (WAN) 552. Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets, and the Internet. When implemented in a LAN networking environment, the computing device 502 is connected to a local network 550 via a network interface or adapter 554. When implemented in a WAN networking environment, the computing device 502 typically includes a modem 556 or other means for establishing communications over the wide area network 552. The modem 556, which can be internal or external to computing device 502, can be connected to the system bus 508 via the input/output interfaces 540 or other appropriate mechanisms. The illustrated network connections are exemplary and other means of establishing communication link(s) between the computing devices 502 and 548 can be utilized.
In a networked environment, such as that illustrated with computing environment 500, program modules depicted relative to the computing device 502, or portions thereof, may be stored in a remote memory storage device. By way of example, remote application programs 558 are maintained with a memory device of remote computing device 548. For purposes of illustration, application programs and other executable program components, such as the operating system 526, are illustrated herein as discrete blocks, although it is recognized that such programs and components reside at various times in different storage components of the computing device 502, and are executed by the processors 504 of the computing device.
Although embodiments relating to advanced trick mode have been described in language specific to structural features and/or methods, it is to be understood that the subject of the appended claims is not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as exemplary implementations for advanced trick mode.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US5377051 *||Jan 13, 1993||Dec 27, 1994||Hitachi America, Ltd.||Digital video recorder compatible receiver with trick play image enhancement|
|US5594491 *||Dec 29, 1995||Jan 14, 1997||Vxl/Hcr Technology Corporation||Near-video-on-demand digital video distribution system utilizing asymmetric digital subscriber lines|
|US5724646 *||Jun 15, 1995||Mar 3, 1998||International Business Machines Corporation||Fixed video-on-demand|
|US5864682 *||May 21, 1997||Jan 26, 1999||Oracle Corporation||Method and apparatus for frame accurate access of digital audio-visual information|
|US5917482 *||Mar 18, 1996||Jun 29, 1999||Philips Electronics N.A. Corporation||Data synchronizing system for multiple memory array processing field organized data|
|US5933567 *||Aug 12, 1994||Aug 3, 1999||Hitachi America, Ltd.||Method and apparatus for controlling the position of the heads of a digital video tape recorder during trick play operation and for recording digital data on a tape|
|US6005599 *||Dec 23, 1994||Dec 21, 1999||Hitachi, Ltd.||Video storage and delivery apparatus and system|
|US6064794 *||Mar 8, 1996||May 16, 2000||Thomson Licensing S.A.||Trick-play control for pre-encoded video|
|US6065050 *||Jun 5, 1996||May 16, 2000||Sun Microsystems, Inc.||System and method for indexing between trick play and normal play video streams in a video delivery system|
|US6701528 *||Jan 26, 2000||Mar 2, 2004||Hughes Electronics Corporation||Virtual video on demand using multiple encrypted video segments|
|US7023924 *||Dec 28, 2000||Apr 4, 2006||Emc Corporation||Method of pausing an MPEG coded video stream|
|US7054911 *||Oct 16, 2001||May 30, 2006||Network Appliance, Inc.||Streaming media bitrate switching methods and apparatus|
|US7295608 *||Sep 26, 2002||Nov 13, 2007||Jodie Lynn Reynolds||System and method for communicating media signals|
|US20020087994 *||Jun 14, 2001||Jul 4, 2002||Etienne Gomez||Process and system for video on demand|
|US20020184637 *||May 30, 2001||Dec 5, 2002||Perlman Stephen G.||System and method for improved multi-stream multimedia transmission and processing|
|US20030149975 *||Feb 5, 2002||Aug 7, 2003||Charles Eldering||Targeted advertising in on demand programming|
|US20060171658 *||Dec 8, 2003||Aug 3, 2006||Robert Jochemsen||More user friendly time-shift buffer|
|US20070044010 *||Aug 14, 2006||Feb 22, 2007||Sanghoon Sull||System and method for indexing, searching, identifying, and editing multimedia files|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7558760||Jun 12, 2007||Jul 7, 2009||Microsoft Corporation||Real-time key frame generation|
|US7873261 *||Aug 31, 2006||Jan 18, 2011||At&T Intellectual Property I, L.P.||Methods, systems, and computer-readable media for providing alternative media content|
|US8781305 *||Dec 2, 2008||Jul 15, 2014||Microsoft Corporation||Media streaming with smooth fast-forward and rewind|
|US8819183||Dec 15, 2009||Aug 26, 2014||International Business Machines Corporation||Concurrent execution of request processing and analytics of requests|
|US8874638||Dec 15, 2009||Oct 28, 2014||International Business Machines Corporation||Interactive analytics processing|
|US8892762 *||Dec 15, 2009||Nov 18, 2014||International Business Machines Corporation||Multi-granular stream processing|
|US20100135636 *||Dec 2, 2008||Jun 3, 2010||Microsoft Corporation||Media streaming with smooth fast-forward and rewind|
|US20110145429 *||Jun 16, 2011||International Business Machines Corporation||Multi-granular stream processing|
|WO2008084876A1 *||Jan 9, 2008||Jul 17, 2008||Matsushita Electric Ind Co Ltd||Method for trick playing on streamed and encrypted multimedia|
|U.S. Classification||709/246, 348/E07.07|
|Cooperative Classification||H04L65/4092, H04N7/17309, H04N21/47202, H04N21/6587, H04L29/06027, H04N21/2387|
|European Classification||H04N21/6587, H04N21/2387, H04N21/472D, H04L29/06C2, H04L29/06M4S6, H04N7/173B|
|Dec 1, 2004||AS||Assignment|
Owner name: MICROSOFT CORPORATION, WASHINGTON
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GOPALAKRISHNAN, PALAMALAI;O ROURKE, BRET P.;KNOWLTON, CHRISTOPHER G.;AND OTHERS;REEL/FRAME:015420/0553
Effective date: 20041020
|Feb 25, 2014||FPAY||Fee payment|
Year of fee payment: 4
|Dec 9, 2014||AS||Assignment|
Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTON
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICROSOFT CORPORATION;REEL/FRAME:034541/0477
Effective date: 20141014