Browsing continuous multimedia content

1

BROWSING CONTINUOUS MULTIMEDIA
CONTENT

RELATED APPLICATION

5

This application stems from and claims priority to U.S. Provisional Application Ser. No. 60/142,991, filed on Jul. 8, 1999, the disclosure of which is incorporated by reference herein.

10

TECHNICAL FIELD

The following disclosure relates to the playback of continuous multimedia content such as audio/video content, and to methods and user interfaces for efficiently browsing 15 through such continuous content.

BACKGROUND

VCRs (video cassette recorders) have found nearly uni- 2o versal acceptance as a way to record and distribute audio/ visual entertainment content such as motion pictures. These devices use a magnetic tape medium to store audio and video content. VCRs allow recording of several hours of audio/ video content, which can then be played back as desired. 25

One disadvantage of VCRs and their associated tapes is the difficulty of indexing to an intermediate point in the taped content. Because of the linear nature of magnetic tape, it must be manually advanced to the point containing the desired content. Advancing the tape in this manner can take 30 several minutes.

Another limitation of the linear tape format is that only one portion of the tape can be played or recorded at any given instant—it is not possible to record on one portion of the tape while playing back another portion. 35

These limitations have been addressed in newly emerging devices by storing digitized audio/video content on conventional hard disks. This has been made possible by the continually decreasing cost of hard disk storage. With a hard disk, it is possible to simultaneously record received audio/ 40 video on one portion of the disk, while also playing back audio/video from another portion of the disk.

Hard disk recording of broadcast television signals has enabled several features. One of the simplest features enabled by this technology is the ability to "pause" a 45 broadcast television show and then "resume" it without missing any of the content. When the viewer initiates a pause, the viewing device begins recording the show to disk. When the viewing device receives a subsequent resume command, the recorded content is played back to the viewer, 50 while the device continues to record remaining portions of the show. Thus, although the broadcast transmission has continued normally, the user perceives the ability to control the transmission just as if he or she had been watching the show from a recorded VCR tape. 55

Another feature is the ability for a viewing device or so-called "set-top box" to learn viewing habits and to automatically record favorite shows. The user can then select shows for viewing from an electronic program guide, and watch those shows from hard disk even if the shows 60 have already concluded. This method of interacting with a recording device is much more convenient than present methods of programming VCRs.

These features and others enabled by disk recording technology promise to revolutionize the way broadcast 65 content is distributed and consumed. However, these features also threaten to overwhelm viewers with viewing

2

choices. The technology described below provides ways for a viewer to "skim" through recorded content, providing a new way to deal with the overwhelming volume of broadcast content that is becoming available.

SUMMARY

A content distribution and playback system includes a set-top box or other similar device that receives and records broadcast programming from a plurality of sources. In addition, the set-top box receives segmentation data corresponding to the recorded programming. The segmentation data is used for skimming and indexing through recorded programming.

The system allows a viewer, upon playback, to select both time scale modification and skimming levels. Any selected time scale modification is performed at the set-top box using known time scale modification techniques. Skimming is performed using segmentation data corresponding to the program being played. The segmentation data indicates the most "important" portions of the program—those portions that should be included in a "condensed" or "summarized" version of the program. The set-top box uses this data to select appropriate portions for playback when the user requests skimming.

A user interface is provided that allows control of both time scale modification and skimming. In addition, the user interface allows convenient and intuitive indexing or browsing between different points of programs or shows.

The user interface allows a user to specify concurrent skimming and time-scale modification. Controls are also provided for indexing through a program by various different intervals. A table of contents and displayed still frames provide other ways to index within a program. The user is also able to enter annotations and associate such annotations with specified portions of the program.

DRAWINGS

FIG. 1 is a block diagram of a content distribution and playback system.

FIG. 2 shows acts performed by a network client relating to content skimming.

FIG. 3 shows acts performed by a network server relating to providing skimming data.

FIG. 4 shows a user interface implemented to control playback of distributed content.

DETAILED DESCRIPTION

General Architecture

FIG. 1 shows pertinent components of a computer entertainment system in accordance with an embodiment of the invention. In this embodiment, the system includes a recording device or "set-top box" 10, which is a computerized component that works in conjunction with a conventional television set. However, the various features of the invention described below can be provided in a variety of different configurations, such as in a traditional desktop computer or incorporated in a television set.

Set-top box 10 is configured to receive data, described in more detail below, from one or more service and information providers. In the described embodiment, this is accomplished through a public information system or network 12 such as the public Internet. Through this network, set-top box 10 can communicate and interact with a plurality of 3

different service and information providers 14. These providers include conventional World Wide Web data servers that supply traditional Internet content, as well as additional information to be described below. The physical connection to the Internet is through traditional means such as a modem 5 connection. Other means of connection are also becoming more widely available, such as ISDN, DSL, and other digital connections. Furthermore, information providers might be accessible through private connection means, such as by direct modem connection to specific information providers. 10

Set-top box 10 is also configured to receive broadcast live content from one or more live content sources 16. The term "live content" as used herein refers to continuous content such as audio and video that is provided to an end user at a transmission rate that is not easily varied or interrupted by 15 the user once transmission has begun. This is typically the situation with broadcast television, where a program is broadcast from start to finish without interruption: individual viewers are not able to pause, resume, or otherwise alter the relative time at which specific content is delivered. In the 20 case of broadcast television, transmissions are normally initiated at pre-published times. Other systems, often known as "pay-per-view" or "on-demand" systems, might allow transmission to be initiated at a time specified or selected by a user. Once begun, however, a pay-per-view or on-demand 25 movie plays until its conclusion, with no opportunity to pause and resume or to otherwise change the characteristics of the broadcast stream. Thus, these performances are considered "live" for purposes of this disclosure. Note that the term "live content" in this context does not necessarily refer 30 to content (such as audio or video content) that is transmitted as it is recorded.

In this exemplary embodiment, the live content is delivered through a distribution medium 18 which might comprise a traditional terrestrial RF broadcast system, or a cable 35 or satellite distribution system. These media typically provide broadcast streams at times determined by the content providers. Content might also be delivered over a networking medium such as the public Internet.

Note that the connection to public network 12 is typically 40 bi-directional, allowing the rich interactivity that has made the Internet so popular. The connection to content sources 16, on the other hand, is generally uni-directional. Content is broadcast on a time-table set by the content providers. In many cases, individuals are not able to affect the time at 45 which content is broadcast, and cannot request specific content at specific times.

Note also that the two illustrated forms of content distribution may eventually be merged. For example, a cable television distribution medium might be used for traditional 50 broadcast television signals as well as for bi-directional Internet communications. Alternatively, broadcast or broadcast-like content might be provided through the Internet, using technologies such as multicasting.

Set-top box 10 includes a network communications inter- 55 face 20 to facilitate Internet communications. Set-top box 10 also includes a television demodulator 22 for receiving traditional broadcast television content. Such content might be received in conventional over-the-air formats. In other embodiments, the content might be received in more 60 recently developed digital formats such as MPEG. Appropriate digital decoders are usually needed in embodiments utilizing MPEG data.

Set-top box 10 also includes I/O components 24 that interface to a keyboard 26 and a television set or other 65 display device 28. The display device includes audio components such as speakers (not shown).

4

Set-top box 10 has programmable processor 30 and associated components (not shown) that implement the functions described below. Processor 30 is logically connected to communicate with the other components of set-top box 10.

Set-top box 10 has non-volatile storage 40 in the form of a conventional rotating magnetic medium, commonly referred to as a hard disk. Other types of computer-readable storage media might also be used, such as recordable optical disks and/or electronic memory such as flash memory. One useful characteristic of these types of storage devices is that multiple content streams can be simultaneously written to and read from different logical storage locations. In addition, such storage devices allow non-linear access to stored data (in contrast to video cassette tapes), so that different recorded shows or programs can be easily indexed and retrieved. This allows nearly instant playback in response to viewer commands. It also allows different portions of a recorded program to be quickly accessed.

Set-top box 10 is programmed to implement recording and playback functions as described above in the "Background" section of this document. For example, the set-top box implements a "pause" function, in which live content is stored on disk during the pause. Upon a resume command, the recorded content is presented from disk, while the set-top box continues to record remaining portions of a show. Other functions include the ability to automatically record various different programs or shows for subsequent playback at a viewer's discretion.

This functionality, and the functionality described below, is implemented using conventional programming practices. Specifically, an operating program is stored on one or more storage media of set-top box 10, such as non-volatile storage 40 or other memory accessible to microprocessor 30. Alternatively, at least portions of the described functionality might be implemented in some embodiments using gatetype logic, such as programmable gate arrays or applicationspecific integrated circuits.

Time Scale Modification and Skimming

Set-top box 10 is programmed to facilitate various different types of time scale modification and skimming with respect to recorded content. Time scale modification refers to accelerated playback of content, without removing meaningful content. Skimming refers to omitting less important portions of recorded content during playback.

Most simply, time scale modification can be accomplished by simply speeding up the playback of an audio/ video stream. However, modern time scale modification techniques allow this to be accomplished while maintaining the audio pitch of the original stream, thereby improving intelligibility of the resulting accelerated stream. This is referred to as "linear" time scale modification, because each portion of the original stream is compressed by the same amount.

Time scale modification can also be accomplished by deleting "blank" portions of a stream, such as portions that contain no sound or that contain no video movement. This technique is referred to as "non-linear" time scale modification, because different portions of the original stream are affected by different factors.

Various methods of linear and non-linear time scale modification allow accelerated playback at a factor of two or more times the original speed, without compromising comprehension.

In general, the term "skimming" is used to describe viewing only highlights of a presentation, and skipping other