WO1996013125A1 - Method and apparatus for encoding and formatting data representing a video program to provide multiple overlapping presentations of the video program - Google Patents

Method and apparatus for encoding and formatting data representing a video program to provide multiple overlapping presentations of the video program Download PDF

Info

Publication number
WO1996013125A1
WO1996013125A1 PCT/US1995/013513 US9513513W WO9613125A1 WO 1996013125 A1 WO1996013125 A1 WO 1996013125A1 US 9513513 W US9513513 W US 9513513W WO 9613125 A1 WO9613125 A1 WO 9613125A1
Authority
WO
WIPO (PCT)
Prior art keywords
die
data stream
subsequences
elements
data
Prior art date
Application number
PCT/US1995/013513
Other languages
French (fr)
Inventor
Edward A. Krause
Paul Shen
Adam S. Tom
Original Assignee
Imedia Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Imedia Corporation filed Critical Imedia Corporation
Priority to EP95938280A priority Critical patent/EP0787408B1/en
Priority to DE69511570T priority patent/DE69511570T2/en
Priority to AU38966/95A priority patent/AU3896695A/en
Priority to CA002202350A priority patent/CA2202350C/en
Priority to JP8514059A priority patent/JPH10507889A/en
Publication of WO1996013125A1 publication Critical patent/WO1996013125A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • H04N21/2312Data placement on disk arrays
    • H04N21/2315Data placement on disk arrays using interleaving
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/23614Multiplexing of additional data and video streams
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/2365Multiplexing of several video streams
    • H04N21/23655Statistical multiplexing, e.g. by controlling the encoder to alter its bitrate to optimize the bandwidth utilization
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/434Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
    • H04N21/4347Demultiplexing of several video streams
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/434Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
    • H04N21/4348Demultiplexing of additional data and video streams
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/16Analogue secrecy systems; Analogue subscription systems
    • H04N7/173Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal
    • H04N7/17309Transmission or handling of upstream communications
    • H04N7/17336Handling of requests in head-ends

Definitions

  • the present invention relates to the provision of video programming on- demand, and more particularly to a method and apparatus that encodes, formats, stores and retrieves data representing a video program as a plurality of concurrent, overlapping presentations of the video program to facilitate virtual on-demand access to a single copy of the video program by virtually any number of subscribing viewers.
  • Such video programming can be rented in many forms, such as video cassette tapes for playback using a video cassette recorder (VCR), video disks for playback on video disk players, or as CD ROM's for playback using personal computers and other forms of CD ROM players.
  • Renting video programming in this manner is desirable because it permits the user to view the programming at any time and in any manner. For example, the user may view some portion of the program at one time and the remainder of the program at some different time. Further, the user may replay certain portions of the program or view the program in its entirety several times. The user may access the program from any point in the program by simply fast-forwarding or reversing through the program. The user is thereby freed from the constraints of available network or cable television programming.
  • Cable television and direct broadcast satellite (DBS) companies would like to compete in this arena by providing users with the same freedom of use enjoyed through video rental. This service would be known as "video-on-demand.” Such companies would clearly enjoy an advantage over video rental establishments in providing such a service because users would not be required to leave the comfort of their own homes to rent a copy of the video program (nor would they have to return it when finished). These companies have been heretofore constrained, however, by existing playback and distribution technology.
  • One possible compromise would be to produce multiple, overlapping playbacks (i.e. presentations) of the same video program, such that a new presentation of the program would begin, for example, every five minutes.
  • a total of twenty-four overlapping presentations of the program would be made available to subscribers.
  • Each subscriber would then have a receiver capable of selectively receiving any one of the twenty-four presentations.
  • the subscriber would have to wait at most five minutes to begin viewing the program in its entirety (or to access any point within the program).
  • the subscriber could fast-forward or reverse through the program by accessing a different one of the overlapping presentations, although he would be constrained to do so over the five minute intervals.
  • VCR-like functions such as fast-forward and reverse, would not only increase the complexity of the servers, but it would also impinge on available bandwidth because each subscriber must be able to communicate commands back to his or her dedicated server.
  • Such "back channels" are not even available in the context of existing DBS systems, and most existing cable distribution systems.
  • the best service that cable television and DBS companies have been able to offer thus far is a pay-per-view service that permits users to request (either over the telephone or directly through the cable network) an offered video program for a fee.
  • the company then permits the subscriber to receive the selected transmission of the video program.
  • These services are far from video-on-demand, however, as the number of available programs and the number of starting times for the programs are severely limited. Thus, the subscriber must still wait for a scheduled start time at which a desired program will be transmitted over the distribution network. Further, the subscriber does not have the freedom provided by an in-home playback resource such as a VCR; the program is still just passively received.
  • the present invention provides a method and apparatus for encoding and formatting data representing a single presentation of a video program for storage and for transmission that simulates multiple overlapping presentations of the video program using a single playback resource.
  • the video program is transmitted as a digital data stream that has been formatted in such a way that it appears to a subscriber that a number of segments of the same program are being continuously distributed over a plurality of subchannels concurrently.
  • an entire presentation of the video program can be assembled.
  • the subscriber can fast-forward or reverse through the program by advancing or decrementing the selected subchannel and thus receive a later or earlier segment of the program.
  • a one-hour program formatted in accordance with the present invention could be made to simulate, for example, twenty overlapping presentations of the program with each presentation (i.e. program segment) being three minutes ahead of the previous one.
  • a subscriber would have to wait only a maximum of three minutes to begin receiving the program in its entirety (i.e. until the segments begin again), and would be able to fast forward or reverse through the program at three-minute intervals.
  • the maximum delay that a subscriber would have to experience to randomly access any point in the program i.e. the access time
  • the present invention resides in its ability to provide this functionality with a single playback resource producing a single formatted data stream that represents a single presentation of the video program.
  • Video programs are typically organized as a series of scenes or frames, each frame comprising a two-dimensional array of picture elements or pixels. Each pixel has characteristics such as color and brightness which can be quantified as binary data. Audio information associated with the video program can also be converted to a binary representation.
  • the image and audio portions of a video program are converted to digital data streams using known techniques and standards.
  • the binary data generated to represent a video program can often be compressed considerably, thereby minimizing requisite memory storage and transmission bandwidth.
  • the video data streams are preferably compressed (i.e encoded) using any known video data compression technique to produce compressed video data streams.
  • the binary data comprising these data streams are grouped into arbitrary units called elements; an element can refer to one or more bits of video data where video data refers to all data required to represent a video program, compressed or not, and including but not limited to image, audio and other associated information.
  • the video data streams (compressed or not) are partitioned into n subsequences of elements representing segments of the video program, with each segment comprising an ordered sequence of m elements.
  • the ordered sequence of elements making up each subsequence are interleaved to produce a single interleaved data stream that preferably begins with the first element of each of the n segments, then the second element of each segment and so on in segment order until it ends with the m'th element of the n'th segment.
  • This interleaved data stream is continuously transmitted over the distribution medium from beginning to end.
  • a subscriber with an appropriate receiver can reconstruct the entire program, starting when the transmission is at the beginning of the interleaved data stream, by sequentially selecting and assembling the m elements of the first segment as the receiver serially parses through the interleaved data stream.
  • the receiver reconverts the selected and assembled elements back into image and/or audio in real time for presentation of the first segment to the subscriber.
  • the receiver selects and assembles all of the elements of the second segment for reconstruction, and repeats this process until it completes its n'th pass through the data stream to select and assemble the m elements comprising the n'th segment.
  • the receiver continuously decompresses (i.e. decodes) and reconverts the assembled segments in real time to reconstruct the video program in segment order for viewing by the subscriber.
  • the rate of transmission of the interleaved data stream is at least "n" times the average data rate "r" of the individual segments, the system will operate properly.
  • the throughput of the resource used to transmit the interleaved data stream defines the number of possible segments into which the program may be divided.
  • the viewing time of one of the n segments defines the access time "T" of the system, which is the minimum time interval between accessible points in the program. Further, the time necessary to transmit the entire interleaved data stream once must be less than or equal to T.
  • a subscriber receives access to an ordered sequence of n segments of the video program concurrently over n subchannels, which means any number of subscribers can be concurrently reconstructing n overlapping presentations of the video program, each presentation running ahead of its predecessor by an amount of time T required to reconstruct one video segment.
  • the formatting of the data stream representing the video program operates analogously to the process of time-division multiplexing information received from a plurality of communications channels.
  • each channel carries a different conversation or program, whereas the present invention exploits similar principles to break up and transmit a single program over separate subchannels of the same channel. It is this unique application of TDM principles to the context of video-on-demand which forms the basis for the instant invention.
  • An alternate preferred embodiment of the invention adapts the idea of statistical encoding to the interleaving process so that video segments that require more data to maintain desired picture quality are allocated more data while other segments of the program requiring less data are allocated less data such that the overall allocated bandwidth remains the same.
  • the video data streams are partitioned into subsequences representative of the segments first and then each subsequence is compressed and interleaved through a statistical multiplexer.
  • This embodiment although more complex in implementation, provides more uniform picture quality throughout the program.
  • the compression and interleaving processes are performed interactively through a combination of software and hardware, and the resulting encoded and formatted (i.e. interleaved) data stream representing the program is intermediately stored on a disk or some other form of mass storage.
  • the interleaved data stream can then be sequentially retrieved and cyclically transmitted over the distribution medium to subscribers by some relatively simple embodiment of a video server.
  • Figure 1 illustrates the interleaving and de-interleaving of a data stream representative of a video program.
  • Figure 2 is a block diagram illustrating a first preferred embodiment of the invention.
  • Figure 3 is a data structure diagram illustrating an interleaved sequence of elements formed by the system of Figure 2.
  • Figure 4 is a diagram illustrating a possible layout of video data representative of a program as formatted and stored on a mass storage device by the first preferred embodiment of the invention.
  • Figure 5 is a block diagram illustrating a second preferred embodiment of the invention that employs statistical encoding and interleaving.
  • Figure 6 is a diagram illustrating the statistical encoding and interleaving of video data according to the second preferred embodiment of the invention.
  • Figure 7 illustrates, in more detail, the statistical encoding and interleaving process as performed by the second preferred embodiment of the invention.
  • Figure 8 is a block diagram showing an encoder of the second preferred embodiment of me invention in greater detail.
  • Figure 9 is block diagram illustrating a possible layout for mass storage of a sequence of elements as statistically encoded and formatted by the second embodiment of the invention.
  • one way to closely approximate video-on-demand is to provide subscribers with access to multiple overlapping presentations of the same video program.
  • Complete random access might be achieved using this scheme if the number of overlapping presentations is made, for example, equal to the number of frames in the video program.
  • the number of overlapping presentations does not have to be nearly that numerous.
  • even a satisfactory approximation of random access to a video program using this technique would be prohibitively expensive using known or even envisioned technology.
  • the present invention resides in a unique application of time-division multiplexing principles (TDM) to provide subscribers with continuous access to multiple segments of a video program, thereby facilitating reconstruction of multiple overlapping presentations of the same video program.
  • TDM time-division multiplexing principles
  • This functionality is accomplished through the repeated transmission of a single, specially formatted data stream representative of one complete presentation of the video program.
  • the number of overlapping presentations of the video program that can be reconstructed from the formatted data stream, and thus the minimum access time between points in the program, is directly related to the rate at which the data stream can be transmitted over the distribution network as well as the rate at which data must be received by the receiver to permit real-time reconstruction of the video segments at the desired level of picture quality.
  • the present invention is unique because TDM principles are typically applied where a number of distinct data streams, each representing a different conversation or program, are to be transmitted over the same high-speed trunk such that all of the conversations or programs are simultaneously accessible by a receiver.
  • the present invention applies these principles to transmit n different segments of the same program over n different subchannels of the same channel, thus making each segment of the video program simultaneously accessible to a subscriber's receiver over the same channel of the distribution network.
  • the receiver can therefore be made to reconstruct and present any of the n segments to the subscriber by selecting the subchannel carrying the desired segment.
  • Figure 1 illustrates the essential principles of the instant invention.
  • a video program is typically organized as an ordered sequence of scenes or frames, with each frame defined by a two-dimensional array of picture elements or pixels.
  • a pixel has characteristics of color and intensity of illumination which, when combined with other pixels, creates an image or frame. For a given frame, the characteristics of each pixel can be represented digitally.
  • a video program can be converted into a digital data stream 10 that is an ordered sequence of bits which describes pixel values for each pixel of the aray during each frame of the video program.
  • audio associated with the program can also be converted to digital data, and can be identified with the image data on a frame-by-frame basis.
  • a video program is first converted to a digital data stream 10 in accordance with well-known standards and procedures.
  • the data stream 10 can represent images, audio or both, although in the preferred embodiment, separate data streams 10 are produced for the image and audio portions of the program. It will be assumed for ease of discussion and illustration that the data stream 10 throughout the
  • Figures represents the image portion of the video program, although audio data streams can be processed in accordance with the present invention as described for image data.
  • the data stream 10 is typically organized into an ordered sequence of elements, where as previously defined, an element is an arbitrary quantum of data that can be one bit, the number of bits representative of a single pixel, or the number of bits representative of one or more frames of pixels. In the case of audio, an element can be from one to several bits of audio data.
  • the video program is then divided into a number of video segments n, of approximately equal duration. Preferably, as many segments as possible given the rate at which the receiver must receive data to present the program in real time, and the maximum transmission rate which the playback and distribution resources are capable of sustaining.
  • the segments are created by partitioning the data stream 10 into n ordered subsequences of m elements, each subsequence representative of one of the n segments of the video program.
  • the video program as represented by data stream 10, starts at the beginning of the sequence of elements (i.e., the first element of segment 1 as denoted by reference numeral 12), and ends with the last element of segment n (i.e., denoted by reference numeral 16).
  • each segment is interleaved to produce an interleaved sequence of elements called an interleaved data stream 18 as shown in Fig. 1.
  • the interleaved data stream is constructed by arranging into frames identically numbered elements of each segment in segment order, beginning with the first element of each segment, and ending with the m'th element of each segment.
  • the n video segments which make up the video program are essentially time-division multiplexed over n "subchannels" of the channel over which the video program is broadcast.
  • the interleaved data stream 18 is organized into a series of m "frames” 13, with each frame partitioned into n "time slots” in segment order (i.e. one time slot is assigned to each segment). During each frame, one element belonging to each segment is transmitted during the time slot assigned to that segment.
  • the interleaved data stream 18 is repeatedly transmitted over a single channel of the television cable to subscribers. Provided that the transmission rate of the interleaved data stream 18 is at least n times the average data rate r of the individual segments, it will appear as though all n segments of the program are available to the receiver simultaneously. Each new transmission of the interleaved data stream 18 provides a subscriber an opportunity to begin restoration of the video program from its beginning, or to continue restoration of the program initiated during previous transmissions of the interleaved data stream.
  • the subscriber can view the program from its very beginning by simply programming the receiver to select and reconstruct elements associated with segment 1 (i.e. the first "subchannel") when transmission of the interleaved data stream is at the beginning of the interleaved data stream 18 (i.e at time tl).
  • the subscriber's receiver selects and assembles (i.e. de-interleaves or demultiplexes) the elements which form the first segment 14.
  • the receiver converts the elements comprising segment 1 into a presentation of the segment for viewing by the subscriber. From the beginning of the next transmission of (or pass through) the interleaved data stream 18 (i.e. at time t2), the receiver is altered (i.e.
  • the invention achieves the significantly important result of providing multiple subscribers virtually random access to a video program while minimizing the storage and distribution infrastructure necessary to provide this access.
  • a subscriber can begin accessing and reconstructing the program from any segment, even at a time when the transmission is currently in the middle of the interleaved data stream 18 and therefore in the middle of each of the n segments.
  • a subscriber can fast forward or reverse through the program by advancing or decrementing the segment currently being selected by the subscriber's receiver. Such steps forward or backward in the program are, however, constrained to multiples of the access time T.
  • n is referred to as the
  • interleaving factor As previously discussed, the number of video segments into which a program can be divided is limited by the bandwidth available for transmitting the n interleaved segments and die maximum data rate that can be sustained by die playback resource. One way to reduce the data rate that must be sustained by the playback resource for a given interleaving factor is to reduce the total amount of data necessary to adequately represent the video program.
  • Well-known techniques in the art can be used to encode the data stream 10 of Figure 1 to compress the amount of data necessary to adequately represent die program.
  • Two examples of well-known digital compression standards for video data are die MPEG-1 and MPEG-2 standards for
  • a first preferred embodiment of the present invention will be described which compresses the video data stream 10 and then interleaves the resulting compressed data elements.
  • a second preferred embodiment employs die principles of statistical multiplexing to statistically encode and interleave the video data stream 10 to ensure a more uniform picture quality for the reconstructed video program as viewed by a subscriber.
  • Nonstatistical encoding and interleaving may be simpler to implement with existing encoder and decoder products and can be easily applied to independently compressed video data streams.
  • Statistical encoding and interleaving may result in significantly improved uniformity of picture quality, but may require more sophisticated encoding structures than non-statistical encoding and interleaving.
  • Figure 2 illustrates a first embodiment for performing nonstatistical encoding and interleaving according to die invention.
  • a video program source 20 provides a data stream 10 that represents, for example, the image portion of the video program.
  • the data stream 10 which is typically an ordered sequence of digital data representing pixels embodying me images of the video program, is compressed by a digital encoder 22, employing any well-known procedure for compressing digitized video data.
  • the digital encoder 22 produces a compressed video data stream 23 that is organized as an ordered sequence of elements of compressed data representing the video program as embodied in the data stream 10.
  • d e elements are made up of equal numbers of bits which may not be organized based on pixel boundaries.
  • the compressed video data stream 23 is then partitioned into n multi-element subsequences of elements, each representative of one of n video segments.
  • the elements of the n subsequences are then interleaved by a partition and interleaving circuit 24 in accordance with the method of the invention as previously described.
  • the partition and interleaving circuit 24 produces an interleaved data stream 25 that can be directly transmitted to subscribers or that can be stored on a mass data storage medium 27 by data storage unit 26 for future retrieval and transmission.
  • Data storage medium 27 may consist of one or more hard disk, optical, tape, or electronic memory devices.
  • the interleaved data stream 25 will typically be stored on data storage device 27 for future playbacks of the video program embodied in the interleaved data stream 25.
  • the data storage device 27 is placed in a system that provides on-demand video programming including a server 31, and a plurality of receivers 32, 34 that are coupled to die server 31 by an appropriate distribution network 35.
  • the server repeatedly transmits the interleaved sequence of elements stored on die data storage device 27 to all receivers on die distribution network 35.
  • Each of die receivers 32, 34 includes die processing capacity necessary to reconstruct the video program in the manner described above by selecting, for each video segment, die sequence of elements comprising that segment.
  • each receiver includes the processing capacity to de-interleave die interleaved data stream 25, to reassemble and decode die resulting compressed data stream and to reconvert the resulting decompressed data stream 10 back into the sequence of pixels that embodies die images of the video program.
  • Figure 3 illustrates the partitioning of the compressed data stream 23 into subsequences representing die video segments.
  • the sequence of elements is encoded (i.e. compressed) by the digital encoder 22, the resulting compressed data stream 23 is produced as illustrated in Figure 2.
  • the compressed data stream 23 is then partitioned by the partition and interleaving circuit 24 into n ordered subsubsequences of elements each representative of one of n video segments.
  • Each subsequence is further partitioned into m elements so that the total number of video program elements is n , and each element is denoted by Sij, where 1 ⁇ i ⁇ n and 1 ⁇ j ⁇ m.
  • partitioning renders die compressed data stream 23 into an ordered sequence of n subsequences, each subsequence including one or more elements, each element including one or more bits of compressed video data.
  • Figure 3 also illustrates the interleaved data stream 25 which results from the interleaving process.
  • Figure 4 illustrates the storage of die interleaved sequence of elements comprising the interleaved data stream 25 on data storage device 27.
  • one or more data storage devices 27 can be employed to store the interleaved data stream 25.
  • the required number of data storage devices may be determined by design considerations taking into account die size of die compressed bit stream 23, die interleaving factor n, and d e access speed and capacity of individual data storage devices. Assuming mat die data storage device is a hard disk drive, the interleaved sequence 25 can be split into different, but equally-sized sections, and each section can be recorded onto one of N d different hard disks. The capacity of each data storage device 27 is assumed to be sufficient to contain nm/N d elements. A more detailed discussion of those considerations will be presented below.
  • FIG. 5 illustrates a second preferred embodiment of the invention which performs statistical encoding and interleaving.
  • a video program source 60 provides a data stream 10 representative of a sequence of pixels that embodies d e image portion of die video program.
  • the data stream 10 is organized as an ordered sequence of elements, each element comprising data representing an equal number of pixels.
  • the video program is partitioned into n segments, with each video segment represented by a subsequence of data stream 10, each subsequence representative of an equal number of pixels.
  • the n subsequences representative of the n video segments are aligned in time by the partition and alignment circuit 62 and each subsequence is concurrendy provided on a signal line 63 to a respective digital encoder 64.
  • Each digital encoder converts its respective subsequence into a compressed subsequence of elements, each element consisting of one or more bits of compressed video data.
  • Elements of the compressed video data stream 65 are interleaved by an interleaving circuit 66 to produce an interleaved data stream 67.
  • a data storage unit 68 receives the interleaved data stream 67 and transfers it to a data storage device 70 for later retrieval and transmission.
  • die interleaved data stream 67 can also be transmitted immediately to subscribers over a distribution network in realtime.
  • the data storage device 70 is placed in a system mat provides on-demand video programming.
  • the system preferably includes a server 72, and a plurality of receivers 73, 74 that are coupled to the server 72 by an appropriate distribution network 75.
  • the server repeatedly transmits the interleaved data stream 67 stored on die data storage device 70 to all receivers connected to the distribution network 75.
  • the interleaved data stream 67 is repeatedly transmitted to all receivers.
  • Each of the receivers 73, 74 includes die processing capacity necessary to reconstruct the video program by selecting and assembling, for each video segment, the subsequence of elements representative of the segment.
  • each receiver includes die processing capacity to de-interleave (i.e. demultiplex) die interleaved data stream 67 to thereby reproduce compressed data stream 65, to decode compressed data stream 65 into data stream 10, and to convert the data stream 10 to the sequence of pixels mat embody die video program.
  • de-interleave i.e. demultiplex
  • the data stream 10 which is a digitized representation of the sequence of pixels embodying die images of die video program, is partitioned into n subsequences of elements representative of n video segments. Each element of the subsequence consists of data diat represents the same number of pixels. Each subsequence has an equal number of elements and therefore represents the same number of total pixels.
  • Each pixel of the subsequence representing die first segment 84 of partitioned data stream 80 has a video intensity value s(t), where each pixel is defined by its position in the subsequence as a function of time. Because the duration of a segment is defined to be T, die video intensity value for each pixel of die second subsequence representing die second segment 86 is defined by s(t + T), and for all pixels is defined by s(t + i « T) were O ⁇ i ⁇ n.
  • Each of die subsequences of partitioned data stream 80 are then separately assembled and input to their respective decoders simultaneously as shown in Figures 6 and 7. Thus, the segments are time shifted for purposes of concurrent processing.
  • Each encoder 64 of Figure 7 processes its own subsequence of digital pixel data and die preferred embodiment of such an encoder is shown in more detail in Figure 8.
  • Such encoders are well known in the art and die details of die encoder disclosed are not relied on for patentability of the invention. Those of skill in the art will recognize that any encoder suitable for compressing video data within the context of the prevent invention can be substituted therefor.
  • each encoder compresses die data comprising its respective subsequence of pixels, producing a sequence of elements consisting of compressed data called blocks 82.
  • the encoder performs this function by first organizing its subsequence of pixel data into elements called blocks with a blocking circuit 100 ( Figure 8).
  • Each block consists of data representing an equal number of pixels.
  • Each block represents some subset of die total number of pixels embodying the segment.
  • a block can consist of data representing all of die pixels that make up a single frame or image of the segment.
  • a frame may contain 480 lines of 720 pixels, or 345,600 pixels total.
  • Blocks can arbitrarily consist of pixels making up fractions of frames, single frames or even multiples of frames.
  • Each element or block of pixel data is dien processed dirough a transform circuit 102, which uses well-known compression transforms, such as the discrete cosine transform (DCT) to convert the binary pixel data to transform coefficients.
  • DCT discrete cosine transform
  • the coefficients are quantized or normalized by adaptive quantizer 103, which improves coding efficiency by truncating die coefficients in accordance widi the desired level of picture quality to be produced when die information is decompressed for presentation. Finally, the quantized coefficients are passed dirough a statistical encoder 110 which further compresses die data based on a variable length encoding technique such as Huffman coding.
  • Each of die encoders is synchronized so that each produces a block of compressed data simultaneously, and each encoder is preset to a coding level that reflects e desired quality level of the reproduced picture.
  • Each compressed block (i.e. element) of data produced by an encoder may vary in its amount of binary data because some blocks of digital pixel data will compress more easily than others.
  • Block multiplexer 66 ( Figure 7) then interleaves the blocks by selecting a block from the first encoder (and thus from the first segment), then the second and so on, up to encoder n, and then repeats the process for the next set of compressed blocks.
  • an interleaved data stream 94 is produced as shown in Figure 6, widi each block 82 designated first by a segment number and then by block (i.e. element) number.
  • the interleaved data stream 94 is then input to a first-in-first-out (FIFO) channel buffer 92 ( Figure 7) and clocked out of die buffer at the transmission rate necessary for proper operation (i.e. n • r). If the buffer begins to fill up, a signal is provided over line 93 ( Figure 7) which instructs each of the adaptive quantizers 103
  • the picture quality should be reasonably consistent throughout the video program and the number of changes in the compression ratio during encoding and interleaving should be minimal.
  • mis encoding technique as applied to multiple programs being multiplexed over a multichannel trunk is disclosed in U.S. Patent No. 5,216,503 to Paik et al.. which is incorporated herein by this reference.
  • this technique is applied to ensure that die sum of die data transmitted at any time over all of the channels of a trunk does not exceed die total bandwidth of the trunk.
  • Application of this technique to die instant invention is unique because it is applied to segments of the same program to achieve uniform picture quality during playback of die single program.
  • each of the "n" encoders of Figure 7 could maintain an independent channel buffer, and the compression ratio of each individual encoder would be adjusted in order to maintain a pre-assigned fraction of the total throughput.
  • the disadvantage of the nonstatistical system is that the encoders will tend to deliver unnecessarily high picture quality at times when the video program is easily compressed, and poor picture quality during certain complex scenes when compression becomes difficult.
  • An encoding process implemented in software may use a video compression standard such as the one implemented by the encoder of Fig. 8.
  • die software compresses the data stream 10 in accordance widi a desired quantization level, and then measures the total amount of data that results. If the amount of data exceeds that which can be transmitted within the specified bandwidth, the quantization level is increased and die data stream 10 is compressed again. This process continues until the resulting compressed data stream falls within die requisite bandwidth.
  • the software partitions the compressed data stream and interleaves the elements of the segments of the partitioned subsequences to produce an interleaved data stream representative of the video program.
  • Iterative compression through software is particularly advantageous when statistically encoding and interleaving because it does not require die use of costly hardware to split die ordered sequence of pixels represented by die data stream 10 into video segments, nor does it require multiple encoders to simultaneously compress each video segment.
  • the data stream 10 is first partitioned into subsequences of elements representing segments with die same number of pixels.
  • the partitioned data stream is then compressed in accordance with a specified encoding level using a standard such as die one implemented by die encoder of Fig. 8.
  • the software then interleaves the compressed elements of the subsequences in accordance widi die invention.
  • die software interleaves the elements, it sums the total amount of data across die segments widiin a certain window, much like die channel buffer 92 of Fig. 7. Any sections of the interleaved data stream for which the amount of data exceeds die specified bandwidth over the defined window are compressed again using a higher coding level. For any sections which exceed some minimum amount of data, die program can recompress the section using a lower coding level.
  • One way to save some computer time would be to use a hardware encoder such as die one shown in Fig. 8 to compress die data stream 10 at a number of different coding levels and to store die different compressed versions.
  • the software can perform the steps of interleaving and summing, and for any section of the interleaved data stream that is detected to be sufficiently over or under die specified bandwiddi, the software can simply replace that section with a section of one of the stored interleaved data streams that has been compressed at die appropriate level of quantization.
  • a receiver During each transmission of the interleaved data stream, a receiver must be able to identify die elements of the particular video segment to be accessed, reconstructed and displayed.
  • One possible implementation by which this can be accomplished is by inserting video segment identifiers into die interleaved sequence of elements as they are being stored for future transmission. For example, sequence numbers (i.e., subchannels) ranging from 1 to n can be assigned to video segments 1 to n respectively. The appropriate sequence number can then be associated widi die video data comprising each element prior to interleaving and storage.
  • the receiver selects and assembles each element having a sequence number of 1 during die first pass, followed by die elements having a sequence number of 2 during die second pass and so on.
  • the reconstruction process is complete when all of the elements having a sequence number of "n" are selected during die final pass.
  • a subscriber could program die receiver to jump ahead or backward in die program by manually programming the selected sequence number (i.e., by changing the subchannel).
  • Another possible implementation is to program the playback resource to insert the video segment identification information into the interleaved sequence of elements as they are being transmitted to die receivers.
  • the same sequence numbers ranging from 1 to n can be initially assigned to video segments 1 to n respectively for a first transmission.
  • the playback resource will decrement each segment's assigned sequence number such that segment number two currently has sequence number one assigned to it, while segment one currently has sequence number "n" assigned to it. Therefore, once the receiver begins to reconstruct the video program based on a particular subchannel or sequence number, it can remain tuned to that number throughout the entire reconstruction.
  • One disadvantage of this implementation is that the first segment, which always represents the beginning of the video program, is no longer associated widi a single sequence number. Therefore, the playback resource must insert additional information into die interleaved data stream prior to its transmission to demarcate the first segment of the video program.
  • Another possible implementation would be to demultiplex the elements at the receiver (i.e. 32, Fig. 2 or 74 Fig. 5) synchronously, with each time slot in a frame assigned to a particular segment of die program.
  • die receiver i.e. 32, Fig. 2 or 74 Fig. 5
  • die receiver also must be able to decode die compressed data and display die right pixels in die right sequence and at die right time.
  • die receiver In die case of audio data, die receiver must also be able to synchronize the audio widi the images.
  • die receiver must know what the coding level of die compressed data is at all times.
  • diere are a number of well-known standards for representing video data digitally as well as compressing it mat specify how such information be provided in die data stream.
  • the interleaved sequence of blocks comprising the interleaved data stream 94 may be packetized by a packet encoder 121 (shown in Fig. 7).
  • die interleaved sequence of blocks 94 may be divided into groups of packets, with each packet in the group containing compressed video data representing pixel values for die same sequence of pixels during each of die n segments.
  • each packet might be comprised of a block from each segment, each block containing the pixel values s(t) for corresponding pixels of frames occurring at time t, t+T, t+2T ... t+(n-l)T.
  • a single packet may be organized as shown in Figure 9. It should be noted diat, in the preferred embodiment, die size of a packet may vary because some blocks of video may be more difficult to compress man others. Odier information included in a packet might be at which defines packet and block boundaries, as well as die coding level of the data in the packet.
  • Anodier way of organizing the data is to create packets of data during the interleaving process, such that each packet contains some arbitrary amount of data from a single subsequence representing a segment, which may or may not be on block boundaries.
  • the packets are interleaved radier than the blocks, and information defining die segment, die particular pixels represented by the data and a time stamp which defines a display time for the pixels defined by die data in die packet are included as a header.
  • the order of interleaving is determined by increasing time stamp value instead of by segment number.
  • die required amount of storage and die necessary throughput of the storage device for playback of a video program depends on several design parameters.
  • Die access time "T” is the time duration of each video segment as it is reconstructed by a receiver for viewing by a subscriber.
  • This parameter defines die maximum delay diat a subscriber must endure when die subscriber randomly accesses the program for playback from its beginning.
  • Playback is a real-time process involving continuous retrieval and transmission of the same interleaved video data stream, and die continuous reconstruction of a video program from that interleaved data stream. Because all n video segments are presented for access by a receiver simultaneously, it follows that any of the n video segments can be accessed at any time during playback.
  • die access time T also defines die maximum delay diat will be endured to access any point in the video program. Stated differently, it defines the minimum interval of time over which the program can be randomly accessed.
  • Anodier parameter to be specified is die average data rate "r" of die compressed video segments of the video program.
  • This parameter defines die rate at which segment data must be received by a receiver so that it can reconstruct die video program in real-time at the desired level of quality.
  • the rate at which the interleaved data stream is transmitted over the distribution must be equal to the number of segments n times the average data rate of one segment r (i.e. n-r).
  • the distribution system and die storage facility from which die interleaved data program is to be retrieved (or the encoding and interleaving apparatus used to generate the interleaved data stream if interim storage is not used), must be able to sustain this data rate.
  • One of the advantages of the present invention is that the cylinders and tracks of a hard disk can be accessed sequentially or in any sequence which maximizes the throughput performance of the unit. Sequential access is made possible by the fact that the playback process accesses, retrieves and transmits the same interleaved sequence of elements over and over.
  • the minimum access interval may be further reduced by splitting the interleaved data stream into equal portions, storing the portions on separate disks, and then accessing each disk concurrently to retrieve and transmit each portion of the interleaved data stream simultaneously.
  • each video segment has been subdivided into equal portions, with each portion of each segment made available to a receiver simultaneously over a different communications channel. For example, suppose that the interleaved data stream representative of the two-hour program of the previous example is divided into diree equal portions, with each portion stored on a separate disk.
  • multiple disks also may be used in die event diat die total data comprising the interleaved data stream exceeds die storage capacity of a single storage device such as a disk, or if the required throughput of the single storage device is not sufficient to meet iat specified by n-r.
  • Two preferred mediods of partitioning the interleaved sequence of blocks for writing on multiple disks are now described. The purpose of these mediods is to create overlapping data streams from each disk that remain relatively constant in time relative to one another.
  • Those of skill in the art will appreciate that there are many possible schemes for partitioning the data which are contemplated to be widiin the scope of the present invention.
  • One partitioning method is to split a packetized interleaved sequence of elements at packet boundaries such diat the same number of packets are written to each disk. For example, if each packet contains data diat is representative of a fixed number of pixels, then the first K packets may be allocated to a first disk, the next K packets to a second disk, and so on, until finally die last K packets are assigned to the last disk. The total number of packets must be equal to die product of K and die number of disks.
  • This partitioning memod may be referred to as die "constant pixels" method, because all of the disks contain video data diat is representative of the same number of pixels.
  • a second preferred method of splitting the packetized interleaved sequence of blocks for writing on multiple disks may be termed die "constant bits" method.
  • die constant bits method die same amount of compressed video data is allocated to each disk.
  • the interleaved sequence of blocks 94 is divided into equal sections, one for each disk, but die partitioning is not constrained to occur at a packet boundary.
  • dien die sequence may be extended by inserting fill data at die end or at certain points where they will not affect the decoding process.
  • encoders based on odier known compression techniques may be substituted for that disclosed in die preferred embodiment.
  • die interleaved data stream could be generated and transmitted in real time without need for intermediate storage.
  • the elements of the interleaved data stream can be any arbitrary amount of data, die boundaries of which can be defined by any number of predefined standards. Therefore, this invention is to be limited only by die following claims, which include all such embodiments and modifications.

Abstract

Virtually random and on-demand access is provided to a virtually unlimited number of subscribers by partitioning the video program into an ordered sequence of n segments and providing the subscribers concurrent access to each of the n subsequences. A data stream representative of the video program is positioned into n subsequences, each representative of one of the n segments. The data of each of the n subsequences is organized as ordered sequence of elements. The elements of each of the n subsequences are interleaved and the interleaved data stream is continuously transmitted over a video program distribution medium at a rate which renders the data representing each segment concurrently available to any subscriber having a receiver capable of selecting, assembling, and displaying the data of a particular segment. The data stream can be compressed prior to interleaving using one of many known video data compression standards and techniques. Data compression can be performed in real time, or iteratively using software. The interleaved data stream can be transmitted in real time, or it can be stored on a storage device such as a hard disk or optical disk for later retrieval and transmission. The interleaved data stream can be ordered using any known standard by which video data is transmitted for reconstruction and display by a receiver. Data can be inserted into the interleaved data stream to inform the receiver to which of the n segments a portion of the interleaved data stream belongs, as well as the encoding level necessary for decompression of the data and time stamps to indicate order of display. Thus, the receiver may begin reconstruction from the beginning of the video program by selecting, assembling, decompressing and displaying the data making up the first segment during a first complete transmission of the interleaved data stream, and performing the same function for the succeeding segments during subsequent transmissions. The receiver can also be programmed to reconstruct any of the n segments in no particular order, approximating random access to the video program.

Description

METHOD AND APPARATUS FOR ENCODING AND FORMATTING DATA REPRESENTING A VIDEO PROGRAM TO PROVIDE MULTIPLE OVERLAPPING PRESENTATIONS OF THE VIDEO
PROGRAM
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to the provision of video programming on- demand, and more particularly to a method and apparatus that encodes, formats, stores and retrieves data representing a video program as a plurality of concurrent, overlapping presentations of the video program to facilitate virtual on-demand access to a single copy of the video program by virtually any number of subscribing viewers.
2. Description of the Related Art
People in the United States spend roughly $7.5 billion annually to rent movies and other pre-recorded video programming for private playback at their convenience.
Such video programming can be rented in many forms, such as video cassette tapes for playback using a video cassette recorder (VCR), video disks for playback on video disk players, or as CD ROM's for playback using personal computers and other forms of CD ROM players. Renting video programming in this manner is desirable because it permits the user to view the programming at any time and in any manner. For example, the user may view some portion of the program at one time and the remainder of the program at some different time. Further, the user may replay certain portions of the program or view the program in its entirety several times. The user may access the program from any point in the program by simply fast-forwarding or reversing through the program. The user is thereby freed from the constraints of available network or cable television programming.
Cable television and direct broadcast satellite (DBS) companies would like to compete in this arena by providing users with the same freedom of use enjoyed through video rental. This service would be known as "video-on-demand." Such companies would clearly enjoy an advantage over video rental establishments in providing such a service because users would not be required to leave the comfort of their own homes to rent a copy of the video program (nor would they have to return it when finished). These companies have been heretofore constrained, however, by existing playback and distribution technology.
It would be prohibitively expensive for a cable television company to provide true video-on-demand using currently known technology. To duplicate the advantages of video rental and in-home playback, the company would have to provide a dedicated playback resource to each cable subscriber, along with an expensive memory array containing a library of video programs from which the subscriber could select programs for playback through the dedicated resource. Further, the cable distribution infrastructure would be required to have sufficient bandwidth to distribute a different video program, or at least a different playback of a video program, to each subscriber connected to the network. Of course this would be impossible without a leap in technology and replacement of the current distribution infrastructure.
One possible compromise would be to produce multiple, overlapping playbacks (i.e. presentations) of the same video program, such that a new presentation of the program would begin, for example, every five minutes. For a two hour video program, a total of twenty-four overlapping presentations of the program would be made available to subscribers. Each subscriber would then have a receiver capable of selectively receiving any one of the twenty-four presentations. Although a subscriber would not enjoy full video-on-demand, the subscriber would have to wait at most five minutes to begin viewing the program in its entirety (or to access any point within the program). Further, the subscriber could fast-forward or reverse through the program by accessing a different one of the overlapping presentations, although he would be constrained to do so over the five minute intervals.
Although such a compromise would decrease both the requisite number of playback resources and the necessary bandwidth, the costs of implementing such a system in currently known technology would still be prohibitive. For the above example, twenty-four playback resources would be required to produce twenty-four separate presentations, each being transmitted over one of a limited number of channels comprising the distribution medium. Further, without sophisticated server technology, such a system might require twenty-four separate copies of the program.
Complex disk-drive arrays or video servers have been recently proposed, each having thousands of video programs stored in their memory and each capable of serving up to two hundred subscribers. The cost of implementing a video-on-demand system for the 57 million current cable subscribers, assuming that such advanced technology could be implemented, would still require an estimated $20 billion in capital investments (about $350.00 per subscriber). Further, full implementation of a service based on such proposed server technology would require that the current cable and telephone distribution network infrastructure be restructured and upgraded over the next several years at a cost of an additional $2 billion per year to increase its bandwidth. Implementing VCR-like functions, such as fast-forward and reverse, would not only increase the complexity of the servers, but it would also impinge on available bandwidth because each subscriber must be able to communicate commands back to his or her dedicated server. Such "back channels" are not even available in the context of existing DBS systems, and most existing cable distribution systems.
The best service that cable television and DBS companies have been able to offer thus far is a pay-per-view service that permits users to request (either over the telephone or directly through the cable network) an offered video program for a fee.
The company then permits the subscriber to receive the selected transmission of the video program. These services are far from video-on-demand, however, as the number of available programs and the number of starting times for the programs are severely limited. Thus, the subscriber must still wait for a scheduled start time at which a desired program will be transmitted over the distribution network. Further, the subscriber does not have the freedom provided by an in-home playback resource such as a VCR; the program is still just passively received.
Thus, there is a need in the art for technology that can provide virtually an unlimited number of viewers with virtually random access to as few as one copy of a video program through as few as one playback resource and that is operable with the existing telephone and cable distribution infrastructure.
SUMMARY OF THE INVENTION
The present invention provides a method and apparatus for encoding and formatting data representing a single presentation of a video program for storage and for transmission that simulates multiple overlapping presentations of the video program using a single playback resource. Put another way, the video program is transmitted as a digital data stream that has been formatted in such a way that it appears to a subscriber that a number of segments of the same program are being continuously distributed over a plurality of subchannels concurrently. By selecting successive segments for presentation over the receiver (e.g. by advancing the subchannel to which the receiver is tuned), an entire presentation of the video program can be assembled. Further, the subscriber can fast-forward or reverse through the program by advancing or decrementing the selected subchannel and thus receive a later or earlier segment of the program.
Thus, a one-hour program formatted in accordance with the present invention could be made to simulate, for example, twenty overlapping presentations of the program with each presentation (i.e. program segment) being three minutes ahead of the previous one. A subscriber would have to wait only a maximum of three minutes to begin receiving the program in its entirety (i.e. until the segments begin again), and would be able to fast forward or reverse through the program at three-minute intervals. Thus, the maximum delay that a subscriber would have to experience to randomly access any point in the program (i.e. the access time) would be three minutes. The present invention resides in its ability to provide this functionality with a single playback resource producing a single formatted data stream that represents a single presentation of the video program. It is well-known in the art that a video program can be converted to a digital data stream for purposes of transmitting the program over a digital distribution medium to subscribers. Video programs are typically organized as a series of scenes or frames, each frame comprising a two-dimensional array of picture elements or pixels. Each pixel has characteristics such as color and brightness which can be quantified as binary data. Audio information associated with the video program can also be converted to a binary representation. In accordance with the present invention, the image and audio portions of a video program are converted to digital data streams using known techniques and standards.
It is also well-known that much of the information contained in a video program is redundant (i.e. pixels in certain regions of the pixel matrix may not change over considerable numbers of frames). Further, areas where changes occur rapidly can often tolerate artifacts that result from truncation of data representing pixel characteristics. Accordingly, the binary data generated to represent a video program can often be compressed considerably, thereby minimizing requisite memory storage and transmission bandwidth. Thus, the video data streams are preferably compressed (i.e encoded) using any known video data compression technique to produce compressed video data streams. The binary data comprising these data streams (both before and after compression) are grouped into arbitrary units called elements; an element can refer to one or more bits of video data where video data refers to all data required to represent a video program, compressed or not, and including but not limited to image, audio and other associated information. The video data streams (compressed or not) are partitioned into n subsequences of elements representing segments of the video program, with each segment comprising an ordered sequence of m elements. The ordered sequence of elements making up each subsequence are interleaved to produce a single interleaved data stream that preferably begins with the first element of each of the n segments, then the second element of each segment and so on in segment order until it ends with the m'th element of the n'th segment. This interleaved data stream is continuously transmitted over the distribution medium from beginning to end.
A subscriber with an appropriate receiver can reconstruct the entire program, starting when the transmission is at the beginning of the interleaved data stream, by sequentially selecting and assembling the m elements of the first segment as the receiver serially parses through the interleaved data stream. The receiver reconverts the selected and assembled elements back into image and/or audio in real time for presentation of the first segment to the subscriber. As transmission of the interleaved data stream begins again, the receiver selects and assembles all of the elements of the second segment for reconstruction, and repeats this process until it completes its n'th pass through the data stream to select and assemble the m elements comprising the n'th segment. The receiver continuously decompresses (i.e. decodes) and reconverts the assembled segments in real time to reconstruct the video program in segment order for viewing by the subscriber.
As long as the rate of transmission of the interleaved data stream is at least "n" times the average data rate "r" of the individual segments, the system will operate properly. Thus, for a given value of r, the throughput of the resource used to transmit the interleaved data stream defines the number of possible segments into which the program may be divided. The viewing time of one of the n segments defines the access time "T" of the system, which is the minimum time interval between accessible points in the program. Further, the time necessary to transmit the entire interleaved data stream once must be less than or equal to T.
Thus, a subscriber receives access to an ordered sequence of n segments of the video program concurrently over n subchannels, which means any number of subscribers can be concurrently reconstructing n overlapping presentations of the video program, each presentation running ahead of its predecessor by an amount of time T required to reconstruct one video segment. The formatting of the data stream representing the video program operates analogously to the process of time-division multiplexing information received from a plurality of communications channels. In the context of communications, however, each channel carries a different conversation or program, whereas the present invention exploits similar principles to break up and transmit a single program over separate subchannels of the same channel. It is this unique application of TDM principles to the context of video-on-demand which forms the basis for the instant invention. An alternate preferred embodiment of the invention adapts the idea of statistical encoding to the interleaving process so that video segments that require more data to maintain desired picture quality are allocated more data while other segments of the program requiring less data are allocated less data such that the overall allocated bandwidth remains the same. In this embodiment, the video data streams are partitioned into subsequences representative of the segments first and then each subsequence is compressed and interleaved through a statistical multiplexer. This embodiment, although more complex in implementation, provides more uniform picture quality throughout the program.
In the preferred embodiment of the invention, the compression and interleaving processes are performed interactively through a combination of software and hardware, and the resulting encoded and formatted (i.e. interleaved) data stream representing the program is intermediately stored on a disk or some other form of mass storage. The interleaved data stream can then be sequentially retrieved and cyclically transmitted over the distribution medium to subscribers by some relatively simple embodiment of a video server.
It is therefore an objective of the present invention to provide services which approximate video-on-demand to cable television and direct broadcast satellite subscribers while minimizing the requisite playback resources and minimizing the requisite bandwidth to remain compatible with the existing distributional infrastructure.
It is further an objective of the present invention to format a video data stream representative of a single presentation of a video program such that multiple overlapping presentations of the video program can be reconstructed from the repeated transmission of the formatted data stream by a virtually unlimited number of subscribers.
It is still further an objective of the present invention to provide subscribers with the ability to selectively access and reconstruct any one of n segments of the video program such that a subscriber may fast forward or reverse through the program between segments such as with a VCR. It is still further an objective of the present invention to use statistical techniques in encoding and formatting the data stream to minimize any variation in picture quality throughout the presentation of the video program.
These and other advantages of the present invention will be appreciated in view of the following detailed description and the figures referenced therein.
BRIEF DESCRIPTION OF THE DRAWINGS
Figure 1 illustrates the interleaving and de-interleaving of a data stream representative of a video program.
Figure 2 is a block diagram illustrating a first preferred embodiment of the invention.
Figure 3 is a data structure diagram illustrating an interleaved sequence of elements formed by the system of Figure 2.
Figure 4 is a diagram illustrating a possible layout of video data representative of a program as formatted and stored on a mass storage device by the first preferred embodiment of the invention.
Figure 5 is a block diagram illustrating a second preferred embodiment of the invention that employs statistical encoding and interleaving.
Figure 6 is a diagram illustrating the statistical encoding and interleaving of video data according to the second preferred embodiment of the invention. Figure 7 illustrates, in more detail, the statistical encoding and interleaving process as performed by the second preferred embodiment of the invention.
Figure 8 is a block diagram showing an encoder of the second preferred embodiment of me invention in greater detail.
Figure 9 is block diagram illustrating a possible layout for mass storage of a sequence of elements as statistically encoded and formatted by the second embodiment of the invention.
DETAILED DESCRIPTION OF THE INVENTION
As previously discussed, one way to closely approximate video-on-demand is to provide subscribers with access to multiple overlapping presentations of the same video program. The greater the number of overlapping presentations available to a subscriber, the more starting times there will be and the closer the approximation to complete random access of the video program. Complete random access might be achieved using this scheme if the number of overlapping presentations is made, for example, equal to the number of frames in the video program. As a practical matter, however, the number of overlapping presentations does not have to be nearly that numerous. As previously discussed, however, even a satisfactory approximation of random access to a video program using this technique would be prohibitively expensive using known or even envisioned technology.
The present invention resides in a unique application of time-division multiplexing principles (TDM) to provide subscribers with continuous access to multiple segments of a video program, thereby facilitating reconstruction of multiple overlapping presentations of the same video program. This functionality is accomplished through the repeated transmission of a single, specially formatted data stream representative of one complete presentation of the video program. The number of overlapping presentations of the video program that can be reconstructed from the formatted data stream, and thus the minimum access time between points in the program, is directly related to the rate at which the data stream can be transmitted over the distribution network as well as the rate at which data must be received by the receiver to permit real-time reconstruction of the video segments at the desired level of picture quality.
The present invention is unique because TDM principles are typically applied where a number of distinct data streams, each representing a different conversation or program, are to be transmitted over the same high-speed trunk such that all of the conversations or programs are simultaneously accessible by a receiver. The present invention, however, applies these principles to transmit n different segments of the same program over n different subchannels of the same channel, thus making each segment of the video program simultaneously accessible to a subscriber's receiver over the same channel of the distribution network. The receiver can therefore be made to reconstruct and present any of the n segments to the subscriber by selecting the subchannel carrying the desired segment.
The following detailed description is made with reference to Figures 1-9, in which like reference numerals indicate identical elements throughout all of the
Figures. Figure 1 illustrates the essential principles of the instant invention.
A video program is typically organized as an ordered sequence of scenes or frames, with each frame defined by a two-dimensional array of picture elements or pixels. A pixel has characteristics of color and intensity of illumination which, when combined with other pixels, creates an image or frame. For a given frame, the characteristics of each pixel can be represented digitally. Thus, it is well known in the art that a video program can be converted into a digital data stream 10 that is an ordered sequence of bits which describes pixel values for each pixel of the aray during each frame of the video program. It is also known that audio associated with the program can also be converted to digital data, and can be identified with the image data on a frame-by-frame basis.
Thus, a video program is first converted to a digital data stream 10 in accordance with well-known standards and procedures. The data stream 10 can represent images, audio or both, although in the preferred embodiment, separate data streams 10 are produced for the image and audio portions of the program. It will be assumed for ease of discussion and illustration that the data stream 10 throughout the
Figures represents the image portion of the video program, although audio data streams can be processed in accordance with the present invention as described for image data.
The data stream 10 is typically organized into an ordered sequence of elements, where as previously defined, an element is an arbitrary quantum of data that can be one bit, the number of bits representative of a single pixel, or the number of bits representative of one or more frames of pixels. In the case of audio, an element can be from one to several bits of audio data.
The video program is then divided into a number of video segments n, of approximately equal duration. Preferably, as many segments as possible given the rate at which the receiver must receive data to present the program in real time, and the maximum transmission rate which the playback and distribution resources are capable of sustaining. The segments are created by partitioning the data stream 10 into n ordered subsequences of m elements, each subsequence representative of one of the n segments of the video program. Thus, the video program, as represented by data stream 10, starts at the beginning of the sequence of elements (i.e., the first element of segment 1 as denoted by reference numeral 12), and ends with the last element of segment n (i.e., denoted by reference numeral 16).
In the preferred embodiment, the elements of each segment are interleaved to produce an interleaved sequence of elements called an interleaved data stream 18 as shown in Fig. 1. The interleaved data stream is constructed by arranging into frames identically numbered elements of each segment in segment order, beginning with the first element of each segment, and ending with the m'th element of each segment.
Thus, the n video segments which make up the video program are essentially time-division multiplexed over n "subchannels" of the channel over which the video program is broadcast. The interleaved data stream 18 is organized into a series of m "frames" 13, with each frame partitioned into n "time slots" in segment order (i.e. one time slot is assigned to each segment). During each frame, one element belonging to each segment is transmitted during the time slot assigned to that segment.
The interleaved data stream 18 is repeatedly transmitted over a single channel of the television cable to subscribers. Provided that the transmission rate of the interleaved data stream 18 is at least n times the average data rate r of the individual segments, it will appear as though all n segments of the program are available to the receiver simultaneously. Each new transmission of the interleaved data stream 18 provides a subscriber an opportunity to begin restoration of the video program from its beginning, or to continue restoration of the program initiated during previous transmissions of the interleaved data stream.
The subscriber can view the program from its very beginning by simply programming the receiver to select and reconstruct elements associated with segment 1 (i.e. the first "subchannel") when transmission of the interleaved data stream is at the beginning of the interleaved data stream 18 (i.e at time tl). As the interleaved data stream 18 is transmitted, the subscriber's receiver selects and assembles (i.e. de-interleaves or demultiplexes) the elements which form the first segment 14. Concurrently, the receiver converts the elements comprising segment 1 into a presentation of the segment for viewing by the subscriber. From the beginning of the next transmission of (or pass through) the interleaved data stream 18 (i.e. at time t2), the receiver is altered (i.e. is tuned) to the next subchannel to select and assemble me elements representative of the second segment 15. Meanwhile, another subscriber can simultaneously begin accessing the program by selecting and assembling data representing the first (or any other) segment 17. This process continues for the first subscriber until the receiver has received and reconstructed all n segments of the video program, or until the subscriber manually alters the subchannel to which the receiver is tuned. Reconstruction will continue from the newly selected segment.
Thus, virtually any number of subscribers can simultaneously access any of the n segments, all of which are accessible from the transmission of a single data stream representative of a single presentation of the video program. Accordingly, the invention achieves the significantly important result of providing multiple subscribers virtually random access to a video program while minimizing the storage and distribution infrastructure necessary to provide this access. As previously discussed, a subscriber can begin accessing and reconstructing the program from any segment, even at a time when the transmission is currently in the middle of the interleaved data stream 18 and therefore in the middle of each of the n segments. A subscriber can fast forward or reverse through the program by advancing or decrementing the segment currently being selected by the subscriber's receiver. Such steps forward or backward in the program are, however, constrained to multiples of the access time T. The granularity of access points in the video program is a direct function of the number of video segments into which the video program has been divided. The greater the number of video segments n, the shorter their duration and thus the finer the steps between them. Thus, n is referred to as the
"interleaving factor." As previously discussed, the number of video segments into which a program can be divided is limited by the bandwidth available for transmitting the n interleaved segments and die maximum data rate that can be sustained by die playback resource. One way to reduce the data rate that must be sustained by the playback resource for a given interleaving factor is to reduce the total amount of data necessary to adequately represent the video program. Well-known techniques in the art can be used to encode the data stream 10 of Figure 1 to compress the amount of data necessary to adequately represent die program. Two examples of well-known digital compression standards for video data are die MPEG-1 and MPEG-2 standards for
Generic Coding of Moving Pictures and Associated Audio. The ITU-T (International Telecommunications Union Telecommunications Standardization Sector) Draft Recommendation H. 262 (10: 18 Friday 25 March 1994) is incorporated herein by this reference. A further benefit of data compression is that the amount of storage space necessary to store interleaved data streams 18 representative of video programs for later transmission is also reduced.
A first preferred embodiment of the present invention will be described which compresses the video data stream 10 and then interleaves the resulting compressed data elements. A second preferred embodiment employs die principles of statistical multiplexing to statistically encode and interleave the video data stream 10 to ensure a more uniform picture quality for the reconstructed video program as viewed by a subscriber. Nonstatistical encoding and interleaving may be simpler to implement with existing encoder and decoder products and can be easily applied to independently compressed video data streams. Statistical encoding and interleaving may result in significantly improved uniformity of picture quality, but may require more sophisticated encoding structures than non-statistical encoding and interleaving. 125
1. First Preferred Embodiment: Non-Statistical Encoding and
Interleaving
Figure 2 illustrates a first embodiment for performing nonstatistical encoding and interleaving according to die invention. In Figure 2, a video program source 20 provides a data stream 10 that represents, for example, the image portion of the video program. The data stream 10, which is typically an ordered sequence of digital data representing pixels embodying me images of the video program, is compressed by a digital encoder 22, employing any well-known procedure for compressing digitized video data. The digital encoder 22 produces a compressed video data stream 23 that is organized as an ordered sequence of elements of compressed data representing the video program as embodied in the data stream 10. In the preferred embodiment, d e elements are made up of equal numbers of bits which may not be organized based on pixel boundaries.
The compressed video data stream 23 is then partitioned into n multi-element subsequences of elements, each representative of one of n video segments. The elements of the n subsequences are then interleaved by a partition and interleaving circuit 24 in accordance with the method of the invention as previously described. The partition and interleaving circuit 24 produces an interleaved data stream 25 that can be directly transmitted to subscribers or that can be stored on a mass data storage medium 27 by data storage unit 26 for future retrieval and transmission. Data storage medium 27 may consist of one or more hard disk, optical, tape, or electronic memory devices.
In a preferred embodiment of the invention, the interleaved data stream 25 will typically be stored on data storage device 27 for future playbacks of the video program embodied in the interleaved data stream 25. The data storage device 27 is placed in a system that provides on-demand video programming including a server 31, and a plurality of receivers 32, 34 that are coupled to die server 31 by an appropriate distribution network 35. The server repeatedly transmits the interleaved sequence of elements stored on die data storage device 27 to all receivers on die distribution network 35. Each of die receivers 32, 34 includes die processing capacity necessary to reconstruct the video program in the manner described above by selecting, for each video segment, die sequence of elements comprising that segment. In this regard, each receiver includes the processing capacity to de-interleave die interleaved data stream 25, to reassemble and decode die resulting compressed data stream and to reconvert the resulting decompressed data stream 10 back into the sequence of pixels that embodies die images of the video program. Figure 3 illustrates the partitioning of the compressed data stream 23 into subsequences representing die video segments. As the sequence of elements is encoded (i.e. compressed) by the digital encoder 22, the resulting compressed data stream 23 is produced as illustrated in Figure 2. The compressed data stream 23 is then partitioned by the partition and interleaving circuit 24 into n ordered subsubsequences of elements each representative of one of n video segments. Each subsequence is further partitioned into m elements so that the total number of video program elements is n , and each element is denoted by Sij, where 1 ≤ i ≤ n and 1 ≤ j ≤ m. As Figure 3 illustrates, partitioning renders die compressed data stream 23 into an ordered sequence of n subsequences, each subsequence including one or more elements, each element including one or more bits of compressed video data. Figure 3 also illustrates the interleaved data stream 25 which results from the interleaving process. Figure 4 illustrates the storage of die interleaved sequence of elements comprising the interleaved data stream 25 on data storage device 27. Referring once again to d e interleaved data stream 25 shown in Figures 3 and 4, and to die combination for storage and formatting illustrated in Figure 2, it should be evident that one or more data storage devices 27 can be employed to store the interleaved data stream 25. The required number of data storage devices may be determined by design considerations taking into account die size of die compressed bit stream 23, die interleaving factor n, and d e access speed and capacity of individual data storage devices. Assuming mat die data storage device is a hard disk drive, the interleaved sequence 25 can be split into different, but equally-sized sections, and each section can be recorded onto one of Nd different hard disks. The capacity of each data storage device 27 is assumed to be sufficient to contain nm/Nd elements. A more detailed discussion of those considerations will be presented below.
2. Second Preferred Embodiment of the Invention; Statistical Encoding and Interleaving
Figure 5 illustrates a second preferred embodiment of the invention which performs statistical encoding and interleaving. A video program source 60 provides a data stream 10 representative of a sequence of pixels that embodies d e image portion of die video program. The data stream 10 is organized as an ordered sequence of elements, each element comprising data representing an equal number of pixels. The video program is partitioned into n segments, with each video segment represented by a subsequence of data stream 10, each subsequence representative of an equal number of pixels. The n subsequences representative of the n video segments are aligned in time by the partition and alignment circuit 62 and each subsequence is concurrendy provided on a signal line 63 to a respective digital encoder 64. Each digital encoder converts its respective subsequence into a compressed subsequence of elements, each element consisting of one or more bits of compressed video data. Elements of the compressed video data stream 65 are interleaved by an interleaving circuit 66 to produce an interleaved data stream 67. In the preferred embodiment, a data storage unit 68 receives the interleaved data stream 67 and transfers it to a data storage device 70 for later retrieval and transmission. As previously discussed, die interleaved data stream 67 can also be transmitted immediately to subscribers over a distribution network in realtime.
For playback of the video program embodied in me interleaved sequence of elements 67, the data storage device 70 is placed in a system mat provides on-demand video programming. The system preferably includes a server 72, and a plurality of receivers 73, 74 that are coupled to the server 72 by an appropriate distribution network 75. The server repeatedly transmits the interleaved data stream 67 stored on die data storage device 70 to all receivers connected to the distribution network 75. The interleaved data stream 67 is repeatedly transmitted to all receivers. Each of the receivers 73, 74 includes die processing capacity necessary to reconstruct the video program by selecting and assembling, for each video segment, the subsequence of elements representative of the segment. In this regard, each receiver includes die processing capacity to de-interleave (i.e. demultiplex) die interleaved data stream 67 to thereby reproduce compressed data stream 65, to decode compressed data stream 65 into data stream 10, and to convert the data stream 10 to the sequence of pixels mat embody die video program.
A more detailed discussion of die statistical encoding and interleaving process performed by die combination illustrated in Figure 5 is now presented with reference to Figures 6 and 7. The data stream 10, which is a digitized representation of the sequence of pixels embodying die images of die video program, is partitioned into n subsequences of elements representative of n video segments. Each element of the subsequence consists of data diat represents the same number of pixels. Each subsequence has an equal number of elements and therefore represents the same number of total pixels.
Each pixel of the subsequence representing die first segment 84 of partitioned data stream 80 has a video intensity value s(t), where each pixel is defined by its position in the subsequence as a function of time. Because the duration of a segment is defined to be T, die video intensity value for each pixel of die second subsequence representing die second segment 86 is defined by s(t + T), and for all pixels is defined by s(t + i«T) were O ≤ i < n. Each of die subsequences of partitioned data stream 80 are then separately assembled and input to their respective decoders simultaneously as shown in Figures 6 and 7. Thus, the segments are time shifted for purposes of concurrent processing.
Each encoder 64 of Figure 7 processes its own subsequence of digital pixel data and die preferred embodiment of such an encoder is shown in more detail in Figure 8. Such encoders are well known in the art and die details of die encoder disclosed are not relied on for patentability of the invention. Those of skill in the art will recognize that any encoder suitable for compressing video data within the context of the prevent invention can be substituted therefor.
As illustrated in Figure 6, each encoder compresses die data comprising its respective subsequence of pixels, producing a sequence of elements consisting of compressed data called blocks 82. The encoder performs this function by first organizing its subsequence of pixel data into elements called blocks with a blocking circuit 100 (Figure 8). Each block consists of data representing an equal number of pixels. Each block represents some subset of die total number of pixels embodying the segment. A block can consist of data representing all of die pixels that make up a single frame or image of the segment. As an example, a frame may contain 480 lines of 720 pixels, or 345,600 pixels total. Blocks can arbitrarily consist of pixels making up fractions of frames, single frames or even multiples of frames.
Each element or block of pixel data is dien processed dirough a transform circuit 102, which uses well-known compression transforms, such as the discrete cosine transform (DCT) to convert the binary pixel data to transform coefficients.
The coefficients are quantized or normalized by adaptive quantizer 103, which improves coding efficiency by truncating die coefficients in accordance widi the desired level of picture quality to be produced when die information is decompressed for presentation. Finally, the quantized coefficients are passed dirough a statistical encoder 110 which further compresses die data based on a variable length encoding technique such as Huffman coding.
Each of die encoders is synchronized so that each produces a block of compressed data simultaneously, and each encoder is preset to a coding level that reflects e desired quality level of the reproduced picture. Each compressed block (i.e. element) of data produced by an encoder, although still representative of an equal number of pixels, may vary in its amount of binary data because some blocks of digital pixel data will compress more easily than others. Block multiplexer 66 (Figure 7) then interleaves the blocks by selecting a block from the first encoder (and thus from the first segment), then the second and so on, up to encoder n, and then repeats the process for the next set of compressed blocks. Thus, an interleaved data stream 94 is produced as shown in Figure 6, widi each block 82 designated first by a segment number and then by block (i.e. element) number.
The interleaved data stream 94 is then input to a first-in-first-out (FIFO) channel buffer 92 (Figure 7) and clocked out of die buffer at the transmission rate necessary for proper operation (i.e. n • r). If the buffer begins to fill up, a signal is provided over line 93 (Figure 7) which instructs each of the adaptive quantizers 103
(Figure 8) in each of die encoders 64 (Figure 7) to increase the coding level which will increase the compression ratio (i.e. be less accurate with respect to the coefficient values) until the buffer starts to empty out. Likewise, if the buffer becomes too empty, the channel buffer 92 can instruct the adaptive quantizers over signal line 93 to decrease die coding level which will decrease the compression ratio.
If the number of segments "n" and the channel buffer 92 are sufficiently large, then it is statistically probable that die number of data intensive compressed blocks (i.e. those blocks which were difficult to compress) will balance out with those compressed blocks containing less information (i.e. those blocks which were more easily compressed) over any window of time. Thus, the picture quality should be reasonably consistent throughout the video program and the number of changes in the compression ratio during encoding and interleaving should be minimal.
An example of mis encoding technique as applied to multiple programs being multiplexed over a multichannel trunk is disclosed in U.S. Patent No. 5,216,503 to Paik et al.. which is incorporated herein by this reference. In Paik et al., this technique is applied to ensure that die sum of die data transmitted at any time over all of the channels of a trunk does not exceed die total bandwidth of the trunk. Application of this technique to die instant invention is unique because it is applied to segments of the same program to achieve uniform picture quality during playback of die single program.
The same encoder diat is illustrated in Figure 8 can also be used in a nonstatistical encoding combination as in die first embodiment. In this case, each of the "n" encoders of Figure 7 could maintain an independent channel buffer, and the compression ratio of each individual encoder would be adjusted in order to maintain a pre-assigned fraction of the total throughput. The disadvantage of the nonstatistical system is that the encoders will tend to deliver unnecessarily high picture quality at times when the video program is easily compressed, and poor picture quality during certain complex scenes when compression becomes difficult.
3. Additional Implementation Details Common to Both Preferred Embodiments The application of video compression does not necessitate real-time digital encoding as implied in die first and second embodiments, nor does it require multiple encoders. Another preferred way of interleaving and compressing a data stream 10 representative of a video program to take advantage of die statistical variations in the video program is to iteratively compress and interleave the video program using an algorithm that is wholly or partly implemented in software and uses multiple encoding passes through die video program. The algorithm can more efficiently compress the video program by tuning die compression algorithm during each iteration.
An encoding process implemented in software may use a video compression standard such as the one implemented by the encoder of Fig. 8. For the first embodiment, die software compresses the data stream 10 in accordance widi a desired quantization level, and then measures the total amount of data that results. If the amount of data exceeds that which can be transmitted within the specified bandwidth, the quantization level is increased and die data stream 10 is compressed again. This process continues until the resulting compressed data stream falls within die requisite bandwidth. The software then partitions the compressed data stream and interleaves the elements of the segments of the partitioned subsequences to produce an interleaved data stream representative of the video program.
Iterative compression through software is particularly advantageous when statistically encoding and interleaving because it does not require die use of costly hardware to split die ordered sequence of pixels represented by die data stream 10 into video segments, nor does it require multiple encoders to simultaneously compress each video segment. Radier, the data stream 10 is first partitioned into subsequences of elements representing segments with die same number of pixels. The partitioned data stream is then compressed in accordance with a specified encoding level using a standard such as die one implemented by die encoder of Fig. 8. The software then interleaves the compressed elements of the subsequences in accordance widi die invention. As die software interleaves the elements, it sums the total amount of data across die segments widiin a certain window, much like die channel buffer 92 of Fig. 7. Any sections of the interleaved data stream for which the amount of data exceeds die specified bandwidth over the defined window are compressed again using a higher coding level. For any sections which exceed some minimum amount of data, die program can recompress the section using a lower coding level.
One way to save some computer time would be to use a hardware encoder such as die one shown in Fig. 8 to compress die data stream 10 at a number of different coding levels and to store die different compressed versions. The software can perform the steps of interleaving and summing, and for any section of the interleaved data stream that is detected to be sufficiently over or under die specified bandwiddi, the software can simply replace that section with a section of one of the stored interleaved data streams that has been compressed at die appropriate level of quantization.
During each transmission of the interleaved data stream, a receiver must be able to identify die elements of the particular video segment to be accessed, reconstructed and displayed. One possible implementation by which this can be accomplished is by inserting video segment identifiers into die interleaved sequence of elements as they are being stored for future transmission. For example, sequence numbers (i.e., subchannels) ranging from 1 to n can be assigned to video segments 1 to n respectively. The appropriate sequence number can then be associated widi die video data comprising each element prior to interleaving and storage. Thus, as a receiver begins to reconstruct a video program from its beginning, the receiver selects and assembles each element having a sequence number of 1 during die first pass, followed by die elements having a sequence number of 2 during die second pass and so on. The reconstruction process is complete when all of the elements having a sequence number of "n" are selected during die final pass. Of course, a subscriber could program die receiver to jump ahead or backward in die program by manually programming the selected sequence number (i.e., by changing the subchannel).
Another possible implementation is to program the playback resource to insert the video segment identification information into the interleaved sequence of elements as they are being transmitted to die receivers. For example, the same sequence numbers ranging from 1 to n can be initially assigned to video segments 1 to n respectively for a first transmission. At the beginning of the next pass through the interleaved data stream, the playback resource will decrement each segment's assigned sequence number such that segment number two currently has sequence number one assigned to it, while segment one currently has sequence number "n" assigned to it. Therefore, once the receiver begins to reconstruct the video program based on a particular subchannel or sequence number, it can remain tuned to that number throughout the entire reconstruction. One disadvantage of this implementation is that the first segment, which always represents the beginning of the video program, is no longer associated widi a single sequence number. Therefore, the playback resource must insert additional information into die interleaved data stream prior to its transmission to demarcate the first segment of the video program.
Another possible implementation would be to demultiplex the elements at the receiver (i.e. 32, Fig. 2 or 74 Fig. 5) synchronously, with each time slot in a frame assigned to a particular segment of die program.
As previously discussed, die receiver (i.e. 32, Fig. 2 or 74 Fig. 5) also must be able to decode die compressed data and display die right pixels in die right sequence and at die right time. In die case of audio data, die receiver must also be able to synchronize the audio widi the images. Finally, die receiver must know what the coding level of die compressed data is at all times. As previously discussed, diere are a number of well-known standards for representing video data digitally as well as compressing it mat specify how such information be provided in die data stream.
For example, the interleaved sequence of blocks comprising the interleaved data stream 94 (Fig. 6) as generated by the second embodiment combination shown in Figures 6, 7, and 8 may be packetized by a packet encoder 121 (shown in Fig. 7). As an example, die interleaved sequence of blocks 94 may be divided into groups of packets, with each packet in the group containing compressed video data representing pixel values for die same sequence of pixels during each of die n segments. Put another way, each packet might be comprised of a block from each segment, each block containing the pixel values s(t) for corresponding pixels of frames occurring at time t, t+T, t+2T ... t+(n-l)T.
A single packet may be organized as shown in Figure 9. It should be noted diat, in the preferred embodiment, die size of a packet may vary because some blocks of video may be more difficult to compress man others. Odier information included in a packet might be at which defines packet and block boundaries, as well as die coding level of the data in the packet.
Anodier way of organizing the data is to create packets of data during the interleaving process, such that each packet contains some arbitrary amount of data from a single subsequence representing a segment, which may or may not be on block boundaries. The packets are interleaved radier than the blocks, and information defining die segment, die particular pixels represented by the data and a time stamp which defines a display time for the pixels defined by die data in die packet are included as a header. In this case, the order of interleaving is determined by increasing time stamp value instead of by segment number.
As previously discussed, die required amount of storage and die necessary throughput of the storage device for playback of a video program depends on several design parameters. One of the most important parameters to be specified is die access time "T", which is the time duration of each video segment as it is reconstructed by a receiver for viewing by a subscriber. This parameter defines die maximum delay diat a subscriber must endure when die subscriber randomly accesses the program for playback from its beginning. Playback is a real-time process involving continuous retrieval and transmission of the same interleaved video data stream, and die continuous reconstruction of a video program from that interleaved data stream. Because all n video segments are presented for access by a receiver simultaneously, it follows that any of the n video segments can be accessed at any time during playback. Playback of all video segments begins at the same time and completes at the same time. Thus, die access time T also defines die maximum delay diat will be endured to access any point in the video program. Stated differently, it defines the minimum interval of time over which the program can be randomly accessed.
Anodier parameter to be specified is die average data rate "r" of die compressed video segments of the video program. This parameter defines die rate at which segment data must be received by a receiver so that it can reconstruct die video program in real-time at the desired level of quality. The data rate "r" is determined by die initial image resolution and frame rate of the video program and by die compression ratio that was applied during its encoding. It is expected diat most applications of this invention will not require better than VHS picture quality and it is reasonable to assume that this can be achieved if r = 1.5 Mbits/second. If statistical multiplexing is used, men the data rate of each compressed video segment will vary because some data segments will compress more easily than others. As previously discussed, however, die cumulative data rates of all n segments will tend to be relatively constant from one program to the next.
For a receiver to have simultaneous access to each segment, the rate at which the interleaved data stream is transmitted over the distribution must be equal to the number of segments n times the average data rate of one segment r (i.e. n-r). Thus, the distribution system and die storage facility from which die interleaved data program is to be retrieved (or the encoding and interleaving apparatus used to generate the interleaved data stream if interim storage is not used), must be able to sustain this data rate. One of the advantages of the present invention is that the cylinders and tracks of a hard disk can be accessed sequentially or in any sequence which maximizes the throughput performance of the unit. Sequential access is made possible by the fact that the playback process accesses, retrieves and transmits the same interleaved sequence of elements over and over.
Thus, there is no random access requirement imposed on the interim storage unit. Assuming that a particular disk drive device can sustain a data rate of 30 Mbits/second, if r = 1.5 Mbits/second, then n cannot exceed 20. As previously defined, die total duration of die video program is n-T. Assuming that d e video program is two hours in duration, the duration T of each segment is 6 minutes. The storage capacity required for 120 minutes of video compressed to an average data rate of r = 1.5 Mbits/second is dierefore, 1.35 GBytes.
In addition to compressing the data and maximizing the data rate of interim storage, the minimum access interval may be further reduced by splitting the interleaved data stream into equal portions, storing the portions on separate disks, and then accessing each disk concurrently to retrieve and transmit each portion of the interleaved data stream simultaneously. Effectively, each video segment has been subdivided into equal portions, with each portion of each segment made available to a receiver simultaneously over a different communications channel. For example, suppose that the interleaved data stream representative of the two-hour program of the previous example is divided into diree equal portions, with each portion stored on a separate disk. If die portions from each disk are retrieved and transmitted simultaneously over three separate channels, then the first one-third of each segment will be available on a different subchannel of die first channel, the second one-third of each segment will be available on a different subchannel of die second channel, and die diird one-third of each segment will be available on a different subchannel of the diird channel. The access interval is thereby reduced from T = 6 minutes to Td = 2 minutes, where Td = T/Nd (Nd = the number of disks). The minimum capacity of each disk is likewise reduced to 40 minutes of video compressed to an average data rate of r = 1.5 Mbits/second, or 450 MBytes.
It will be appreciated by diose of skill in the art that multiple disks also may be used in die event diat die total data comprising the interleaved data stream exceeds die storage capacity of a single storage device such as a disk, or if the required throughput of the single storage device is not sufficient to meet iat specified by n-r. Two preferred mediods of partitioning the interleaved sequence of blocks for writing on multiple disks are now described. The purpose of these mediods is to create overlapping data streams from each disk that remain relatively constant in time relative to one another. Those of skill in the art will appreciate that there are many possible schemes for partitioning the data which are contemplated to be widiin the scope of the present invention.
One partitioning method is to split a packetized interleaved sequence of elements at packet boundaries such diat the same number of packets are written to each disk. For example, if each packet contains data diat is representative of a fixed number of pixels, then the first K packets may be allocated to a first disk, the next K packets to a second disk, and so on, until finally die last K packets are assigned to the last disk. The total number of packets must be equal to die product of K and die number of disks. This partitioning memod may be referred to as die "constant pixels" method, because all of the disks contain video data diat is representative of the same number of pixels.
A second preferred method of splitting the packetized interleaved sequence of blocks for writing on multiple disks may be termed die "constant bits" method. For die constant bits method, die same amount of compressed video data is allocated to each disk. The interleaved sequence of blocks 94 is divided into equal sections, one for each disk, but die partitioning is not constrained to occur at a packet boundary.
If the total amount of video data in die interleaved data stream is not evenly divisible by die number of disks to be used, dien die sequence may be extended by inserting fill data at die end or at certain points where they will not affect the decoding process. Clearly, other embodiments and modifications of die present invention will occur readily to diose of ordinary skill in the art in view of these teachings. For example, encoders based on odier known compression techniques may be substituted for that disclosed in die preferred embodiment. In addition, die interleaved data stream could be generated and transmitted in real time without need for intermediate storage. The elements of the interleaved data stream can be any arbitrary amount of data, die boundaries of which can be defined by any number of predefined standards. Therefore, this invention is to be limited only by die following claims, which include all such embodiments and modifications.

Claims

What is claimed is:
1. A method of providing concurrent access to an ordered sequence of a plurality of n segments comprising a video program, said me iod comprising the steps of:
partitioning a data stream comprising an ordered sequence of data representative of the video program into n subsequences, each of the n subsequences comprising a portion of the data stream representative of one of the n segments, the data of each of die n subsequences organized as an ordered sequence of elements; and
interleaving die elements of the n subsequences to produce an interleaved data stream.
2. The method of Claim 1 wherein die ordered sequence of elements of each of the n subsequences comprises a plurality of m elements.
3. The method of Claim 2 further comprising the step of compressing the data stream whereby each of the m elements of each of the n subsequences comprises compressed data.
4. The method of Claim 3 wherein each of the m elements comprises an equal amount of compressed data.
5. The method of Claim 4 wherein said interleaving step further comprises die step of grouping die first elements of each of the n subsequences togedier in segment order, then likewise grouping the second through mth elements of each of the n subsequences.
6. The method of Claim 1, 5 or 15 further comprising the step of repeatedly transmitting the interleaved data stream over a video program distribution medium for a predetermined period of time.
7. The method of Claim 1, 5 or 15 further comprising the step of storing the interleaved data stream on a storage device.
8. The method of Claim 7 further comprising die steps of:
retrieving the interleaved data stream from the storage device;
transmitting the retrieved interleaved data stream over a video program distribution medium; and
repeating said retrieving and transmitting steps continuously for a predetermined period of time.
9. The method of claim 6 wherein a receiver must receive die ordered sequence of elements of one of the n subsequences at a rate r to present the segment represented by die one of die n subsequences to a viewer in real time without interruption, and wherein said step of repeatedly transmitting is performed at rate equal to or greater dian the product of n and r.
10. The method of claim 8 wherein a receiver must receive die ordered sequence of elements of one of the n subsequences at a rate r to present die segment represented by die one of die n subsequences to a viewer in real time without interruption, and wherein said step of transmitting the interleaved data stream is performed at rate equal to or greater dian the product of n and r.
11. The method of Claim 3 wherein said step of compressing the data stream is performed statistically.
12. The method of Claim 11 wherein the data comprising die data stream is representative of an ordered sequence of pixels comprising die video program, and wherein each of the m elements of each of the n subsequences comprises a portion of the data stream representative of an equal number of pixels.
13. The method of Claim 12 wherein said step of statistically compressing further comprises die step of encoding each of die n subsequences in accordance widi a specified compression ratio to produce n encoded subsequences of elements, each of the elements comprising compressed data representative of its ordered sequence of pixels.
14. The method of Claim 13 wherein said interleaving step further comprises the step of grouping die first elements of each of die n encoded subsequences togedier in segment order, men likewise grouping die second dirough mdi elements of each of the n encoded subsequences to produce an interleaved data stream comprising m groupings of n elements.
15. The method of Claim 14 wherein said step of statistically compressing further comprises the steps of:
summing the amount of compressed data in each of the m groupings widi die data in a predetermined number of its preceding groupings;
increasing die specified compression ratio whenever the summed data exceeds a first predetermined diroughput; and
decreasing die specified compression ratio whenever die summed data falls below a second predetermined dirough put.
16. The method of Claim 15 wherein said step of encoding each of die n subsequences is performed on each of die n subsequences concurrently and wherein each of the n encoded subsequences of elements are produced synchronously.
17. The method of Claim 16 wherein said interleaving step, said summing step and said steps of increasing and decreasing die compression ratio are performed in real time on said n encoded subsequences of elements.
18. The method of Claim 14 wherein said step of statistically compressing further includes die steps of:
summing the amount of data comprising each of a plurality of portions of the interleaved data stream, each of the plurality of portions comprising a different one of die m groupings and a predetermined number of its succeeding groupings;
increasing the specified compression ratio for each of the plurality of portions having a sum of data diat exceeds a first predetermined diroughput; decreasing die specified compression ratio for each of the plurality of portions having a sum of data diat falls below a second predetermined diroughput; and
repeating said steps of encoding, interleaving, summing, increasing the specified compression ratio and decreasing the specified compression ratio until no further increases or decreases in die specified compression ratio are required.
19. The method of Claim 5 or 14 wherein the grouped elements are organized into packets, the packets comprising one or more of the grouped elements of the interleaved data stream.
20. The mediod of Claim 19 wherein the packets further comprise, information to facilitate the decoding and presentation of the video program.
21. The method of Claim 20 further comprising the step of storing die packets on one or more storage devices.
22. The method of Claim 21 further comprising the steps of:
partitioning the interleaved data stream into two or more portions, each of the portions comprising an equal number of the packets; and
storing each of the two or more portions on a different storage device.
23. The method of claim 22 further comprising the steps of:
concurrently retrieving the two or more portions from the two or more storage devices;
transmitting each of the concurrently retrieved two or more portions simultaneously over separate channels of a video distribution medium; and
repeating said concurrently retrieving and said transmitting steps continuously for a predetermined time.
24. The method of claim 23 further comprising the steps of:
concurrently retrieving the two or more portions from the two or more storage devices;
transmitting each of the concurrently retrieved two or more portions simultaneously over separate channels of a video distribution medium; and
repeating said concurrently retrieving and said transmitting steps continuously for a predetermined time.
25. The method of claim 13 wherein said interleaving step further comprises the steps of:
apportioning each of the n encoded subsequences of elements into a plurality of x packets, each packet comprising a portion of the compressed data comprising the n encoded subsequences of elements and representative of the same number of pixels; and
ordering die packets by first selecting and grouping the first packets from each of the n encoded subsequences in segment order, men likewise selecting and grouping the second through the xth packets.
26. The method of claim 1 or 3 wherein said interleaving step further comprises die steps of:
apportioning each of die n subsequences into an ordered sequence of a plurality of x packets, each packet comprising a portion of the data stream representative of an equal number of pixels; and
ordering die packets by first selecting and grouping into segment order die first packet from each of the n subsequences and likewise selecting and grouping into segment order die second dirough xth packets from each of the n subsequences.
27. The method of claims 1, 5, or 14 wherein said step of interleaving further comprises the steps of:
assigning a unique segment identification number to each of the n subsequences; and
inserting data representative of the unique segment identification numbers into the interleaved data stream to identify to which of die n subsequences each of die elements belongs.
28. The mediod of claim 8 further comprising the steps of:
assigning a unique segment identification number to each of the n subsequences; and
inserting data representative of the unique segment identification numbers into the retrieved interleaved data stream to identify to which of die n subsequences each of the elements belongs.
29. The method of claim 29 wherein the unique segment identification numbers indicate die sequence order of die segments comprising the video program.
30. The method of claim 29 further comprising the steps of:
for each retrieval and transmission of the interleaved data stream, inserting a flag to denote die one of n subsequences representative of die first segment of die video program; and
decrementing the currently assigned segment identification numbers for each of the n subsequences upon completion of each retrieval and transmission of the interleaved data stream.
PCT/US1995/013513 1994-10-19 1995-10-19 Method and apparatus for encoding and formatting data representing a video program to provide multiple overlapping presentations of the video program WO1996013125A1 (en)

Priority Applications (5)

Application Number Priority Date Filing Date Title
EP95938280A EP0787408B1 (en) 1994-10-19 1995-10-19 Method and apparatus for encoding and formatting data representing a video program to provide multiple overlapping presentations of the video program
DE69511570T DE69511570T2 (en) 1994-10-19 1995-10-19 METHOD AND DEVICE FOR CODING AND FORMATTING VIDEO PROGRAM DATA FOR PROVIDING MULTIPLE OVERLAPPING VIDEO PROGRAMS
AU38966/95A AU3896695A (en) 1994-10-19 1995-10-19 Method and apparatus for encoding and formatting data representing a video program to provide multiple overlapping presentations of the video program
CA002202350A CA2202350C (en) 1994-10-19 1995-10-19 Method and apparatus for encoding and formatting data representing a video program to provide multiple overlapping presentations of the video program
JP8514059A JPH10507889A (en) 1994-10-19 1995-10-19 Method and apparatus for encoding and formatting data representing a video program and providing multiple overlapping presentations of the video program

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US08/326,511 US5612742A (en) 1994-10-19 1994-10-19 Method and apparatus for encoding and formatting data representing a video program to provide multiple overlapping presentations of the video program
US08/326,511 1994-10-19

Publications (1)

Publication Number Publication Date
WO1996013125A1 true WO1996013125A1 (en) 1996-05-02

Family

ID=23272525

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US1995/013513 WO1996013125A1 (en) 1994-10-19 1995-10-19 Method and apparatus for encoding and formatting data representing a video program to provide multiple overlapping presentations of the video program

Country Status (7)

Country Link
US (1) US5612742A (en)
EP (1) EP0787408B1 (en)
JP (1) JPH10507889A (en)
AU (1) AU3896695A (en)
CA (1) CA2202350C (en)
DE (1) DE69511570T2 (en)
WO (1) WO1996013125A1 (en)

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0731468A2 (en) * 1995-03-06 1996-09-11 Sony Corporation Video data recording and reproducing
EP0748123A2 (en) * 1995-06-07 1996-12-11 International Business Machines Corporation Multimedia server system and method for communicating multimedia information
EP0749242A1 (en) * 1995-06-15 1996-12-18 International Business Machines Corporation Hybrid video-on-demand
EP0766473A2 (en) * 1995-09-29 1997-04-02 Hewlett-Packard Company Digital layout method suitable for near video on demand system
WO1997013369A1 (en) * 1995-09-29 1997-04-10 Philips Electronics N.V. Method and system for storing and reading data blocks for a number of user groups
GB2317082A (en) * 1996-09-04 1998-03-11 Ibm Distribution of digitally encoded presentations
US6005620A (en) * 1997-01-31 1999-12-21 Hughes Electronics Corporation Statistical multiplexer for live and pre-compressed video
US6078958A (en) * 1997-01-31 2000-06-20 Hughes Electronics Corporation System for allocating available bandwidth of a concentrated media output
US6084910A (en) * 1997-01-31 2000-07-04 Hughes Electronics Corporation Statistical multiplexer for video signals
US6097435A (en) * 1997-01-31 2000-08-01 Hughes Electronics Corporation Video system with selectable bit rate reduction
US6188436B1 (en) 1997-01-31 2001-02-13 Hughes Electronics Corporation Video broadcast system with video data shifting
EP1164796A1 (en) * 2000-06-14 2001-12-19 Astrium SAS Process and system for video on demand
WO2002054768A1 (en) * 2001-01-05 2002-07-11 Deyang Song Method for digital media playback in a broadcast network
EP2494482A2 (en) * 2009-10-29 2012-09-05 Microsoft Corporation Assembling streamed content for on-demand presentation
ITMI20130785A1 (en) * 2013-05-13 2014-11-14 Sky Italia S R L NEAR-REAL TIME CODING OF A LINEAR SIGNAL
US11540302B2 (en) * 2006-10-22 2022-12-27 Viasat, Inc. Cyclical obstruction communication system

Families Citing this family (121)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5926205A (en) * 1994-10-19 1999-07-20 Imedia Corporation Method and apparatus for encoding and formatting data representing a video program to provide multiple overlapping presentations of the video program
US7349976B1 (en) 1994-11-30 2008-03-25 Realnetworks, Inc. Audio-on-demand communication system
US5793980A (en) 1994-11-30 1998-08-11 Realnetworks, Inc. Audio-on-demand communication system
WO1996020566A1 (en) 1994-12-23 1996-07-04 Imedia Corporation Method and apparatus for providing vcr-like trick mode functions for viewing distributed video data
AU5027796A (en) 1995-03-07 1996-09-23 Interval Research Corporation System and method for selective recording of information
US5961614A (en) * 1995-05-08 1999-10-05 Apple Computer, Inc. System for data transfer through an I/O device using a memory access controller which receives and stores indication of a data status signal
US7917922B1 (en) * 1995-06-08 2011-03-29 Schwab Barry H Video input switching and signal processing apparatus
GB9517806D0 (en) * 1995-08-31 1995-11-01 Philips Electronics Uk Ltd Information handling for interactive apparatus
US5956088A (en) * 1995-11-21 1999-09-21 Imedia Corporation Method and apparatus for modifying encoded digital video for improved channel utilization
US5966120A (en) * 1995-11-21 1999-10-12 Imedia Corporation Method and apparatus for combining and distributing data with pre-formatted real-time video
JP3648577B2 (en) * 1995-12-07 2005-05-18 カシオ計算機株式会社 Image processing device
US5729292A (en) * 1995-12-21 1998-03-17 Thomson Multimedia, S.A. Optimizing performance in a packet slot priority packet transport system
US5930526A (en) * 1996-01-24 1999-07-27 Intel Corporation System for progressive transmission of compressed video including video data of first type of video frame played independently of video data of second type of video frame
US5940073A (en) * 1996-05-03 1999-08-17 Starsight Telecast Inc. Method and system for displaying other information in a TV program guide
US5774677A (en) * 1996-06-28 1998-06-30 Intel Corporation Dual interleaved block motion compensation
JPH1023375A (en) * 1996-07-04 1998-01-23 Matsushita Electric Ind Co Ltd Sequential digital video decoder
US5867230A (en) * 1996-09-06 1999-02-02 Motorola Inc. System, device, and method for streaming a multimedia file encoded at a variable bitrate
BR9707158A (en) * 1996-11-21 1999-04-06 Philips Electronics Nv Process and arrangement for the transmission of television programs through a plurality of channels and additional data services process for the reception of television programs that are transmitted through a plurality of television channels and additional data services television receiver and television signal
US6263507B1 (en) * 1996-12-05 2001-07-17 Interval Research Corporation Browser for use in navigating a body of information, with particular application to browsing information represented by audiovisual data
US5893062A (en) 1996-12-05 1999-04-06 Interval Research Corporation Variable rate video playback with synchronized audio
JPH10173674A (en) * 1996-12-13 1998-06-26 Hitachi Ltd Digital data transmission system
US5953506A (en) * 1996-12-17 1999-09-14 Adaptive Media Technologies Method and apparatus that provides a scalable media delivery system
WO1998028915A2 (en) * 1996-12-23 1998-07-02 Koninklijke Philips Electronics N.V. Method and system for supplying data streams
EP0903042B1 (en) 1997-02-08 2002-05-29 Matsushita Electric Industrial Co., Ltd. Quantization matrix for still and moving picture coding
US9113122B2 (en) * 1997-04-21 2015-08-18 Rovi Guides, Inc. Method and apparatus for time-shifting video and text in a text-enhanced television program
US6021120A (en) * 1997-04-30 2000-02-01 Siemens Information And Communication Networks, Inc. System and method for creating full duplex virtual circuits based on multiple asymmetrical links
US6314102B1 (en) * 1997-07-10 2001-11-06 Alcatel Telecommunications system for providing both narrowband and broadband services to subscribers
US6018359A (en) * 1998-04-24 2000-01-25 Massachusetts Institute Of Technology System and method for multicast video-on-demand delivery system
US6874161B1 (en) * 1998-06-30 2005-03-29 Webtv Networks, Inc. Utilizing a high bandwidth communications channel for broadcasting network data during off-peak hours
JP2000032327A (en) * 1998-07-10 2000-01-28 Matsushita Electric Ind Co Ltd Picked-up image transmission equipment
EP0973129A3 (en) * 1998-07-17 2005-01-12 Matsushita Electric Industrial Co., Ltd. Motion image data compression system
AU758390B2 (en) * 1998-07-27 2003-03-20 Webtv Networks, Inc. Remote computer access
US8189662B2 (en) 1999-07-27 2012-05-29 Microsoft Corporation Selection compression
US7360230B1 (en) 1998-07-27 2008-04-15 Microsoft Corporation Overlay management
US6898762B2 (en) 1998-08-21 2005-05-24 United Video Properties, Inc. Client-server electronic program guide
US7068724B1 (en) * 1999-10-20 2006-06-27 Prime Research Alliance E., Inc. Method and apparatus for inserting digital media advertisements into statistical multiplexed streams
US20020083441A1 (en) 2000-08-31 2002-06-27 Flickinger Gregory C. Advertisement filtering and storage for targeted advertisement systems
US7653923B2 (en) * 2000-02-18 2010-01-26 Prime Research Alliance E, Inc. Scheduling and presenting IPG ads in conjunction with programming ads in a television environment
US7185353B2 (en) 2000-08-31 2007-02-27 Prime Research Alliance E., Inc. System and method for delivering statistically scheduled advertisements
US7039932B2 (en) * 2000-08-31 2006-05-02 Prime Research Alliance E., Inc. Queue-based head-end advertisement scheduling method and apparatus
US7228555B2 (en) * 2000-08-31 2007-06-05 Prime Research Alliance E., Inc. System and method for delivering targeted advertisements using multiple presentation streams
US20020083439A1 (en) * 2000-08-31 2002-06-27 Eldering Charles A. System for rescheduling and inserting advertisements
US20020144263A1 (en) * 2000-08-31 2002-10-03 Eldering Charles A. Grouping of advertisements on an advertising channel in a targeted advertisement system
US7328448B2 (en) * 2000-08-31 2008-02-05 Prime Research Alliance E, Inc. Advertisement distribution system for distributing targeted advertisements in television systems
US8151295B1 (en) 2000-08-31 2012-04-03 Prime Research Alliance E., Inc. Queue based advertisement scheduling and sales
US8180675B2 (en) * 2000-08-31 2012-05-15 Prime Research Alliance E., Inc. System and method for automatically managing avail inventory data and avail pricing
DE60039861D1 (en) * 1999-04-20 2008-09-25 Samsung Electronics Co Ltd ADVERTISING MANAGEMENT SYSTEM FOR DIGITAL VIDEO TONES
EP1177674A4 (en) * 1999-05-10 2004-12-08 Samsung Electronics Co Ltd Advertisement subgroups for digital streams
US7155735B1 (en) 1999-10-08 2006-12-26 Vulcan Patents Llc System and method for the broadcast dissemination of time-ordered data
US6618367B1 (en) * 1999-12-16 2003-09-09 Agere Systems Inc. Transmission frame structure for a satellite digital audio radio system
US6757682B1 (en) 2000-01-28 2004-06-29 Interval Research Corporation Alerting users to items of current interest
DE10004829B4 (en) * 2000-01-31 2004-07-22 Tellique Kommunikationstechnik Gmbh Method and device for transmitting data units of a data stream
US20020013948A1 (en) * 2000-03-13 2002-01-31 Erwin Aguayo Video data management, transmission, and control system and method emloying distributed video segments microcasting
US20040148625A1 (en) 2000-04-20 2004-07-29 Eldering Charles A Advertisement management system for digital video streams
US20020059620A1 (en) * 2000-05-31 2002-05-16 Hoang Khoi Nhu Selective inactivation and copy-protection
US6557030B1 (en) * 2000-05-31 2003-04-29 Prediwave Corp. Systems and methods for providing video-on-demand services for broadcasting systems
US20020175998A1 (en) * 2000-05-31 2002-11-28 Hoang Khoi Nhu Data-on-demand digital broadcast system utilizing prefetch data transmission
US20020066100A1 (en) * 2000-05-31 2002-05-30 Khoi Hoang Method for providing data services for a large number of small-sized data files
US20020059635A1 (en) * 2000-05-31 2002-05-16 Hoang Khoi N. Digital data-on-demand broadcast cable modem termination system
US20020023267A1 (en) * 2000-05-31 2002-02-21 Hoang Khoi Nhu Universal digital broadcast system and methods
US20020049980A1 (en) * 2000-05-31 2002-04-25 Hoang Khoi Nhu Controlling data-on-demand client access
US20020138845A1 (en) * 2000-05-31 2002-09-26 Khoi Hoang Methods and systems for transmitting delayed access client generic data-on demand services
US7346698B2 (en) * 2000-12-20 2008-03-18 G. W. Hannaway & Associates Webcasting method and system for time-based synchronization of multiple, independent media streams
US7331057B2 (en) 2000-12-28 2008-02-12 Prime Research Alliance E, Inc. Grouping advertisement subavails
US8091112B1 (en) * 2001-02-28 2012-01-03 Keen Personal Technologies, Inc. System and a method for transmitting and receiving a program with improved efficiency
EP1413134A1 (en) * 2001-04-24 2004-04-28 Prediwave Corp. A method for providing data services for a large number of small -sized data files
US20030084461A1 (en) * 2001-10-25 2003-05-01 Khoi Hoang Method and apparatus for transmitting non-VOD services
US20030051249A1 (en) * 2001-08-20 2003-03-13 Khoi Hoang System and method for data insertion (commercials) in client generic data-on-demand broadcast transmissions
US8713623B2 (en) 2001-09-20 2014-04-29 Time Warner Cable Enterprises, LLC Technique for effectively providing program material in a cable television system
US20030138237A1 (en) * 2002-01-23 2003-07-24 Stewart John Sidney Multimedia on demand for use in a near on demand environment
US7962944B2 (en) 2002-01-23 2011-06-14 John Sidney Stewart Providing multimedia on demand in a near on demand environment
US7142565B2 (en) * 2002-02-21 2006-11-28 Qualcomm, Incorporated Method and apparatus for radio link control of signaling messages and short messages data services in a communication system
US8272020B2 (en) * 2002-08-17 2012-09-18 Disney Enterprises, Inc. System for the delivery and dynamic presentation of large media assets over bandwidth constrained networks
WO2004019602A2 (en) * 2002-08-21 2004-03-04 Disney Enterprises, Inc. Digital home movie library
JP2006509454A (en) * 2002-12-06 2006-03-16 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ Channel tapping in near video on demand system
CN1720737A (en) * 2002-12-06 2006-01-11 皇家飞利浦电子股份有限公司 Near-video-on-demand stream filtering
US20060031401A1 (en) * 2002-12-06 2006-02-09 Verhaegh Wilhelmus F Multi-point service injection in a broadcast system
US20040143850A1 (en) * 2003-01-16 2004-07-22 Pierre Costa Video Content distribution architecture
US20040199660A1 (en) * 2003-02-14 2004-10-07 Nokia Corporation Method of multiplexing compressed and uncompressed internet protocol packets
US7610603B2 (en) 2004-03-26 2009-10-27 Broadcom Corporation Multistream video communication with staggered access points
US20060031916A1 (en) * 2004-04-30 2006-02-09 Vulcan Inc. Management and non-linear presentation of broadcasted or streamed multimedia content
US20060031879A1 (en) * 2004-04-30 2006-02-09 Vulcan Inc. Management and non-linear presentation of news-related broadcasted or streamed multimedia content
US20060031885A1 (en) * 2004-04-30 2006-02-09 Vulcan Inc. Management and non-linear presentation of music-related broadcasted or streamed multimedia content
US7564874B2 (en) 2004-09-17 2009-07-21 Uni-Pixel Displays, Inc. Enhanced bandwidth data encoding method
US9723267B2 (en) 2004-12-15 2017-08-01 Time Warner Cable Enterprises Llc Method and apparatus for wideband distribution of content
DE102005001286A1 (en) * 2005-01-11 2006-07-20 Siemens Ag Method and device for transmitting scalable data
US20060235883A1 (en) 2005-04-18 2006-10-19 Krebs Mark S Multimedia system for mobile client platforms
US8627390B2 (en) * 2006-02-01 2014-01-07 Arris Solutions, Inc. Method and device for providing programs to multiple end user devices
US7734579B2 (en) * 2006-02-08 2010-06-08 At&T Intellectual Property I, L.P. Processing program content material
US8108563B2 (en) * 2006-02-24 2012-01-31 Qualcomm Incorporated Auxiliary writes over address channel
US8107492B2 (en) 2006-02-24 2012-01-31 Qualcomm Incorporated Cooperative writes over the address channel of a bus
US8582663B2 (en) * 2006-08-08 2013-11-12 Core Wireless Licensing S.A.R.L. Method, device, and system for multiplexing of video streams
US9143329B2 (en) * 2008-01-30 2015-09-22 Adobe Systems Incorporated Content integrity and incremental security
US20110008021A1 (en) * 2008-03-26 2011-01-13 David Alan Casper Method for rapid archiving and restoring of a video store
JP2009303064A (en) * 2008-06-16 2009-12-24 Denso Corp Data delivery system, receiver, program, and method for delivering data
US20090319547A1 (en) * 2008-06-19 2009-12-24 Microsoft Corporation Compression Using Hashes
JP2010219669A (en) * 2009-03-13 2010-09-30 Toshiba Corp Video server
KR101750048B1 (en) 2009-11-13 2017-07-03 삼성전자주식회사 Method and apparatus for providing trick play service
US8868575B2 (en) * 2010-01-13 2014-10-21 International Business Machines Corporation Method and system for transformation of logical data objects for storage
KR20110105710A (en) * 2010-03-19 2011-09-27 삼성전자주식회사 Method and apparatus for adaptively streaming content comprising plurality of chapter
US9300445B2 (en) 2010-05-27 2016-03-29 Time Warner Cable Enterprise LLC Digital domain content processing and distribution apparatus and methods
US9762639B2 (en) 2010-06-30 2017-09-12 Brightcove Inc. Dynamic manifest generation based on client identity
AU2010202740B1 (en) * 2010-06-30 2010-12-23 Brightcove Inc. Dynamic indexing for ad insertion in media streaming
US8954540B2 (en) 2010-06-30 2015-02-10 Albert John McGowan Dynamic audio track selection for media streaming
US9838450B2 (en) 2010-06-30 2017-12-05 Brightcove, Inc. Dynamic chunking for delivery instances
US8301733B2 (en) 2010-06-30 2012-10-30 Unicorn Media, Inc. Dynamic chunking for delivery instances
AU2010202741B1 (en) 2010-06-30 2010-12-23 Brightcove Inc. Dynamic chunking for media streaming
US9185341B2 (en) 2010-09-03 2015-11-10 Time Warner Cable Enterprises Llc Digital domain content processing and distribution apparatus and methods
AU2011201404B1 (en) 2011-03-28 2012-01-12 Brightcove Inc. Transcodeless on-the-fly ad insertion
US8625789B2 (en) 2011-09-26 2014-01-07 Unicorn Media, Inc. Dynamic encryption
US8239546B1 (en) 2011-09-26 2012-08-07 Unicorn Media, Inc. Global access control for segmented streaming delivery
US8165343B1 (en) 2011-09-28 2012-04-24 Unicorn Media, Inc. Forensic watermarking
US9860604B2 (en) 2011-11-23 2018-01-02 Oath Inc. Systems and methods for internet video delivery
US9112939B2 (en) 2013-02-12 2015-08-18 Brightcove, Inc. Cloud-based video delivery
JP6317696B2 (en) * 2015-03-16 2018-04-25 株式会社東芝 Communication apparatus and communication system
US11153585B2 (en) 2017-02-23 2021-10-19 Netflix, Inc. Optimizing encoding operations when generating encoded versions of a media title
US10742708B2 (en) 2017-02-23 2020-08-11 Netflix, Inc. Iterative techniques for generating multiple encoded versions of a media title
US11166034B2 (en) 2017-02-23 2021-11-02 Netflix, Inc. Comparing video encoders/decoders using shot-based encoding and a perceptual visual quality metric
US10715814B2 (en) 2017-02-23 2020-07-14 Netflix, Inc. Techniques for optimizing encoding parameters for different shot sequences
US10666992B2 (en) 2017-07-18 2020-05-26 Netflix, Inc. Encoding techniques for optimizing distortion and bitrate
WO2023242590A1 (en) * 2022-06-16 2023-12-21 Mbda Uk Limited Method for image encoding

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1991003112A1 (en) * 1989-08-23 1991-03-07 Delta Beta Pty. Ltd. Program transmission optimisation
WO1992011713A1 (en) * 1990-12-21 1992-07-09 Gte Service Corporation Method for providing video programming nearly on demand
WO1994010775A1 (en) * 1992-10-30 1994-05-11 Scientific-Atlanta, Inc. A multiplex control packet for digital services
EP0605115A2 (en) * 1992-12-28 1994-07-06 AT&T Corp. Program server for interactive television system

Family Cites Families (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB103438A (en) * 1916-07-24 1917-01-25 Alice Sophia Collard Adjustable Bed Foot-rest.
US4343042A (en) * 1979-07-10 1982-08-03 Cablebus Systems Corporation Bi-directional data transmission and control system
US4862268A (en) * 1980-03-31 1989-08-29 General Instrument Corporation Addressable cable television control system with video format data transmission
US4590516A (en) * 1982-06-01 1986-05-20 World Video Library, Inc. Recorded program communication system
US4567512A (en) * 1982-08-19 1986-01-28 World Video Library, Inc. Recorded program communication system
US4521806A (en) * 1982-08-19 1985-06-04 World Video Library, Inc. Recorded program communication system
US4616263A (en) * 1985-02-11 1986-10-07 Gte Corporation Video subsystem for a hybrid videotex facility
JPS6477069A (en) * 1987-05-09 1989-03-23 Ricoh Kk Density control system for digital copying machine
US5191410A (en) * 1987-08-04 1993-03-02 Telaction Corporation Interactive multimedia presentation and communications system
US4829372A (en) * 1987-08-20 1989-05-09 Telaction Corporation Presentation player
US4975771A (en) * 1989-02-10 1990-12-04 Kassatly Salim A Method and apparatus for TV broadcasting
US5119188A (en) * 1988-10-25 1992-06-02 Telaction Corporation Digital audio-video presentation display system
US4901367A (en) * 1988-11-30 1990-02-13 Victor Nicholson Cable communications system with remote switching and processing converters
US5014125A (en) * 1989-05-05 1991-05-07 Cableshare, Inc. Television system for the interactive distribution of selectable video presentations
US5421031A (en) * 1989-08-23 1995-05-30 Delta Beta Pty. Ltd. Program transmission optimisation
US5051822A (en) * 1989-10-19 1991-09-24 Interactive Television Systems, Inc. Telephone access video game distribution center
US5130792A (en) * 1990-02-01 1992-07-14 Usa Video Inc. Store and forward video system
KR950001439B1 (en) * 1990-04-30 1995-02-24 삼성전자주식회사 Coding apparatus of error correction
US5220420A (en) * 1990-09-28 1993-06-15 Inteletext Systems, Inc. Interactive home information system for distributing compressed television programming
FR2670969B1 (en) * 1990-12-19 1993-04-16 France Etat TIME-FREQUENCY DISTRIBUTION DATA TRANSMISSION SYSTEM, WITH CHANNEL STRUCTURING.
US5168353A (en) * 1990-12-21 1992-12-01 Gte Laboratories Incorporated Video distribution system allowing viewer access to time staggered indentical prerecorded programs
US5253341A (en) * 1991-03-04 1993-10-12 Rozmanith Anthony I Remote query communication system
EP0527632B1 (en) * 1991-08-13 1998-04-08 Canon Kabushiki Kaisha Image transmission apparatus
US5243629A (en) * 1991-09-03 1993-09-07 At&T Bell Laboratories Multi-subcarrier modulation for hdtv transmission
JPH0568243A (en) * 1991-09-09 1993-03-19 Hitachi Ltd Variable length coding controlling system
JP2002504271A (en) * 1991-09-10 2002-02-05 ハイブリッド・ネットワークス・インコーポレイテッド Remote link adapter for TV broadcast data transmission system
US5231486A (en) * 1992-07-27 1993-07-27 General Electric Company Data separation processing in a dual channel digital high definition television system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1991003112A1 (en) * 1989-08-23 1991-03-07 Delta Beta Pty. Ltd. Program transmission optimisation
WO1992011713A1 (en) * 1990-12-21 1992-07-09 Gte Service Corporation Method for providing video programming nearly on demand
WO1994010775A1 (en) * 1992-10-30 1994-05-11 Scientific-Atlanta, Inc. A multiplex control packet for digital services
EP0605115A2 (en) * 1992-12-28 1994-07-06 AT&T Corp. Program server for interactive television system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
GELMAN A D ET AL: "A STORE-AND-FORWARD ARCHITECTURE FOR VIDEO-ON-DEMAND SERVICE", COMMUNICATIONS - RISING TO THE HEIGHTS, DENVER, JUNE 23 - 26, 1991, vol. 2 OF 3, 23 June 1991 (1991-06-23), INSTITUTE OF ELECTRICAL AND ELECTRONICS ENGINEERS, pages 842 - 846, XP000269608 *

Cited By (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5799127A (en) * 1995-03-06 1998-08-25 Sony Corporation Video data recording apparatus and video data reproducing apparatus
EP0731468A3 (en) * 1995-03-06 1997-02-26 Sony Corp Video data recording and reproducing
EP0731468A2 (en) * 1995-03-06 1996-09-11 Sony Corporation Video data recording and reproducing
EP0748123A2 (en) * 1995-06-07 1996-12-11 International Business Machines Corporation Multimedia server system and method for communicating multimedia information
EP0748123A3 (en) * 1995-06-07 2005-12-21 International Business Machines Corporation Multimedia server system and method for communicating multimedia information
EP0749242A1 (en) * 1995-06-15 1996-12-18 International Business Machines Corporation Hybrid video-on-demand
EP0766473A2 (en) * 1995-09-29 1997-04-02 Hewlett-Packard Company Digital layout method suitable for near video on demand system
WO1997013369A1 (en) * 1995-09-29 1997-04-10 Philips Electronics N.V. Method and system for storing and reading data blocks for a number of user groups
EP0766473A3 (en) * 1995-09-29 1997-05-28 Hewlett Packard Co Digital layout method suitable for near video on demand system
US5721823A (en) * 1995-09-29 1998-02-24 Hewlett-Packard Co. Digital layout method suitable for near video on demand system
GB2317082A (en) * 1996-09-04 1998-03-11 Ibm Distribution of digitally encoded presentations
US5838912A (en) * 1996-09-04 1998-11-17 International Business Machines Corporation Distribution of digitally encoded presentations
GB2317082B (en) * 1996-09-04 2001-04-18 Ibm Method and system for the distribution of digitally encoded presentations
US6097435A (en) * 1997-01-31 2000-08-01 Hughes Electronics Corporation Video system with selectable bit rate reduction
US6005620A (en) * 1997-01-31 1999-12-21 Hughes Electronics Corporation Statistical multiplexer for live and pre-compressed video
US6188436B1 (en) 1997-01-31 2001-02-13 Hughes Electronics Corporation Video broadcast system with video data shifting
US6078958A (en) * 1997-01-31 2000-06-20 Hughes Electronics Corporation System for allocating available bandwidth of a concentrated media output
US6084910A (en) * 1997-01-31 2000-07-04 Hughes Electronics Corporation Statistical multiplexer for video signals
US6988278B2 (en) 2000-06-14 2006-01-17 Astrium Sas Process and system for video on demand
EP1164796A1 (en) * 2000-06-14 2001-12-19 Astrium SAS Process and system for video on demand
WO2002054768A1 (en) * 2001-01-05 2002-07-11 Deyang Song Method for digital media playback in a broadcast network
US11540302B2 (en) * 2006-10-22 2022-12-27 Viasat, Inc. Cyclical obstruction communication system
EP2494482A2 (en) * 2009-10-29 2012-09-05 Microsoft Corporation Assembling streamed content for on-demand presentation
EP2494482A4 (en) * 2009-10-29 2013-07-24 Microsoft Corp Assembling streamed content for on-demand presentation
US9002881B2 (en) 2009-10-29 2015-04-07 Microsoft Technology Licensing, Llc Assembling streamed content for on-demand presentation
ITMI20130785A1 (en) * 2013-05-13 2014-11-14 Sky Italia S R L NEAR-REAL TIME CODING OF A LINEAR SIGNAL
WO2014184632A1 (en) * 2013-05-13 2014-11-20 Sky Italia S.R.L. Near-real time encoding of a linear signal
US9973770B2 (en) 2013-05-13 2018-05-15 Sky Italia S.R.L. Near-real time encoding of linear signal

Also Published As

Publication number Publication date
AU3896695A (en) 1996-05-15
CA2202350A1 (en) 1996-05-02
EP0787408A1 (en) 1997-08-06
EP0787408B1 (en) 1999-08-18
JPH10507889A (en) 1998-07-28
US5612742A (en) 1997-03-18
DE69511570T2 (en) 2000-02-17
CA2202350C (en) 2002-12-31
DE69511570D1 (en) 1999-09-23

Similar Documents

Publication Publication Date Title
EP0787408B1 (en) Method and apparatus for encoding and formatting data representing a video program to provide multiple overlapping presentations of the video program
EP0954925B1 (en) Method and apparatus for encoding and formatting data representing a video program to provide multiple overlapping presentations of the video program
US6233735B1 (en) Near video-on-demand system and broadcasting method therefor
US20020015576A1 (en) Method and apparatus for simultaneously producing compressed play and trick play bitstreams from a video frame sequence
US7565681B2 (en) System and method for the broadcast dissemination of time-ordered data
US8091112B1 (en) System and a method for transmitting and receiving a program with improved efficiency
US6327421B1 (en) Multiple speed fast forward/rewind compressed video delivery system
CA2199360C (en) Method and apparatus for electronic distribution of digital multi-media information
US5691986A (en) Methods and apparatus for the editing and insertion of data into an encoded bitstream
US5754773A (en) Multimedia on-demand server having different transfer rates
US6282207B1 (en) Method and apparatus for storing and accessing multiple constant bit rate data
US20030163824A1 (en) Method and apparatus for producing demand real-time television
EP0676898A1 (en) Frame sampling scheme for video scanning in a video-on-demand system
EP1797723A1 (en) A video compression system
KR20030034081A (en) Methods for providing video-on-demand services for broadcasting systems
KR100639428B1 (en) System for delivering data over a network
JPH09121339A (en) Video data transmitter, video data receiver and broadcasting system
JPH0974535A (en) Video data recording and reproducing device
US6754441B1 (en) Data recording device, data recording/reproducing device, data recording method, and data recording/reproducing method for recording image/voice data
KR100865683B1 (en) Data placement scheme for mulit-dimensional scalable video data
JPH09200729A (en) Device and method for near video on demand system
Kermode A novel method for video-on-demand via digital broadcast
WO2001086955A1 (en) System and method for the efficient utilization of bandwidth in the broadcast dissemination of time-ordered data
WO1997023091A1 (en) Vcr-type controls for video server system
JPH08317368A (en) Data transmitter

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AL AM AT AU BB BG BR BY CA CH CN CZ DE DK EE ES FI GB GE HU IS JP KE KG KP KR KZ LK LR LT LU LV MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK TJ TM TT UA UG UZ VN

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): KE MW SD SZ UG AT BE CH DE DK ES FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN ML MR NE SN TD TG

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
ENP Entry into the national phase

Ref document number: 2202350

Country of ref document: CA

Ref country code: CA

Ref document number: 2202350

Kind code of ref document: A

Format of ref document f/p: F

WWE Wipo information: entry into national phase

Ref document number: 1995938280

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 1995938280

Country of ref document: EP

REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

WWG Wipo information: grant in national office

Ref document number: 1995938280

Country of ref document: EP