A video compression system and method for compressing video data for transmission or storage by reducing the temporal redundancy in the video data is described. A frame of video data is divided into a variable number of blocks of pixel data of varying size, and each block of data is compared to a window of pixel data in a reference frame of pixel data, typically the previous frame. A best matched block of pixel data is selected from the window of pixel data in the reference frame, and a displacement vector is assigned to describe the selected block location in the reference frame relative to the current block of pixel data. The number and size of the blocks of pixel data are permitted to vary, in order to adapt to motion discontinuities in the sequential frames of pixel data. This is to allow prediction blocks of pixel data in the current frame to be smaller in areas of high activity, while maintaining high levels of compression, achieved by using larger prediction blocks, in areas... |
Citations|
| US4179709 | Jan 10, 1978 | Dec 18, 1979 | Bell & Howell Company | Video information bandwidth compression | | US4293920 | Sep 4, 1979 | Oct 6, 1981 | | Two-dimensional transform processor | | US4504860 | Jul 15, 1982 | Mar 12, 1985 | British Telecommunications | Method and process for transmitting an image | | US4774574 | Jun 2, 1987 | Sep 27, 1988 | Eastman Kodak Company | Adaptive block transform image coding method and apparatus | | US4776030 | Mar 28, 1986 | Oct 4, 1988 | GTE Laboratories Incorporated | Block quantizer for transform coding | | US4796087 | Jun 1, 1987 | Jan 3, 1989 | | Process for coding by transformation for the transmission of picture signals | | US4807042 | Jan 27, 1987 | Feb 21, 1989 | Fuji Photo Film Co., Ltd. | Method of image signal encoding by orthogonal transformation | | US4816906 | Aug 31, 1987 | Mar 28, 1989 | AEG Aktiengesellschaft | Method for motion-compensated frame-to-frame prediction coding | | US4821119 | May 4, 1988 | Apr 11, 1989 | Bell Communications Research, Inc. | Method and apparatus for low bit-rate interframe video coding | | US4922341 | Sep 26, 1988 | May 1, 1990 | Siemens Aktiengesellschaft | Method for scene-model-assisted reduction of image data for digital television signals | | US4924309 | Aug 23, 1988 | May 8, 1990 | Deutche Thomson-Brandt GmbH | Method and circuit arrangement for improving the resolution of the coefficients of digital signals, particularly digital TV signals | | US4984076 | Jul 26, 1989 | Jan 8, 1991 | Kabushiki Kaisha Toshiba | Image compression coding system | | US5021891 | Feb 27, 1990 | Jun 4, 1991 | Qualcomm, Inc. | Adaptive block size image compression method and system | | US5045938 | Jul 24, 1990 | Sep 3, 1991 | Victor Company of Japan, Ltd. | Method and apparatus for encoding using variable length codes | | US5073820 | Oct 23, 1990 | Dec 17, 1991 | Olympus Optical Co., Ltd. | Image data coding apparatus and coding method | | US5107345 | May 28, 1991 | Apr 21, 1992 | Qualcomm Incorporated | Adaptive block size image compression method and system | | US5126857 | May 29, 1990 | Jun 30, 1992 | Fuji Photo Film Co., Ltd. | Device for coding a picture signal by compression | | US5151784 | Apr 30, 1991 | Sep 29, 1992 | AT&T Bell Laboratories | Multiple frame motion estimation | | US5235419 | Oct 24, 1991 | Aug 10, 1993 | General Instrument Corporation | Adaptive motion compensation using a plurality of motion compensators |
Referenced by|
| US5731835 | Feb 26, 1996 | Mar 24, 1998 | David Sarnoff Research Center, Inc. | Dynamic coding rate control in a block-based video coding system | | US5748788 | Aug 28, 1996 | May 5, 1998 | Cannon Kabushiki Kaisha | Image processing method and apparatus | | US5754240 | Oct 1, 1996 | May 19, 1998 | Matsushita Electric Industrial Co., Ltd. | Method and apparatus for calculating the pixel values of a block from one or two prediction blocks | | US5764921 | Oct 26, 1995 | Jun 9, 1998 | Motorola Northwestern University | Method, device and microprocessor for selectively compressing video frames of a motion compensated prediction-based video codec | | US5809174 | Oct 15, 1996 | Sep 15, 1998 | C-Cube Microsystems | Decompression processor for video applications | | US5838827 | Oct 27, 1995 | Nov 17, 1998 | Graphics Communication Laboratories | Apparatus and method for searching motion vector | | US5883674 | Aug 15, 1996 | Mar 16, 1999 | Sony Corporation | Method and apparatus for setting a search range for detecting motion vectors utilized for encoding picture data | | US5903673 | Mar 14, 1997 | May 11, 1999 | Microsoft Corporation | Digital video signal encoder and encoding method | | US5936670 | May 19, 1997 | Aug 10, 1999 | U.S. Philips Corporation | Method and device for decoding coded digital video signals | | US5946425 | Jun 3, 1996 | Aug 31, 1999 | Massachusetts Institute of Technology | Method and apparatus for automatic alingment of volumetric images containing common subject matter | | US6008847 | Mar 3, 1997 | Dec 28, 1999 | Connectix Corporation | Temporal compression and decompression for video | | US6023296 | Jun 15, 1998 | Feb 8, 2000 | Sarnoff Corporation | Apparatus and method for object based rate control in a coding system | | US6031573 | Oct 31, 1996 | Feb 29, 2000 | Sensormatic Electronics Corporation | Intelligent video information management system performing multiple functions in parallel | | US6058210 | Sep 15, 1997 | May 2, 2000 | Xerox Corporation | Using encoding cost data for segmentation of compressed image sequences | | US6084908 | Oct 23, 1996 | Jul 4, 2000 | Sarnoff Corporation Sharp Corporation | Apparatus and method for quadtree based variable block size motion estimation | | US6101220 | Jul 24, 1997 | Aug 8, 2000 | Victor Company of Japan, Ltd. | Method and apparatus for limiting band of moving-picture signal | | US6115420 | May 5, 1997 | Sep 5, 2000 | Microsoft Corporation | Digital video signal encoder and encoding method | | US6118817 | Mar 14, 1997 | Sep 12, 2000 | Microsoft Corporation | Digital video signal encoder and encoding method having adjustable quantization | | US6160846 | Oct 23, 1996 | Dec 12, 2000 | Sarnoff Corporation Sharp Corporation | Apparatus and method for optimizing the rate control in a coding system | | US6192081 | Oct 26, 1995 | Feb 20, 2001 | Sarnoff Corporation | Apparatus and method for selecting a coding mode in a block-based coding system | | US6243497 | Feb 11, 1998 | Jun 5, 2001 | Sarnoff Corporation | Apparatus and method for optimizing the rate control in a coding system | | US6317459 | Aug 25, 2000 | Nov 13, 2001 | Microsoft Corporation | Digital video signal encoder and encoding method | | US6320906 | May 20, 1997 | Nov 20, 2001 | Matsushita Electric Industrial Co., Ltd. | Motion vector detecting circuit | | US6349152 | Jun 30, 1997 | Feb 19, 2002 | Microsoft Corporation | Table-based compression with embedded coding | | US6356665 | Dec 9, 1998 | Mar 12, 2002 | Sharp Laboratories of America, Inc. | Quad-tree embedded image compression and decompression method and apparatus | | US6404923 | Mar 29, 1996 | Jun 11, 2002 | Microsoft Corporation | Table-based low-level image classification and compression system | | US6473461 | Apr 4, 2000 | Oct 29, 2002 | Microsoft Corporation | Half-pixel motion estimation in motion video signal encoding | | US6510249 | May 8, 2000 | Jan 21, 2003 | Matsushita Electric Industrial Co., Ltd. | Object-base digital image predictive coding transfer method and apparatus, and decoding apparatus | | US6529613 | May 26, 1999 | Mar 4, 2003 | Princeton Video Image, Inc. | Motion tracking using image-texture templates | | US6539122 | Mar 30, 1998 | Mar 25, 2003 | General Dynamics Decision Systems, Inc. | Adaptive wavelet coding of hyperspectral imagery | | US6560364 | Oct 11, 2002 | May 6, 2003 | Matsushita Electric Industrial Co., Ltd. | Object-based digital image predictive coding transfer method and apparatus, and decoding apparatus | | US6571016 | May 5, 1997 | May 27, 2003 | Microsoft Corporation | Intra compression of pixel blocks using predicted mean | | US6584226 | Mar 14, 1997 | Jun 24, 2003 | Microsoft Corporation | Method and apparatus for implementing motion estimation in video compression | | US6639945 | Jan 12, 2001 | Oct 28, 2003 | Microsoft Corporation | Method and apparatus for implementing motion detection in video compression | | US6687408 | Mar 14, 2003 | Feb 3, 2004 | Matsushita Electric Industrial Co., Ltd. | Object-based digital image predictive coding transfer method and apparatus | | US6690833 | Jun 18, 1998 | Feb 10, 2004 | Sarnoff Corporation | Apparatus and method for macroblock based rate control in a coding system | | US6707852 | Aug 31, 2000 | Mar 16, 2004 | Microsoft Corporation | Digital video signal encoder and encoding method | | US6741725 | Dec 10, 2002 | May 25, 2004 | Princeton Video Image, Inc. | Motion tracking using image-texture templates | | US6914939 | Nov 13, 2000 | Jul 5, 2005 | International Business Machines Corporation | Method and apparatus for producing pseudo-constant bits per picture video bit-streams for low-delay compression system | | US6937657 | Mar 16, 2004 | Aug 30, 2005 | Microsoft Corporation | Motion video signal encoder and encoding method | | US6968396 | Jul 26, 2001 | Nov 22, 2005 | Openwave Systems Inc. | Reloading of hypermedia pages by sending only changes | | US6972868 | Nov 9, 2000 | Dec 6, 2005 | Hewlett-Packard Development Company, L.P. | Image data compression method | | US7039110 | Jul 1, 2002 | May 2, 2006 | Koninklijke Philips Electronics N.V. | Methods of and units for motion or depth estimation and image processing apparatus provided with such motion estimation unit | | US7072396 | Mar 16, 2004 | Jul 4, 2006 | Microsoft Corporation | Motion video signal encoder and encoding method | | US7139313 | Jun 1, 2005 | Nov 21, 2006 | Microsoft Corporation | Digital video signal encoder and encoding method | | US7154951 | Mar 16, 2004 | Dec 26, 2006 | Microsoft Corporation | Motion video signal encoder and encoding method | | US7162091 | Mar 31, 2003 | Jan 9, 2007 | Microsoft Corporation | Intra compression of pixel blocks using predicted mean | | US7170934 | Dec 20, 2002 | Jan 30, 2007 | LSI Logic Corporation | Method and/or apparatus for motion estimation using a hierarchical search followed by a computation split for different block sizes | | US7181072 | Mar 31, 2003 | Feb 20, 2007 | Microsoft Corporation | Intra compression of pixel blocks using predicted mean | | US7203369 | May 22, 2003 | Apr 10, 2007 | Samsung Electronics Co., Ltd. | Method for estimating motion by referring to discrete cosine transform coefficients and apparatus therefor | | US7372903 | Sep 22, 1999 | May 13, 2008 | MediaTek, Inc. | Apparatus and method for object based rate control in a coding system | | US7397857 | Mar 16, 2001 | Jul 8, 2008 | Robert Bosch GmbH | Method for transformation-coding full motion image sequences | | US7424058 | Sep 28, 2000 | Sep 9, 2008 | Autodesk, Inc. | Variable bit-rate encoding | | US7606310 | Apr 14, 2008 | Oct 20, 2009 | On2 Technologies, Inc. | Video compression and encoding method | | US7643559 | Sep 12, 2002 | Jan 5, 2010 | NTT DoCoMo, Inc. | Coding method, decoding method, coding apparatus, decoding apparatus, image processing system, coding program, and decoding program | | US7715479 | Jul 21, 2003 | May 11, 2010 | International Business Machines Corporation | Power-aware on-chip memory management for video coding algorithms | | US7792188 | Mar 1, 2005 | Sep 7, 2010 | Apple Inc. | Selecting encoding types and predictive modes for encoding video data | | US7852916 | Mar 1, 2005 | Dec 14, 2010 | Apple Inc. | Efficient use of storage in encoding and decoding video data streams | | US7856149 | Jul 29, 2003 | Dec 21, 2010 | QUALCOMM Incorporated | Digital image encoding | | US7940844 | Feb 21, 2003 | May 10, 2011 | Qualcomm Incorporated | Video encoding and decoding techniques | | US8018994 | Jul 26, 2010 | Sep 13, 2011 | Apple Inc. | Selecting encoding types and predictive modes for encoding video data | | US8081678 | Feb 18, 2004 | Dec 20, 2011 | Panasonic Corporation | Picture coding method and picture decoding method | | US8111752 | Apr 28, 2005 | Feb 7, 2012 | Apple Inc. | Encoding mode pruning during video encoding | | US8194730 | Sep 8, 2010 | Jun 5, 2012 | Apple Inc. | Efficient use of storage in encoding and decoding video data streams | | US8194987 | Aug 22, 2007 | Jun 5, 2012 | QUALCOMM Incorporated | Digital image encoding |
Claims1. An interframe video compression system comprising: - a first motion predictor having an input for receiving a block of pixel data and having an output for providing a first image prediction;
- a first distortion calculator having a first input for receiving said block of pixel data and having a second input for receiving said first image prediction and having an output for providing a first distortion value;
- at least one additional motion predictor provided in parallel with said first motion predictor having an input for receiving said block of pixel data and having an output for providing additional image predictions;
- a second distortion calculator having a first input for receiving said block of pixel data and having a second input for receiving said additional image predictions and having a first output for providing a second distortion value;
- an encoding format selector having a first input coupled to said first distortion calculator output and having a second input coupled to said second distortion calculator output and having an output for providing a selected encoding format; and
- encoder having a first input coupled to said encoding format selector output and having an output for providing a selectively encoded residual frame, having a second input for receiving a first displaced frame difference generated in accordance with said first image prediction and having a third input for receiving a second displaced frame difference generated in accordance with said second image prediction and for selectively encoding said first displaced frame difference and said second displaced frame difference in accordance with said selected encoding format.
2. The system of claim 1 wherein said first motion predictor compares an N.times.N block of pixel data, where N is an integer, with N.times.N blocks of pixel data in a first reference block of data. 3. The system of claim 2 wherein said first motion predictor compares a 32.times.32 block of pixel data with 32.times.32 blocks of pixel data in a first reference block of data. 4. The system of claim 2 wherein said at least one additional motion predictor compares N/2.times.N/2 sub-blocks of pixel data with N/2.times.N/2 sub-blocks of data in said additional reference blocks of data. 5. The system of claim 1 wherein said first motion predictor has a second input for receiving a first reference block of data comprising pixel data from a previous frame of pixel data. 6. The system of claim 1 wherein said first motion predictor has a second input for receiving a combination block of data determined in accordance with previous frames of pixel data. 7. The system of claim 1 wherein said first motion predictor has a second output for providing a first motion vector. 8. The system of claim 1 wherein said at least one additional motion predictor has a second output for providing additional motion vectors. 9. The system of claim 1 further comprising a weighting value multiplier disposed between said first distortion calculator and said encoding format selector. 10. An interframe video compression system comprising: - first motion compensation means for receiving a block of video data and for comparing said block of video data with blocks of pixel data of a first block size to provide displaced frame difference (DFD) blocks, and for measuring distortion values for said DFD blocks, and for selecting a most similar block of pixel data of said first block size in accordance with said distortion values for said DFD blocks to provide a first motion vector, a first DFD block and a first distortion value;
- additional motion compensation means for receiving said block of video data and for comparing said block of video data with a plurality blocks of pixel data of a second block size to provide second DFD blocks, and for measuring distortion values for said second DFD blocks, and for selecting a set of most similar block of pixel data of said second block size in accordance with said distortion values for said second DFD blocks to provide a set of additional motion vectors, a set of additional second DFD blocks, and a second distortion value;
- decision means for comparing said first distortion value and said second distortion value and for selecting an encoding format in accordance with said comparison; and
- encoding means for selectively encoding said first motion vector and said first DFD block and said set of additional motion vectors and said second DFD blocks in accordance with said selected encoding format.
11. The system of claim 10 wherein said block of video data is an N.times.N block where N is an integer, and wherein said first block size is N.times.N. 12. The system of claim 11 wherein said block of video data is a 32.times.32 block and wherein first block size is 32.times.32. 13. The system of claim 11 wherein said second block size is N/2=N/2. 14. The system of claim 10 wherein said blocks of pixel data of a first block size comprise pixel data from a previous frame of pixel data. 15. The system of claim 10 wherein said blocks of pixel data of a first block size are combinations of pixel data from previous frames of pixel data. 16. The system of claim 10 wherein said decision means performs said comparison responsive to a weighting format. 17. The system of claim 10 further comprising a weighting means for receiving said additional distortion value by a predetermined weighting format. 18. An interframe video compression method for compressing a block of video data comprising the steps of: - comparing said block of video data with blocks of pixel data of a first block size to provide displaced frame difference (DFD) blocks;
- measuring distortion values for said DFD blocks;
- selecting a most similar block of pixel data of said first block size in accordance with said distortion values to provide a first motion vector, a first DFD block and a first distortion value;
- comparing said block of video data with a plurality blocks of pixel data of a second block size to provide additional DFD blocks;
- measuring distortion values for said DFD blocks;
- selecting a set of most similar block of pixel data of said second block size in accordance with said distortion values for said additional DFD blocks to provide a set of additional motion vectors, a set of additional DFD blocks and a second distortion value;
- selecting an encoding format in accordance with said first distortion value and said second distortion value; and
- selectively encoding said first motion vector and said first DFD block and said set of additional motion vectors and said set of additional DFD blocks in accordance with said selected encoding format.
19. The method of claim 18 wherein said block of video data is an N.times.N block, where N is an integer, and wherein said first block size is N.times.N. 20. The method of claim 19 wherein said block of video data is a 32.times.32 block and wherein first block size is 32.times.32. 21. The method of claim 19 wherein said second block size is N/2.times.N/2. 22. The method of claim 18 wherein said blocks of pixel data of a first block size comprise pixel data from a previous frame of video data. 23. The method of claim 18 wherein said blocks of pixel data of a first block size are combinations of blocks of data from previous frames of video data. 24. The method of claim 18 further comprising the step of weighting said additional distortion value by a predetermined weighting format. |