|Publication number||US7822125 B2|
|Application number||US 11/153,243|
|Publication date||Oct 26, 2010|
|Filing date||Jun 14, 2005|
|Priority date||Jun 14, 2004|
|Also published as||CN1969556A, CN100542288C, EP1757105A1, US20060018386, WO2005122588A1|
|Publication number||11153243, 153243, US 7822125 B2, US 7822125B2, US-B2-7822125, US7822125 B2, US7822125B2|
|Original Assignee||Tandberg Telecom As|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (7), Non-Patent Citations (4), Referenced by (6), Classifications (16), Legal Events (3)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This application claims priority under 35 U.S.C. §119 or 365 to Norwegian Application No. 20042477, filed Jun. 14, 2004. The entire teachings of the above application are incorporated herein by reference.
Transmission of moving pictures in real-time is employed in several applications like e.g. video conferencing, net meetings, TV broadcasting and video telephony.
However, representing moving pictures requires bulk information as digital video typically is described by representing each pixel in a picture with 8 bits (1 Byte). Such uncompressed video data results in large bit volumes, and can not be transferred over conventional communication networks and transmission lines in real time due to limited bandwidth.
Thus, enabling real time video transmission requires a large extent of data compression. Data compression may, however, compromise with picture quality. Therefore, great efforts have been made to develop compression techniques allowing real time transmission of high quality video over bandwidth limited data connections.
In video compression systems, the main goal is to represent the video information with as little capacity as possible. Capacity is defined with bits, either as a constant value or as bits/time unit. In both cases, the main goal is to reduce the number of bits.
The most common video coding method is described in the MPEG* and H.26* standards. The video data undergo four main processes before transmission, namely prediction, transformation, quantization and entropy coding.
The prediction process significantly reduces the amount of bits required for each picture in a video sequence to be transferred. It takes advantage of the similarity of parts of the sequence with other parts of the sequence. Since the predictor part is known to both encoder and decoder, only the difference has to be transferred. This difference typically requires much less capacity for its representation. The prediction is mainly based on vectors representing movements. The prediction process is typically performed on square block sizes (e.g. 16×16 pixels). Note that in some cases, predictions of pixels based on the adjacent pixels in the same picture rather than pixels of preceding pictures are used. This is referred to as intra prediction, as opposed to inter prediction. Consequently, when the pixels in a block are coded by means of intra prediction, the block is said to be an intra coded.
The residual represented as a block of data (e.g. 4×4 pixels) still contains internal correlation. A well-known method of taking advantage of this is to perform a two dimensional block transform. In H.263 an 8×8 Discrete Cosine Transform (DCT) is used, whereas H.264 uses a 4×4 integer type transform. This transforms 4×4 pixels into 4×4 transform coefficients and they can usually be represented by fewer bits than the pixel representation. Transform of a 4×4 array of pixels with internal correlation will probably result in a 4×4 block of transform coefficients with much fewer non-zero values than the original 4×4 pixel block.
Direct representation of the transform coefficients is still too costly for many applications. A quantization process is carried out for a further reduction of the data representation. Hence the transform coefficients undergo quantization. A simple version of quantisation is to divide parameter values by a number—resulting in a smaller number that may be represented by fewer bits. It should be mentioned that this quantization process has as a result that the reconstructed video sequence is somewhat different from the uncompressed sequence. This phenomenon is referred to as “lossy coding”. The outcome from the quantisation part is referred to as quantized transform coefficients.
Entropy coding implies lossless representation of different types of parameters such as overhead data or system description, prediction data (typically motion vectors), and quantized transform coefficients from the quantisation process. The latter typically represent the largest bit consumption.
The coding is performed on block wise parts of the video picture. A macro block consists of several sub blocks for luminance (luma) as well as for chrominance (chroma). There are typically two chrominance components (Cr, Cb) with half the resolution both horizontally and vertically compared with luminance. In
The present video standards H.261, H.262, H.263, H.264/AVC, MPEG1, MPEG2, MPEG4 all use blockbased coding. This means blockbased prediction from previously encoded and decoded pictures as well as blockbased coding of residual signal.
Blockbased coding/decoding has proven to be very efficient. However, one of the drawbacks is that the reconstructed image may have visible artifacts corresponding to the blocks used for prediction and residual signal coding. This phenomenon is usually referred to as blocking or blocking artifacts.
The present invention is related to block wise coded video pictures. The present invention discloses a method in video decoding for reducing blocking artifacts between a first and a second chroma block in a block wise coded video picture, wherein if at least one of the first and second chroma block is intra coded, executing a de-blocking filter operation on pixels adjacent to a boundary between the first and the second chroma block on each side of said boundary, or, if at least one of the first and the second chroma block is included in a collection of adjacent blocks including one or more blocks having non-zero transform coefficients, executing a de-blocking filter operation on pixels adjacent to said boundary between the first and the second chroma block on each side of said boundary.
The foregoing and other objects, features and advantages of the invention will be apparent from the following more particular description of preferred embodiments of the invention, as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention.
In order to make the invention more readily understandable; the discussion that follows will refer to the accompanying drawings and tables.
The present invention is an improvement of prior art method for removal of blocking and quantization noise. This method is described in the specification ITU-T Rec. H.264|ISO/IEC 14496-10 AVC, the basics of which are described in the following.
In H.264/AVC an adaptive deblocking filter is applied in the coding loop. This means that further prediction is performed on filtered images. The filter is designed to remove as much as possible of the blocking and quantization noise and still maintain as much of the image content as possible. It is often a challenge to separate quantization noise and picture content. This is why the filter is highly content adaptive and therefore complex concerning computational operations.
When the above test indicates blocking artifacts, a smoothing operation is carried out for each pixel line crossing the boundary. The operation starts by calculating a delta value from the pixels close to the boundary. Using the pixel notation defined in
This delta value then undergoes a clipping procedure resulting in a delta marked value. The delta marked value is dependent on the quantization value that has been used in current coding, i.e. the size of the quantization interval, and on the above-listed criteria. According to H.264/AVC, the clipping is found by a look up in a specified table.
Finally, the delta marked value is used to modify the pixels closed to the boundary. As an example, the new values of d and e are calculated as follows:
d′=d+Δ′, e′=e−A′ (2)
The lists of criteria above are performed on luma blocks, but the result is decided also to apply for the corresponding chroma blocks. The same filtering procedure as described above is used, but on fewer pixels.
Performing deblocking filtering as described above is one of the most resource demanding operations in H.264. The chroma filtering represents a large part of this computational burden, even if the chroma filtering is somewhat simpler. As an example, the chroma filtering has to consider as many edge characteristics as luma filtering since luma edge characteristics is used also for chroma.
It is also usually the case that chroma deblocking filtering result in considerably less subjective quality improvement compared to luma deblocking filtering.
Lastly the present definition of chroma deblocking filtering in H.264 with the dependence of corresponding luma deblocking filtering result in a suboptimal chroma deblocking filtering.
Based on the fact that the need for chroma deblocking filtering is less than for luma deblocking filtering, the present invention discloses a simplified method of determining the boundary characteristics. Chroma deblocking filtering is used less frequently than corresponding luma deblocking filtering, and it should only depend on the characteristics of the chroma coding itself, and not on the luma filtering, which is the case for current H.264/AVC.
According to the present invention, the deciding criteria listed above are replaced by new deciding criteria that are used to decide whether to perform de-blocking filtering or not. As opposed to current H.264/AVC, the criteria apply to chroma block characteristics. In one embodiment of the invention, artifacts are said likely to occur between two blocks when the following situation is detected:
In a second embodiment of the invention, artifacts are said likely to occur between two blocks when the following situations are detected:
Note that the term collection of blocks in the e)-criterion could be one single block for one specific chroma component to a collection of blocks (macroblock) for each chroma component (Cr, Cb) or for several chroma components.
In a third embodiment of the invention, artifacts are said likely to occur between two blocks when a combination of d) and e) occurs.
In preferred embodiments of the invention, for 8×8 Chroma block coding of residual signal a maximum of 2 pixels on each side of the block boundary, i.e., c d e f in
The calculation of the smoothed edge pixels including delta and clipping is carried out e.g. as described in current H.264/AVC.
However, a simplified table compared to the one shown in
While this invention has been particularly shown and described with references to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention encompassed by the appended claims.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US6215425||Apr 22, 1998||Apr 10, 2001||Netergy Networks, Inc.||Deblocking filter for encoder/decoder arrangement and method with divergence reduction|
|US6690838||Apr 27, 2001||Feb 10, 2004||Equator Technologies, Inc.||Image processing circuit and method for reducing a difference between pixel values across an image boundary|
|US7319415 *||Feb 11, 2003||Jan 15, 2008||Thomson Licensing||Chroma deblocking filter|
|US20010020906||Dec 11, 2000||Sep 13, 2001||Andrews Barry D.||Deblocking filter for encoder/decoder arrangement and method with divergence reduction|
|US20030206587||Feb 11, 2003||Nov 6, 2003||Cristina Gomila||Chroma deblocking filter|
|US20040190626||Mar 11, 2004||Sep 30, 2004||Shijun Sun||Adaptive filtering based upon boundary strength|
|JP2002330436A||Title not available|
|1||Advanced video coding for generic audiovisual services, International Telecommunication Union, H.264 (May 2003), pp. 144-153.|
|2||Anthony Joch, "Loop Filter Simplification and Improvement", Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG(ISO/IEC JTC1/SC29/WG11 and ITU-T SG16Q.6), No. JVT-D037, XP-030005311, Jul. 26, 2002, pp. 1-10.|
|3||Peter List, et al., "Adaptive Deblocking Filter", IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, No. 7, XP-011221094, Jul. 1, 2003, pp. 614-619.|
|4||Thomas Wiegand, "Joint Committee Draft (CD)", Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, 3rd Meeting, Fairfax, Virginia, USA, May 6-10, 2002, pp. 58-63.|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US8165223||Apr 24, 2012||Cisco Technology, Inc.||Deblocking filter|
|US8326064 *||Jan 17, 2008||Dec 4, 2012||Nec Corporation||Image re-encoding method to decode image data which is orthogonally transformed per first block and encoded by a first encoding method|
|US8644394||Dec 27, 2011||Feb 4, 2014||Cisco Technology, Inc.||Deblocking filter|
|US9277243 *||Oct 25, 2007||Mar 1, 2016||Thomson Licensing||Methods and apparatus for in-loop de-artifact filtering|
|US20090257664 *||Oct 25, 2007||Oct 15, 2009||Meng-Ping Kao||Methods and apparatus for in -loop de-artifact filtering|
|US20100086225 *||Jan 17, 2008||Apr 8, 2010||Shoji Yachida||Image re-encoding device, image re-encoding method, and image encoding program|
|U.S. Classification||375/240.29, 375/240.25|
|International Classification||H04N7/26, H04N7/50, H04N7/18, H04N|
|Cooperative Classification||H04N19/44, H04N19/85, H04N19/186, H04N19/527, H04N19/61|
|European Classification||H04N7/26A6C8, H04N7/26P, H04N7/26D, H04N7/26M2G, H04N7/50|
|Oct 5, 2005||AS||Assignment|
Owner name: TANDBERG TELECOM AS, NORWAY
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:BJONTEGAARD, GISI E.;REEL/FRAME:016852/0941
Effective date: 20050915
|Dec 1, 2011||AS||Assignment|
Owner name: CISCO TECHNOLOGY, INC., CALIFORNIA
Free format text: CONFIRMATORY ASSIGNMENT;ASSIGNORS:TANDBERG TELECOM AS;CISCO SYSTEMS INTERNATIONAL SARL;SIGNING DATES FROM 20111110 TO 20111129;REEL/FRAME:027307/0451
|Apr 28, 2014||FPAY||Fee payment|
Year of fee payment: 4