WO2000046999A1 - Quantizer selection based on region complexities derived using a rate distortion model - Google Patents
Quantizer selection based on region complexities derived using a rate distortion model Download PDFInfo
- Publication number
- WO2000046999A1 WO2000046999A1 PCT/US2000/003092 US0003092W WO0046999A1 WO 2000046999 A1 WO2000046999 A1 WO 2000046999A1 US 0003092 W US0003092 W US 0003092W WO 0046999 A1 WO0046999 A1 WO 0046999A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- region
- frame
- macroblock
- current frame
- quantizer
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/587—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
- H04N19/126—Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
- H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
- H04N19/152—Data rate or code amount at the encoder output by measuring the fullness of the transmission buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
- H04N19/192—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding the adaptation method, adaptation tool or adaptation type being iterative or recursive
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
Definitions
- the present invention relates to image processing, and, in particular, to video compression.
- the goal of video compression processing is to encode image data to reduce the number of bits used to represent a sequence of video images while maintaining an acceptable level of quality in the decoded video sequence.
- This goal is particularly important in certain applications, such as videophone or video conferencing over POTS (plain old telephone service) or ISDN (integrated services digital network) lines, where the existence of limited transmission bandwidth requires careful control over the bit rate, that is, the number of bits used to encode each image in the video sequence.
- POTS plain old telephone service
- ISDN integrated services digital network
- images may be designated as the following different types of frames for compression processing: o An intra (I) frame which is encoded using only intra-frame compression techniques, o A predicted (P) frame which is encoded using inter-frame compression techniques based on a previous I or P frame, and which can itself be used as a reference frame to encode one or more other frames, o A bi-directional (B) frame which is encoded using bi-directional inter-frame compression techniques based on a previous I or P frame, a subsequent I or P frame, or a combination of both, and which cannot itself be used to encode another frame, and o A PB frame which corresponds to two images — a P frame and a subsequent B frame ⁇ that are encoded as a single frame (as in the H.263 video compression algorithm).
- I intra
- P predicted
- B bi-directional
- I frames typically require the greatest numbers of bits
- B frames typically require the least number of bits.
- a block-based transform such as a discrete cosine transform (DCT)
- DCT discrete cosine transform
- the resulting transform coefficients for each block are then quantized for subsequent encoding (e.g., run-length encoding followed by variable-length encoding).
- the degree to which the transform coefficients are quantized directly affects both the number of bits used to represent the image data and the quality of the resulting decoded image. This degree of quantization is also referred to as the quantization level, which is often represented by a specified quantizer value that is used to quantize all of the transform coefficients.
- the quantization level refers to a particular table of quantizer values that are used to quantize the different transform coefficients, where each transform coefficient has its own corresponding quantizer value in the table.
- higher quantizer values imply more severe quantization and therefore fewer bits in the encoded bitstream at the cost of lower playback quality of the decoded images.
- the quantizer is often used as the primary variable for controlling the tradeoff between bit rate and image quality.
- Visual quality of video depends not only on global measures (like pixel signal-to- noise ratio (PSNR)), but also on how the error is distributed in space and time. Thus, it is important to maintain spatial smoothness of the quantizer (which is closely related to the local distortion) across the picture.
- PSNR pixel signal-to- noise ratio
- the ideal quantizer selection is a uniform value across the scene.
- such a scheme will not support the moving of bits to a more-important region from less-important regions, and furthermore, will provide very little control over the bits used to encode the picture. Thus, it cannot be used in constant (or near-constant) bit-rate applications (like videophone and video-conferencing over POTS or ISDN).
- the quantizer value can change by a value of at most 2 in either direction from one macroblock to the next when following a raster scan pattern through the image).
- Examples of such schemes are given in the H.263+ TMN8 (Test Model Near-Term 8) and TMN9 documents (see, e.g., ITU - Telecommunications Standardization Sector, "Video Codec Test Model, Near-Term, Version 9 (TMN9)", Document Q15-C-15, December 1997).
- TMN8 Transmission Model Near-Term 8
- TMN9 See, e.g., ITU - Telecommunications Standardization Sector, "Video Codec Test Model, Near-Term, Version 9 (TMN9)", Document Q15-C-15, December 1997).
- the present invention is directed to a technique for assigning quantization levels (e.g., quantizer values) used during video compression processing.
- quantization levels e.g., quantizer values
- an image is segmented into one or more different regions, and a temporal prediction model is separately applied to each region to assign a quantization level to be used to quantize the transform coefficients for the macroblocks of that region. Because the temporal prediction model is applied separately to each region, a different quantization level may be - but does not have to be - assigned to each different region.
- the present invention is a method for encoding a current frame in a video sequence, comprising the steps of (a) segmenting the current frame into one or more different regions; (b) generating an encoding complexity measure for each corresponding region of a previously encoded frame in the video sequence; (c) using the encoding complexity measure for each region of the previous frame to select a quantization level for the corresponding region of the current frame; and (d) encoding the current frame using the one or more selected quantization levels.
- Fig. 2 shows a flow diagram of the processing involved in applying a temporal prediction model during video compression processing, according to one embodiment of the present invention.
- Fig. 1 shows an example of a typical image 100 that can be encoded using the present invention.
- the image in Fig. 1 consists of the head and shoulders of a person 102 positioned in front of background imagery 104, where the image data corresponding to the head of person 102 varies more in time (i.e., from frame to frame) than the background imagery.
- Such a "talking head" scene is typical of videophone and video-conferencing applications.
- the person in the foreground is of greater importance to the viewer of image 100 than is the background imagery.
- image 100 is encoded such that, during playback, the video quality of the more-important foreground imagery is greater than the video quality of the less-important background imagery.
- This variation in playback video quality within an image is achieved by allowing the quantizers used during the video compression processing to encode the macroblocks of image 100 to vary within the image.
- the selection of quantizers follows a particular scheme, described as follows.
- image 100 is divided into three different regions: a foreground region 106 (also referred to as the most-important region) consisting of those macroblocks corresponding to the head of person 102, a background region 108 (also referred to as the least-important region) consisting of macroblocks corresponding to background imagery 104 (including, in this particular example, the shoulders of person 102), and a transition region 110 consisting of macroblocks located between the foreground region and the background region.
- a foreground region 106 also referred to as the most-important region
- background region 108 also referred to as the least-important region
- transition region 110 consisting of macroblocks located between the foreground region and the background region.
- all of the macroblocks corresponding to the foreground region 106 are encoded using the same quantizer QP2
- all of the macroblocks corresponding to the background region 108 are encoded using the same quantizer QPO
- the most-important, foreground region 106 is typically encoded using more bits per macroblock than the transition region 110, which in turn is typically encoded using more bits per macroblock than the least-important, background region 108.
- Fig. 2 shows a flow diagram of the processing involved in applying a temporal prediction model during video compression processing, according to one embodiment of the present invention.
- the processing of Fig. 2 may be applied to a video compression scheme that segments each image into three regions, such as those shown in Fig. 1.
- the processing of Fig. 2 may be applied in video compression schemes that segment images into any number of regions, including those that treat each image as a single region.
- the processing of Fig. 2 will be assumed to be for a video compression scheme that is based on the three regions of Fig. 1.
- Processing begins by initializing the quantizer values for the three regions (step 202 of Fig. 2).
- the quantizers for all three regions may be initialized to the same value, for example, conservatively selecting a high (or even the highest possible) quantizer value to ensure initial compliance with target bit rates.
- the most-important region i.e., foreground region 106 in Fig. 1
- the least-important region i.e., background region 108
- the magnitude of the difference in quantizers between successive macroblocks may be limited, e.g., ⁇ 2, as in the H.263 video compression standard. In such applications, this constraint is taken into account during the initialization of quantizer values.
- segmentation analysis is applied to the first image (i.e., frame) in the video sequence to identify the three subsets of macroblocks corresponding to the foreground, background, and transition regions, respectively (step 204). Any suitable type of segmentation analysis may be used.
- each of the three subsets of macroblocks may, but need not correspond to a contiguous set of macroblocks in the image.
- the foreground region may consist of two different contiguous sets of macroblocks, each corresponding to the face of a different video-conference participant.
- the general principle of the segmentation scheme of Fig. 1 is that at least one macroblock of the transition region be located between a macroblock of the foreground region and a macroblock of the background region when following a raster scan pattern through the image.
- the transition region for a particular image may be made up of a relatively large number of non-contiguous sets of macroblocks, as in transition region 110 of Fig. 1
- the first image is encoded, where the transform coefficients for all of the macroblocks of each region are quantized using the particular quantizer value selected for that region (step 206).
- This encoding may, but does not have to, involve the selection of alternative quantizer values and the requantization of the transform coefficients, if and when the target bit rate is not met for the frame using the initial selection of quantizers.
- steps 208-216 which are applied sequentially to each of the remaining images in the scene. The processing preferably starts all over again at a scene change from step 202.
- the I frame is encoded using a uniform quantizer which is the average quantizer from the preceding P frame, and no model updates are done.
- the subsequent P frame uses the model update from the last P frame and processing proceeds. It is as if the I frame did not exist for the purpose of choosing and constraining quantizers for the subsequent P frame.
- data from the previous frame is used to generate a value for the encoding complexity measure X of each macroblock in the previous frame using Equation (1) as follows:
- the distortion measure S may be any suitable measure of the distortion of the macroblock.
- one possible distortion measure is based on the sum of the absolute differences (SAD) of the inter-frame pixel difference data for the macroblock.
- SAD absolute differences
- intra-frame encoded data one possible distortion measure is based on the SAD between the individual pixel values and the mean pixel value for the corresponding block or macroblock. In either case, in Equation (1), the distortion measure S is the average SAD per pixel in the macroblock, translated to the DCT domain.
- the DCT coefficients (before quantization) have a different dynamic range from the original pixel data due to a scaling factor (which is generally 8).
- a scaling factor which is generally 8
- S Until S reaches a certain threshold, the block will quantize to zero (i.e., none of the coefficients will be significant).
- This threshold is modeled as 2.5Q, since the actual quantization step size is 2Q, with the rounding used by H.263. This means that S has to be greater than 2.5Q for it to yield at least one non-zero quantized coefficient. Note that this thresholding can be avoided by setting C to 0.
- each complexity value X is derived as an average of the corresponding regions of the forward and backward reference frames.
- the number of header bits (H) is the sum of Hmb (the number of bits in the macroblock mode headers for the previous frame) and Hmv (the number of bits for the motion vectors for the current frame). Any of the parameters that are based on the previous frame (i.e., Xave, Q, and/or Hmb) may be derived from more than just the one previous frame. Equation (2) is applied to different quantizer values in an iterative manner until the frame target bit rate is satisfied to within a specified tolerance range. This can be done iteratively for each region while assuming the maximum difference between quantizer values for different regions allowable under the compression algorithm. Note that the first P frame does not have a history and the average encoding complexities for the previous frame (Xave) are not available. In this case, a "fall- back" mode is preferably used to adapt the quantizers over the frame to meet the target. At that point, the actual bits used for each macroblock within a region are used to initialize the region complexity.
- the quantizer values are adjusted if necessary for both spatial and temporal considerations (step 212). Spatially, the quantizer values generated using Equation (2) may need to be adjusted to ensure that the magnitude of change in quantizer value from one macroblock in one region to the next macroblock in a different region does not exceed the limit, if one exists, imposed by the video compression algorithm. Temporally, the magnitude of change in quantizer values from one region in one frame to the corresponding region in the next frame as generated using Equation (2) is preferably limited to ensure temporal smoothness in quality.
- the temporal variation of the quantizer for each region may be constrained to 15% of Q when the quantizer value is reduced and to 60% of Q when the quantizer value is increased, where Q can vary from 1 to 31.
- step 214 segmentation analysis is applied to the current image to identify the three subsets of macroblocks corresponding to the foreground, background, and transition regions, respectively.
- step 216 the current image is encoded, where the transform coefficients for all of the macroblocks of each region are quantized using the particular quantizer value selected for that region.
- steps 214 and 216 for the current image is analogous to the processing of steps 202 and 204 for the first image.
- the temporal prediction model of Equations (1) and (2) is based on a first-order rate distortion model.
- higher-order models such as a closed-form solution for a quadratic rate distortion model, may be used.
- the present invention can be embodied in the form of methods and apparatuses for practicing those methods.
- the present invention can also be embodied in the form of program code embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage medium, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention.
- the present invention can also be embodied in the form of program code, for example, whether stored in a storage medium, loaded into and/or executed by a machine, or transmitted over some transmission medium or carrier, such as over electrical wiring or cabling, through fiber optics, or via electromagnetic radiation, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention.
- program code When implemented on a general-purpose processor, the program code segments combine with the processor to provide a unique device that operates analogously to specific logic circuits.
Abstract
For video compression processing, each frame in a video sequence is segmented into one or more different regions, where the macroblocks of each region are to be encoded using the same quantizer value, but the quantizer value can vary between regions in a frame. For example, for the videophone or video-conferencing paradigm of one or more 'talking heads' in front of a relatively static background, each frame is segmented into a foreground region corresponding to the talking head, a background region corresponding to the static background, and an intervening transition region. An encoding complexity measure is generated for each macroblock of the previous frame using a (e.g., first-order) rate distortion model and the resulting macroblock-level encoding complexities are used to generate an average encoding complexity for each region. These region complexities are then used to select quantizer values for each region in the current frame, e.g., iteratively until the target bit rate for the frame is satisfied to within a specified tolerance range. The selected quantizer values may be modified based on spatial and/or temporal constraints to satisfy spatial requirements of the video compression algorithm and/or to provide temporal smoothness in quality, respectively.
Description
QUANTIZER SELECTION BASED ON REGION COMPLEXITIES DERIVED USING A RATE DISTORTION MODEL
BACKGROUND OF THE INVENTION Field of the Invention
The present invention relates to image processing, and, in particular, to video compression.
Cross-Reference to Related Applications This application claims the benefit of the filing date of U.S. provisional application no. 60/118,359, filed on 02/03/99 as attorney docket no. SAR 1333 IP.
Description of the Related Art
The goal of video compression processing is to encode image data to reduce the number of bits used to represent a sequence of video images while maintaining an acceptable level of quality in the decoded video sequence. This goal is particularly important in certain applications, such as videophone or video conferencing over POTS (plain old telephone service) or ISDN (integrated services digital network) lines, where the existence of limited transmission bandwidth requires careful control over the bit rate, that is, the number of bits used to encode each image in the video sequence. Furthermore, in order to satisfy the transmission and other processing requirements of a video conferencing system, it is often desirable to have a relatively steady flow of bits in the encoded video bitstream. That is, the variations in bit rate from image to image within a video sequence should be kept as low as practicable. Achieving a relatively uniform bit rate can be very difficult, especially for video compression algorithms that encode different images within a video sequence using different compression techniques. Depending on the video compression algorithm, images may be designated as the following different types of frames for compression processing: o An intra (I) frame which is encoded using only intra-frame compression techniques, o A predicted (P) frame which is encoded using inter-frame compression techniques based on a previous I or P frame, and which can itself be used as a reference frame to encode one or more other frames,
o A bi-directional (B) frame which is encoded using bi-directional inter-frame compression techniques based on a previous I or P frame, a subsequent I or P frame, or a combination of both, and which cannot itself be used to encode another frame, and o A PB frame which corresponds to two images — a P frame and a subsequent B frame ~ that are encoded as a single frame (as in the H.263 video compression algorithm).
Depending on the actual image data to be encoded, these different types of frames typically require different numbers of bits to encode. For example, I frames typically require the greatest numbers of bits, while B frames typically require the least number of bits.
In a typical transform-based video compression algorithm, a block-based transform, such as a discrete cosine transform (DCT), is applied to blocks of image data corresponding either to pixel values or pixel differences generated, for example, based on a motion- compensated inter-frame differencing scheme. The resulting transform coefficients for each block are then quantized for subsequent encoding (e.g., run-length encoding followed by variable-length encoding). The degree to which the transform coefficients are quantized directly affects both the number of bits used to represent the image data and the quality of the resulting decoded image. This degree of quantization is also referred to as the quantization level, which is often represented by a specified quantizer value that is used to quantize all of the transform coefficients. In some video compression algorithms, the quantization level refers to a particular table of quantizer values that are used to quantize the different transform coefficients, where each transform coefficient has its own corresponding quantizer value in the table. In general, higher quantizer values imply more severe quantization and therefore fewer bits in the encoded bitstream at the cost of lower playback quality of the decoded images. As such, the quantizer is often used as the primary variable for controlling the tradeoff between bit rate and image quality. Visual quality of video depends not only on global measures (like pixel signal-to- noise ratio (PSNR)), but also on how the error is distributed in space and time. Thus, it is important to maintain spatial smoothness of the quantizer (which is closely related to the local distortion) across the picture. In fact, in many scenes, the ideal quantizer selection is a uniform value across the scene. However, such a scheme will not support the moving of bits to a more-important region from less-important regions, and furthermore, will provide very little control over the bits used to encode the picture. Thus, it cannot be used in constant (or
near-constant) bit-rate applications (like videophone and video-conferencing over POTS or ISDN).
The other possibility is to vary the quantizer from macroblock to macroblock within the constraints of the coding standard being used (for example, in H.263, the quantizer value can change by a value of at most 2 in either direction from one macroblock to the next when following a raster scan pattern through the image). Examples of such schemes are given in the H.263+ TMN8 (Test Model Near-Term 8) and TMN9 documents (see, e.g., ITU - Telecommunications Standardization Sector, "Video Codec Test Model, Near-Term, Version 9 (TMN9)", Document Q15-C-15, December 1997). In these schemes, while the frame-level bit target can be accurately met, there are many, possibly large quantizer changes, both spatially and temporally, which show up annoyingly in the moving video as undesirable artifacts.
SUMMARY OF THE INVENTION The present invention is directed to a technique for assigning quantization levels (e.g., quantizer values) used during video compression processing. According to the present invention, an image is segmented into one or more different regions, and a temporal prediction model is separately applied to each region to assign a quantization level to be used to quantize the transform coefficients for the macroblocks of that region. Because the temporal prediction model is applied separately to each region, a different quantization level may be - but does not have to be - assigned to each different region.
According to one embodiment, the present invention is a method for encoding a current frame in a video sequence, comprising the steps of (a) segmenting the current frame into one or more different regions; (b) generating an encoding complexity measure for each corresponding region of a previously encoded frame in the video sequence; (c) using the encoding complexity measure for each region of the previous frame to select a quantization level for the corresponding region of the current frame; and (d) encoding the current frame using the one or more selected quantization levels.
BRIEF DESCRIPTION OF THE DRAWINGS Other aspects, features, and advantages of the present invention will become more fully apparent from the following detailed description, the appended claims, and the accompanying drawings in which: Fig. 1 shows an example of a typical image that can be encoded using the present invention; and
Fig. 2 shows a flow diagram of the processing involved in applying a temporal prediction model during video compression processing, according to one embodiment of the present invention. DETAILED DESCRIPTION
Fig. 1 shows an example of a typical image 100 that can be encoded using the present invention. The image in Fig. 1 consists of the head and shoulders of a person 102 positioned in front of background imagery 104, where the image data corresponding to the head of person 102 varies more in time (i.e., from frame to frame) than the background imagery. Such a "talking head" scene is typical of videophone and video-conferencing applications. In general, during playback, the person in the foreground is of greater importance to the viewer of image 100 than is the background imagery. According to the present invention, image 100 is encoded such that, during playback, the video quality of the more-important foreground imagery is greater than the video quality of the less-important background imagery. This variation in playback video quality within an image is achieved by allowing the quantizers used during the video compression processing to encode the macroblocks of image 100 to vary within the image. According to the present invention, the selection of quantizers follows a particular scheme, described as follows.
As shown in Fig. 1, image 100 is divided into three different regions: a foreground region 106 (also referred to as the most-important region) consisting of those macroblocks corresponding to the head of person 102, a background region 108 (also referred to as the least-important region) consisting of macroblocks corresponding to background imagery 104 (including, in this particular example, the shoulders of person 102), and a transition region 110 consisting of macroblocks located between the foreground region and the background region. According to the present invention, all of the macroblocks corresponding to the foreground region 106 are encoded using the same quantizer QP2, all of the macroblocks
corresponding to the background region 108 are encoded using the same quantizer QPO, and all of the macroblocks corresponding to the transition region 110 are encoded using the same quantizer QP1, where, typically, QPO >= QP1 >= QP2. As such, the most-important, foreground region 106 is typically encoded using more bits per macroblock than the transition region 110, which in turn is typically encoded using more bits per macroblock than the least-important, background region 108. As a result, the foreground region 106 will typically have the highest quality and the background region 108 will typically have the lowest quality during playback of the decoded image data, as is usually desired for applications such as videophone and video-conferencing. Fig. 2 shows a flow diagram of the processing involved in applying a temporal prediction model during video compression processing, according to one embodiment of the present invention. Depending on the application, the processing of Fig. 2 may be applied to a video compression scheme that segments each image into three regions, such as those shown in Fig. 1. In general, however, the processing of Fig. 2 may be applied in video compression schemes that segment images into any number of regions, including those that treat each image as a single region. For the following discussion, the processing of Fig. 2 will be assumed to be for a video compression scheme that is based on the three regions of Fig. 1.
Processing begins by initializing the quantizer values for the three regions (step 202 of Fig. 2). Depending on the particular implementation, the quantizers for all three regions may be initialized to the same value, for example, conservatively selecting a high (or even the highest possible) quantizer value to ensure initial compliance with target bit rates. Alternatively, the most-important region (i.e., foreground region 106 in Fig. 1) may be assigned a relatively low quantizer value (corresponding to less-severe quantization) and the least-important region (i.e., background region 108) may be assigned a relatively high quantizer value (corresponding to more-severe quantization) to initialize processing at more desirable quantization levels. In some video compression algorithms, the magnitude of the difference in quantizers between successive macroblocks (e.g., following a raster scan pattern through the image) may be limited, e.g., ±2, as in the H.263 video compression standard. In such applications, this constraint is taken into account during the initialization of quantizer values.
After initializing the quantizer values, segmentation analysis is applied to the first image (i.e., frame) in the video sequence to identify the three subsets of macroblocks corresponding to the foreground, background, and transition regions, respectively (step 204). Any suitable type of segmentation analysis may be used. Depending on the segmentation analysis employed and the characteristics of the video imagery, each of the three subsets of macroblocks may, but need not correspond to a contiguous set of macroblocks in the image. For example, in a video-conferencing application in which two "talking heads" appear in the image corresponding to two different individuals, the foreground region may consist of two different contiguous sets of macroblocks, each corresponding to the face of a different video-conference participant. Moreover, the general principle of the segmentation scheme of Fig. 1 is that at least one macroblock of the transition region be located between a macroblock of the foreground region and a macroblock of the background region when following a raster scan pattern through the image. As such, the transition region for a particular image may be made up of a relatively large number of non-contiguous sets of macroblocks, as in transition region 110 of Fig. 1
(where sets of macroblocks touching at only a single corner are not considered to be contiguous).
After the first image has been segmented into three sets of macroblocks corresponding to the three regions, the first image is encoded, where the transform coefficients for all of the macroblocks of each region are quantized using the particular quantizer value selected for that region (step 206). This encoding may, but does not have to, involve the selection of alternative quantizer values and the requantization of the transform coefficients, if and when the target bit rate is not met for the frame using the initial selection of quantizers. After the first image is encoded, processing continues to steps 208-216, which are applied sequentially to each of the remaining images in the scene. The processing preferably starts all over again at a scene change from step 202. For an I frame introduced in the middle of a scene, the I frame is encoded using a uniform quantizer which is the average quantizer from the preceding P frame, and no model updates are done. The subsequent P frame uses the model update from the last P frame and processing proceeds. It is as if the I frame did not exist for the purpose of choosing and constraining quantizers for the subsequent P frame.
In step 208, data from the previous frame is used to generate a value for the encoding complexity measure X of each macroblock in the previous frame using Equation (1) as follows:
X = (R-H) * Q / (S - CQ) (1) where R is the number of bits used to encode the macroblock, H is the number of header bits
(including motion vector coding bits) used to encode the macroblock, Q is the quantizer level used to encode the macroblock, S is a measure of the distortion of the macroblock, and C is a specifiable constant (e.g., preferably 2.5). The distortion measure S may be any suitable measure of the distortion of the macroblock. For inter-frame encoded data, one possible distortion measure is based on the sum of the absolute differences (SAD) of the inter-frame pixel difference data for the macroblock. For intra-frame encoded data, one possible distortion measure is based on the SAD between the individual pixel values and the mean pixel value for the corresponding block or macroblock. In either case, in Equation (1), the distortion measure S is the average SAD per pixel in the macroblock, translated to the DCT domain. In many cases, the DCT coefficients (before quantization) have a different dynamic range from the original pixel data due to a scaling factor (which is generally 8). Until S reaches a certain threshold, the block will quantize to zero (i.e., none of the coefficients will be significant). This threshold is modeled as 2.5Q, since the actual quantization step size is 2Q, with the rounding used by H.263. This means that S has to be greater than 2.5Q for it to yield at least one non-zero quantized coefficient. Note that this thresholding can be avoided by setting C to 0.
For each region, the encoding complexity values X for the different macroblocks in the region are averaged to generate an average encoding complexity Xave for each region in the previous frame. The previous frame refers to the previously encoded frame. For bi- directionally encoded frames, each complexity value X is derived as an average of the corresponding regions of the forward and backward reference frames.
The average encoding complexity Xave for each region is then used to determine a quantizer value to be used to encode the macroblocks of the corresponding region in the current frame (step 210), using Equation (2) as follows: (R-H) = max {Xave*(Save-CQ)/Q, 0} (2)
where (R-H) corresponds to the target number of bits for encoding each macroblock of the corresponding region of the current frame, Xave is the average encoding complexity for the corresponding region of the previous frame generated using Equation (1), Save is the average distortion measure for the corresponding region of the current frame, and Q is the quantizer value for the corresponding region of the previous frame. The number of header bits (H) is the sum of Hmb (the number of bits in the macroblock mode headers for the previous frame) and Hmv (the number of bits for the motion vectors for the current frame). Any of the parameters that are based on the previous frame (i.e., Xave, Q, and/or Hmb) may be derived from more than just the one previous frame. Equation (2) is applied to different quantizer values in an iterative manner until the frame target bit rate is satisfied to within a specified tolerance range. This can be done iteratively for each region while assuming the maximum difference between quantizer values for different regions allowable under the compression algorithm. Note that the first P frame does not have a history and the average encoding complexities for the previous frame (Xave) are not available. In this case, a "fall- back" mode is preferably used to adapt the quantizers over the frame to meet the target. At that point, the actual bits used for each macroblock within a region are used to initialize the region complexity.
After a quantizer value is selected for each region in the current frame using Equation (2), the quantizer values are adjusted if necessary for both spatial and temporal considerations (step 212). Spatially, the quantizer values generated using Equation (2) may need to be adjusted to ensure that the magnitude of change in quantizer value from one macroblock in one region to the next macroblock in a different region does not exceed the limit, if one exists, imposed by the video compression algorithm. Temporally, the magnitude of change in quantizer values from one region in one frame to the corresponding region in the next frame as generated using Equation (2) is preferably limited to ensure temporal smoothness in quality. For example, the temporal variation of the quantizer for each region may be constrained to 15% of Q when the quantizer value is reduced and to 60% of Q when the quantizer value is increased, where Q can vary from 1 to 31. As a particular example, if the quantizer value for an region in the previous frame was 16, then the quantizer value for the corresponding region in the current frame cannot be less than Q-15%Q = 16-(0.15)(16)
= 13.6 or greater than Q+60%Q = 16+(0.60)16 = 25.6.
After adjusting the quantizer values, segmentation analysis is applied to the current image to identify the three subsets of macroblocks corresponding to the foreground, background, and transition regions, respectively (step 214). After the current image has been segmented into three sets of macroblocks corresponding to the three regions, the current image is encoded, where the transform coefficients for all of the macroblocks of each region are quantized using the particular quantizer value selected for that region (step 216). The processing of steps 214 and 216 for the current image is analogous to the processing of steps 202 and 204 for the first image.
In the embodiment described in conjunction with Fig. 2, the temporal prediction model of Equations (1) and (2) is based on a first-order rate distortion model. In alternative embodiments, higher-order models, such as a closed-form solution for a quadratic rate distortion model, may be used.
The present invention can be embodied in the form of methods and apparatuses for practicing those methods. The present invention can also be embodied in the form of program code embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage medium, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention. The present invention can also be embodied in the form of program code, for example, whether stored in a storage medium, loaded into and/or executed by a machine, or transmitted over some transmission medium or carrier, such as over electrical wiring or cabling, through fiber optics, or via electromagnetic radiation, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention. When implemented on a general-purpose processor, the program code segments combine with the processor to provide a unique device that operates analogously to specific logic circuits.
It will be further understood that various changes in the details, materials, and arrangements of the parts which have been described and illustrated in order to explain the nature of this invention may be made by those skilled in the art without departing from the principle and scope of the invention as expressed in the following claims.
Claims
1. A method for encoding a current frame in a video sequence, comprising the steps of:
(a) segmenting the current frame into one or more different regions;
(b) generating an encoding complexity measure for each corresponding region of a previously encoded frame in the video sequence;
(c) using the encoding complexity measure for each region of the previous frame to select a quantization level for the corresponding region of the current frame; and
(d) encoding the current frame using the one or more selected quantization levels.
2. The invention of claim 1, wherein the encoding complexity measure is based on a first-order temporal prediction model.
3. The invention of claim 1, wherein the encoding complexity measure for each region in the previous frame is generated based on:
X = (R-H) * Q / (S - CQ) wherein X is an encoding complexity measure for a macroblock in the previous frame, R is a number of bits used to encode the macroblock, H is a number of header bits used to encode the macroblock, Q is a quantizer level used to encode the macroblock, S is a measure of distortion of the macroblock, and C is a constant.
4. The invention of claim 3, wherein the encoding complexity measure for each region in the previous frame is generated by averaging the encoding complexity measure X for all of the macroblocks in the region.
5. The invention of claim 1, wherein step (c) comprises the step of iteratively selecting one or more different quantization levels for each region until a frame target bit rate is satisfied to within a specified tolerance range according to:
(R-H) = max {Xave*(S-CQ)/Q, 0} wherein (R-H) corresponds to a target number of bits for encoding each macroblock of the corresponding region of the current frame, Xave is an average encoding complexity for the corresponding region of the previous frame, S is an average distortion measure for the corresponding region of the current frame, and Q is a quantizer value for the corresponding region of the current frame.
6. The invention of claim 1, wherein step (c) comprises the step of applying at least one constraint to modify, if appropriate, the selected quantization level for each region of the current frame.
7. The invention of claim 6, wherein a spatial constraint is applied to ensure that a magnitude of change in quantization level from one macroblock to a next macroblock following a raster scan pattern is not greater than a specified maximum spatial change in quantization level.
8. The invention of claim 6, wherein a temporal constraint is applied to limit a magnitude of change in quantization level from one region in the previous frame to the corresponding region in the current frame.
9. The invention of claim 8, wherein the temporal constraint when quantization level is increasing from the previous frame to the current frame is different from the temporal constraint when quantization level is decreasing from the previous frame to the current frame.
10. The invention of claim 9, wherein the temporal constraint allows greater percentage increases in quantization level than percentage decreases from the previous frame to the current frame.
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11835999P | 1999-02-03 | 1999-02-03 | |
US60/118,359 | 1999-02-03 | ||
US09/376,733 US6539124B2 (en) | 1999-02-03 | 1999-08-17 | Quantizer selection based on region complexities derived using a rate distortion model |
US09/376,733 | 1999-08-17 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2000046999A1 true WO2000046999A1 (en) | 2000-08-10 |
Family
ID=26816247
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2000/003092 WO2000046999A1 (en) | 1999-02-03 | 2000-02-03 | Quantizer selection based on region complexities derived using a rate distortion model |
Country Status (3)
Country | Link |
---|---|
US (1) | US6539124B2 (en) |
TW (1) | TW463505B (en) |
WO (1) | WO2000046999A1 (en) |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6539124B2 (en) | 1999-02-03 | 2003-03-25 | Sarnoff Corporation | Quantizer selection based on region complexities derived using a rate distortion model |
US7027506B2 (en) * | 2001-11-17 | 2006-04-11 | Lg Electronics Inc. | Object-based bit rate control method and system thereof |
KR100643454B1 (en) * | 2001-11-17 | 2006-11-10 | 엘지전자 주식회사 | Method for video data transmission control |
WO2007045555A1 (en) * | 2005-10-20 | 2007-04-26 | International Business Machines Corporation | Method and apparatus for digital image redundancy removal by selective quantization |
FR2914124A1 (en) * | 2007-03-21 | 2008-09-26 | Assistance Tech Et Etude De Ma | METHOD AND DEVICE FOR CONTROLLING THE RATE OF ENCODING VIDEO PICTURE SEQUENCES TO A TARGET RATE |
EP2041984A1 (en) * | 2006-07-17 | 2009-04-01 | THOMSON Licensing | Method and apparatus for adapting a default encoding of a digital video signal during a scene change period |
WO2010078758A1 (en) * | 2009-01-09 | 2010-07-15 | 深圳市融创天下科技发展有限公司 | Method for encoding video signal |
GB2513932A (en) * | 2013-05-03 | 2014-11-12 | Imagination Tech Ltd | Encoding an image |
EP2882200A1 (en) * | 2013-12-04 | 2015-06-10 | Samsung Electronics Co., Ltd | Display apparatus and method of displaying an image being divided into a plurality of regions |
EP3171596A3 (en) * | 2015-11-23 | 2017-08-02 | Xiaomi Inc. | Image compression with adaptive quantization of regions of interest (roi) |
WO2018136129A1 (en) * | 2017-01-20 | 2018-07-26 | Google Llc | Image compression based on semantic relevance |
Families Citing this family (46)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR100335055B1 (en) * | 1999-12-08 | 2002-05-02 | 구자홍 | Method of removal block effect and ringing effect of compressed video signal |
US6891566B2 (en) * | 2000-03-14 | 2005-05-10 | Joseph Robert Marchese | Digital video system using networked cameras |
US6873654B1 (en) * | 2000-05-16 | 2005-03-29 | Redrock Semiconductor, Inc | Method and system for predictive control for live streaming video/audio media |
JP4256574B2 (en) * | 2000-08-04 | 2009-04-22 | 富士通株式会社 | Image signal encoding method and image signal encoding apparatus |
KR20020064925A (en) * | 2000-10-10 | 2002-08-10 | 코닌클리케 필립스 일렉트로닉스 엔.브이. | Video bitrate control method and device for digital video recording |
US6948127B1 (en) * | 2001-12-10 | 2005-09-20 | Cisco Technology, Inc. | Interface for compressed video data analysis |
US7027982B2 (en) * | 2001-12-14 | 2006-04-11 | Microsoft Corporation | Quality and rate control strategy for digital audio |
GB2387058A (en) * | 2002-03-28 | 2003-10-01 | Sony Uk Ltd | Method of selecting a quantisation parameter using trial parameters |
CA2491522C (en) * | 2002-07-01 | 2014-12-09 | E G Technology Inc. | Efficient compression and transport of video over a network |
EP1602242A2 (en) * | 2003-03-03 | 2005-12-07 | Koninklijke Philips Electronics N.V. | Video encoding |
JP2004289254A (en) * | 2003-03-19 | 2004-10-14 | Matsushita Electric Ind Co Ltd | Videophone terminal |
US7327786B2 (en) * | 2003-06-02 | 2008-02-05 | Lsi Logic Corporation | Method for improving rate-distortion performance of a video compression system through parallel coefficient cancellation in the transform |
US7383180B2 (en) * | 2003-07-18 | 2008-06-03 | Microsoft Corporation | Constant bitrate media encoding techniques |
US7343291B2 (en) | 2003-07-18 | 2008-03-11 | Microsoft Corporation | Multi-pass variable bitrate media encoding |
KR100936034B1 (en) * | 2003-08-11 | 2010-01-11 | 삼성전자주식회사 | Deblocking method for block-coded digital images and display playback device thereof |
KR100987765B1 (en) * | 2003-09-30 | 2010-10-13 | 삼성전자주식회사 | Prediction method and apparatus in video encoder |
US7889792B2 (en) | 2003-12-24 | 2011-02-15 | Apple Inc. | Method and system for video encoding using a variable number of B frames |
US20050231596A1 (en) * | 2004-03-31 | 2005-10-20 | Marchese Joseph R | Testing apparatus for digital video camera anomalies |
US20050286629A1 (en) * | 2004-06-25 | 2005-12-29 | Adriana Dumitras | Coding of scene cuts in video sequences using non-reference frames |
US20060062481A1 (en) * | 2004-09-21 | 2006-03-23 | Markus Suvanto | Apparatuses, computer program product and method for bit rate control of digital image encoder |
US7924915B2 (en) * | 2005-01-28 | 2011-04-12 | Broadcom Corporation | Method and system for encoding video data |
US7974193B2 (en) | 2005-04-08 | 2011-07-05 | Qualcomm Incorporated | Methods and systems for resizing multimedia content based on quality and rate information |
US8345768B1 (en) * | 2005-07-28 | 2013-01-01 | Teradici Corporation | Progressive block encoding using region analysis |
US8582905B2 (en) | 2006-01-31 | 2013-11-12 | Qualcomm Incorporated | Methods and systems for rate control within an encoding device |
US9166883B2 (en) | 2006-04-05 | 2015-10-20 | Joseph Robert Marchese | Network device detection, identification, and management |
US8077775B2 (en) * | 2006-05-12 | 2011-12-13 | Freescale Semiconductor, Inc. | System and method of adaptive rate control for a video encoder |
JP2008193410A (en) * | 2007-02-05 | 2008-08-21 | Matsushita Electric Ind Co Ltd | Image encoding device, recording device, moving image encoding method, and moving image encoding program |
US8711926B2 (en) * | 2007-02-08 | 2014-04-29 | Qualcomm Incorporated | Distortion estimation for quantized data |
CN100461217C (en) * | 2007-03-29 | 2009-02-11 | 杭州电子科技大学 | Method for cutting complexity measure image grain |
US8724698B2 (en) * | 2007-04-13 | 2014-05-13 | Apple Inc. | Method and system for video rate control |
JP2011511554A (en) * | 2008-01-30 | 2011-04-07 | ヒューレット−パッカード デベロップメント カンパニー エル.ピー. | Method for streaming video data |
US8325800B2 (en) | 2008-05-07 | 2012-12-04 | Microsoft Corporation | Encoding streaming media as a high bit rate layer, a low bit rate layer, and one or more intermediate bit rate layers |
US8379851B2 (en) | 2008-05-12 | 2013-02-19 | Microsoft Corporation | Optimized client side rate control and indexed file layout for streaming media |
US7925774B2 (en) * | 2008-05-30 | 2011-04-12 | Microsoft Corporation | Media streaming using an index file |
US8325796B2 (en) * | 2008-09-11 | 2012-12-04 | Google Inc. | System and method for video coding using adaptive segmentation |
US8265140B2 (en) * | 2008-09-30 | 2012-09-11 | Microsoft Corporation | Fine-grained client-side control of scalable media delivery |
US8503527B2 (en) | 2008-10-03 | 2013-08-06 | Qualcomm Incorporated | Video coding with large macroblocks |
EP2302845B1 (en) | 2009-09-23 | 2012-06-20 | Google, Inc. | Method and device for determining a jitter buffer level |
US8630412B2 (en) | 2010-08-25 | 2014-01-14 | Motorola Mobility Llc | Transport of partially encrypted media |
US8477050B1 (en) | 2010-09-16 | 2013-07-02 | Google Inc. | Apparatus and method for encoding using signal fragments for redundant transmission of data |
US8751565B1 (en) | 2011-02-08 | 2014-06-10 | Google Inc. | Components for web-based configurable pipeline media processing |
US20120281756A1 (en) * | 2011-05-04 | 2012-11-08 | Roncero Izquierdo Francisco J | Complexity change detection for video transmission system |
US9262670B2 (en) * | 2012-02-10 | 2016-02-16 | Google Inc. | Adaptive region of interest |
US20150071343A1 (en) * | 2013-09-12 | 2015-03-12 | Magnum Semiconductor, Inc. | Methods and apparatuses including an encoding system with temporally adaptive quantization |
JP6537396B2 (en) * | 2015-08-03 | 2019-07-03 | キヤノン株式会社 | IMAGE PROCESSING APPARATUS, IMAGING APPARATUS, AND IMAGE PROCESSING METHOD |
CN114339196A (en) * | 2020-09-30 | 2022-04-12 | 华为技术有限公司 | Video coding method, device, equipment and medium |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5532940A (en) * | 1993-11-24 | 1996-07-02 | Intel Corporation | Process, apparatus and system for selecting quantization levels for encoding video signals |
US5552832A (en) * | 1994-10-26 | 1996-09-03 | Intel Corporation | Run-length encoding sequence for video signals |
WO1999003279A1 (en) * | 1997-07-10 | 1999-01-21 | Sarnoff Corporation | Apparatus and method for object based rate control in a coding system |
WO1999004359A1 (en) * | 1997-07-14 | 1999-01-28 | Sarnoff Corporation | Apparatus and method for macroblock based rate control in a coding system |
WO2000018131A1 (en) * | 1998-09-18 | 2000-03-30 | Sarnoff Corporation | Intra-frame quantizer selection for video compression |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR0150955B1 (en) * | 1992-05-27 | 1998-10-15 | 강진구 | Compressive and extensive method of an image for bit-fixing and the apparatus thereof |
US5528238A (en) | 1993-11-24 | 1996-06-18 | Intel Corporation | Process, apparatus and system for decoding variable-length encoded signals |
US5539662A (en) * | 1993-11-24 | 1996-07-23 | Intel Corporation | Process, apparatus and system for transforming signals using strength-reduced transforms |
US5933450A (en) * | 1994-04-20 | 1999-08-03 | Thomson Consumer Electronics, Inc. | Complexity determining apparatus |
US5933451A (en) | 1994-04-22 | 1999-08-03 | Thomson Consumer Electronics, Inc. | Complexity determining apparatus |
US5802213A (en) * | 1994-10-18 | 1998-09-01 | Intel Corporation | Encoding video signals using local quantization levels |
US5694170A (en) * | 1995-04-06 | 1997-12-02 | International Business Machines Corporation | Video compression using multiple computing agents |
US5852475A (en) * | 1995-06-06 | 1998-12-22 | Compression Labs, Inc. | Transform artifact reduction process |
US6094455A (en) * | 1996-09-25 | 2000-07-25 | Matsushita Electric Industrial Co., Ltd. | Image compression/encoding apparatus and system with quantization width control based on bit generation error |
US6539124B2 (en) | 1999-02-03 | 2003-03-25 | Sarnoff Corporation | Quantizer selection based on region complexities derived using a rate distortion model |
-
1999
- 1999-08-17 US US09/376,733 patent/US6539124B2/en not_active Expired - Lifetime
-
2000
- 2000-01-04 TW TW089100056A patent/TW463505B/en not_active IP Right Cessation
- 2000-02-03 WO PCT/US2000/003092 patent/WO2000046999A1/en active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5532940A (en) * | 1993-11-24 | 1996-07-02 | Intel Corporation | Process, apparatus and system for selecting quantization levels for encoding video signals |
US5552832A (en) * | 1994-10-26 | 1996-09-03 | Intel Corporation | Run-length encoding sequence for video signals |
WO1999003279A1 (en) * | 1997-07-10 | 1999-01-21 | Sarnoff Corporation | Apparatus and method for object based rate control in a coding system |
WO1999004359A1 (en) * | 1997-07-14 | 1999-01-28 | Sarnoff Corporation | Apparatus and method for macroblock based rate control in a coding system |
WO2000018131A1 (en) * | 1998-09-18 | 2000-03-30 | Sarnoff Corporation | Intra-frame quantizer selection for video compression |
Cited By (23)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6539124B2 (en) | 1999-02-03 | 2003-03-25 | Sarnoff Corporation | Quantizer selection based on region complexities derived using a rate distortion model |
US7027506B2 (en) * | 2001-11-17 | 2006-04-11 | Lg Electronics Inc. | Object-based bit rate control method and system thereof |
KR100643453B1 (en) * | 2001-11-17 | 2006-11-10 | 엘지전자 주식회사 | Bit rate control based on object |
KR100643454B1 (en) * | 2001-11-17 | 2006-11-10 | 엘지전자 주식회사 | Method for video data transmission control |
WO2007045555A1 (en) * | 2005-10-20 | 2007-04-26 | International Business Machines Corporation | Method and apparatus for digital image redundancy removal by selective quantization |
EP2041984A1 (en) * | 2006-07-17 | 2009-04-01 | THOMSON Licensing | Method and apparatus for adapting a default encoding of a digital video signal during a scene change period |
US8179961B2 (en) | 2006-07-17 | 2012-05-15 | Thomson Licensing | Method and apparatus for adapting a default encoding of a digital video signal during a scene change period |
EP2041984A4 (en) * | 2006-07-17 | 2009-08-05 | Thomson Licensing | Method and apparatus for adapting a default encoding of a digital video signal during a scene change period |
GB2459633A (en) * | 2007-03-21 | 2009-11-04 | Ateme Sa | Method and device for adjusting the encoding flowrate of video image sequences relative to a target flowrate |
WO2008132378A3 (en) * | 2007-03-21 | 2008-12-24 | Ateme Sa | Method and device for adjusting the encoding flowrate of video image sequences relative to a target flowrate |
WO2008132378A2 (en) * | 2007-03-21 | 2008-11-06 | Ateme Sa | Method and device for adjusting the encoding flowrate of video image sequences relative to a target flowrate |
FR2914124A1 (en) * | 2007-03-21 | 2008-09-26 | Assistance Tech Et Etude De Ma | METHOD AND DEVICE FOR CONTROLLING THE RATE OF ENCODING VIDEO PICTURE SEQUENCES TO A TARGET RATE |
US8279944B2 (en) | 2007-03-21 | 2012-10-02 | Ateme Sa | Method and device for regulating the encoding bit rate of video image sequences relative to a target bit rate |
WO2010078758A1 (en) * | 2009-01-09 | 2010-07-15 | 深圳市融创天下科技发展有限公司 | Method for encoding video signal |
GB2513932B (en) * | 2013-05-03 | 2015-03-25 | Imagination Tech Ltd | Encoding an image |
GB2513932A (en) * | 2013-05-03 | 2014-11-12 | Imagination Tech Ltd | Encoding an image |
US9525870B2 (en) | 2013-05-03 | 2016-12-20 | Imagination Technologies Limited | Encoding an image |
US9525869B2 (en) | 2013-05-03 | 2016-12-20 | Imagination Technologies Limited | Encoding an image |
EP2882200A1 (en) * | 2013-12-04 | 2015-06-10 | Samsung Electronics Co., Ltd | Display apparatus and method of displaying an image being divided into a plurality of regions |
EP3171596A3 (en) * | 2015-11-23 | 2017-08-02 | Xiaomi Inc. | Image compression with adaptive quantization of regions of interest (roi) |
KR101848191B1 (en) * | 2015-11-23 | 2018-04-11 | 시아오미 아이엔씨. | Image compression method and apparatus, and server |
WO2018136129A1 (en) * | 2017-01-20 | 2018-07-26 | Google Llc | Image compression based on semantic relevance |
US10405003B2 (en) | 2017-01-20 | 2019-09-03 | Google Llc | Image compression based on semantic relevance |
Also Published As
Publication number | Publication date |
---|---|
TW463505B (en) | 2001-11-11 |
US20020034245A1 (en) | 2002-03-21 |
US6539124B2 (en) | 2003-03-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6539124B2 (en) | Quantizer selection based on region complexities derived using a rate distortion model | |
US6256423B1 (en) | Intra-frame quantizer selection for video compression | |
Ribas-Corbera et al. | Rate control in DCT video coding for low-delay communications | |
Fung et al. | New architecture for dynamic frame-skipping transcoder | |
US6590936B1 (en) | Coded data transform method, transcoding method, transcoding system, and data storage media | |
US5969764A (en) | Adaptive video coding method | |
US6343098B1 (en) | Efficient rate control for multi-resolution video encoding | |
US5790196A (en) | Adaptive video coding method | |
US6765962B1 (en) | Adaptive selection of quantization scales for video encoding | |
US9258567B2 (en) | Method and system for using motion prediction to equalize video quality across intra-coded frames | |
Youn et al. | Video transcoder architectures for bit rate scaling of H. 263 bit streams | |
US20060227868A1 (en) | System and method of reduced-temporal-resolution update for video coding and quality control | |
US7095784B2 (en) | Method and apparatus for moving picture compression rate control using bit allocation with initial quantization step size estimation at picture level | |
US5974185A (en) | Methods and apparatus for encoding video data using motion vectors for decoding by regular or downconverting decoders | |
US7373004B2 (en) | Apparatus for constant quality rate control in video compression and target bit allocator thereof | |
US5880784A (en) | Method and apparatus for adaptively switching on and off advanced prediction mode in an H.263 video coder | |
Youn et al. | Video transcoding with H. 263 bit-streams | |
JP2000261800A (en) | Variable bit rate moving picture encoding method and device | |
US7254176B2 (en) | Apparatus for variable bit rate control in video compression and target bit allocator thereof | |
US7133448B2 (en) | Method and apparatus for rate control in moving picture video compression | |
Nakagawa et al. | Dynamic resolution conversion method for low bit rate video transmission | |
Chen et al. | Encoder Control Enhancement in HEVC Based on R-Lambda Coefficient Distribution | |
JPH0775095A (en) | Rate control circuit | |
KR100595144B1 (en) | An adaptive quantization algorithm for video coding | |
Cho et al. | An adaptive quantization algorithm for video coding |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): BR CA CN JP KR |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
DFPE | Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101) | ||
122 | Ep: pct application non-entry in european phase |