« PreviousContinue »
8% mi 6.3: 2.; B24 ...§E56w 2.; E2 mom B 2._2§.E>
RC-B values 255 -------------------------------------------------------------- - [F 600 ‘T F61 1161 PIXOI6 16-I 666114 (<01. H6666 16.6.1 M.--' "O 2_,__“..6---~~>g?‘TT'§_Og must be move cl lefl: along their respective lines J" ,.,~<""‘—' _ I rgbfl *“"_ till red reaches O. \“~,\ IJ, g, r -—- input values "‘\.\ I3’, g’, r’ — output values “\.\ desat — required liI.6SEiI.LlI‘HI2l0I1 factor ..\_ __\,,,\ 0 Q des6iL%\IR _. I1 Sflfi'El:Ct0l"
1 INTELLIGENT SATURATION OF VIDEO DATA
CROSS REFERENCE TO RELATED APPLICATION
This application is a continuation of U.S. patent application Ser. No. 11/295,882, filed Dec. 7, 2005, which is incorporated herein by reference in its entirety.
The present invention relates generally to image and video processing, and more particularly to saturation of color in image signals.
BACKGROUND OF THE INVENTION
For video applications, the YCbCr format is typically used because it is the industry standard for video transmission and storage. The YCbCr format is also typically easier to convert to and from RGB than from HSV. The ease in conversion is important because the final display is usually configured to accept data in RGB format. Increasing the saturation of a video signal results in a more vivid, and hence a more colorful picture. However, increasing the saturation of aYCbCr pixel is a processing step that is very prone to generating blatantly invalid pixels which result in invalid RGB values (and potentially visually armoying artifacts).
An appreciation of the present invention and its improvements can be obtained by reference to the accompanying drawings, which are briefly summarized below, to the following detailed description of illustrated embodiments of the invention, and to the appended claims.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a block diagram of an example intelligent saturation controller for increasing saturation.
FIG. 2 is a block diagram of an example intelligent controller for decreasing saturation.
FIG. 3 is an example illustration of a valid space of YUV vectors in an 8-bit system.
FIG. 4 is an example illustration of the effect of video saturation on an RGB color vector.
FIG. 5 is another example illustration of the effect of video saturation on an RGB color vector.
FIG. 6 shows an example of de-saturating an invalid pixel. The invalid pixel has one or more of its RGB values outside the 8-bit valid range.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
In the following detailed description of exemplary embodiments of the invention, reference is made to the accompanied drawings, which form a part hereof, and which is shown by way of illustration, specific exemplary embodiments of which the invention may be practiced. These embodiments are described in suflicient detail to enable those skilled in the art to practice the invention, and it is to be understood that other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the present invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is defined only by the appended claims.
Throughout the specification and claims, the following tenns take the meanings explicitly associated herein, unless the context clearly dictates otherwise. The meaning of “a,” “an,” and “the” includes plural reference, the meaning of “in” includes “in” and “on.” The tenn “comiected” means a direct electrical connection between the items connected, without any intermediate devices. The tenn “coupled” means either a direct electrical connection between the items connected, or an indirect connection through one or more passive or active intermediary devices. The tenn “circuit” means either a single component or a multiplicity of components, either active and/ or passive, that are coupled together to provide a desired function. The tenn “signal” means at least one current, voltage, or data signal. Referring to the drawings, like numbers indicate like parts throughout the views.
The present invention is directed to an intelligent saturation controller that calculates the exact maximum saturation any validYCbCr (or more generally, any RGB) pixel can undergo before it becomes invalid in RGB space. The controller models the saturation operation in RGB color space and calculates the maximum saturation level at which the RGB values falls outside the valid range. The saturation operation is performed independently for every pixel of the incoming video frame and ensures that each output pixel is a valid. The intelligent saturation controller approach can be used at various stages of the video processing pipeline such as increase of video saturation for picture enhancement and correction of invalid pixels that arise due to other processing.
Various embodiments as discussed below can be incorporated into a video processing system such as described in U.S. patent application Ser. No. 11/296,185 filed Dec. 7, 2005, entitled “Color Management Unit,” the disclosure of which is expressly incorporated herein by reference.
The intelligent saturation controller approach can be used in two directions to check RGB values to make sure they are remaining valid for any given saturation factor. First, intelligent saturation can be used to increase saturation to make images look more vivid, without causing side-effects of clipping or hue shifts. Secondly, intelligent saturation can be used as a gamut-remapping algoritlnn that decreases saturation to make the image fit within the given gamut.
The controller finds the maximum saturation for each input pixel and checks whether it is less than a predetermined saturation factor (e.g., an amount of saturation adjustment that is desired). If so, then this calculated maximum saturation value is applied. If not, the predetermined saturation factor is applied. Accordingly, the output RGB pixels are valid and no clamping is necessary if no other video processing is done in YCbCr space. Increasing the saturation of the video signal results in a more vivid and more colorful picture.
FIG. 1 is a block diagram of an example intelligent saturation controller for increasing saturation. The intelligent saturation controller (100) comprises a saturation evaluation unit (110) and a saturation processor (120). The example intelligent controller (100) calculates a maximum saturation for any YCbCr pixel that remains valid in RGB space. The calculation is accomplished by modeling the saturation operation in RGB color space and calculating the maximum saturation at which one of the RGB values falls outside the valid range.
The calculation (described below with reference to FIG. 5, for example) is typically performed independently for every pixel of the incoming video frame to ensure that each output pixel is valid in RGB space. The intelligent saturation controller finds the maximum saturation for each input pixel and verifies whether the input pixel is less than the predetermined saturation factor. If so, then this calculated maximum satura
tion value is applied. If not, the predetermined saturation factor is applied. In the proposed system, the output RGB pixels are guaranteed to be valid and no clamping is necessary if no other video processing is done inYCbCr space (this can be ensured by placing the color management unit at the backend of the video processing pipeline.) Converter 130 converts YCbCr pixels to RGB pixels.
FIG. 2 is a block diagram of an example intelligent controller for decreasing saturation. The intelligent saturation controller (200) comprises aYCbCr-to-RGB converter (210) and a desaturation processor (220). In operation, the intelligent saturation controller can “correct” invalid pixels by desaturating the invalid pixels. To correct (rather than to merely clamp) invalid RGB pixels, the intelligent saturation controller calculates (described below with reference to FIG. 6, for example) the amount of desaturation needed to place invalid pixels within a valid range. In an embodiment, the desaturation is directly applied to RGB pixels instead of saturating pixels within YCbCr space. Valid pixels are typically umnodified. To represent the invalid pixels, the output data of the color space converter uses a higher resolution than the input. After the desaturation procedure, the original resolution is maintained.
Enhancement of video data is perfonned in all modern displays and video processing equipment. Video is usually transmitted and processed in YCbCr color space due to a number of reasons such as lower bandwidth and separation of brightness and color information. However, most display panels use RGB as their input data fonnat. Because RGB input is commonly used as the input fonnat, the YCbCr data is converted to RGB-fonnatted data at some stage of the video processing pipeline.
The conversion to RGB is done using a set of linear equations that calculate R, G and B given Y, Cb and Cr. The problem that arises during this conversion is that any valid YCbCr triplet (say 24 or 30 bits) does not always map to a valid RGB triplet (assuming the same number of bits). Processing and enhancement of YCbCr data may transform a valid YCbCr pixel into an invalid one (a valid YCbCr pixel is one that maps to a valid RGB pixel). Conventional conversion mechanisms typically clamp the invalid RGB values to the extremes of their valid range, which often leads to visual anomalies in the output signal. A disadvantage of clamping is that brightness and hue of the pixel are not preserved, and a loss of details and patchiness are present in the output picture.
The YCbCr color space is typically used for contrast enhancement because the luminance and color infonnation are separate and can be processed independently. The YCbCr color space is also one of the common video exchange formats that is currently used. “Y” is the luminance (usually referred to as luma) of the video while “Cb” and “Cr” are the chroma or color components. Another advantage of transmitting signals in YCbCr fonnat is that the bandwidth of the signal can also be reduced by down-sampling the cl1rominance signals. Down-sampling is visually acceptable because the human vision system is typically more sensitive to brightness than to color. Y, Cb and Cr are usually assigned values ranging from 0 to 255 in an 8-bit system.
When an offset of 128 is removed from Cb and Cr, the YCbCr space is referred to as theYUV space. InYUV space, U and V range from -128 to 127. The value U is the offset removed representation of Cb and V is the offset removed representation of Cr. Although transmission of video data is often more convenient in YCbCr form (due to the unsigned nature of data), most color enhancement operations are usually defined in tenns of the U and V components. For example, the location of the UV vector determines the hue of
When an RGB pixel is converted to aYUV pixel, the valid signal space is compressed. Namely, while every valid n-bit RGB vector maps to a valid n-bitYUV vector, the converse is not true. This can be seen by inspection of theYUV to RGB transfonnation equations (2). WhenY, U andV are all at their maximum values for example (Y is 255 and U andV are 127), the resulting R and B values exceed 255.
The compression of the valid pixel range is significant because if data for an input YUV pixel is valid, video enhancement of the pixel may change the pixel data to the point where the data may no longer be valid. The possibility of the pixel being transformed to an invalid space can substantially affect the quality of video processing algoritlnns that are usually perfonned onYUV data. Examples of such processing include deinterlacing, noise removal, sharpening, and color and contrast enhancements. Increasing the saturation ofYUV data is particularly prone to generation of invalid data.
FIG. 3 is an example illustration of a valid space of YUV vectors in an 8-bit system. The figure illustrates a valid space of YUV vectors is a cuboid shaped region (310) inside the cube of 8-bit YUV values (300). The figure shows that the identification of validYUV data is relatively computationally intense. Prediction of when valid YUV vectors become invalid can be even more complex.
Enhancing the color of an incoming video data stream is one of the most commonly used enhancement procedures in consumer electronics. This can be done by increasing the saturation of the video data using the following equation:
U0,“ sat 0 U,-,1 (3) (V...) lo ml(v..]
Increasing the saturation is relatively simple—the input U and V values are multiplied by some saturation factor sat. With respect to the U-V axis, changing the saturation can be visualized as moving the input [u v]’ vector along a line through the origin and the original point. If sat is Zero, the