US 20060007460 A1
A method for re-mapping out of gamut colors into a display device color space. The method includes determining a maximum color gamut of the display device in a perceptual color space that represents the combination of hue, lightness and saturation; and determining if a target color is out of gamut for the display device. Accordingly, an out of gamut target color is provided. One may then obtain a hue angle for the out of gamut target color. Using the hue angle, one determines the lightness value of the out of gamut target color. Subsequently, one would hold the one holding the hue angle constant, altering the lightness and saturation, as necessary to bring the out of gamut target color into the display device color space.
1. A method for re-mapping out of gamut colors into a display device color space, said method comprising the steps of:
determining a maximum color gamut of the display device in a perceptual color space that represents the combination of hue, lightness and saturation;
determining if a target color is out of gamut for the display device, thereby providing an out of gamut target color;
obtaining a hue angle for the out of gamut target color;
using the hue angle, determining the lightness value of the out of gamut target color; and
holding the hue angle constant, altering the lightness and saturation, as necessary to bring the out of gamut target color into the display device color space.
2. The method as claimed in
3. The method as claimed in
This is a Divisional application of U.S. Ser. No. 09/797,891; filed Mar. 2, 2001; of Mitchell J. Bogdanowicz, Carlo V. Hume and Charles P. Hagmaier, entitled “Method Of Digital Processing For Digital Cinema Projection Of Tone Scale And Color”.
The present invention relates generally to the field of video processing or digital image processing to emulate motion picture film tonescale and color. More specifically, the invention relates to rendering an electronic image to emulate a film system reproduction, that is, to provide the appearance of an image captured and displayed with motion picture film.
The digital era that is now upon us has the potential to change the way everyone operates in the motion picture field. The traditional method for projecting motion picture film was relatively simple because there were not very many options available to be controlled. Basically, film has a certain appearance or “look” that can be altered by processing, lighting and laboratory methods. The digital age has brought to the motion picture industry the ability to capture a motion picture print image and structure it in the form of data. Once the data has been captured, it can be manipulated practically to any degree that is desired. However, the more variables one has to work with, the harder it is to maintain or reconstruct the original intent.
In the prior art, digital image processing has taken place on variables associated with video images, as described in U.S. Pat. Nos. 5,335,013, 5,475,425 and 5,831,673, in order to emulate the broadcast look of film or the look of film after it has been through a telecine transfer. These patents describe systems for rendering the output of a video camera to simulate the visual appearance of motion picture film that has been transferred or converted to a video signal to be output directly for television broadcasting or recording on video tape. Further, the above-cited prior art teaches the alteration of the apparent contrast of the video image so the desired broadcast film appearance may be obtained. More specifically, in the '013 patent a gray scale modifier is used as a look-up table (LUT) and the operator can choose between a variety of curves (% light level vs. video level) stored in programmable read-only memory (PROM) to reflect different film types or achieve different photographic effects. The desired curve is selected by pressing a switch on the hardware.
U.S. Pat. Nos. 5,140,414; 5,374,954; 5,406,326; 5,457,491 and 5,687,011 (each issued to Mowry) represent a family of related post-production video technology that seeks to arrive at an aesthetically acceptable simulation of the appearance that images originated on different motion picture film stocks would embody after telecine “flying spot scanner” transfer to video from taped high definition video originated images. One component of this prior art technology deals with the conversion of the video-originated material through a LUT that is based on color temperature of the scene lighting, scene brightness and selected f-stop setting. The conversion values in the LUT are derived by filming color charts and grey scale charts, obtaining a digital representation of the film component responses of the charts from telecine transfer of the film to videotape, and then charting the telecine-derived component responses against video originated images of the same charts under identical lighting conditions. Another component of this prior art technology allows for physically instilling selected film grain patterns to the video images. The final simulated video image is either recorded as a high definition signal, or converted to an NTSC signal and broadcast or displayed.
In the latter two of the aforementioned Mowry patents, the digitized video signal may be sent to a film recorder, which reproduces the component-modified images onto a selected, reversal film stock. The film is chemically processed with a film processor and then optically projected, or scanned to video, digital video, or other electronic media. If the film recording option is employed, these patents specify that it is important that the telecine-derived LUT used in the component modification involves response data which compensates for the inherent color response of the film stock on which the images are being digitally recorded.
Prior art also exists for generating a broadcast film appearance with video or digitally captured images, where the processing occurs inside of the camera. Specifically, setup cards have been described for beta-camcorders and also digital camcorders (see L. J. Thorpe et al, “The HDTV Camcorder and the March to Marketplace Realty”, SMPTE Journal, March 1998, pp. 164-177). These miniature plug-in setup cards facilitate prealignment of the camera to achieve an HD image having attributes similar to those that might otherwise be created from a film origination followed by telecine transfer to HD video. The setup cards can be pre-programmed to store desired digital data settings for aesthetic choices to be made concerning, among other characteristics, color reproduction, tonal reproduction and skin-tone detail.
In a combined approach, U.S. Pat. No. 5,319,465 describes a method using modified camera production and modified post-production processes and equipment to create film-like images. Specifically, the method includes the steps of shooting a benchmark comprising a gray scale chart, a color test chart and two backfocus charts with both a film and a video camera with comparable scene lighting and depth of field. Once the film test benchmark is shot, the film is transferred to videotape utilizing a telecine apparatus, with settings indicative of an industry standard set-up film. The video camera image's hue, saturation, luminance and gamma levels are manipulated to color correct the video camera image to look visually like the transferred film image. After the videotape has been edited, the videotape undergoes color correction in which the videotape benchmark is corrected to match the film test benchmark.
The aforementioned prior art is generally trying to emulate the look of film after it has been telecine-transferred to video. This is desirable to some extent because the telecine system does have some film attributes when broadcasted. However, the prior art neglects the emulation of the look of film origination, as if a negative film has been directly printed and projected through a motion picture system. This is particularly desirable where the digital output is recorded on film for projection. Where the prior art does deal with film recording, as in the latter two of the above-mentioned Mowry patents, it does so in the context of a telecine-transferred benchmark. Moreover, when the prior art alters the tone scale and color of a video or digitally captured image to emulate a film, it is done on the telecine-transferred benchmark. This is an imperfect alteration because it cannot operate upon the scene exposure as seen by a film.
Copending, commonly-assigned U.S. Ser. No. 09/712,500, entitled “System and Method for Processing Electronically Captured Images to Emulate Film Tonescale and Color”, filed Nov. 14, 2000 in the names of K. M. Cirulli, J. C. Brewer and N. M. Rodriguez, provides a system and method for rendering electronically captured images to emulate the appearance in tonescale and color of images captured with motion picture film, and subsequently printed onto another motion picture film and displayed theatrically (negative-positive system). The tonescale and color of an electronically captured image is altered only after it has been shifted to emulate scene exposure as seen by film. Then the shifted electronic exposures are mapped through a predetermined LUT and matrix that imparts a film tonescale and color to the electronically captured image. This method is advantageous because it allows for automation of the image processing with predetermined algorithms. In addition, it does not require color-timing decisions to be made by an operator on a per scene basis, if the video or digital camera set-ups remain constant.
Other prior art also exists for manipulation of digital images to emulate a particular “look”. U.S. Pat. Nos. 5,809,164 and 5,239,370 both describe a color management system for characterizing and re-mapping colors. Both describe the ability to re-map the colors in a linear and non-perceptual process. U.S. Pat. No. 4,839,721 teaches that transformation between a capture medium the selected color is essentially a linear response.
The prior art generally cited above describes the conversion of an electronically captured image from an electronic camera to data that can be written to a film recorder and subsequently printed. Ideally, as represented in the Cirulli et al application Ser. No. 09/712,500, the image chain allows for a match between the print generated by this process and a print from a film captured scene (“print” as used herein refers to the final transparency used for projection).
What is needed is a system that converts a scanned film image into data that can be displayed on an electronic display device, such as a digital projector, and still look like a projected film image. The projected digital image would correctly replicate the look of film origination, particularly as to film tonescale and color reproduction, as if a negative film has been directly printed and projected through a motion picture system.
The present invention is directed to overcoming one or more of the problems set forth above. Briefly summarized, according to one aspect of the present invention, a method for re-mapping out of gamut colors into a display device color space. The method includes determining a maximum color gamut of the display device in a perceptual color space that represents the combination of hue, lightness and saturation; and determining if a target color is out of gamut for the display device. Accordingly, an out of gamut target color is provided. One may then obtain a hue angle for the out of gamut target color. Using the hue angle, one determines the lightness value of the out of gamut target color. Subsequently, one would hold the one holding the hue angle constant, altering the lightness and saturation, as necessary to bring the out of gamut target color into the display device color space.
This invention is a straightforward, reliable, and practical process for converting digital image sequences so that their appearance matches that of projected motion picture print film. In addition, the technique can be used to assist capturing of image sequences.
This invention provides a standard process for digital cinema presentation. The process can be employed not only at projection, but also at capture. Current industry methods are time consuming, inconsistent, and unpredictable. Colorists spend countless hours each year to adjust image sequences in order to match a desired output response. This invention provides a fast and reliable process for matching a digital source to a motion picture print film release. It therefore greatly minimizes the work involved by a colorist to adjust the flesh tone and colorimetry of image sequences, therefore saving time and enabling the use of digital cinema projection.
This invention provides the control and flexibility to be put in use in digital cinema projection. This invention transforms the digital image sequence by simulating a “film model” for characterizing how film dyes appear when viewed with light. The film dye amounts are then converted and mapped to the target digital projector. Any out of gamut colors are remapped in perceptual color space to minimize the color appearance to the viewer.
These and other aspects, objects, features and advantages of the present invention will be more clearly understood and appreciated from a review of the following detailed description of the preferred embodiments and appended claims, and by reference to the accompanying drawings.
Because video or digital image processing systems that emulate the look of motion picture film, particularly after telecine conversion, are well known, the present description will be directed in particular to attributes forming part of, or cooperating more directly with, a system and a method in accordance with the present invention. Attributes not specifically shown or described herein may be selected from those known in the art. In the following description, a preferred embodiment of the present invention would ordinarily be implemented at least in part as hardware, although those skilled in the art will readily recognize that the equivalent of such hardware may also be designed in software. Given the system as described according to the invention in the following materials, hardware and software not specifically shown, suggested or described herein that is useful for implementation of the invention is conventional and within the ordinary skill in such arts.
If the invention is implemented as a computer program, the program may be stored in conventional computer readable storage medium, which may comprise, for example; magnetic storage media such as a magnetic disk (such as a floppy disk or a hard drive) or magnetic tape; optical storage media such as an optical disc, optical tape, or machine readable bar code; solid state electronic storage devices such as random access memory (RAM), or read only memory (ROM); or any other physical device or medium employed to store a computer program. If the invention is implemented in hardware, it may be implemented as an integrated circuit, such as an application specific integrated circuit (ASIC), or by other means known in the art.
This invention is designed to provide a straightforward and robust method for transforming and projecting digital data to replicate what is seen by conventional projected film release prints. Accordingly, the invention is designed to meet the following objectives:
The core of this invention is a mathematical process that transforms input digital image data into another digital metric that is suitable to be projected on a digital projector or a CRT display device. The objective is to obtain a “target look” in the viewing environment, such as the “look” of a motion picture print film. Beginning with
As mentioned above, the desired “look” is preferably that of a motion picture print film projection system. There are different ways of trying to get this “look”, or something approximating this “look”. The original data from the telecine scanner 15 may be displayed on a CRT 45 calibrated to simple 1D look up tables (LUT) 50 that impart the “film tone scale”; however, this technique is an approximation at best since it can get the neutral scale close but it fails on the color scale. Alternatively, the original data may be written by a film recorder 55 to a film element 60 as a negative that is subsequently printed onto motion picture print film 65, which inherently provides the correct “film look”. In either case, the output system is designed to include, or approximate, the characteristics of the motion picture print film. The original data, consequently, is devoid of the characteristics of the motion picture print film projection system. Another way to obtain this “look”, which is the subject of this invention, is to specify photographic parameters in the computer model 20 that achieve a representation of a motion picture print film projection system. Any changes in the “look” that is different from the “standard film look” will then appear in the data as a perturbation that achieves the desired result on the “standard film output”.
The target “look” is achieved by mathematical modeling of the input to output data, which is embodied in the computer model 20. The calculations rely upon some physical measurements such as a target dye set spectrophotometry (which is illustrated by
In an important aspect of the invention, the tone scale and tone scale manipulations are operated on in a channel independent space, preferably a space referred to herein as an Analytical Dye Amount space. This space assures the orthogonality of the operations. Analytical dye amounts are linear proportions of the individual cyan, magenta and yellow dye spectrophotometry. The target “look” for a film based emulation is constructed from the appearance of the image made from cyan, magenta and yellow dyes.
The choice of Analytical Dye Amount space allows the balancing (shifting) of the tone scale curves without imparting errors due to cross talk between the channels. Consequently, this invention uses analytical densities to transform a digital image sequence to the target film model. Since imparting a film look is necessarily a non-linear operation, the inventive transformation is not linear nor does it need to be a linear combination between the capture medium and the selected output color space. Furthermore, out of gamut colors (input defined colors that can not be accurately displayed on the output display device) are remapped in such a way to minimize the perception of color change. The output display device is a red, green and blue primary device. Consequently, this invention also treats the difficulties in the transformation between a subtractive (CMY) and an additive (RGB) system.
The image processing sequence starts with digitized data that corresponds to logarithmic printing densities. These printing densities are the exposure densities to a motion picture print material. The modeling incorporates multiple characteristics of the print, printing and viewing process. The states of these characteristics determine the desired “look” obtained. Many choices are presented that affect the result, such as balance position, print exposure flashing, gamma alteration, achromatic vs. non-achromatic tone scales, incomplete or skipped bleaching and display device dynamic range. The internal representation of the colorimetric state of a RGB input triad is matched to the display device primaries and tone scale characteristics. If a color is not achievable (out of gamut), it is gracefully molded into the domain of achievable colors without imparting artificial artifacts in the appearance in the image.
The first step is the construction of the final density log exposure (DLE) curves, as shown in
The DLE series contains a step patch that is used as a control patch for adjusting color during film printing. The color timing of the print is adjusted to match a particular target. This target is normally referred to as LAD (the laboratory aim density patch). The Status-A RGB density position is 1.09, 1.06, and 1.03. These values are the Kodak Vision Print Film #2383 dye set Status-A RGB densities that are required to achieve a visual neutral density of 1.0 with a standard motion picture projector light source. The digital DLE series is also adjusted to LAD. The Status-A LAD values are converted to analytical dye amounts before the adjustment is made. This provides a common starting point for calculating a model for any film print or negative.
If necessary the DLE series is then adjusted to account for viewing flare, print material exposure flashing and gamma changes in the system. For example, as also shown in
A block diagram of the image processing chain for the conversion of image sequences provided by the scanner 15 is shown in
The image data is then mapped though the constructed DLE series (constructed according to
At this point, if the digital projector RGB values are outside the data range then the desired color could not be directly mapped. A simple re-mapping technique would be to clip the RGB values to the minimum and/or maximum data range of the projector. The model according to this invention employs a more sophisticated re-mapping stage 150 for color re-mapping. The color re-mapping technique is described in detail in a subsequent section.
After the normalized linear digital projector RGB values are properly defined and obtained in a linear conversion stage 160, the RGB values are scaled in a rescale stage 170 to the actual bit depth of the projector. At this point the minimum and maximum data ranges are specified. The values do not have to be defined at minimum or maximum of the chosen bit depth. This allows for headroom in the system. The final gamma application stage 180 applies an inverse gamma function to the RGB data for the digital projector, as follows:
The color re-mapping method employed in the re-mapping stage 150 employs the following methodology when the color set is out of gamut. A RGB color set is out of color gamut if any of the following assumptions are true:
First, the maximum color gamut of the system is determined in perceptual color space. The chosen perceptual color space is CIE LAB, which is well understood by those of skill in these arts, but any other perceptual color space, such as XYZ tristimulus or CIE LUV space, would work as well. These color spaces are device-independent color spaces that are designed to directly describe the color that is perceived by a human observer. They are generally based on the system developed by the Commission International de l'Eclairage (CIE). In CIE LAB space, the maximum color gamut is determined by setting the maximum saturation (in terms of the CIE LAB a* and b* coordinates) at each of the hue angles (as shown in
Each color has a profile depending on the hue (as shown in
The gamut remapping algorithm attempts to conserve the hue angle of the target color. Perceptually, changes in hue angle are more apparent than changes in lightness or saturation. Since the hue angle is maintained, the lightness and saturation must be altered to bring the target color into the display device color space. The hue angle profiles (
While the foregoing gamut mapping technique is preferred, other gamut mapping techniques may be used in this imaging chain to assign valid RGB code values to otherwise undisplayable colors.
As described earlier, the image processing chain shown in
The modeling values were calculated at various reference points in the system. These reference points are referred to as nodes in such tetrahedral 3D interpolation calculations. A node is referred to as a set of RGB values. For example, if a system has a 17 node set, the combination would 173 or 4913 values. Values that don't fall on a node are then interpolated. The nodes were also generated to be non-equal distance throughout the system. The incremental change in code values from one node to the next was smaller in the middle region of the code value range. This allowed finer accuracy in the region where changes in the luminance are visually more sensitive. (see
The invention has been described with reference to a preferred embodiment. However, it will be appreciated that variations and modifications can be effected by a person of ordinary skill in the art without departing from the scope of the invention.
10 photographic element
20 computer model
25 digital processor
30 CRT monitor
35 digital projector
45 CRT monitor
50 1D LUTs
55 film recorder
60 film element
65 motion picture print film
80 RGB film density curves
82 density conversion stage
84 printing flare stage
86 gamma adjustment stage
88 contrast ratio adjustment stage
90 neutrality adjustment stage
92 print type transfer curves
100 print density data input
110 conversion matrix
120 DLE mapping stage
130 dye conversion stage
140 space conversion matrix
150 re-mapping method
160 linear conversion stage
170 rescale stage
180 gamma application stage
190 display data