Search Images Maps Play YouTube News Gmail Drive More »
Sign in
Screen reader users: click this link for accessible mode. Accessible mode has the same essential features but works better with your reader.

Patents

  1. Advanced Patent Search
Publication numberUS20090022417 A1
Publication typeApplication
Application numberUS 12/241,029
Publication dateJan 22, 2009
Filing dateSep 29, 2008
Priority dateJan 6, 2003
Also published asUS7447374, US7916963
Publication number12241029, 241029, US 2009/0022417 A1, US 2009/022417 A1, US 20090022417 A1, US 20090022417A1, US 2009022417 A1, US 2009022417A1, US-A1-20090022417, US-A1-2009022417, US2009/0022417A1, US2009/022417A1, US20090022417 A1, US20090022417A1, US2009022417 A1, US2009022417A1
InventorsRussell Reid
Original AssigneeRussell Reid
Export CitationBiBTeX, EndNote, RefMan
External Links: USPTO, USPTO Assignment, Espacenet
Method and Apparatus for An Intuitive Digital Image Processing System That Enhances Digital Images
US 20090022417 A1
Abstract
To better realize the great potential of amateur digital photography, the present invention introduces an intuitive system for digital processing. The system of the present invention allows a user to enhance digital images that appear dark or dull. In one embodiment, the user activates an image enhancement button on a graphical user interface using cursor control device such as a mouse button. In one embodiment, the digital image enhancement system is implemented using a nonlinear brightness level transformation correction and a pseudo gamma correction. The parameters for both the nonlinear brightness level transformation correction and a pseudo gamma correction are determined from examining the properties of the image to be enhanced.
Images(10)
Previous page
Next page
Claims(24)
1-20. (canceled)
21. A method of enhancing a digital image defined by reference to a plurality of color components, said method comprising:
analyzing the content of said digital image to determine black and white cutoff values, wherein analyzing the content of said digital image comprises analyzing color values of said digital image along at least two color components; and
performing a non linear brightness level correction based upon said black and white cutoff values.
22. The method of claim 21, wherein analyzing the content of said image comprises generating a histogram.
23. The method of claim 21, wherein performing the non linear brightness level correction comprises:
changing all brightness levels darker than said black cutoff value to black,
changing all brightness levels brighter than said white cutoff value to white, and
linearly spreading out remaining brightness levels across the full range of brightness levels.
24. The method of claim 21, wherein performing the non linear brightness level correction comprises generating a first lookup table for a first color component, a second lookup table for a second color component, and a third lookup table for a third color component.
25. The method of claim 21 further comprising:
analyzing the content of said digital image to determine a skin brightness and an average brightness; and
performing a pseudo gamma correction based upon said skin brightness and said average brightness.
26. A method of enhancing a digital image, said method comprising:
analyzing the content of said digital image to determine a skin brightness and an average brightness; and
performing a pseudo gamma correction based upon said skin brightness and an average brightness.
27. The method of claim 26, wherein analyzing the content of said image comprises generating a histogram.
28. The method of claim 26, wherein said pseudo gamma correction comprises a first linear transform and a second linear transform that meet at an output midpoint.
29. The method of claim 26, wherein a performing the pseudo gamma correction comprises generating a lookup table.
30. The method of claim 26, wherein analyzing the content of said digital image further comprises:
analyzing the content of said digital image to locate a set of skin colored pixels.
31. A computer readable medium storing computer instructions for enhancing a digital image defined by reference to a plurality of color components, said computer instructions comprising sets of instructions for:
analyzing the content of said digital image to determine black and white cutoff values, wherein analyzing the content of said digital image comprises analyzing color values of said digital image along at least two color components; and
performing a non linear brightness level correction based upon said black and white cutoff values.
32. The computer readable medium of claim 31, wherein the set of instructions for analyzing the content of said image comprises generating a histogram.
33. A computer readable medium storing computer instructions for enhancing a digital image defined by reference to a plurality of color components, said computer instructions comprising sets of instructions for:
analyzing the content of said digital image to determine a skin brightness and an average brightness; and
constructing a pseudo gamma correction based upon said skin brightness and an average brightness.
34. The computer readable medium of claim 33, wherein the set of instructions for analyzing the content of said image comprises generating a histogram.
35. A method of enhancing a digital image defined by reference to a plurality of color components, said method comprising:
a) analyzing the content of said digital image along at least two color components;
b) determining a pair of black and white cutoff values for each of the analyzed components; and
c) enhancing the digital image, wherein enhancing the digital image comprises synthesizing a separate transformation for each of the analyzed color components based on the corresponding pair of black and white cutoff values.
36. The method of claim 35, wherein enhancing the digital image further comprises applying the synthesized transformations to the content of said digital image.
37. The method of claim 35, wherein said color components comprise red, green, and blue components, wherein said analyzing comprises determining a particular black and white cutoff values for each of the red, green, and blue components.
38. The method of claim 35, wherein performing the nonlinear brightness level correction comprises generating lookup tables for at least the two color components.
39. A method of enhancing digital images, said method comprising:
a) providing an image display area for displaying an image, said image defined by reference to a plurality of color components;
b) providing a color correction control for initiating a color correction operation on said image, wherein a user selection of said color correction control initiates said color correction operation, said color correction operation comprising analyzing content of said image along at least two color component in order to determine black and white cutoff values for said image.
40. The method of claim 39, wherein a pair of black and white cutoff values is determined for each of the components that is analyzed, and a look-up table is generated for each of the components based the corresponding black and white cutoff values.
41. A method of providing a user interface for an application that enhances digital images, said method comprising:
a) providing a display area for displaying a digital image;
b) providing an image enhancement control for enhancing said digital image, wherein a single user selection of said image enhancement control initiates an image enhancement operation that comprises a brightness level correction and a gamma correction.
42. The method of claim 41, wherein the image enhancement operation is a color correction operation that is a hybrid of said brightness level correction combined with said gamma correction.
43. The method of claim 41, wherein the single user selection of said image enhancement control further initiates analysis of the content of said digital image in order to derive parameter settings for both the brightness level correction and the gamma correction.
Description
    FIELD OF THE INVENTION
  • [0001]
    The present invention relates to the field of digital photography. In particular, the present invention discloses a system for enhancing digital images.
  • BACKGROUND OF THE INVENTION
  • [0002]
    With improved technology and sharply reduced digital camera pricing, digital photography is rapidly replacing traditional film photography. With digital photography, there is no waiting for photographic film to be processed. Instead, most digital cameras allow the photographer to instantly review the digitally captured images. With this ability, the photographer may take another shot if he or she is not satisfied with the first digital images.
  • [0003]
    Digital photography has given amateur digital photographers new avenues of image distribution. Amateur digital photographs can easily be published on the world-wide web (WWW) to allow anyone on the Internet to view the digital photographs. Internet users can also email digital photographs as attachments or embedded within HTML formatted email messages.
  • [0004]
    One of the most powerful aspects of digital photography is ability to perform digital image processing. Digital mage processing programs can be used to manipulate digital photographs in order to correct flaws, modify colors, etc. However, the potential of digital image processing by amateurs remains largely unrealized. The only digital image processing used by most amateur digital photographers is the rotation of digital images to correct the orientation of the image. Digital image processing largely remains the domain of experts since quality image processing software is often too expensive for the amateur photographer and too difficult to use for the casual user.
  • [0005]
    To better realize the great potential of amateur digital photography, it would be very desirable to have improved digital image processing tools that are available for amateur digital photography enthusiasts. These digital image processing tools should be very simple to use yet provide powerful features for manipulating digital images.
  • SUMMARY OF THE INVENTION
  • [0006]
    To better realize the great potential of amateur digital photography, the present invention introduces an intuitive system for digital processing. The system of the present invention allows a user to enhance digital images that appear dark or dull. In one embodiment, the user activates an image enhancement button on a graphical user interface using cursor control device such as a mouse button.
  • [0007]
    In one embodiment, the digital image enhancement system is implemented using a nonlinear brightness level transformation correction and a pseudo gamma correction. The parameters for both the nonlinear brightness level transformation correction and a pseudo gamma correction are determined from examining the properties of the image to be enhanced.
  • [0008]
    Other objects, features, and advantages of present invention will be apparent from the company drawings and from the following detailed description.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • [0009]
    The objects, features, and advantages of the present invention will be apparent to one skilled in the art in view of the following detailed description in which:
  • [0010]
    FIG. 1 illustrates a block diagram of a computer system that may be used to implement the teachings of the present invention.
  • [0011]
    FIG. 2 illustrates one embodiment of an example graphical user interface for the image enhancement system of the present invention.
  • [0012]
    FIG. 3 a graphically illustrates a look-up table containing the identity transform.
  • [0013]
    FIG. 3 b graphically illustrates a first example of a look-up table containing a contrast change.
  • [0014]
    FIG. 3 c graphically illustrates a second example of a look-up table containing a contrast change.
  • [0015]
    FIG. 3 d graphically illustrates a first example of a look-up table containing a gamma color correction.
  • [0016]
    FIG. 3 e graphically illustrates a second example of a look-up table containing a gamma color correction.
  • [0017]
    FIG. 4 a graphically illustrates an example of a brightness histogram.
  • [0018]
    FIG. 4 b graphically illustrates the brightness histogram of FIG. 4 a with a set of dark pixels that have been deemed black and a set of light pixels that have been deemed white.
  • [0019]
    FIG. 4 c graphically illustrates a look-up table containing a contrast change built from the set of dark pixels that have been deemed black and the set of light pixels that have been deemed white of FIG. 4 b.
  • [0020]
    FIG. 5 graphically illustrates an example of a look-up table containing a pseudo gamma color correction.
  • [0021]
    FIG. 6 graphically illustrates an example of a look-up table containing a combined contrast and pseudo gamma color correction.
  • DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
  • [0022]
    A digital image enhancement system is disclosed. In the following description, for purposes of explanation, specific nomenclature is set forth to provide a thorough understanding of the present invention. However, it will be apparent to one skilled in the art that these specific details are not required in order to practice the present invention. For example, the present invention has been described with reference to one particular skin identification method. However, the system of the present invention may be implemented with other types of skin identification methods.
  • Digital Image Processing
  • [0023]
    Digital photography is rapidly replacing traditional film photography. One of the most powerful aspects of digital photography is ability to perform digital image processing upon captured digital images. Digital image processing programs can be used to manipulate digital photographs in order to correct image flaws, modify image colors, and perform other image corrections. Most digital image processing can be performed on a typical personal computer system such as the iMac personal computers from Apple Computer, Inc. of Cupertino, Calif.
  • Image Processing Computer System
  • [0024]
    FIG. 1 illustrates an exemplary embodiment of a computer system 100 that may be used to implement the present invention. Computer system 100 may be a typical personal computer system with a processor 110, a memory 120, and a bus controller 130 for controlling a system bus 135. Various different Input/Output devices may be coupled to the system bus 135 to allow the computer system 100 to interact with the outside world. For example, computer system 100 includes the input/output devices of a hard disk drive (HDD) 169, a display device 161 (such as a CRT or flat panel display), a keyboard 151, a cursor control device 153 (such as a computer mouse, track pad, or trackball), a fixed media drive 180, and network interface 191 for coupling to a network such as local area network (LAN) 195.
  • [0025]
    The image manipulation system 125 of the present invention resides in the memory 120 of computer system 100 as a set of computer instructions. The image manipulation system may also reside as a program on a fixed media device 181 such as Compact Disc Read-Only-Memory (CD-ROM) or Digital Versatile Disc Read-Only-Memory (DVD-ROM). The fixed media device is used with an associated fixed media drive 180 such as a CD-ROM drive or DVD-ROM drive, respectively. Alternatively, the image manipulation system 125 may be downloaded from across a network such as the global Internet 199 accessible through network interface 191.
  • [0026]
    The first task of the digital image processing system of the present invention is to acquire digital images to process. Referring to FIG. 1, the acquired images are normally stored onto a mass storage device such as hard disk drive (HDD) 169. In one embodiment of the present invention, the digital image processing system 125 may acquire digital images from a number of different sources.
  • [0027]
    To acquire images, the system of the present invention may include driver software necessary to automatically identify and interoperate with several different types of digital cameras that may be coupled to the host computer system. For example, FIG. 1 illustrates a digital camera system 147 coupled to serial bus 145 controlled by serial bus interface 140 of the computer system 100. Serial bus 145 may be an IEEE 1394 Firewire serial bus, a Universal Serial Bus (USB), or any other type of serial bus. In a preferred embodiment, the system is able to automatically recognize when digital camera system 147 is coupled to computer system 100 such that the digital images in digital camera system 147 can be acquired.
  • [0028]
    The system can also acquire images in a number of other manners. For example, film photographs may be placed onto a floppy disc (not shown) or onto a fixed media device 181 by a film developer at the request of the film photographer. Such photos may be acquire by image manipulation system 125 by having the user place the floppy disc (not shown) or the fixed media device 181 into the appropriate drive such as floppy disc drive (not show) or the fixed media drive 180.
  • [0029]
    Digital images may also be obtained through a computer network interface 191 of computer system 100. For example, a user may obtain digital images from a World Wide Web (WWW) site on the global Internet 199. Digital images may also be obtained using a digital scanner (not shown) coupled to computer system 100 as is well known in the art.
  • Dull Images
  • [0030]
    Captured digital images do not always appear as the photographers would like the digital images to appear. Photographs taken through automobile or airplane windows may have washed out colors due to the filter effects of the windows. An image captured with insufficient light may appear too dark. Such common undesirable flaws often plague otherwise excellent digital images and make those digital images appear dull.
  • [0031]
    Film photographs are also susceptible to having such problems. Insufficient lighting also causes dull looking film photographs. Furthermore, a user may set the film speed incorrectly such that many pictures receive the wrong exposure time for the type of film in the camera. These problems can distract the viewer from completely enjoying the captured images.
  • [0032]
    It would therefore be desirable to provide users with a tool for enhancing dull images. Ideally, the image enhancement solution should be simple and intuitive such that any computer user could easily use the image enhancement tool.
  • Digital Image Enhancement Overview
  • [0033]
    The present invention introduces a simple and intuitive image enhancement tool that may be used to brighten dull digital images. For example, FIG. 2 illustrates one embodiment of an example graphical user interface for the blemish removal system of the present invention. In FIG. 2, an image window 210 contains an image of bicyclist. However, the image was taken through a car window such that the colors appear muted thus disturbing the perfect image. To enhance the image, the user may activate the image enhancement tool by clicking on image enhancement button 220 using cursor control device such as a mouse button.
  • Color Adjustment
  • [0034]
    One method of modifying an image is to adjust the colors in the image. Color adjustment is often performed with a look-up table that outputs an output color for every input color. The effects of a color look-up table can be displayed graphically. FIG. 3 a graphically illustrates the results of a look-up table containing the identity transform. Specifically, each input brightness level on the horizontal axis is assigned the same output brightness level. If this is done separately for the display device's constituent colors, such as red, green, and blue in a luminous display or cyan, magenta, and yellow in a printing device, such a look-up table will result in modifications to displayed colors in the images.
  • [0035]
    A common method of attempting to improve an image is to increase the contrast of the image. Increasing the contrast of an image may be performed by making all the very dark pixels black, making all the very light pixels white, and spreading out the remaining pixel colors across the widened color space. FIG. 3 b graphically illustrates the look-up table of an example contrast. As illustrated in FIG. 3 b, the dark colors on the left are turned into black, the light colors on the right turned into white, and the remaining colors are spread across the available colors. The contrast function may be adjusted by changing parameters such as the amount of dark colors that become black and light colors. FIG. 3 c illustrates a contrast function having different parameter settings. By applying a different transformation to each of the display device's constituent colors, color adjustments are simultaneously performed.
  • [0036]
    Another method of attempting to improve an image is to use gamma control to bright the colors. Gamma correction is a nonlinear color correction that increases darker colors more than it increases lighter colors. FIG. 3 d graphically illustrates the look-up table of an example gamma correction. As illustrated in FIG. 3 d, the darker colors are more dramatically affected as illustrated by the steeper slope of the transform function above the darker colors. The gamma correction may also be adjusted by changing parameters. FIG. 3 e illustrates a gamma correction function having different parameter settings that provides less of a boost.
  • [0037]
    To improve image appearance, the present invention performs a color correction that is a hybrid of a nonlinear brightness level correction combined with a pseudo gamma correction. The parameter settings of both the nonlinear brightness level correction and the pseudo gamma correction are derived by analyzing the contents of the image.
  • Digital Image Enhancement Embodiment Details
  • [0038]
    The first step of the image enhancement system of the present invention is to analyze the digital image. To analyze the digital image, one or more histograms are created. The primary histogram used by the present invention is a brightness histogram that specifies how many pixels of each different brightness level exists in the digital image. FIG. 4 a graphically illustrates an example of a brightness histogram. In an image with eight bits for each color component (red, green, and blue), there are 256*3 different brightness levels.
  • The Nonlinear Brightness Level Correction
  • [0039]
    To perform a nonlinear brightness level correction, the present invention examines a brightness histogram and determines a first set of brightness levels that should be deemed black and a second set of brightness levels that should be deemed white. A number of different methods may be used to determine these sets.
  • [0040]
    In one embodiment, the system selects all the brightness levels from one end up to a certain fixed percentage. For example, the darkest 2% of pixels may be deemed black and the lightest 2% of pixels may be deemed white. The lightest pixel deemed a black pixel is known as the black cut-off pixel and the darkest pixel deemed white is known as the white cut-off pixel. Such an implementation is functional but a bit rigid.
  • [0041]
    In another embodiment, the system selects all the brightness levels until a significant change in numbers pixels is detected. For example, all dark brightness level pixels starting from black are deemed black until a brightness level that makes up at least 1% of the pixels in the image is detected. Similarly, all light brightness level pixels starting from white are deemed white until a brightness level that makes up at least 1% of the pixels in the image is detected.
  • [0042]
    Once the black cut-off and white cut-off pixels have been selected, a nonlinear brightness level correction function may be constructed. For example, FIG. 4 b illustrates the histogram of FIG. 4 a wherein a set of pixels at the darkest brightness levels have been deemed black and a set of pixels at the lightest brightness levels have been deemed white. This information can be used to construct the nonlinear brightness level correction function illustrated in FIG. 4 c.
  • A Complex Nonlinear Brightness Level Correction Implementation
  • [0043]
    A number of different methods of determining the black and white cut-off pixels have been investigated. After a long investigation a combination of different approaches was selected. Specifically, the following code specifies how a black cut-off pixel (blackcutoffs) and a white cut-off pixel (whitecutoffs) pixel were selected in a preferred embodiment that provided good results. The red, green, and blue (or other constituent color) values from the cutoff pixels are then used to synthesize separate contrast level transformations for each constituent color, and applying those transforms simultaneously optimizes contrast and adjusts color balance.
  • [0000]
    void histogram (unsigned char *image, unsigned long bytesPerPixel,
     unsigned long width, unsigned long height, //Dimensions of image
     int flag1, float param1,
     float *imageHistogram, // The global histogram array
     int *blackcutoffs, // The first point deemed black
     int *whitecutoffs, // The first point deemed white
     int *blackpoint, // The representative black point
     int *whitepoint, // The representative white point
     int *averageFaceBrightness)
    {
     unsigned char *pix = image;
     int i, minimumReasonableWhiteCutoff, firstSignificantBlack;
     int npixels = width*height; // Number of pixels in the image
     unsigned long k; // Loop counter
     int brightness;
     double averageBrightness, faceBrightnessAccumulator = 0;
     unsigned long brightnessAccumulator = 0;
     double hist1[3*255+1];
     long histbrit[3*255+1], histr[256], histg[256], histb[256], histSkin[256];
     int tmp;
     int xmax, ymax, xmin, ymin;
     int brightmin = 255*3;
     int brightmax = 0;
     float darkest = 255.0*3.0;
     float whitest = 0.0;
     int row, col;
     int offsetArray[ ] = {−width−1, −width, −width+1, −1, +1, width−1, width, width+1};
     int blackThreshold = 0;
     int blackFirstPercentile,blackSecondPercentile,blackThirdPercentile;
     int whiteThreshold = 255;
     int reducedWhiteThreshold[3];
     unsigned char redW = 255, greenW = 255, blueW = 255;
     float samplingFraction = 0.5;
     int samplingStep = (int) (1.0/samplingFraction);
     samplingFraction = samplingFraction*samplingFraction; //Because we skip pixels in two
    dimensions
     *averageFaceBrightness = −1;
     whitecutoffs[0] = whitecutoffs[1] = whitecutoffs[2] = 250;
     blackcutoffs[0] = blackcutoffs[1] = blackcutoffs[2] = 10;
     xmax = ymax= 0;
       xmin = width − 1;
       ymin = height − 1;
     /* Empty the histograms */
     for (i = 0; i < 3*255+1; i++) { hist1[i] = 0.0; histbrit[i] = 0; }
     for (i = 0; i <= 255;   i++) { histr[i] = 0; histg[i] = 0; histb[i] = 0; histSkin[i] = 0; }
     // Make initial pass, looking at every nth pixel, construct sub sampled histogram.
     // Number of pixels considered here depends on setting of samplingFraction above.*/
     brightnessAccumulator = 0;
     faceBrightnessAccumulator = 0.0;
     for( row = 0; row < height; row += samplingStep ){
      for( col = 0; col < width; col += samplingStep){
       k = row*width+col; // Determine pixel index
       pix = image + k*bytesPerPixel;  // Determine pixel address
       brightness = (int) (pix[0]+pix[1]+pix[2]); // Calculate pixel brightness
       histbrit[brightness] += 1; // Add brightness to histogram
       brightnessAccumulator += brightness; // Add pixel's brightness to accumulator
       if (row>height/4 && row<3*height/4 && col>width/5 && col<4*width/5 ){
        if(pixel_is_flesh_colored(image, k*bytesPerPixel, width, height, 0, 0,
            0.0 , 0, 0, 0, 0, 0 , 1)) {
         nSkinPixels ++;
         faceBrightnessAccumulator += (double) brightness;
         histSkin[brightness/3]++;
        }
       }
      }
     }
      // Calculate average brightness
     averageBrightness = brightnessAccumulator/(3.0*npixels*samplingFraction);
     //Store this in image histogram array. First slots go to r,g,b, and brightness
     imageHistogram[6*256 + 0] = averageBrightness; // Store averageBrightness in global hist array
     if (nSkinPixels > 30) *averageFaceBrightness = faceBrightnessAccumulator/(3.0*nSkinPixels);
     // Determine first significant black as darkest 1/100% pixel
     i = 0;   while(histbrit[i] < 0.0005 * npixels * samplingFraction ){ i++;}
     firstSignificantBlack = i;
     // Don't recalibrate more than 15% of the entire range as black.
     if (firstSignificantBlack > 0.15 * 765.0 ) firstSignificantBlack = .15 * 765.0;
     /*Replace histbrit frequencies with cumulative sum */
     for (i = 1; i <= 3*255; i++) histbrit[i] = histbrit[i−1] + histbrit[i];
     /* Find threshold where cumulative distribution exceeds 4% of the # of pixels sampled above. */
     i = 0;   while( histbrit[i] < 0.04 * npixels * samplingFraction ){ i++;} blackThreshold = i;
     /* Find threshold where cumulative distribution exceeds 1%, 2%, 3%, and 4% of the # of pixels*/
     i = 0;   while( histbrit[i] < 0.01 * npixels * samplingFraction ){ i++;}
     blackFirstPercentile = i;
     i = 0;   while( histbrit[i] < 0.02 * npixels * samplingFraction ){ i++;}
     blackSecondPercentile = i;
     i = 0;   while( histbrit[i] < 0.03 * npixels * samplingFraction ){ i++;}
     blackThirdPercentile = i;
     /* Determine the brightness histogram quartiles */
     i = 0;   while( histbrit[i] < 0.25 * npixels * samplingFraction ){ i++;}
     imageHistogram[6*256 + 1] = i;
     i = 0;   while( histbrit[i] < 0.5 * npixels * samplingFraction ){ i++;}
     imageHistogram[6*256 + 2] = i;
     i = 0;   while( histbrit[i] < 0.75 * npixels * samplingFraction ){ i++;}
     imageHistogram[6*256 + 3] = i;
       /* Set 90% as the minimum reasonable white cut-off */
     i = 0; while (histbrit[i] < 0.90 * npixels * samplingFraction ){ i++;}
     minimumReasonableWhiteCutoff = i;
     i = 3*255; while (histbrit[i] > 0.99 * npixels * samplingFraction) { i−−;} whiteThreshold = i;
     imageHistogram[6*256 + 4] = i; // Set 99th percentile
     //Store 55th percentile.
     i = 0; while (histbrit[i] < 0.55 * npixels * samplingFraction) { i++;}
     imageHistogram[6*256 + 5] = i;
     // Determine more reduced possible white thresholds
     i = 3*255; while (histbrit[i] > 0.995 * npixels * samplingFraction ){ i−−;}
     reducedWhiteThreshold[0] = i;
     i = 3*255; while (histbrit[i] > 0.999 * npixels * samplingFraction ){ i−−;}
     reducedWhiteThreshold[1] = i;
     // Reduce the thresholds if there are too few brightness levels in the middle.
     if (whiteThreshold − blackThreshold < 400) {  // If < 400 brightness levels between then,
      whiteThreshold = reducedWhiteThreshold[1];  // select the most reduced white threshold and
      blackThreshold = blackFirstPercentile;  // the first black percentile
     }
     else if (whiteThreshold − blackThreshold < 570 ) { // if < 570 brightness levels between then,
      whiteThreshold = reducedWhiteThreshold[1];  // select the most reduced white threshold and
      blackThreshold = blackSecondPercentile;  // the *second* black percentile.
     }
     else if (whiteThreshold − blackThreshold < 650) { // If < 650 brightness levels between then,
      whiteThreshold = reducedWhiteThreshold[0];  // select the less reduced white threshold and
      blackThreshold = blackThirdPercentile;  // the *third* black percentile.
     };
     //  But if it is less than 10 past the first significant black move it forward.
     if (firstSignificantBlack < blackThreshold − 10) blackThreshold = firstSignificantBlack + 5;
     for(i = 0; i < 3*255 +1; i++) { hist1[i] = 0.0; histbrit[i] = 0;} // Reset histograms to zero.
     /* Now compute a full histogram using all the pixels in the image */;
     for (row = 0; row < height; row++) { // Scan across all rows
      for (col = 0; col < width; col++) { // Scan across all columns
       k = row*width + col;      // Determine pixel index
       pix = image + k*bytesPerPixel; // Determine pixel address
       brightness = pix[0] + pix[1] + pix[2]; // Calculate pixel brightness
       histr[pix[0]]+=1; histg[pix[1]]+=1; histb[pix[2]]+=1; // Tally this pixel in histogram.
       histbrit[brightness] += 1; // Add pixel's brightness to accumulator
       if (brightness > brightmax && brightness <= whiteThreshold)   brightmax = brightness;
       else if (brightness < brightmin && brightness >= blackThreshold ) brightmin = brightness;
       if ((brightness <= brightmin + 10 && brightness >= blackThreshold) // if pixel is dark and
          && col>1 && row>1 && row<height−1 && col<width−2) { // not within 1 pixel from edge
        tmp = brightness; // Start with current pixel brightness
        for (i = 0; i < 8; i++) {    // look at the 8 pixels surrounding the current one
         pix =image+(k + offsetArray[i])*bytesPerPixel;   // Calculate pixel address
         tmp += (pix[0] + pix[1] + pix[2]); // Add pixel's brightness
        }
        tmp = (int) ((float) tmp)/9.0 +0.5; // Calculate average brightness
       if (tmp < darkest) {   // If brightness avg < current darkest, then make new darkest
         pix = image + k*bytesPerPixel; // Calculate pixel address
         darkest = tmp; // Set current pixel as darkest
         xmin = col; ymin = row; // Set this point as the black point
         blackpoint[0] = xmin; blackpoint[1] = ymin;
         // change black cut off
         blackcutoffs[0] = pix[0]; blackcutoffs[1] = pix[1]; blackcutoffs[2] = pix[2];
         tmp = pix[0];  //block some kinds of color correction for black point.
         if (pix[1] > tmp) tmp = pix[1];  // find largest (color which will be reduced the most)
         if (pix[2] > tmp) tmp = pix[2];
         if (tmp == pix[1]) blackcutoffs[1] = darkest/3.0;
         if (tmp == pix[2]) blackcutoffs[2] = darkest/3.0;
        }
       }
       if ((brightness >= 0.94* brightmax && brightness <= whiteThreshold) // if light pixel and
         && col>1 && row>1 && row<height−1 && col<width−2 ){ // not 1 pixel of an edge,
        tmp = brightness; // Start with current pixel brightness
        for(i = 0; i < 8; i++) { // look at the 8 pixels surrounding the current one
         pix = image (k + offsetArray[i])*bytesPerPixel; // Calculate pixel address
         tmp += (pix[0] + pix[1] + pix[2]); // Add pixel's brightness
        }
        tmp = ((float) tmp)/9.0 + 0.5; // Calculate average brightness
        if (tmp > whitest) {   // If brightness avg > current whitest, then make new whitest
         pix = image + k*bytesPerPixel; // Calculate pixel address
         whitest = tmp; // Set current pixel as whitest
         xmax = col; ymax = row; // Set this point as the white point
         whitepoint[0] = xmax; whitepoint[1] = ymax;
         redW  = whitecutoffs[0] = pix[0]; // change white cut off
         greenW = whitecutoffs[1] = pix[1];
         blueW  = whitecutoffs[2] = pix[2];
        }
       }
      }
     }
     if (blueW + 50 < redW && blueW + 40 < greenW ) { //probably found candle or dim bulb
      whitecutoffs[2] = (redW + greenW)/2.0; // don't boost the blue an inordinate amount.
      whitecutoffs[0] = (redW + greenW)/2.0; // don't reduce the red too much
     }
     /*Modify hist1, dividing by pixel count, so hist1 holds percentages (or rather fractions)
    instead of absolute counts.*/
     for(i = 0; i <= 3*255; i++){
      hist1[1] = ((double)histbrit[i])/((double)width*height);
      imageHistogram[3*256+i] = hist1[i];
     }
     /* Store histogram data in global histogram array */
     for(i = 0; i < 256; i++){
      imageHistogram[i    ] = (float)histr[i]/((float) width*height );
      imageHistogram[i+ 256 ] = (float)histg[i]/((float) width*height );
      imageHistogram[i+2*256] = (float)histb[i]/((float) width*height );
     }
     return;
    }
  • The Pseudo Gamma Correction
  • [0044]
    To further improve the image, a pseudo gamma correction is performed. One embodiment of the pseudo gamma correction uses a pair of correction slopes instead of a gamma correction curve. Specifically, a pair of slopes that meet at the middle output point as illustrated in FIG. 5 may be used to create a pseudo gamma correction. The pseudo gamma correction can be defined using a single gamma (‘γ’) parameter.
  • [0045]
    To highlight the people present in an image, it would be desirable to include the main skin colors in the lower slope that provides a greater brightness boost than the higher slope. In order to perform this, the skin colors in the image must be determined. A number of different skin color determining methods have been devised in the art. One method of identifying skin pixels is provided in the following code:
  • [0000]
    static int pixel_is_flesh_colored (unsigned char *pixel, int offset,
       int image_width, int image_height,
       float par1, float par2, float par3, float par4,
       int bool1, int bool2, int bool3, int bool4, int sensitivity )
    /* This has a lot of unnecessary parameters right now. */
    {
     float r,g,b;
     float r0,g0,b0,A0;
     float whitenessThreshold = .04;
     int isSkinColored = FALSE; // Assume pixel is not skin
     r = (float)pixel[0+offset];
     g = (float)pixel[1+offset];
     b = (float)pixel[2+offset];
     /* calculate normalized color coordinates,
     and also A0, the distance from white */
     r0 = r/(r+g+b + 0.001);
     g0 = g/(r+g+b + 0.001);
     b0 = b/(r+g+b + 0.001);
     A0 = sqrt((r0 − 1/3.0)*(r0 − 1/3.0)+
     (g0 − 1/3.0)*(g0 − 1/3.0)+ (b0 − 1/3.0)*(b0 − 1/3.0));
     if (sensitivity == 1 && r+g+b > 100 &&
      g0 > 0.5367164519094756 − 0.6128005657708601*r0 &&
      g0 > 0.38 − 0.2353293562638825*r0 &&
      g0 > 0.015583841668305133 + 0.5392560632803767*r0 &&
      g0 < 1.484094829686838 − 2.426683826683842*r0 &&
      g0 < 0.5409076086570325 − 0.4758136154405282*r0 &&
      g0 < 0.4177904911910172 − 0.20741151415949488*r0 &&
      g0 < −0.3795460192202154 + 2.103239978034066*r0
     ) isSkinColored = TRUE;
     whitenessThreshold = 0.03;
     if (sensitivity == 2 ) whitenessThreshold = 0.02;
     // Exclude colors that are too white;
     if (sensitivity > 0 && A0 < whitenessThreshold) isSkinColored =
     FALSE;
       // 0.09 more accurate than 0.02 overall but misses very fair skin.
     return isSkinColored;
    }
  • [0046]
    Once the skin pixels have been determined, a brightness of the skin pixels is then determined. The system then determines a ‘gamma’ parameter that ideally includes the skin pixels in the lower slope. However, the average brightness of the overall image must be taken into consideration. Specifically, if the average brightness of the image is high, then the gamma value may need to be reduced.
  • [0047]
    The following code provides one example of a method of determining a gamma value for color system that uses eight bits per color component (red, green, and blue):
  • [0000]
    averageBrightness = imageHistogram[6*256 + 0];
    firstBrightnessQuartile = imageHistogram[6*256 + 1];
    secondBrightnessQuartile = imageHistogram[6*256 + 2];
    thirdBrightnessQuartile = imageHistogram[6*256 + 3];
    brightness99thPercentile = imageHistogram[6*256 + 4];
    sixtiethPercentile = imageHistogram[6*256 + 5];
    // Perform a nonlinear pseudo gamma brightness correction.
    gammaValue = 0.0;
    gammaFlag = 1;
    facebrightness = *averageFaceBrightness;
    if (*averageFaceBrightness > 0 && *averageFaceBrightness < 100
        && (*averageFaceBrightness < sixtiethPercentile/3.0 ∥ averageBrightness < 100)
        && ((*averageFaceBrightness > firstBrightnessQuartile/3 −10 )
           ∥ (*averageFaceBrightness > averageBrightness −20) ))
     {
     gammaValue = 0.6*(150.0 − *averageFaceBrightness);
     }
    else if (*averageFaceBrightness > 0 && *averageFaceBrightness < 80 )
     gammaValue = 0.6*(150.0 − *averageFaceBrightness);
    else if (*averageFaceBrightness > 0 && *averageFaceBrightness < 100 )
     gammaValue = 0.6*(135.0 − *averageFaceBrightness);
    else if (*averageFaceBrightness > 150)
     gammaValue = 0.66*(160.0 − *averageFaceBrightness);
    else if (*averageFaceBrightness > 100)
     {
     brightnessTarget = 150.0;
     if (*averageFaceBrightness > averageBrightness− 10)
        brightnessTarget = (sixtiethPercentile/3.0 + *averageFaceBrightness)/2;
        gammaValue = 0.6*(brightnessTarget − *averageFaceBrightness);
     }
    if (gammaValue > 0 && brightness99thPercentile < 600 && *averageFaceBrightness > 130)
        gammaValue = 0;
    if (averageBrightness < 90) gammaValue = fabs(gammaValue);
    if (gammaValue > 30 ) gammaValue = 30;
    if (averageBrightness > 75 && gammaValue > 27) gammaValue = 27;
    if (averageBrightness > 100 && gammaValue > 20) gammaValue = 20;
    if (averageBrightness > 100 && facebrightness > 85 && gammaValue > 15)  gammaValue = 15;
    if (facebrightness > 65 && gammaValue > 27 ) gammaValue = 27;
    // Select the smallest midpoint of red, green, and blue.
    midpoint = whitecutoffs[0]/2.0;
    if (whitecutoffs[1]/2.0 < midpoint) midpoint = whitecutoffs[1]/2.0;
    if (whitecutoffs[2]/2.0 < midpoint) midpoint = whitecutoffs[2]/2.0;
    maxPossibleGamma = midpoint − 20;
    if (gammaValue > maxPossibleGamma) gammaValue = maxPossibleGamma;
    changeColorLevels(bwBuffer, bytesPerPixel, rowBytes, width, height, blackcutoffs,
        whitecutoffs, gammaValue, gammaFlag, imageHistogram);

    Once the gamma value has been determine, a pseudo gamma correction look-up table may be created such as the pseudo gamma correction of FIG. 5.
  • [0048]
    The contrast change and the pseudo gamma correction may be performed in a single step. FIG. 6 graphically illustrates a look-up table for a combined nonlinear brightness level correction and gamma correction. Note that since the nonlinear brightness level correction is actually different for the red, green, and blue constituent colors, three different combined look-up tables will be created.
  • [0049]
    The foregoing has described a method and apparatus for performing digital image enhancement. It is contemplated that changes and modifications may be made by one of ordinary skill in the art, to the materials and arrangements of elements of the present invention without departing from the scope of the invention.
Patent Citations
Cited PatentFiling datePublication dateApplicantTitle
US4661843 *Jul 9, 1985Apr 28, 1987Kabushiki Kaisha ToshibaColor image copying apparatus
US4931864 *Sep 27, 1989Jun 5, 1990Canon Kabushiki KaishaImage forming apparatus which performs gamma correction on the basis of a cumulative frequency distribution produced from a histogram of image data representing a selected area of an image
US5216493 *Feb 19, 1992Jun 1, 1993Eastman Kodak CompanyMultipoint digital automatic white balance for a video system
US5245432 *Nov 4, 1991Sep 14, 1993Imageware Research And Development Inc.Apparatus and method for transforming a digitized signal of an image to incorporate an airbrush effect
US5509086 *Dec 23, 1993Apr 16, 1996International Business Machines CorporationAutomatic cross color elimination
US5579446 *Jan 27, 1994Nov 26, 1996Hewlett-Packard CompanyManual/automatic user option for color printing of different types of objects
US5796861 *Jul 11, 1997Aug 18, 1998Frim International, Inc.Mosaic construction, processing, and review of very large electronic micrograph composites
US5808697 *Jun 14, 1996Sep 15, 1998Mitsubishi Denki Kabushiki KaishaVideo contrast enhancer
US6014464 *Oct 21, 1997Jan 11, 2000Kurzweil Educational Systems, Inc.Compression/ decompression algorithm for image documents having text graphical and color content
US6236751 *Sep 23, 1998May 22, 2001Xerox CorporationAutomatic method for determining piecewise linear transformation from an image histogram
US6384836 *Aug 27, 1997May 7, 2002Canon Inc.Color gamut clipping
US6433898 *Aug 16, 1996Aug 13, 2002Heidelberger Druckmaschinen AktiengesellschaftMethod and apparatus for the conversion of color values
US6535648 *Dec 8, 1998Mar 18, 2003Intel CorporationMathematical model for gray scale and contrast enhancement of a digital image
US6999604 *Aug 7, 1998Feb 14, 2006Korea Institute Of Science And TechnologyApparatus and method for detecting a moving object in a sequence of color frame images
US7139425 *Aug 28, 2001Nov 21, 2006Fuji Photo Film Co., Ltd.Method and apparatus for correcting white balance, method for correcting density and a recording medium on which a program for carrying out the methods is recorded
US7263221 *Jun 23, 2000Aug 28, 2007Minolta Co., Ltd.Image processing for image correction
US7283683 *Nov 20, 1998Oct 16, 2007Sharp Kabushiki KaishaImage processing device and image processing method
US20030012414 *Jun 29, 2001Jan 16, 2003Huitao LuoAutomatic digital image enhancement
US20030152283 *Aug 3, 1999Aug 14, 2003Kagumi MoriwakiImage correction device, image correction method and computer program product in memory for image correction
US20030179911 *Jun 7, 1999Sep 25, 2003Edwin HoFace detection in digital images
US20040109604 *Oct 11, 2001Jun 10, 2004Shuichi KagawaColor converter and color converting method
Referenced by
Citing PatentFiling datePublication dateApplicantTitle
US8184176Dec 9, 2009May 22, 2012International Business Machines CorporationDigital camera blending and clashing color warning system
US8693777 *Apr 21, 2011Apr 8, 2014Samsung Electro-Mechanics Co., Ltd.Apparatus and method for estimating edge areas of digital image
US20110134261 *Dec 9, 2009Jun 9, 2011International Business Machines CorporationDigital camera blending and clashing color warning system
US20120033884 *Feb 9, 2012Dong-A University Research Foundation For Industry-Academy CooperationApparatus and method for estimating edge areas of digital image
WO2011034525A1 *Sep 16, 2009Mar 24, 2011Hewlett-Packard Development Company, LpSystem and method for assessing photographer competence
Classifications
U.S. Classification382/274
International ClassificationG06K9/40
Cooperative ClassificationH04N1/4074, G06T5/40, H04N1/628, G06T5/009
European ClassificationG06T5/40, H04N1/62E, H04N1/407B2
Legal Events
DateCodeEventDescription
Dec 6, 2011CCCertificate of correction
Sep 3, 2014FPAYFee payment
Year of fee payment: 4