|Publication number||USRE41454 E1|
|Application number||US 10/769,418|
|Publication date||Jul 27, 2010|
|Filing date||Jan 30, 2004|
|Priority date||Apr 26, 1996|
|Also published as||US5614948, US5712682, US20040183924, USRE40654|
|Publication number||10769418, 769418, US RE41454 E1, US RE41454E1, US-E1-RE41454, USRE41454 E1, USRE41454E1|
|Inventors||Eric C. Hannah|
|Original Assignee||Intel Corporation|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (11), Non-Patent Citations (4), Referenced by (1), Classifications (10)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This is a continuation of application application Ser. No. 08/638/528 09/491,672 filed Jan. 26, 2000 which is a reissue of U.S. Pat. No. 5,712,682 whose parent application has Ser. No. 08/638,528, filed Apr. 26, 1996, and that issued as U.S. Pat. No. 5,614,948.
1. Field of the Invention
The present invention relates to image processing. More specifically, a system for capturing and processing video images.
Digital cameras are used in a variety of applications requiring image capture and image processing. Many applications require cameras which are economical, yet generate a high quality video signal. Typical solid state sensors used in digital cameras have a dynamic range (or light intensity range) of 1000:1 or greater. However, many existing digital cameras utilize an inexpensive 8 bit or 6 bit analog-to-digital (A/D) converter to generate a digital output signal representing the captured image. The use of an 8 bit A/D converter limits the dynamic range of the output signal by providing a maximum of 256 possible luminance levels. Therefore, the dynamic range capabilities of the sensor (1000:1) are compressed to 256:1 by the A/D converter. This reduction in dynamic range results in “clipping” of the image; i.e., loss of image detail in bright areas and dark areas of the image. When clipping occurs, dark areas of the image become black (e.g., luminance level 0) and bright areas of the image become white (e.g., luminance level 255).
Existing cameras attempt to compensate for this dynamic range reduction by using an automatic gain control (AGC) amplifier having different gain settings. The gain setting of the AGC amplifier is determined based on the total luminance entering the camera. Since a single luminance level is determined, the selected gain setting is applied uniformly to the entire image.
An example of an existing camera is illustrated in
AGC amplifier 18 typically has several different gain settings which may be applied to the sensor output signal. The required gain setting for a particular image is selected based on the total light entering camera 10. If the total light level is low, the gain setting is increased. Similarly, if the total light level is high, the gain setting is decreased. The particular gain setting selected is applied uniformly to the entire image.
A/D converter 20 generates a digital video output signal on a signal line 22. As discussed above, the dynamic range of the digital video output signal is limited by the capacity of A/D converter 22.
Problems associated with dynamic range reduction may be solved by utilizing a 10 bit or 12 bit A/D converter to preserve the dynamic range of the sensor output signal. However, 10 bit and 12 bit A/ D converters are expensive and substantially increase the cost of the camera. In an application requiring an inexpensive camera, the use of 10 bit or 12 bit AID converters is not practical.
It is therefore desirable to provide a low-Cost digital camera utilizing an inexpensive A/D converter, yet capable of generating a video signal containing image detail in bright areas and dark areas of the image.
The present invention provides a digital camera having an inexpensive A/D converter and including a processor for enhancing the dynamic range of the camera. The processor instructs a gain control amplifier to reduce the gain in bright areas of the image and increase the gain in dark areas of the image. These changes in gain settings for different portions of the image increase the image detail provided by the camera, thereby improving the overall image quality.
An embodiment of the present invention provides a sensor for capturing an image and generating a sensor output signal. A gain control amplifier is coupled to the sensor and receives the sensor output signal. The gain control amplifier has controls for applying various levels of gain to the sensor output signal. An analog-to-digital converter is coupled to the gain control amplifier and generates a digital output signal representing the captured image. A processor is coupled to the analog-to-digital converter and the gain control amplifier. The processor provides a control signal to the gain control amplifier for adjusting the level of gain applied by the amplifier.
Another feature of the present invention provides a gain map containing gain settings applied to the sensor output signal by the gain control amplifier. The gain map is continually updated by the processor to include changes in the captured image.
Other embodiments of the invention provide a register coupled to the processor and the gain control amplifier. The gain map containing gain settings is stored in the register and the gain control amplifier reads the gain settings from the register.
The present invention is illustrated by way of example in the following drawings in which like references indicate similar elements. The following drawings disclose various embodiments of the present invention for purposes of illustration only and are not intended to limit the scope of the invention.
The following detailed description sets forth numerous specific details to provide a thorough understanding of the invention. However, those skilled in the art will appreciate that the invention may be practiced without these specific details. In other instances, well known methods, procedures, components, and circuits have not been described in detail so as not to obscure the invention.
The present invention provides a system for enhancing the dynamic range of a digital camera having an inexpensive A/D converter. An adaptive gain control mechanism is provided for adjusting gain settings applied to a captured image.
A first embodiment of the invention is illustrated in
As discussed above, A/D converter 108 may be a relatively inexpensive 6 bit or 8 bit A/D converter. To simplify the explanation of the invention, the specification describes camera 100 and its operation when using an 8 bit A/D converter 108.
A/D converter 108 converts the signal received from gain control amplifier 106 from an analog signal to a digital signal. The digital output of A/D converter 108 is provided on signal line 110 and represents an 8 bit digital video signal of the captured image. As noted above, the dynamic range of the signal on line 110 is limited by the 8 bit capacity of A/D converter 108.
The digital video signal on signal line 110 is also transmitted to a processor 114 using signal line 112. Processor 114 may be any type of processor capable of receiving a digital video signal and performing various operations or calculations related to the received signal. Processor 114 can be a dedicated image processing system or part of a system servicing other devices or performing other functions. In a preferred embodiment of the invention, processor 114 is a personal computer capable of performing a variety of operations and servicing numerous devices. In the preferred embodiment, the personal computer utilizes a PentiumŪ processor manufactured by Intel Corporation of Santa Clara, Calif.
The actual operations and processing steps performed by processor 114 may be implemented in software executed by processor 114. Those skilled in the art will appreciate that processor 114 may be a single processor device capable of processing digital data (e.g., a digital signal processor) or processor 114 may be a complete computer system containing a variety of devices and capable of performing multiple simultaneous operations. Thus, the invention is capable of being implemented on a wide variety of processing devices and computer platforms.
By way of example, the invention will be described in an implementation utilizing a personal computer as processor 114. Numerous type of computers may be used to practice the invention. The computer must be capable of receiving the digital video signal on signal line 112, performing the necessary operations and calculations, and transmitting a control signal to gain control amplifier 106 using a communication line 116. Line 116 functions as a control bus used to communicate control signals and data between processor 114 and gain control amplifier 106. In a specific embodiment of the invention, line 116 is a high-speed serial bus such as a Universal Serial Bus (USB). Although the invention will be described with reference to a high-speed serial bus, any communication line having sufficient bandwidth and low latency may be used to implement the invention.
In the embodiment of
At step 124 of
Referring again to
Image 152 is divided into eight rows and eight columns, creating 64 image regions 154. Each image region 154 is associated with a particular gain setting contained in a corresponding cell of gain map 148 (FIG. 6A). Image region (2, 4) is a notation indicating the image region at the intersection of row 2 and column 4. The gain level applied to image region (2, 4) is stored in the gain map at cell (2, 4). Thus, each gain map cell provides a gain level to be applied to the corresponding image region. Although
At step 136 of
If image detail has been clipped from the image region at step 138 due to brightness, then the gain map setting for that region is reduced at step 140 to provide increased image detail; i.e., instead of producing all white regions, the gain setting is reduced to prevent clipping and provide enhanced image detail.
If significant bright portions are not identified at step 138, then step 142 determines whether the image region being analyzed contains significant dark portions; i.e., whether image detail has been lost or clipped by the 8 bit A/D converter. If image detail has been lost at step 142, then the gain map setting for that region is increased at step 140 to provide increased image detail. By increasing the gain setting, previously all black portions of the region may begin to show some image detail, thereby increasing the overall image quality.
If the image region being analyzed does not contain significant bright portions or significant dark portions, then the routine continues to step 144 without updating the gain map setting for the particular image region. Step 144 determines whether the current row and column counters indicate the last row and column of the captured image. If all image regions have been analyzed, then the routine ends. Otherwise, the routine branches to step 146 where the row and/or column counters are incremented to select the next image region of the captured image. The routine then returns to step 136 to analyze the next image region. The image regions may be analyzed in any order. Row and column counters represent an example of a mechanism for analyzing each image region in a systematic manner.
When all regions of a captured image have been analyzed according to the procedure illustrated in
A particular example of the operations performed in
After capturing the first image, processor 114 divides the image into an array of image regions 154, as shown in FIG. 5. Each region 154 is analyzed by processor 114 to determine whether the gain setting for the region should be adjusted to provide increased image detail. Using
The gain settings illustrated in
The process of capturing an image, applying settings contained in a gain map to the image, and analyzing the digital video output signal to update gain settings is performed repeatedly by the system illustrated in FIG. 2. The gain settings contained in the gain map are updated in response to changes in the captured image; e.g., changes in light level, movement of objects in the image, and the like. Thus, the processor is repeatedly updating the gain map settings to enhance the overall image detail provided in the digital video output signal.
Those skilled in the art will appreciate that various methods may be used by processor 114 to analyze light intensity in each image region. An embodiment of the invention applies a histogram equalization algorithm to each image region. The histogram equalization algorithm generates a histogram of pixel luminance values in a particular region. The histogram is used to identify areas of high luminance and low luminance; i.e., areas requiring gain adjustments. Several exemplary histograms are illustrated in
In addition to the histogram equalization algorithm discussed above, various methods and algorithms for smoothing sharp edges and transitions between adjacent image regions will be known to those skilled in the art. These methods and algorithms may be utilized with the present invention to provide smoothing in transition areas and between adjacent image regions.
As noted above,
The addition of register 160 to camera 156 increases the cost of the camera slightly, but eliminates the dependence of gain control amplifier 106 on communication line 116 (
From the above description and drawings, it will be understood by those skilled in the art that the particular embodiments shown and described are for purposes of illustration only and are not intended to limit the scope of the invention. Those skilled in the art will recognize that the invention may be embodied in other specific forms without departing from its spirit or essential characteristics. Referring to details of particular embodiments are not intended to limit the scope of the claims.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US4962426||Apr 7, 1989||Oct 9, 1990||Hitachi, Ltd.||Dynamic noise reduction circuit for image luminance signal|
|US5157493||Sep 21, 1990||Oct 20, 1992||Laboratoire Europeen De Recherches Electroniques Appliquees Societe En Nom Collectif||Synch responsive agc utilizing a-d converter|
|US5282022||Nov 13, 1990||Jan 25, 1994||Sanyo Electric Co., Ltd.||White balance adjusting apparatus for automatically adjusting white balance in response to luminance information signal and color information signal obtained from image sensing device|
|US5295001||Oct 28, 1991||Mar 15, 1994||Canon Kabushiki Kaisha||Image sensing apparatus having tone control function|
|US5400074||Feb 25, 1993||Mar 21, 1995||Sony Corporation||Video camera device having an exposure control circuit|
|US5402249||Sep 21, 1993||Mar 28, 1995||Kabushiki Kaisha Toshiba||Integrated circuit with exterior gain control adjustment|
|US5420630||Sep 8, 1993||May 30, 1995||Canon Kabushiki Kaisha||Image pickup apparatus performing white balance control based on data from regions of a frame|
|US5448306||Jun 18, 1993||Sep 5, 1995||Canon Kabushiki Kaisha||Image processing apparatus with variable clamping|
|US5471241||Mar 2, 1994||Nov 28, 1995||Canon Kabushiki Kaisha||Image sensing method and apparatus for producing a color video signal and luminance video signal|
|US5512948||May 24, 1995||Apr 30, 1996||Fuji Photo Film Co., Ltd.||Negative-image signal processing apparatus|
|US6163342||Oct 29, 1997||Dec 19, 2000||Canon Kabushiki Kaisha||Image sensing method and apparatus|
|1||Non-Final Office Action (mailed Jan. 19, 2001)-U.S. Appl. No. 09/491,672, filed Jan. 26, 2000 ( 7 pages).|
|2||Non-Final Office Action (mailed Jan. 19, 2001)—U.S. Appl. No. 09/491,672, filed Jan. 26, 2000 ( 7 pages).|
|3||Non-Final Office Action (mailed Nov. 15, 2002)-U.S. Appl. No. 09/491,672, filed Jan. 26, 2000 ( 6 pages).|
|4||Non-Final Office Action (mailed Nov. 15, 2002)—U.S. Appl. No. 09/491,672, filed Jan. 26, 2000 ( 6 pages).|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US20070165122 *||Jan 18, 2007||Jul 19, 2007||Keiichi Tsumura||Video signal clamping circuit|
|U.S. Classification||348/255, 348/222.1|
|International Classification||H04N5/243, H04N5/52, H04N5/20|
|Cooperative Classification||H04N5/20, H04N5/243, H04N5/52|
|European Classification||H04N5/243, H04N5/20|