|Publication number||US6614562 B1|
|Application number||US 09/343,390|
|Publication date||Sep 2, 2003|
|Filing date||Jun 30, 1999|
|Priority date||Jun 30, 1999|
|Publication number||09343390, 343390, US 6614562 B1, US 6614562B1, US-B1-6614562, US6614562 B1, US6614562B1|
|Inventors||Ronald K. Minemier|
|Original Assignee||Intel Corporation|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (11), Referenced by (21), Classifications (9), Legal Events (6)|
|External Links: USPTO, USPTO Assignment, Espacenet|
The invention relates to reducing dark current noise during image capture operations in a digital imaging system.
Referring to FIG. 1, a common digital image processing system includes a camera 100, a computer 102 coupled to the camera 100 by a communication link 104, and a display unit 106. The communication link 104 may be a serial bus such as the universal serial bus (USB). The display unit 106 may be any convenient display device such as a cathode ray tube (CRT) or liquid crystal display (LCD).
The camera 100 may use the imager 108 to generate an electrical representation of an optical image 110. To accomplish this, the imager 108 may include a sensor having an array of photon sensing elements. During an integration time or interval, each sensor element accumulates light energy from that portion of optical image 110 that is focused on it by camera 100 optics (not shown in FIG. 1). At the expiration of the integration interval, sensor elements indicate the intensity of the received light energy by, for example, an analog voltage signal. Camera 100 typically processes the indications from sensor elements to form a frame of digital data which may then be stored in memory internal to the camera 100 (not shown in FIG. 1), and/or transferred to the computer 102.
Typically, the frame of data does not indicate an exact duplicate of the optical image 110 due to imperfections introduced by camera 100. The camera 100 may introduce optical distortion and noise such as dark current noise. Dark current noise may be defined as sensor element current that is present even when the sensor element is not illuminated. Dark current noise tends to reduce the captured image's dynamic range and signal to noise ratio (SNR) and places a limit on sensor element integration time. Dark current noise also tends to increase as the temperature increases, rendering many cameras substantially unusable in high temperature environments.
Some cameras attempt to cancel out dark current noise by subtracting a predetermined noise level from the intensity that is indicated by the sensor element. The predetermined noise level may be determined from, for example, an extra sensor element that is not exposed to light during image capture operations. Similarly, some cameras attempt to compensate for offset noise by subtracting a predetermined value from an analog to digital converter output.
With conventional dark current compensation schemes, a full frame exposure in the dark condition is normally used to provide for dark current compensation. This increases the capture time thereby resulting in slower back-to-back captures. Allowing for dark frame capture may also result in an additional shutter opening and closing cycle. Extra opening and closing cycles increase battery power consumption in some situations. In addition, multiple shutter opening and closing sounds per capture may cause operator confusion.
Thus, there is a continuing need for better ways for reducing dark current noise.
In accordance with one aspect, a method for reducing dark current noise in an image sensor includes measuring the forward voltage of a diode on the integrated circuit including the image sensor. An image is captured using the sensor. The forward voltage measurement is used to correct the image sensor for dark current noise.
FIG. 1 shows a prior art digital imaging system;
FIG. 2 shows a digital camera in accordance with one embodiment of the invention;
FIG. 3 shows one embodiment of the imager of FIG. 2;
FIG. 4 shows one embodiment of the signal conditioning unit of FIG. 3;
FIG. 5 shows a schematic of an active pixel sensor in accordance with one embodiment of the invention;
FIG. 6 is the flow for developing dark current reference information during camera assembly time; and
FIG. 7 is a flow chart for software for making a dark current correction at image capture time.
Techniques to reduce the effect of dark current noise in an imaging system are described. While the following embodiments of this inventive concept are described in terms of a portable personal computer (PC) camera, they are illustrative only and are not to be considered limiting in any respect.
Referring to FIG. 2, a digital portable PC camera 200 in accordance with one embodiment of the invention may include optics unit 202 to focus an optical image onto the focal plane of imager 204. Image data (e.g., frames) generated by imager 204 may be transferred to a random access memory (RAM) 206 (through memory controller 208) or flash memory 210 (through memory controller 212) via the bus 214. In one embodiment of the invention, RAM 206 is a nonvolatile memory.
The camera 200 may also include a compression unit 216 that interacts with the imager 204 to compress the size of a generated frame before storing it in a camera memory (RAM 206 and/or flash memory 210). To transfer a frame of data to a computer, the camera 200 may include a serial bus interface 218 to couple the camera memory (RAM 206 and flash memory 210) to a serial bus 220. One illustrative serial bus is the universal serial bus (USB).
The camera 200 may also include a processor 222 coupled to a bus 214 via a bus interface unit 224. In some embodiments, the processor 222 interacts with the imager 204 to adjust image capture parameters to reduce the effect of dark current noise of the captured image or frame.
Referring to FIG. 3, the imager 204 may include a rectangular grid or array 300 of pixel sensors 302. This arrangement allows column and row decoders, 304 and 306 respectively, to selectively retrieve indications from the sensors 302. In one embodiment, the sensor array 300 is a 768×576 array of complementary metal oxide semiconductor (CMOS) active pixel sensors.
Decoders 304 and 306 route the selected indications to a signal conditioning circuit 308 which may, among other functions, amplify and digitize the received signals. The signal conditioning circuit 308 may also furnish the resultant data signals to an output interface 310 which includes circuitry for interfacing the imager 204 to the bus 214. Control unit 312, through circuitry such as state machines and timers, may coordinate and control the scanning (e.g., selection by row and column decoders 306 and 308) of pixel sensor 302 indications, their subsequent processing by signal conditioning circuit 308, and their transmission to other elements of the camera 200 through an output interface 310.
Camera operations may include normal image capture and calibration. During normal image capture, each pixel sensor 302 accumulates light energy from that portion of an image that is focused on it by optics 202 for a period of time referred to as the integration time or interval. At the expiration of the integration interval, pixel sensors 302 indicate the intensity of the received light energy by, for example, an analog voltage signal. Control circuit 312 routes the pixel sensor indications through column and row decoders 304 and 306 to the signal conditioning circuit 308 where they may be amplified and digitized to form a frame—digital data signals representing the captured image. A frame may be compressed by compression unit 216 and transmitted to memory (e.g., RAM memory 206 or flash memory 210), and/or a computer system via the serial bus interface 218 and serial bus 220.
In general, during calibration camera parameters may be set. Example parameters include pixel sensor integration time, pixel sensor signal gain, and illuminant determination. Calibration may include taking measurements and making settings both in the camera manufacturing facility and during normal use of the camera.
Referring to FIG. 4, a pixel sensor output signal 400 may be amplified (by one or more gain units 402) and digitized (by one or more analog to digital converters 404) by signal conditioning circuit 308 to generate a digital signal 406. In a dark capture environment, such as during calibration, the pixel sensor output signal 400 represents pixel sensor noise (e.g., dark current noise).
In a calibration mode, a measure of the temperature of a silicon diode on the same integrated circuit as the imager 204 may be used to compensate for dark current noise. The diode 410 may be any diode which is available for use during a calibration mode. Advantageously, the diode 410 is also used for another non-calibration function. For example, an embedded flicker diode for the image sensor may be used as the diode 410 in one embodiment of the present invention. Alternatively, where a separate non-arrayed photodiode is not available, an electrostatic discharge (ESD) protection diode from an otherwise momentarily unused input or output pin of the image sensor may be utilized as the temperature sensing diode 410 for calibration.
By measuring the forward voltage of the diode 410, the image sensor junction temperature may be obtained. The voltage drop across the diode 410 varies inversely with its absolute temperature. Since dark current is a function of temperature, temperature based dark current correction information can be obtained by measuring the voltage across the diode 410.
For most image sensors the temperature is isothermal across the whole die. Therefore, even reading a temperature near an edge of the die, for example using a device pin ESD structure, can provide an accurate measure of junction temperature for the whole imaging array. The analog to digital converter 404 used to digitize the analog voltages from the sensor array may also be used during calibration to digitize the measure of the forward voltage of the diode 410.
The voltage on the node 414 may be adjusted by the gain control circuit 412 and then provided to the gain circuit 402. A current source 408 drives the diode 410.
At camera assembly time, an initial dark frame, called a reference frame, may be taken and stored in non-volatile memory such as the FLASH memory 210 as indicated at 600 in FIG. 6. At the same time, the forward voltage of the diode 410 is measured and that digitized voltage is saved in non-volatile memory (block 602). The reference gain and the reference integration time used for the dark current reference frame are also stored in the non-volatile memory, as illustrated by block 604.
Referring to FIG. 7, a software routine 700 may control the calibration sequence. Whenever the camera is utilized to capture an image, metering determines what gain and exposure time should be used for the actual scene frame. Those parameters along with a new measure of the digitized forward voltage of the reference diode 410 form the scalars for synthesizing a dark frame (block 702).
To synthesize a dark frame that matches the scene frame conditions, each pixel of the reference dark frame may be multiplied by a correction factor. The correction factor may be equal to the product of the values A, B and C. A is the ratio of the actual scene gain to the reference gain, as indicated in block 704. B is the ratio of the scene capture and reference integration times (block 706). C is the difference between actual forward voltage measured at scene capture time and the reference forward diode voltage, then converted to temperature, and finally divided by the dark current doubling temperature of the process used to make the image sensor (block 708). Each final pixel value is determined by multiplying A, B and C times the stored reference pixel value (block 710).
Referring to FIG. 5, the pixel sensor 302 may include a transistor 500, a sample node 502, a photosensitive element such as photodiode 504, a transistor 506, a storage node 508, a storage capacitor 510, transistors 512 and 514, output capacitor 516, and an output node 518. Signals RESET 520, SAMPLE 522, and READ 524 control the operation of sensor element 302 and may be generated by control unit 312 as a result of its own operation or at the bequest of, for example, the processor 222. Transistors 500, 506, 512, and 514 may, in one embodiment, be n-channel CMOS field effect transistors. The voltage Vcc may be a positive supply voltage such as, for example, 3.3 volts.
Prior to capturing an image, sample node 502 may be initialized with a positive initialization voltage (Vcc) by briefly asserting (driving high, for example) RESET signal 520. After a specified time, RESET signal 520 is deasserted (driven low, for example) and SAMPLE signal 522 is asserted to initiate pixel sensor 302 integration. Asserting SAMPLE signal 522 activates transistor 506, selectively coupling sample node 502 to storage node 508, allowing storage capacitor 510 to accumulate charge from sample node 502.
During normal capture operations, part of the accumulated charge may represent sensor noise (e.g., dark current), and part of the accumulated charge may represent the image being captured. During dark image capture operations in the calibration phase, the accumulated charge typically represents sensor noise.
To transfer the analog voltage at the storage node 508 (following deassertion of SAMPLE signal 522) to the output node 518, the transistor 514 may be activated by READ signal 524 (READ signal 524 may be generated by row decoder 306). Because the transistor 512 is arranged in a common source configuration, the voltage signal at storage node 508 is coupled to output node 518 when READ signal 524 is asserted (i.e., when transistor 514 is activated).
In some embodiments of the present invention, an accurate dark frame may be created at image taking time without having to actually run a full frame exposure in the dark condition. This may save valuable click to capture time, thus allowing faster back-to-back captures. Also the number of shutter opening and closing cycles may be reduced, thereby saving battery power. By having only one shutter opening and closing for capture, there is less chance of confusing the user with multiple clicks or multiple shutter cycles.
Various changes in the materials, components, circuit elements, as well as in the details of the illustrated operational method are possible without departing from the scope of the claims. For instance, elements of the illustrative camera of FIG. 2 may be embodied in discrete logic elements, or combined into one or more application specific integrated circuits (ASIC). Further, the invention is not limited to portable PC cameras, but may be used in any digital imaging system. Storage devices suitable for tangibly embodying program instructions include all forms of non-volatile memory including, but not limited to: semiconductor memory devices such as non-volatile RAM (e.g., memory 206), EPROM, EEPROM, and flash devices (e.g., memory 210); magnetic disks (fixed, floppy, and removable); other magnetic media such as tape; and optical media such as CD-ROM disks.
While the present invention has been described with respect to a limited number of embodiments, those skilled in the art will appreciate numerous modifications and variations therefrom. It is intended that the appended claims cover all such modifications and variations as fall within the true spirit and scope of this present invention.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US4194220 *||Mar 27, 1978||Mar 18, 1980||Ball Corporation||Leakage current compensating circuit for semiconductor image sensor|
|US4551762 *||Jan 18, 1984||Nov 5, 1985||Rca Corporation||Dark-current level regulation in solid-state devices|
|US4785353 *||Mar 13, 1987||Nov 15, 1988||Eastman Kodak Company||Block readable video imager having noise compensation|
|US5086343 *||May 11, 1990||Feb 4, 1992||Eastman Kodak Company||Method and apparatus for compensating for sensitivity variations in the output of a solid state image sensor|
|US5113246 *||Sep 24, 1990||May 12, 1992||Canon Kabushiki Kaisha||Image sensing apparatus for correcting a defect in color signals generated by a CCD|
|US5508740 *||Jul 21, 1994||Apr 16, 1996||Hamamatsu Photonics K.K.||Solid-state imaging device having temperature sensor|
|US5604781 *||Feb 8, 1996||Feb 18, 1997||J. Morita Manufacturing Corporation||Medical X-ray imaging apparatus|
|US5880777 *||Apr 15, 1996||Mar 9, 1999||Massachusetts Institute Of Technology||Low-light-level imaging and image processing|
|US6160282 *||Apr 21, 1998||Dec 12, 2000||Foveon, Inc.||CMOS image sensor employing silicide exclusion mask to reduce leakage and improve performance|
|US6184511 *||Apr 14, 1999||Feb 6, 2001||Canon Kabushiki Kaisha||Photoelectric conversion apparatus|
|US6188093 *||Sep 2, 1998||Feb 13, 2001||Nikon Corporation||Photoelectric conversion devices and photoelectric conversion apparatus employing the same|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7102672 *||Feb 8, 2002||Sep 5, 2006||Electro Optical Sciences Inc||Integrated CMOS imaging array dark current monitor|
|US7136100 *||Aug 16, 2000||Nov 14, 2006||Casio Computer Co., Ltd.||Electronic still camera capable of removing noise component in image data and signal processing method thereof|
|US7140766 *||Aug 19, 2003||Nov 28, 2006||Given Imaging Ltd.||Device, system and method for temperature sensing in an in-vivo device|
|US7330208||Mar 8, 2004||Feb 12, 2008||Eastman Kodak Company||Electronic imaging system with adjusted dark floor correction|
|US7349574||Oct 14, 2003||Mar 25, 2008||Sensata Technologies, Inc.||System and method for processing non-linear image data from a digital imager|
|US7813586||Aug 7, 2006||Oct 12, 2010||Mela Sciences, Inc.||Reducing noise in digital images|
|US7907192||Dec 20, 2007||Mar 15, 2011||Eastman Kodak Company||Electronic imaging system with adjusted dark floor correction|
|US8081837 *||Feb 7, 2006||Dec 20, 2011||Intel Corporation||Image sensor array leakage and dark current compensation|
|US8160386||Oct 6, 2010||Apr 17, 2012||Mela Sciences, Inc.||Reducing noise in digital images|
|US8630508||Mar 1, 2012||Jan 14, 2014||Mela Sciences, Inc.||Reducing noise in digital images|
|US20040109488 *||Aug 19, 2003||Jun 10, 2004||Arkady Glukhovsky||Device, system and method for temperature sensing in an in-vivo device|
|US20050195296 *||Mar 8, 2004||Sep 8, 2005||Eastman Kodak Company||Electronic imaging system with adjusted dark floor correction|
|US20060209201 *||Mar 15, 2005||Sep 21, 2006||Spears Kurt E||Charge coupled device|
|US20070195179 *||Feb 7, 2006||Aug 23, 2007||Glenn Robert C||Image sensor array leakage and dark current compensation|
|US20070273775 *||May 24, 2006||Nov 29, 2007||Jutao Jiang||Image sensor with built-in thermometer for global black level calibration and temperature-dependent color correction|
|US20080031537 *||Aug 7, 2006||Feb 7, 2008||Dina Gutkowicz-Krusin||Reducing noise in digital images|
|US20080094490 *||Dec 20, 2007||Apr 24, 2008||Compton John T||Electronic imaging system with adjusted dark floor correction|
|US20080239111 *||Mar 26, 2007||Oct 2, 2008||Micron Technology, Inc.||Method and appratus for dark current compensation of imaging sensors|
|US20110019888 *||Oct 6, 2010||Jan 27, 2011||MELA SCIENCES, Inc., a Delaware corporation||Reducing noise in digital images|
|DE102012011886A1||Jun 15, 2012||Dec 19, 2013||Connaught Electronics Ltd.||Method for operating camera of motor vehicle, involves detecting image of surrounding area of motor vehicle by image sensor of camera, where image processing algorithm is executed on basis of image by image processing unit|
|WO2005088959A1 *||Mar 7, 2005||Sep 22, 2005||Eastman Kodak Co||Imaging system with adjusted dark floor correction|
|U.S. Classification||358/483, 348/E05.081, 358/482|
|International Classification||H04N5/374, H04N5/361, H04N5/378, H04N5/228|
|Jun 30, 1999||AS||Assignment|
Owner name: INTEL CORPORATION, CALIFORNIA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MINEMIER, RONALD K.;REEL/FRAME:010073/0805
Effective date: 19990602
|Feb 26, 2007||FPAY||Fee payment|
Year of fee payment: 4
|Feb 24, 2011||FPAY||Fee payment|
Year of fee payment: 8
|Apr 10, 2015||REMI||Maintenance fee reminder mailed|
|Sep 2, 2015||LAPS||Lapse for failure to pay maintenance fees|
|Oct 20, 2015||FP||Expired due to failure to pay maintenance fee|
Effective date: 20150902