US 7612801 B2
Digital cameras are calibrated so that detailed highlights and shadows in scenes being photographed are reproduced in digital images of those scenes by calibrating holographic data obtained from target images at those scenes. In one embodiment the target image is obtained from an actual target having black and white, and optionally gray, target areas; and in another embodiment, the target image is obtained from the scene being photographed. In both embodiments, histographic data is manipulated so that shadow and highlight data spikes are automatically placed in boundary areas by automatically adjusting lens aperture and/or shutter speed prior to taking the desired photograph of the scene. The resulting photograph is adjusted for highlight and shadow details.
1. An arrangement for calibrating a digital camera so that a digital derived image of a scene being photographed has the same highlights and shadows as the scene, the arrangement comprising:
a target image having light and shadow areas;
within the camera, a generator of brightness data and a histographic data organizer for organizing the brightness data into histographic data, the histographic data organizer having an initial brightness output of uncalibrated shadow and highlight data spikes, wherein the data spikes are not necessarily located at brightness boundaries of the camera, the data organizer further having calibrated data output with the shadow and highlight data spikes located at the brightness boundaries;
a brightness data comparator within the camera and connected to the histogram data organizer for comparing the calibrated highlight and shadow data spikes to the initial highlight and shadow data spikes to generate a difference signal, and
a camera adjustment driver connected between the brightness data comparator and a camera lens control for adjusting settings of lens aperture or lens shutter speed or both aperture and shutter speed until the difference signal indicates substantial correspondence between the initial histogram data boundary and the calibrated histogram data boundary to thereby set the histogram to the calibrated setting, whereby the image being photographed has at least substantially the same highlights and shadows as the scene being photographed.
2. The arrangement of
3. The arrangement of
4. The arrangement of
5. A method of calibrating a digital camera so that an image of a scene being digitally photographed has at least substantially the same highlights and shadows as the scene itself, the method comprising:
photographing a target image having highlight and shadow areas;
generating from the photograph of the target image a highlight data spike and an initial shadow data spike, the initial data spikes being positioned at initial positions with respect to highlight and shadow data boundaries;
generating a calibrated highlight data spike and a calibrated shadow data spike, the calibrated data spikes being positioned in the highlight and shadow data boundaries,
comparing the positions of the initial data spike to that of the calibrated data spikes with respect to the highlight and shadow data boundaries;
if the positions are different, generating a difference signal, and
using the difference signal to adjust lens aperture or shutter speed or both lens aperture and shutter speed so that there is substantial correspondence between the initial histogram data boundaries and the calibrated histogram data boundaries, whereby the digitally photographed image has at least substantially the same highlights and shadows as the scene itself.
6. The method of
7. The method of
8. The method of
This application is a continuation-in-part of U.S. patent application Ser. No. 10/636,738, filed Aug. 8, 2003 now U.S. Pat. No. 7,248,284 which claims the benefit of U.S. Provisional Application Ser. No. 60/402,603 filed Aug. 12, 2002 and incorporated herein in its entirety by reference.
The present invention is directed to calibration methods and apparatus for digital cameras. More particularly, the present invention is directed to calibration methods and apparatus for selecting exposure settings and achieving white balance in digital cameras.
In contrast with the digital photography, film photography traditionally has an exposure tolerance of two to four stops. An original scene exposed within this tolerance range will retain a good quality tonal curve due to the film, which is the recording medium, having a light sensitivity range which exceeds the tonal extremes in an average scene of two to four stops. Exposure for digital photography has a very narrow range, so that when there is over exposure or under exposure, part of the information is lost. Consequently, unless you initially achieve the desired exposure you make an inferior image file, which will not be in full detail either in highlight areas or shadow areas. There is no way to retrieve these lost details for the image file. Consequently, in digital photography techniques have evolved to fix images after they have been taken. This is a time consuming and relatively expensive undertaking in which it is still very difficult to compensate for information missing from an image.
Inside digital cameras there is a sensor on which a latent light impression of an original scene is made. Based on latent information from the sensor, a central processing unit in the camera processes the information into a proper color spectrum and into a proper color curve that retains the detail from highlight to shadow. The quality of image produced using the latent light impression frequently has diminished quality due to exposure error.
All sensors inside cameras have an optimal sensitivity setting (ISO), whether it is due to the sensitivity of film or to the sensitivity of image sensors in digital cameras. Sensitivity has an optimal range where it produces the absolute best image file in terms of color fidelity with the least image defects. In producing an optimal image file, the image file will be given a rated ISO, e.g., 100. Digital cameras have the capability of shooting other ISOs, but as one deviates from the optimum image quality suffers. When deviating from the optimal ISO, noise is introduced into the image files from the CPU and arbitrary abnormalities known as artifacts become visually apparent and the quality and color degrade. Thus, producing optimal digital image files is difficult for the professional photographer and extremely difficult for the consuming public.
Hand held light meters do not adequately compensate for inaccuracies in exposure because tolerances are typically plus or minus a half stop of exposure. Typically light meters select a middle tone, the placement of which varies from one manufacturer to another. Since light meters peg the middle of the tonal curve, light meters select gray rather than the black and white extremes. Pegging the middle of the tonal curve can result in the photographer loosing information at one or the other extreme so that light metering does not work effectively. This forces camera manufactures to develop methods to fix latent information. In order to compensate for inadequate latent information, camera manufacturers provide you with software solutions for manipulating improperly exposed and color-balanced images. But, these “back end fixes” almost invariably produce inferior image files with which before you can even start to produce a print, require very labor intensive efforts having three times the amount of work to process an image file. Accordingly, there is a need for a technique to correctly set exposure and color balance on the front end, i.e. prior to recording an image.
Obtaining correct exposure is part of the problem, the other part being correcting white balance. Most digital cameras provide different options for white balance correction which may be automatic or set by the photographer. Such settings are ballpark settings based on daylight, flash, overcast sky, tungsten filament lighting or fluorescent lighting. Daylight varies depending on the time of day with the color temperature being different at morning, midday and afternoon. There are also differences in white balance due to brightness of tungsten light bulbs because brightness determines color temperature. Since current white balance settings for digital cameras are quite inaccurate, there is a need for improvement.
In view of the aforementioned considerations, in digital photography there is need for improvement in the ability to select correct or desired exposure settings, as well as a need for improving the ability to correct or select desired white balance settings.
The present invention is related to calibration methods and apparatus for digital cameras performed prior to exposing an image of an original scene to an image sensor within the camera by exposing the image sensor to calibration information provided at or by the original scene.
In one embodiment of the calibration, use of a digital calibration target involves adjusting the exposure of digital cameras to retain maximum detail by referencing the brightest important highlight detail (white area of the target) and the darkest important shadow detail (black area of the target) instead of the traditional metering method which references 18% gray. Additionally by ensuring the digital calibration target panels are absolutely neutral (without color bias), the target allows the adjustment of the camera's white point to be set accurately in virtually all lighting situations.
In another embodiment of the calibration, the target is referenced at the location of an original scene and the camera automatically adjusts the exposure and white balance based on the information recorded on the camera's sensor without the photographer's intervention.
In still another embodiment of the calibration, the incorporated information provided by the target is in the scene itself, which information provides important highlight and shadow detail and adjusts automatically so that the need to record an external target is eliminated.
In another embodiment of the invention a calibration target having a substantially white target area that reflects substantially all wavelengths of visible light and a substantially black target area that absorbs substantially all wavelengths of visible light when focused upon, achieves an exposure setting for the digital camera. The exposure setting is then adjusted for the target by a controller in the camera until an adjusted exposure setting for black and white detection is substantially balanced within the camera for a selected intensity and distribution of light at the original scene. The adjusted exposure setting is then applied to at least one image photographed at the scene.
In a further aspect of the invention, at least one substantially gray target area is provided and exposed simultaneously with exposure to the black and white target areas.
In still a further aspect of the invention, the adjusted exposure setting for an exposure balanced image is exposed to a substantially neutral, white or gray target area without color bias that reflects equally substantially all wavelengths of light to produce settings for a color balanced, calibrated image having a selected white balance for the original scene.
In still another aspect of the invention, adjusting the exposure is performed by viewing a histogram display within the digital camera and automatically adjusting the exposure settings until spikes representing black and white detection are not offset laterally with respect to a histogram function.
In still another aspect of the invention, an initial exposure setting is obtained of the location prior to exposing the image sensor to the calibration target. The initial exposure setting is then automatically adjusted without the photographic intervention by subsequent exposures to the calibration target to achieve correct or desired exposure settings.
In a further aspect of the invention, the target further includes at least one middle gray target area, which reflects substantially all wavelengths of light without color bias.
In still a further aspect of the invention, the target includes a neutral white or gray target area without color bias, which neutral area when positioned at locations of scenes prior to recording images of the scenes, determines settings for desired color balances for the images.
In a further aspect of the invention, the target configuration is a panel having black and white target areas on a first side and the neutral target area on a second side of the panel.
The invention further relates to methods of digital photography utilizing the aforedescribed calibration methods and calibration targets.
Various other features and attendant advantages of the present invention will be more fully appreciated as the same becomes better understood when considered in conjunction with the accompanying drawings, in which like reference characters designate the same or similar parts throughout the several views, and wherein:
Referring now to
The scene 11 has shadows 18 and highlights 20. The shadows 18 and highlights 20 may be definite and readily apparent, or may be subtle. For any subject 12, the shadows and highlight are usually intermingled, varied and complex.
The subject 12 and background 11 also has colors and tones, which may vary widely and interact with highlights and shadows to produce a visual image of the original scene 11. A digital camera 24 having a lens focuses the scene 11 on image sensors in the camera that record light electronically. Typically, in digital cameras the image sensors are charged coupled devices (CCD) or complimentary metal oxide semiconductor (CMOS) devices. A computer device in the form of a central processing unit (CPU) converts electrical charges from the image sensors into digital data, which is then stored as digitalized image information in the camera's memory.
As is seen in
As is seen in
The black and white target areas 42 an 44 are neutral in that they contain no color cast, the black target area 42 being substantially pure black and the white target area 44 being substantially pure white. The target 40 is not limited to specific size constraints or materials provided that it's reflective qualities are pure black and pure white.
Referring now to
Preferably in practicing the method of the invention, an initial exposure is made using the camera's internal light meter or a handheld light meter 22 to provide initial exposure settings (see
A second exposure (
While a histogram display 50 is used in accordance with one embodiment of the invention, the characteristics of the histogram display are sensed electronically in another embodiment of the invention (not illustrated) and centered automatically by the CPU of the camera 24. In this alternative embodiment values assigned to the locations of black and white spikes 52 and 54 are compared to values assigned to the boundaries 56 and 58. After exposing the image sensors in the camera 24 initially to the camera's internal light meter or a hand held light meter 22 to obtain values for initial exposure settings corresponding to the positions of values for spikes 52 and 54, the camera is focused on the calibration target 40 and another exposure made. The value for the settings for this exposure are then compared to the values for the boundaries 56 and 58. If the calibration target values fall between the values for the boundaries 56 and 58, the exposure settings are calibrated. If not, an additional exposure is made or several additional exposures are made, until the values for the black and white spikes 52 and 54 do fall between the values for the boundaries 56 and 58.
The additional exposures may be done manually or automatically using a sequence of exposures with the histogram display simply informing the photographer visually or audibly that the exposure settings are correct.
While an initial exposure based on light meter readings would appear desirable, when using an automatic sequence of exposures this step could be either dispensed with or incorporated into the sequence, relying preferably on the camera's internal light meter.
Referring now to
While not its primary purpose or design, because the calibration target 40 or 70 contains known values for highlight, mid tone and shadow, it can be useful for back end correction when included in an original scene. While this is a desirable feature, it is primarily a convenience so that a photographer does not have to carry multiple adjustment tools in case a mistake is made and backend corrections are necessary.
A single gray target area 76 is shown in
Referring now to
Prior to exposing the image of an original scene 11, the menu on the digital camera 24 is accessed to select a custom white balance function illustrated in
As long as the settings for exposure are correct, as obtained by using the calibration targets 40 and 70 of
Referring now more specifically to
As is seen in
The histogram display 50 of
This is accomplished by automating the camera 25 to use exposure control arrangements (already available in the art) but which are according to this invention driven by the histographic data of
Referring now to
This histographic data, which is configured as the spikes may or may not be transmitted to a histogram display, such as display 50 of
The initial histogram data 321 of the scene prior to calibration for brightness is shown in
When and if this match occurs, optionally a signal is sent from the brightness data comparator 316 to a visual or audible indicator 322 indicating to the photographer that brightness detected by the camera 25 is properly calibrated for highlight and shadow.
Referring now to
In the embodiment of
The pattern recognition comparator 356 has outputs 310 and 312 for the data spikes.
From the foregoing description, one skilled in the art can easily ascertain the essential characteristics of this invention, and without departing from the spirit and scope thereof, can make various changes and modifications of the invention to adapt it to various usages and conditions.