|Publication number||US4725881 A|
|Application number||US 06/840,573|
|Publication date||Feb 16, 1988|
|Filing date||May 3, 1985|
|Priority date||May 19, 1984|
|Also published as||DE3418787A1, DE3418787C2, WO1985005526A1|
|Publication number||06840573, 840573, PCT/1985/142, PCT/DE/1985/000142, PCT/DE/1985/00142, PCT/DE/85/000142, PCT/DE/85/00142, PCT/DE1985/000142, PCT/DE1985/00142, PCT/DE1985000142, PCT/DE198500142, PCT/DE85/000142, PCT/DE85/00142, PCT/DE85000142, PCT/DE8500142, US 4725881 A, US 4725881A, US-A-4725881, US4725881 A, US4725881A|
|Original Assignee||Robert Bosch Gmbh|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (4), Referenced by (37), Classifications (7), Legal Events (6)|
|External Links: USPTO, USPTO Assignment, Espacenet|
The invention is based on a process for increasing the resolution of color television cameras of the type which includes three semiconductor-image sensors, each sensor including an array of optoelectric sensing elements spaced apart one from the other in horizontal and vertical directions by an equal distance, means for scanning line-by-line said sensing elements in the respective sensors to produce output signals corresponding to three different colors, the position of a first sensor being offset in the horizontal direction by one third of the equal distance relative to a second sensor and by two third of the equal distance relative to a third sensor, the scanning means including means for generating three scanning signals applied to the respective sensors, a scanning signal for the second sensor being phase shifted by 120░ and a scanning signal for the third sensor by 240░ relative to a scanning signal for the first sensor, means for matrixing the output signals from the respective sensors to generate a standard luminance signal and standard color-difference signals, and means for low-pass filtering the standard signals.
In contrast to video camera tubes, the semiconductor image sensors represent a scanning system wherein the illuminated optical information is scanned in two dimensions by discrete image spots. The resolution of the generated video signal is therefore calculated in accordance with the scanning theorem, namely directly by the number of image spots to be realized on the image sensor.
Therefore, an increase in the resolution is only possible by a technically very problematic increase of the number in image spots of the semiconductor-image sensor.
However, after an accurate testing of the signal frequency spectrum at the output of a semiconductor-image sensor possibilities are available to increase the resolution by a suitable signal processing without technological measures. The frequency spectrum is the one of a pulse amplitude modulated signal which consists of a component in the base band as well as components which are grouped by a multiple of a carrier frequency. The actual limiting of the resolution in accordance with the scanning theorem results from the fact that the carrier frequency signal components generate signal components in the base band which falsify the base band signal. However, if it is possible to eliminate the carrier frequency components a higher resolution is made possible, even with a limited amount of image spots.
A solid state television camera with a plurality of semiconductor-image sensors is already known from West German Offenlegungsschrift 28 46 869, wherein the resolution of the television camera can be also improved by means of a suitable geometric disposition of the semiconductor-image sensors and corresponding signal processing. However, the proposed signal processing serves, primarily for shifting the phase of the output signals of the image sensors in accordance with their horizontal offsetting with respect to each other and this circuit is very expensive.
It is therefore an object of the invention to provide a process for increasing the resolution of television cameras with semiconductor-image sensors, wherein the signal processing can be made simpler and performed more effectively.
The process in accordance with the invention includes the steps of the uniformly weighting and adding said output signals from the respective sensors to generate an additional luminance signal; high-pass filtering said additional luminance signal, adding the high-pass filtered additional luminance signal to said standard luminance signal to produce a resultant luminance signal; and deriving from said high-pass filtered luminance signal a control signal for said standard color-difference signals. The process is advantageous in that it is not only less expensive but any interfering components which still may be present can be practically completely eliminated.
Particularly advantageous is the use of a synchronous demodulator for shunting off the control signal from the high pass filtered additional luminance signal, whereby it is assured that real color being present is not suppressed at high luminance frequencies, while during a black/white oscillation a false chrominance from alias frequencies is prevented.
One exemplified embodiment for performing the process in accordance with the invention is illustrated in the drawings and is explained in more detail in the subsequent description, whereby only these parts are illustrated which are important for the invention. The drawings show:
FIG. 1 a block circuit diagram for performing the process in accordance with the invention,
FIG. 1a a block circuit diagram of a part of the color matrix illustrated in FIG. 1,
FIG. 2 frequency spectra of the video signals shown in FIG. 1,
FIG. 3 spectra and time sequences of the signals in a color or a black/white format.
FIG. 1 illustrates a part of a color television camera with three semiconductor-image sensors 1,2,3 including corresponding color filters (not illustrated) for generating a green signal G, a red signal R and a blue signal B of a single image focused on the respective image sensors 1,2,3. Each of the three image sensors 1,2,3 includes an array of discrete image spots, whose geometric period or equal distance in horizontal and vertical directions is τH. Three output signals G,R and B are phase-shifted due to a horizontal displacement of the individual image sensors 1,2,3 against each other by a third of the image spot period (τH /3), whereby base band components of the same phase and carrier frequency components of the first and second order which are mutually phase shifted by 120░, are picked up on the terminals 4,5,6. Thereby, the reading or scanning cycle control is performed by three reading or scanning cycle frequencies which are phase shifted with respect to each other by 120░ and which are generated by a synchronizing generator 7, whereby these reading cycle frequencies are fed to the image sensors 1,2,3 at control inputs 8,9,10.
The color signals G,R,B from the terminals 4,5,6 are fed to a color matrix unit 11 in which, in a known manner, a standard luminance signal Y=0.30R+0.59G+0.11B as well as standard color difference signals R-Y and B-Y are generated. These standard signals Y and R-Y or B-Y, whose scanning spectra are illustrated in FIGS. 2a and 2d, are fed through corresponding low pass filters 12,13,14.
The before described offset position of the three semiconductor-image sensors 1,2,3 in the horizontal direction by which the obtainable resolution, as known, is triple of that of the individual image sensor, can be used to convert the luminance component of the color signal into a modified luminance signal having a higher resolution. This presumes that the output signals R,G,B of the three semiconductor-image sensors must be uniformly weighted and added in accordance with Y'=0.33R+0.33G+0.33B, so as to generate an additional luminance component. This addition is performed in the color matrix unit 11 by means of the circuit illustrated in FIG. 1a. The color signals G,R,B which are present at the output terminals 4,5,6 are fed through assigned emitter follower stages 24,25,26 and through resistors 27,28,29 having the same value (for example 1 kΩ). The resistors 27,28,29 are connected to one input of a difference amplifier 30, whose other input is connected with ground by means of a resistor 31. The additional luminance signal Y' is then picked up at the output 32. The branching of respective signals as indicated by arrows 34,35,36 and 37,38,39 in FIG. 1a relates to the known further processing of these signals into standard color difference signals R-Y,B-Y and into a standard luminance signal Y. However, the additional luminance signal Y' generated in the color martrix unit 11 is incompatible with the standard color proportioning which requires a luminance matrixing in accordance with Y=0.30R+0.59G+0.11B. In order to satisfy the higher resolution, on the one hand, and also satisfy the color proportioning, on the other hand the additional luminance component Y' is fed through a high pass filter 15 and is added to the standard laminance signal to produce a resulting luminance signal Ys in an adding stage 16. As seen from graphs (a) and (b) in FIG. 2, the 6-dB limit frequency fp1 of the low-pass filtered standard luminance signal Y is equal to 6-dB limit frequency fp2 of the high-pass filtered additional luminance signal Y'.
The frequency spectra of the three signals Y,Y' and Ys are illustrated in FIGS. 2a, b, c. Due to the nonuniform weighting of R,G,B a so-called signal folding occurs in signal Y at the frequency range of all multiples of the reading or scanning frequency fA and the folding superimposes the effective signal (0 to fA). With a uniform weighting of R,G,B the additional luminance signal Y has the triple reading frequency with a corresponding correction of distortion of the partial spectra. The combination of Y and Y' signals from low pass/high pass filters 12,15 produces in adder 16 only a residual folding error (cross hatched portion in FIG. 2c), which is superimposed on the Ys signal up to the limit frequency fP1 of the low pass filter 12.
When recording pure black/white scene images an effect occurs whereby with the increase of the geometric local frequency due to the offset scanning (FIG. 2d), a color interference in the form of an alias frequency is folded to the given optical frequency around the reading or scanning frequency fA. Thereby, the luminance spectrum folds in the chrominance range around fA and 2fA, while no alias components are generated around 0 and 3fA. In order to keep these chrominance interferences very small the low pass filters 13 and 14 are required for the color difference signals R-Y and B-Y. Only one identical limit frequency fP1 of the low pass filters 12,13 and 14 is meaningful for color proportioning purposes. In addition, this eliminates the requirement of the delay or transit time adjustment between the luminance and chrominance signals.
The limit frequency fP1 of the low pass filters 12,13,14 must be smaller than half of the reading frequency fN (Nyquist frequency), since a luminance signal Y of this frequency results in a low frequency beat of the alias components generated in the luminance channel as well as in the chrominance channel around the reading frequency fA. In particular the chrominance folding lets a luminance component with Nyquist frequency fN (0.5fA) appear to be very strongly disturbed on a false color background.
Nonetheless, lower chrominance frequencies which can pass through the described low pass filters 13,14 of the color difference signals R-Y,B-Y, arrive with a luminance signal Y for example in the proximity of the reading frequency fA. As long as the difference frequency of the luminance signal and the scanning rate is smaller than the low pass limit frequency fP1, strong color patterns appear in the image, which cannot be differentiated in the chrominance channel from the genuine color information. Only information from the additional luminance signal Y' behind the high pass filter 15 permits a possibility to distinguish between the appearance of genuine or folded chrominance. Therefore, a control voltage is derived from the signal amplitude generated behind the high pass filter. The control voltage suppresses by means of controlled amplifiers 17 and 18 arranged in the color difference channels the possible alias frequencies in the chrominance signal during the appearances of luminance components above the low pass limit frequency fP1.
By using a synchronous modulator 19 being operated at the frequency fA for deriving the control voltage from the additional luminance signal Y' it is assured that with high luminance frequencies, which have an additional low frequency color background structure, the color contained in the real scene image is not suppressed, while in the case of a black/white oscillation of a uniform frequency erroneous chrominance resulting from alias frequencies is prevented.
This will be explained with the assistance of FIG. 3 in an example of an optical scene image which activates only the sensor for red with a frequency f0 in the proximity of the reading frequency fA. Since only one image sensor 2 takes part in the signal scanning the offset is ineffective, so that the additional luminance signal Y' as well as the standard luminance signal Y have a spectrum which reproduces at the reading frequencies fA (FIG. 3a). In this case, the additional luminance signal Y' contains behind the high pass filter contains 15 spectral components f0 and 2fA -f0 which together represent a beat fS. After the synchronous demodulation a difference oscillation f=fA -f0 of the same frequency as the resulting low frequency luminance component, is generated. An additional uniform chrominance value, which had been derived from the real scene, is fed to the controlled amplifiers 17,18 with a saturation modulation which is superimposed by the present luminance interference. It is important that the interfering luminance structures do not become even more apparent by the missing color. For example, in a red image all image details which represent higher signal frequencies would be decolored and have a light appearance.
However, if the same frequency f0 appears as a pure black/white signal (FIG. 3b), all three semiconductor-image sensors 1,2,3 are taking part in the optoelectrical conversion and scanning and can contribute to an alias improved Y'-signal. Instead of two frequencies fo and 2fA -fo as described in the preceding example only one component is contained at fo. This component also produces after the synchronous demodulation an oscillation at the frequency fA -fo, but only with half of an amplitude. This amplitude difference can be used as a decision making threshold for the controllable amplifiers 17,18, so as to let the chrominance pass in accordance with this example, while in another case it is recognized as an alias structure and is suppressed.
The described process may also be operated in a modified form in that only two of the three available image sensors are brought into a 2-phase offset, preferably for red and green. In this case a resolution increase by the factor of 2 is obtained for the additional luminance signal Y'. This solution is of interest because of the resulting interference distance, since the image sensor 3 for blue generally represents the least sensitive image sensor which is not used for generating the additional luminance signal Y'.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US4220977 *||Oct 24, 1978||Sep 2, 1980||Sony Corporation||Signal transmission circuit|
|US4249203 *||Sep 11, 1978||Feb 3, 1981||Sony Corporation||Solid state television camera|
|US4507677 *||Mar 18, 1983||Mar 26, 1985||Robert Bosch Gmbh||Method of improving the resolution of the semiconductor sensor type of television camera|
|DE2846869A1 *||Oct 27, 1978||May 10, 1979||Sony Corp||Signaluebertragungsschaltung|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US4805011 *||Sep 23, 1987||Feb 14, 1989||Sony Corporation||Luminance signal forming circuit for color tv camera which combines luminance signal derived from primary color signals with luminance signal derived from complementary color signals|
|US5119180 *||Dec 14, 1989||Jun 2, 1992||Fuji Photo Film Co., Ltd.||Image processing system with arbitrary and adjacent picture element signal load factoring|
|US5184212 *||Sep 6, 1991||Feb 2, 1993||Ikegami Tsushinki Co., Ltd.||Circuit for producing brightness signal from output signal of solid state image pick-up apparatus using spatial pixel shift|
|US5223921 *||Apr 24, 1991||Jun 29, 1993||Sanyo Electric Co., Ltd.||White balance adjusting apparatus for automatically adjusting white balance on the basis of a color information signal obtained from an image-sensing device|
|US5289269 *||Apr 3, 1992||Feb 22, 1994||Nippon Television Network Corporation||Color television camera with details of luminance signal formation|
|US5661522 *||Dec 14, 1995||Aug 26, 1997||Hitachi Denshi Kabushiki Kaisha||TV camera provided with solid image pick-up element|
|US6078307 *||Mar 12, 1998||Jun 20, 2000||Sharp Laboratories Of America, Inc.||Method for increasing luminance resolution of color panel display systems|
|US6842297||Aug 31, 2001||Jan 11, 2005||Cdm Optics, Inc.||Wavefront coding optics|
|US6873733||Jan 19, 2001||Mar 29, 2005||The Regents Of The University Of Colorado||Combined wavefront coding and amplitude contrast imaging systems|
|US6911638||Feb 11, 2003||Jun 28, 2005||The Regents Of The University Of Colorado, A Body Corporate||Wavefront coding zoom lens imaging systems|
|US6940649||Apr 4, 2003||Sep 6, 2005||The Regents Of The University Of Colorado||Wavefront coded imaging systems|
|US7025454||Mar 24, 2003||Apr 11, 2006||The Regants Of The University Of Colorado||Extended depth of field optics for human vision|
|US7115849||Jan 31, 2003||Oct 3, 2006||The Regents Of The University Of Colorado||Wavefront coding interference contrast imaging systems|
|US7218448||May 1, 1998||May 15, 2007||The Regents Of The University Of Colorado||Extended depth of field optical systems|
|US7436595||Jan 16, 2004||Oct 14, 2008||The Regents Of The University Of Colorado||Extended depth of field optical systems|
|US7554731||Jun 30, 2009||Omnivision Cdm Optics, Inc.||Wavefront coded imaging systems|
|US7554732||Aug 28, 2006||Jun 30, 2009||Omnivision Cdm Optics, Inc.||Wavefront coded imaging systems|
|US7583442||May 14, 2007||Sep 1, 2009||Omnivision Cdm Optics, Inc.||Extended depth of field optical systems|
|US7732750||Sep 1, 2006||Jun 8, 2010||The Regents Of The University Of Colorado||Wavefront coding interference contrast imaging systems|
|US7746519||Dec 1, 2006||Jun 29, 2010||Oce Printing Systems Gmbh||Method and device for scanning images|
|US7911515 *||Mar 22, 2011||Victor Company Of Japan, Ltd.||Imaging apparatus and method of processing video signal|
|US8004762||Aug 23, 2011||The Regents Of The University Of Colorado, A Body Corporate||Extended depth of field optical systems|
|US20030173502 *||Jan 31, 2003||Sep 18, 2003||Dowski Edward Raymond||Wavefront coding interference contrast imaging systems|
|US20030225455 *||Mar 24, 2003||Dec 4, 2003||Cathey Wade Thomas||Extended depth of field optics for human vision|
|US20040004766 *||Apr 4, 2003||Jan 8, 2004||Dowski Edward Raymond||Wavefront coded imaging systems|
|US20040145808 *||Jan 16, 2004||Jul 29, 2004||Cathey Wade Thomas||Extended depth of field optical systems|
|US20060017662 *||Nov 27, 2003||Jan 26, 2006||Koninklijke Philips Electronics N.V.||Method for improving the perceived resolution of a colour matrix display|
|US20060291058 *||Aug 28, 2006||Dec 28, 2006||Dowski Edward R Jr||Wavefront coded imaging systems|
|US20070001105 *||Sep 1, 2006||Jan 4, 2007||Dowski Edward R Jr||Wavefront coding interference contrast imaging systems|
|US20070076296 *||Aug 28, 2006||Apr 5, 2007||Dowski Edward R Jr||Wavefront coded imaging systems|
|US20080174869 *||May 14, 2007||Jul 24, 2008||Cdm Optics, Inc||Extended Depth Of Field Optical Systems|
|US20090079855 *||Apr 8, 2008||Mar 26, 2009||Victor Company Of Japan, Ltd.||Imaging apparatus and method of processing video signal|
|US20090109535 *||Oct 13, 2008||Apr 30, 2009||Cathey Jr Wade Thomas||Extended Depth Of Field Optical Systems|
|US20090153924 *||Dec 1, 2006||Jun 18, 2009||Bernhard Frei||Method and device for scanning images|
|US20140253734 *||Mar 4, 2014||Sep 11, 2014||Commissariat Ó l'Únergie atomique et aux Únergies alternatives||Image acquisition method and system|
|WO2001099431A2 *||Jun 13, 2001||Dec 27, 2001||University Technology Corporation||Method and apparatus for increasing resolution in digital imaging system by offsetting pixels|
|WO2001099431A3 *||Jun 13, 2001||Mar 28, 2002||Univ Technology Corp||Method and apparatus for increasing resolution in digital imaging system by offsetting pixels|
|U.S. Classification||348/237, 348/264, 348/E09.01|
|International Classification||H04N9/04, H04N9/09|
|Jan 17, 1986||AS||Assignment|
Owner name: ROBERT BOSCH GMBH, 7000 STUTTGART 1, GERMANY
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST.;ASSIGNOR:BUCHWALD, WOLF-PETER;REEL/FRAME:004528/0358
Effective date: 19860110
|Aug 7, 1991||FPAY||Fee payment|
Year of fee payment: 4
|Jul 28, 1995||FPAY||Fee payment|
Year of fee payment: 8
|Sep 7, 1999||REMI||Maintenance fee reminder mailed|
|Feb 13, 2000||LAPS||Lapse for failure to pay maintenance fees|
|Apr 25, 2000||FP||Expired due to failure to pay maintenance fee|
Effective date: 20000216