US 7391925 B2 Abstract A system and method for estimating noise using measurement based parametric fitting non-uniformity correction is disclosed. Fixed pattern noise (“FPN”) is estimating from an overall noise component within a detection system to enhance candidate target detection and tracking. A sensor in the detection system receives energy, such as radiant flux, that is converted to a digital image. A non-uniformity correction device generates an estimated FPN according to an applicable temperate range and integration time. A memory storing an array of coefficients is accessed to determine the estimated FPN. The valves within the array of coefficients are based on actual FPN measurements that are parametrically fitted.
Claims(25) 1. A system for reducing noise in a detection sensor detection, comprising:
a raw digital image of pixels corresponding to energy received at the sensor;
a non-uniformity correction device to remove estimated fixed pattern noise from the pixels of the raw digital image to generate a corrected digital image; and
an array of coefficients to determine the estimated fixed pattern noise, wherein the array of coefficients are based on actual fixed pattern noise measurements that are parametrically fitted over a plurality of temperature ranges, wherein said corrected digital image is generated based on a low estimated fixed pattern noise when a current frame of said raw digital image is less than a middle temperature intensity count, otherwise the corrected digital image is generated based on a high estimated fixed pattern noise.
2. The system of
3. The system of
4. The system of
5. The system of
6. The system of
7. The system of
8. The system of
9. The system of
10. The system of
11. A sensor system for detecting candidate targets from received energy at an array of detectors within the sensor system, comprising:
integration capacitors to control an integration time for the array of detectors to generate a voltage corresponding to the received energy;
an analog-to-digital converter to convert the voltage to a raw digital image having pixel data of the candidate targets;
a non-uniformity correction device to estimate the fixed pattern noise using an array of measurement-based parametrically fitted coefficients corresponding to a temperature range for the sensor system and to remove the estimated fixed pattern noise from the raw digital image, and
a corrected image generated by the non-uniformity correction device that emphasizes the candidate targets in the pixel data, wherein said corrected digital image is generated based on a low estimated fixed pattern noise when a current frame of said raw digital image is less than a middle temperature intensity count, otherwise the corrected digital image is generated based on a high estimated fixed pattern noise.
12. The system of
13. The system of
14. The system of
15. A method for reducing noise in a sensor, comprising:
converting received energy into a raw digital image;
estimating fixed pattern noise in the raw digital image by using an array of coefficients of parametrically fitted measurements of actual fixed pattern noise over a temperature range of a plurality of temperature ranges, and
generating a corrected digital image by removing the estimated fixed pattern noise from the raw digital image, wherein said corrected digital image is generated based on a low estimated fixed pattern noise when a current frame of said raw digital image is less than a middle temperature intensity count, otherwise the corrected digital image is generated based on a high estimated fixed pattern noise.
16. The method of
17. The method of
18. The method of
19. The method of
20. A method for reducing noise in a digital image corresponding to energy received at a sensor, comprising:
estimating fixed pattern noise in the digital image using an array of coefficients for a temperature range of a plurality of temperature ranges, wherein the array of coefficients represent a gain and an offset of the fixed pattern noise, and
removing the estimated fixed pattern noise from the digital image to generate a corrected digital image, wherein said corrected digital image is generated based on a low estimated fixed pattern noise when a current frame of the digital image is less than a middle temperature intensity count, otherwise the corrected digital image is generated based on a high estimated fixed pattern noise.
21. The method of
22. The method of
23. The method of
24. A computer program product comprising a computer useable medium having computer readable code embodied therein for reducing noise in a sensor, the computer program product adapted when run on a computer to effect steps including:
converting received energy into a raw digital image;
estimating fixed pattern noise in the raw digital image by using an array of coefficients of parametrically fitted measurements of actual fixed pattern noise over a temperature range of a plurality of temperature ranges, and
generating a corrected digital image by removing the estimated fixed pattern noise from the raw digital image, wherein said corrected digital image is generated based on a low estimated fixed pattern noise when a current frame of said raw digital image is less than a middle temperature intensity count, otherwise the corrected digital image is generated based on a high estimated fixed pattern noise.
25. A computer program product comprising a computer useable medium having computer readable code embodied therein for reducing noise in a digital image corresponding to energy received at a sensor, the computer program product adapted when run on a computer to effect steps including:
estimating fixed pattern noise in the digital image using an array of coefficients for a temperature range of a plurality of temperature ranges, wherein the array of coefficients represent a gain and an offset of the fixed pattern noise, and
removing the estimated fixed pattern noise from the digital image to generate a corrected digital image, wherein said corrected digital image is generated based on a low estimated fixed pattern noise when a current frame of the digital image is less than a middle temperature intensity count, otherwise the Corrected digital image is generated based on a high estimated fixed pattern noise.
Description The present invention relates to a non-uniform correction system and method to estimate fixed pattern noise from the total noise in a candidate target detection, identification and tracking system. More particularly, the present invention relates to a non-uniform correction system and method to provide higher sensor sensitivity by estimating fixed pattern noise using measurement based parametric fitting non-uniform correction. Fixed pattern noise (“FPN”) is a component of the overall total noise in a candidate target detection and tracking system. For example, an infrared (“IR”) sensor on a missile can have FPN in addition to other noise. A high count of FPN reduces a detection sensor's sensitivity and hampers target tracking and identification due to a large noise component. In typical sensor applications, the standard deviation of FPN from a raw, uncorrected image can be as high as 300-400 counts. A count corresponds to the IR energy received at the IR detectors. A count can be proportional to the output voltage from the detectors for the received IR energy. For example, a weak potential target can have 10 to 20 counts. Temporal noise can be 1 to 2 counts, varying according to the outside temperature. A high count of FPN prevents detection of weak targets. One purpose of any candidate target detection and tracking system is to identify candidate targets as early as possible. A non-uniformity correction (“NUC”) system reduces FPN to allow early target detection and reliable target tracking and recognition/identification. Traditional NUC systems seek to reduce the FPN to around or below the temporal noise (“TN”) level. Known NUC systems incorporate a rotating “chopper-wheel,” or a blurring/deform lens, to separate the outside scene and the inside FPN on the focal plane array (“FPA”) of the sensor. A chopper wheel system rotates the blurring lens to remove the FPN from the total noise component. The chopper wheel system uses a motor to rotate the lens across the FPA. The hardware and software components to implement a chopper wheel with a lens and motor takes up space within the missile or detection device and adds complexity and cost. As detection systems get smaller, space can become a critical constraint on future designs for missiles, aircraft, and the like. Other known NUC systems include scene-based NUC. Scene-based NUC systems use dithering to reduce the FPN. One-pixel level FPA dithering movement is difficult to control within scene-based NUC systems, and the rate of reducing the FPN is slow. Scene-based NUC systems can use a convergent median filter, but this change results in the rate of reducing the FPN being even slower. Other NUC systems use estimated FPN components to remove the FPN from a received frame. Estimating FPN over the temperature ranges needed for today's applications and platforms is complex. Offset and gain values related to the received flux energy can vary greatly, over different temperature ranges. The estimation algorithms include high-order polynomials that result in complex processing loads on the detection sensor. Removal or reduction of the FPN in a detection sensor is needed to maximize potential target identification, to detect targets early and to maintain tracking of an acquired target. Known NUC systems that estimate and reduce FPN, however, tend to be slow or not feasible due to their size. Accordingly, the present invention is directed to a system and method for a measurement based parametric fitting NUC device that solves the deficiencies and shortcomings of the related art. The present invention discloses the use of measurement-based parametric fitting to estimate the FPN from the sensor without the need for complex processing or high-order polynomial algorithms. Additional features and advantages of the embodiments of the present invention are set forth in the detailed description that follows. The additional features and advantages are apparent from the description, or can be learned by practice of the invention. The objectives and other advantages of the embodiments of the present invention are realized and attained by the disclosure particularly pointed out in the written description and claims, as well as the drawings. To achieve these and other advantages, a system for estimating noise in a detection sensor is disclosed. The system can use a digital image of pixels corresponding to energy received at the sensor. The system also includes a non-uniformity correction device to remove estimated fixed pattern noise from the pixels of the digital image to generate a corrected digital image. The system also includes a memory having an array of coefficients to determine the estimated fixed pattern noise. The array of coefficients are based on actual fixed pattern noise measurements that are parametrically fitted over a plurality of temperature ranges. Further, according to embodiments of the present invention, a method for estimating noise in a sensor is disclosed. The method includes converting received energy into a digital image. The method also includes estimating fixed pattern noise in the digital image by using an array of coefficients of parametrically fitted measurements of actual fixed pattern noise over a temperature range of a plurality of temperature ranges. It is understood that both the foregoing general description and the following detailed description are exemplary and explanatory, and are intended to provide further explanation of the invention as claimed below. The accompanying drawings that are included to provide further understanding of the disclosed invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosed invention and together with the description serve to explain the principles of the invention. In the drawings: Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings. System Other applications for system System Radiant flux System Exposure adjustor Output voltage Amplified voltage DIM NUC component Estimator Memory In a preferred embodiment of the present invention, memory Estimated FPN Preferably, NUC component Function DRM function In conventional NUC components using a chopper wheel, a dynamic range management function frequently switches the integration times to keep the pixel intensity at the middle-intensity-counts to avoid pixel starvation and saturation situations. This switching can result in many different integration times and gains, sometimes changing for every frame of received radiant flux For a fixed integration time, the gain and offset coefficients used to estimate the FPN, such as estimated FPN The appropriate integration times and background temperatures can vary according to any number of factors, including requirements or specifications of system Preferably, in the example, four integration times, 2.5, 5.2, 9.0 and 16.6 milliseconds are selected for DRM function Component NUC For example, an integration time equal to about 16.6 milliseconds is preferable for the sub-dynamic range of −33° C. to 0° C. The integration time of 9 milliseconds is preferable for the sub-dynamic range of 0° C. to 35° C. The integration time of 5.2 milliseconds is preferable for the sub-dynamic range 35° C. to 70° C. The integration time for 2.5 milliseconds is preferable for the situation when the target temperature is equal to or above 70° C. Function Measurement based parametric fitting NUC component The estimated FPN can be represented according to Wold's fundamental theorem, which states any stationary discrete-time stochastic process {x(n)} may be expressed as
where frame number n=1, 2, 3, . . . ; u(n) and s(n) are uncorrelated processes; u(n) is a random variable; and s(n) is a deterministic process. If both u(n) and s(n) are zero-mean variables, then
where var stands for variance. Thus, the variance of the stationary discrete-time stochastic process is represented by the sum of the variance of the random variable and the deterministic process. The single frame noise (“SNF”), or total noise, is a stationary discrete-time stochastic process. Using Wold's theorem disclosed above, the single frame noise can be represented by the sum of the temporal noise (“TN”) (a random process) and the fixed pattern noise, or FPN, (a deterministic process), or
In general, the TN and FPN are estimated as their noise standard deviation (std), and thus the quantity of FPN is estimated as:
From Eq. (2), if the SFN and TN are measurable, then the FPN can be estimated. The standard deviation of SFN is calculated from the analog to digital output images, disclosed by DIM The variance of difference of two nearby images is expressed as:
In Eq. (3), the relationship is established that FPN(n)=FPN(n−1). During a short time period, such as between two nearby frames, this relationship is reasonable for a near deterministic process. Therefore, the TN can be estimated as:
In general, the FPN is a deterministic process subject to constant system configuration and environmental conditions, such as the system power-on time, integration time, background temperature, and the like. To quantify the FPN change and any drift amount, a measurement of residual noise (RN) is introduced and defined as:
where, M is a fixed earlier time frame, and n is a later time frame. The later time frame can be minutes, hours, days, or weeks later. The background temperatures and integration times are different at time M and time (n+M). By comparing Eq. (5) with Eq. (4), if there is no FPN drift between frame M and n+M, then std(RN(n))=std(TN(n)) for a stationary temporal noise. Otherwise, if std(RN(n))≠std(TN(n)), the difference std(RTN(n))−std(TN(n)) indicates the FPN drift. Thus, the FPN drift is represented by the difference between the standard deviation of the residual noise and the standard deviation of the temporal noise within a frame, n. The FPN drift is defined as the change or difference in the FPN value at different times. The FPN drift could be mischaracterized as a signal change if not accounted for. As noted above, the empirical measurements preferably are performed at different times. The FPN drift can be caused by variations or problems in the pixels or FPA detector Thus, the FPN is caused by the variation of the gain and offset coefficients of the signal transfer functions in different pixel detectors. In NUC component Using the example disclosed above, for a dynamic range of −33° C. to 70° C., the FPN is measured for each pixel at 38 different temperatures within the range. Standard deviation of the FPN is plotted at different pixel image intensity counts for 100 pixels with the focal plane array integration time of about 5.2 milliseconds. At this integration time condition, an image count of 3,000 is stimulated by a black body temperature of 5° C., and an image count of 12,000 is stimulated by block body temperature of 70° C. The FPN to image count relationship is linear and any curves are fitted using a second order linear polynomial such as y=Ax+B where A and B are the second gain and offset correcting coefficient rates. The disclosed embodiments of the present invention also measure the any applicable FPN drifts using black body measurements at different days under the exactly same testing conditions, such as temperatures, integration times and the like. Drifts can be defined as any changes in the FPN that occurs at different times under otherwise similar conditions. The residual noise caused by two different days is estimated and plotted. The standard deviation and the residual noise are calculated. A reference frame M is the first frame on the first day with all frames being collected for that day. Frames for the second day are collected and a mean standard deviation for the random noise on the first day is calculated and the mean standard deviation of the random noise on the second day is calculated. Therefore, the FPN drift is the difference between the standard deviation of the residual noises for both days. Large portions of the drifts can be caused by a few deviant pixels. Deviant pixels are located from the measured data. Therefore, the FPN drifts are considerably reduced by mapping the deviant pixels out and replacing them with the average value of the surrounding pixels. Thus, the FPN drift is reduced by eliminating the deviant pixels. If the system gain is increased, however, to adjust the sensor sensitivity, the dynamic range requirement of system Referring back to the above-disclosed example, an integration time approximately equal to 2.5 milliseconds is selected when a very hot target, such as over 70° C., is detected to avoid target saturation. An integration time approximately equal to 2.5 milliseconds also is selected when the platform establishes conventional tracking and/or target recognition, or when the platform is in an end-game mode, such as prior to hitting a tracked target. In contrast, some conventional NUC components have the NUC turned off during end-game mode because the system sensitivity is a problem. A purpose of NUC, however, is to reduce high count FPN to enhance target tracking and recognition. By selecting a small integration time, the disclosed embodiments of the present invention avoid strong target saturation, and continue to operate NUC function Preferably, the disclosed embodiments of the present invention include dynamic ranges having selected integration times. According to the disclosed example, an integration time approximately equal to 2.5 milliseconds can be selected for target temperatures greater than 70° C. An integration time approximately equal to 5.2 milliseconds can be selected for background temperatures of 35° C. to 70° C. An integration time approximately equal to 9 milliseconds can be selected for background temperatures of 0 to 35° C. An integration time approximately equal to 16.6 milliseconds can be selected for background temperatures of −33 to 0° C. Alternatively, different integration times can be selected for these temperature ranges. Further, different temperature ranges can be implemented for different integration times. The disclosed embodiments in the present invention are not limited by the example integration times and example ranges given above. Thus, according to the disclosed embodiments, the focal plane array sensitivity can be estimated for times ranging from 5.2 milliseconds to 16.6 milliseconds under different background temperatures from the collected laboratory measurements. Based on the empirical measurement data, the focal plane array noise equivalent delta temperatures are calculated. The noise equivalent delta temperatures then are calculated for system The measured focal plane array image data is used to estimate the gain and offset coefficients for the one-piece and two-piece linear equations disclosed below. These equations are the curved fittings for the FPN-counts relationship used by function The one-piece linear equation can be applicable over a certain temperature range, while a two-piece linear equation is more applicable over another temperature range. Two-piece linear equations are more complex than one-piece linear equations and are used when the curve fitting requires a more complex representation. Thus, the present invention preferably seeks to fit a one-piece linear equation to the laboratory measurements, but can go to two-piece curve-fitting if desired. For the low and very high sub-dynamic ranges, a one-piece linear curve-fitting algorithm is applied. The FPN estimation algorithm is:
For the middle and high sub-dynamic ranges, a two-piece piece-wise linear curve-fitting algorithm is applied. This algorithm differs from the one-piece linear curve-fitting algorithm disclosed above. The FPN estimation algorithm for the middle and high sub-dynamic ranges is: The corrected frame relationship is shown as: The temperature range of −33° to 70° C. is divided into sub-dynamic ranges to provide greater flexibility in determining the offset and gain coefficients for the linear equation. The offset and gain for a pixel intensity value differs over the entire temperature range. By dividing into sub-ranges, the offset and gain relationship is kept linear over the entire temperature range. Thus, the estimated FPN is determined and can be used for further operations, such as reducing the actual FPN for increased target recognition and tracking. By removing the estimated FPN, the actual FPN is suppressed to below or equal to the temporal noise level for the temperatures in the temperature dynamic range for seek and detect systems, such as system Further, the NUC component, such as NUC component The trade-off is that the disclosed embodiments of the present invention can store four two-dimensional coefficient matrices instead of two. Alternatively, the one-piece algorithm or the two-piece algorithm can be used for all temperature ranges, wherein only two two-dimensional coefficient matrices are stored. These coefficient matrices are used by the FPN estimators, such as estimators Further, the disclosed NUC component, such as NUC component The embodiments of the present invention disclose improvements over known NUC components, such as those using a chopper wheel configuration. For example, the disclosed embodiments of the present invention use measurement based parametric fitting that can be implemented in software configurations. Thus, the chopper wheel, its motor driver, and control electronics are eliminated. Further, the disclosed embodiments of the present invention can fit within a smaller space because it can be implemented in software and do not use additional mechanical or electrical components. The disclosed embodiments of the present invention also perform at an increased FPN-reducing rate with no frame delay. The coefficients for estimating the FPNs are calculated using empirical measurements, and are pre-stored in the NUC component according to the disclosed embodiments. The FPN is corrected immediately when the NUC component acquires the image frame. Conventional chopper wheel NUC components subtract only a fraction of the FPN at each frame to avoid an increase of the temporal noise within the frame. For example, known chopper wheel NUC component can take 10 to 40 frames to suppress the FPN to a low value, depending on how much temporal noise increase is allowable. The disclosed embodiments of the present invention also provide high correction accuracy without increasing temporal noise. The estimation of the FPN according to the disclosed embodiments of the present invention is based on the average of the multiple image frame sequence, which is about equal to or less than 600 frames. The estimated FPN coefficients are almost noise-free. Therefore, the temporal noise should not be increased when estimating the FPN from the uncorrected image in the seek and detect system, such as that disclosed by Another feature of the disclosed embodiments of the present invention is that the disclosed NUC component uniformly corrects other measurement and design errors, such as hot-dome shading, cosine It will be apparent to those skilled in the art that various modifications and variation can be made in the disclosed embodiments of the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention covers the modification and variations of this invention provided that they are encompassed by the scope of any claims and their equivalents. Patent Citations
Classifications
Legal Events
Rotate |