US 20020028025 A1 Abstract A spatio-temporal joint filter and a spatial joint filter for noise reduction are disclosed. The spatio-temporal joint filter includes a spatial joint filter including the first and second sub filters having different characteristics and includes a temporal joint filter. When the present invention is adequately used, an edge/detail region of an image is well preserved, an aggressive noise reduction is performed on a flat region, and the temporal flicker problems are eliminated. Additionally, it has an intrinsic motion compensation effect by using the spatio-temporal correlation between the adjacent frames.
Claims(20) 1. A spatial joint filter for noise reduction comprising:
a first filter determining a first weighted average of support region pixel signals corresponding to an input signal using first weighting factors, each of said first weighting factors depending on said input signal and each of said support region pixel signals; and a second filter determining a second weighted sum of said input signal and said first weighted average using a second weighting factor that depends on an original signal variance and a noise variance. 2. The spatial joint filter of 3. The spatial joint filter of where
{overscore (ω)}(l,m,t) is each first weighting factor,
g(l,m,t) is each support region pixel signal, and
S represents a support region of said input signal.
4. The spatial joint filter of where α=1 and ε=2σ
_{n} ^{2}, and g(i,j,t) is said input signal. 5. The spatial joint filter of where c is a predetermined limiting factor, and g(i,j,t) is said input signal.
6. The spatial joint filter of _{s}(i,j,t) is determined using the following equation:f _{s}(i,j,t)=α(i,j,t)g(i,j,t)+[1−α(i,j,t)]g1(i,j,t)and
where
g(i,j,t) is said input signal;
g1(i,j,t) is said first weighted average;
α(i,j,t) is said second weighting factor;
σ
_{f} ^{2}(i,j,t) is said original signal variance; and σ
_{n} ^{2 }is said noise variance. 7. The spatial joint filter of _{f} ^{2}(i,j,t) is determined using the following equation: where σ
_{g} ^{2}(i,j,t) is a local variance of said input signal g(i,j,t). 8. The spatial joint filter of _{g} ^{2}(i,j,t) is determined using the following equation: where S represents a support region of said input signal g(i,j,t).
9. A spatio-temporal joint filter for noise reduction comprising:
a spatial joint filter including
a first filter determining a first weighted average of current support region signals of an input signal using first weighting factors, said current support region signals being included in a current frame, and
a second filter determining a second weighted sum of said input signal and said first weighted average using a second weighting factor that depends on an original signal variance and a noise variance;
a temporal joint filter including
a third filter determining a third weighted average of said second weighted sum and previous support region signals of said input signal using third weighting factors, said previous support region signals being included in a previous frame, and
a forth filter determining a forth weighted sum of said second weighted sum and said third weighted average using a forth weighting factor that depends on a spatio-temporal signal variance and said noise variance; and
a frame memory storing said forth weighted sum and feedbacking said forth weighted sum to said third filter for a next frame. 10. The spatio-temporal joint filter of where
{overscore (ω)}(l,m,t) is each first weighting factor,
g(l,m,t) is each current support region signal, and
S represents a support region of an image.
11. The spatio-temporal joint filter of where α=1 and ε=2σ
_{n} ^{2}, and g(i,j,t) is said input signal. 12. The spatio-temporal joint filter of Where c represents a predetermined limiting factor, and g(i,j,t) is said input signal.
13. The spatio-temporal joint filter of _{s}(i,j,t) is determined using the following equation:f _{s}(i,j,t)=α(i,j,t)g(i,j,t)+[1−α(i,j,t)]g1(i,j,t) and
where
g(i,j,t) is said input signal,
g1(i,j,t) is said first weighted average,
α(i,j,t) is said second weighting factor,
σ
_{f} ^{2}(i,j,t) is said original signal variance, and σ
_{n} ^{2 }is said noise variance. 14. The spatio-temporal joint filter of _{f} ^{2}(i,j,t) is determined using the following equation:σ
_{f} ^{2}(i,j,t)=max[σ _{g} ^{2}(i,j,t)−σ_{n} ^{2},0]where σ
_{g} ^{2}(i,j,t) is a local variance of said input signal g(i,j,t). 15. The spatio-temporal joint filter of _{g} ^{2}(i,j,t) is determined using the following equation: where S represents a support region of an image of said input signal g(i,j,t).
16. The spatio-temporal joint filter of _{T}(i,j,t) is determined using the following equation: where
S represents a support region of an image,
f
_{s}(i,j,t) is said second weighted sum, {overscore (ω)}(l,m,t−1) is each third weighting factor, and
f
_{o}(l,m,t−1) is a previous forth weighted sum included in said support region of said previous frame. 17. The spatio-temporal joint filter of where α=1 and ε=2σ
_{n} ^{2}. 18. The spatio-temporal joint filter of Where c represents a predetermined limiting factor.
19. The spatio-temporal joint filter of _{o}(i,j,t) is determined using the following equation: where
f
_{s}(i,j,t) is set second weighted sum, μ
_{T}(i,j,t) is said third weighted average, σ
_{ST} ^{2}(i,j,t) is said spatio-temporal signal variance, and σ
_{n} ^{2 }is a noise variance. 20. The spatio-temporal joint filter of where
σ
_{f} ^{2}(i,j,t) is a original signal variance, and f
_{o}(l,m,t−1) is a previous forth weighted sum included in said support region of said previous frame.Description [0001] 1. Field of the Invention [0002] The present invention relates to noise reduction filters eliminating a noise signal included in an image signal, and more particularly, to a spatio-temporal and spatial joint noise reduction filters that combines more than one filters having various filter characteristics. [0003] 2. Background of the Related Art [0004] In general, video signals initially received from a video camera and transmitted through a channel unavoidably contain noise signals. A noise signal added to a video signal provides a degraded image to viewers and has negative effects on the image signal processes. Therefore, signal receivers or any other necessary devices have a noise reduction process to avoid such problems. Some of the objectives need to be taken into consideration are as follows. [0005] (1) Noise elimination: [0006] A noise signal should be eliminated so that a clear image can be provided. Particularly, an aggressive noise reduction control is necessary especially for a flat region of an image. [0007] (2) Preservation of edge/detail region: [0008] Unfortunately, some of the edge or detail region of an image may be removed when an every region of the image goes through the aggressive noise reduction process. Therefore, a reasonable degree of noise reduction needs be carefully determined in order to preserve the edge or detail region of the image. [0009] (3) Temporal flickers: [0010] A noise signal sometimes causes temporal flickers to occur between adjacent frames. This also degrades the image quality. [0011] The first and second objectives have a relationship that one can be achieved at the expense of the other. In other words, an aggressive noise reduction performed to achieve the first objective will result in losing some of the edge/detail region of an image. On the other hand, an insufficient degree of noise reduction for achieving the second object will not satisfactorily remove the noise signal for the first objective. [0012] In the signal processing literature, several filters have been reported for image noise suppression and image detail preservation. They are A-LMMSE (Adaptive Linear Minimum Mean Squared Error), AWA (Adaptive Weighted Average), and A-MEAN filters. The related articles are listed below: [0013] D. T. Kuan, A. A. Sawchuk, T. C. Strand, and P. Chavel, “Adaptive Noise Smoothing Filter For Images With Signal-Dependent Noise”, [0014] M. K. Ozkan, M. I. Sezan, and A. M. Tekalp, “Adaptive Motion-Compensated Filtering of Noisy Image Sequences”, [0015] Carlos Polamaza-Raez and Clare D. McGillem, “Digital Phase-Locked Loop Behavior with Clock and Sampler Quantization”, [0016] According to the above mentioned references, a noise-contaminated input pixel signal g(i,j,t), whose horizontal, vertical, and time locations are (i,j,t), can be expressed as [0017] where f(i,j,t) and n(i,j,t) represent an original signal and a noise signal, respectively. Therefore, a performance of a noise reduction filter depends on how accurately f(i,j,t) can be extracted from g(i,j,t). Particularly, a noise signal included in the flat region of an image is easily visible by viewers, but the noise included in the detail/edge region is not. This is called a masking effect. A-LMMSE, AWA, and A-MEAN filters may vary the level of noise reduction based on the characteristics of the noise-contaminated signal. [0018] The A-LMMSE filter, which is originated from a linear predictor, obtains its estimated original signal {circumflex over (f)}(i,j,t) by using
[0019] where σ [0020] In addition, the AWA and A-MEAN filters perform noise reduction using a weighted average, and each weighting factor {overscore (ω)}(l,m,t) is determined based on a difference between an input pixel signal g(i,j,t) and a pixel signal included in the support region g(l,m,t) as shown in Equation 3,4 and 5, where the (l,m,t) corresponds to a support region of the filter.
[0021] Each weighting factor {overscore (ω)}(l,m,t) of the AWA filter is given by
[0022] where α and ε are usually set to α=1 and ε=2σ [0023] where c represents a predetermined limiting factor. According to Equation (5), each weighting factor of the A-MEAN filter is set to one if the difference between g(i,j,t) and g(l,m,t) is less than or equal to c and becomes zero if the difference is larger than c . In general, c is set to c=3σ [0024] In conclusion, the AWA and A-MEAN filters having the characteristics different from the A-LMMSE vary the level of noise reduction based on the characteristics of an image. For example, an aggressive noise reduction is performed on the flat region of an image to eliminate the noise signal, and a moderate noise reduction is used for edge/detail preservation. However, they include the structural limitations due to their two dimensional characteristics and do not eliminate the temporal flicker mentioned earlier. If the support regions of those filters are expanded to three dimensions, the noise reduction performance may be deteriorated, resulting the blurred images. In order to avoid such problems, methods for noise reduction using motion information between image frames such as a motion adaptive noise reducer are introduced. The motion adaptive noise reducer has a reasonably good noise reduction performance while preserving a detail region of an image. However, it introduces other problems. First, the noise reduction performance of the motion adaptive noise reducer depends the accuracy of motion information. However, the noise included in the image makes it difficult to obtain the accurate information. Second, the result of the noise reduction performed on a region having motions is not satisfactory. Especially, the noise included in a region having slow motions needs to be eliminated. [0025] Accordingly, the present invention is directed to a joint spatio-temporal noise-reduction filter that substantially obviates one or more problems due to limitations and disadvantages of the related art. [0026] An object of the present invention is to provide a joint spatio-temporal noise-reduction filter that preserves edge/detail regions of an image, performs a aggressive noise reduction on flat regions, and eliminates temporal flickers occurred between adjacent image frames by combining more than one noise reduction filters having various characteristics. [0027] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings. [0028] To achieve these objects and other advantages and in accordance with the purpose of the invention, as embodied and broadly described herein, a spatial joint filter for noise reduction includes a first filter determining a first weighted average of support region pixel signals corresponding to an input signal using first weighting factors, each of the first weighting factors depending on the input signal and each of the support region pixel signals; and a second filter determining a second weighted sum of the input signal and the first weighted average using a second weighting factor that depends on an original signal variance and a noise variance. [0029] In another aspect of the present invention, a spatial-temporal joint filter for noise reduction includes a spatial joint filter, a temporal joint filter and a frame memory. The spatial joint filter includes a first filter determining a first weighted average of current support region signals of an input signal using first weighting factors, the current support region signals being included in a current frame; and a second filter determining a second weighted sum of the input signal and the first weighted average using a second weighting factor that depends on a original signal variance and a noise variance. The temporal joint filter includes a third filter determining a third weighted average of the second weighted sum and previous support region signals of said input signal using a third weighting factors, the previous support region signals being included in a previous frame; and a forth filter determining a forth weighted sum of the second weighted sum and the third weighted average using a forth weighting factor that depends on a spatio-temporal signal variance and the noise variance. The frame memory stores the forth weighted sum of the current frame and feedbacks the forth weighted sum to the third filter for noise estimation of a next frame. [0030] It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. [0031] The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principle of the invention. In the drawings; [0032]FIG. 1 illustrates a brief structure of a spatial joint noise-reduction filter according to the present invention; [0033]FIG. 2 illustrates a two-dimensional joint spatial noise reduction filter according to the present invention; [0034]FIG. 3 illustrates a spatio-temporal noise reduction filter according to the present invention; and [0035]FIG. 4 illustrates an infinite impulse response (IIR) type spatio-temporal joint noise reduction filtering steps according to the present invention. [0036] Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings. [0037] The present invention relates to a joint spatio-temporal noise-reduction filter that combines an AWA (or A-MEAN) filter characteristic together with an A-LMMSE filter characteristic. FIG. 1 illustrates a brief structure of a spatial joint noise-reduction filter according to the present invention. It includes a first filter ( [0038] The complexity and performance of the joint filter according to the present invention depend on the characteristic and arrangement of each filter being used. FIG. 2 illustrates a two-dimensional joint spatial noise reduction filter according to the present invention where an A_MEAN or AWA filter is used as a first filter, and an A-LMMSE filter is used as a second filter. According to FIG. 2, the A-LMMSE filter being used as the second filter uses the output (g1) from the first filter for an improved edge/detail preserving property. The A-LMMSE filter ( [0039] where the original signal variance σ σ [0040] where σ [0041] The spatial joint noise reduction filter shown in FIGS. 1 and 2 may not have the desired noise reduction property while reasonably preserving the edge/detail region. This problem can be eliminated by using a spatio-temporal joint noise reduction filter that combines a spatial joint filter with a temporal joint filter. In other words, the spatio-temporal noise reduction filter simultaneously can have an effective noise reduction property and preservation of edge/detail regions while eliminating the temporal flickers. [0042]FIG. 3 illustrates the spatio-temporal noise reduction filter according to the present invention. It includes a spatial joint noise reduction filter ( [0043]FIG. 4 illustrates an infinite impulse response (IIR) type spatio-temporal joint noise reduction filtering steps according to the present invention. The spatial joint noise reduction filter ( [0044] where f [0045] Equation (10) [0046] for AWA filter:
[0047] The second filter [0048] where σ [0049] If there is a motion in a region having a large spatial difference such as an edge or detail, σ [0050] Additionally, if there is no motion involved, the variance of the original signal f(i,j,t) in the support region becomes σ [0051] In conclusion, for a region having large spatio-temporal changes (i.e., an edge/detail region having a motion), σ [0052] According to the spatio-temporal joint three dimensional noise reduction filter shown above, the edge/detail preservation, the noise reduction for the flat region and flicker elimination properties are enhanced by combining a spatial joint noise reduction filter and a temporal joint reduction filter, each including two sub filters. Particularly, IIR type spatio-temporal noise reduction filter is used for performing a more aggressive noise reduction on the flat region of the image. It does not involve direct motion estimation, but has an implicit motion compensation effect by using spatio-temporal correlations between the adjacent frames. [0053] The forgoing embodiments are merely exemplary and are not to be construed as limiting the present invention. The present teachings can be readily applied to other types of apparatuses. The description of the present invention is intended to be illustrative, and not to limit the scope of the claims. Many alternatives, modifications, and variations will be apparent to those skilled in the art. Referenced by
Classifications
Legal Events
Rotate |