US RE42367 E1 Abstract A method for illumination-independent change detection in a pair of original registered gray images based on blob extraction. Blobs are extracted from the two original images and their negatives, using an enhanced blob extraction algorithm based on connectivity analysis along gray-levels. Blobs extracted from the first original and negative images are compared with blobs extracted from the second original and negative images to determine whether each blob has a corresponding blob, i.e. whether it is a matched or unmatched blob. All unmatched blobs are tested for significance as “blobs of change” using a fitness measure based on either a ratio of saliency gradients, or a product of this ratio and a gradient distribution measure. The disclosed method is exact, fast, robust, illumination-insensitive, and has low time-complexity
Claims(40) 1. A method for illumination-independent change detection between a pair of registered images, comprising:
a. providing a first original gray-level image, a second original gray-level images, a first negative image related to said first original image and a second negative image related to said second original image,
b. extracting by a computer respective pluralities of blobs from each of said first and second original images and each of said first and second negative images,
c. matching by a computer each extracted blob in said first original and negative images with each extracted blob in said second original and negative images to obtain matched and unmatched blobs, and
d. testing by a computer all said unmatched blobs to identify blobs of change.
2. The method of
i. extracting a first plurality of blobs from said first original image, a second plurality of blobs from said second original image, a third plurality of blobs from said first negative image and a fourth plurality of blobs from said second negative image,
ii. forming a first unified blob list by unifying said first plurality with said third plurality of blobs, and
iii. forming a second unified blob list by unifying said second plurality with said fourth plurality of blobs.
3. The method of
4. The method of
5. The method of
6. The method of
7. The method of
8. The method of
i. calculating a first and a second saliency in each of said first and second original images respectively,
ii. dividing said first saliency by said second saliency to obtain a saliency ratio,
iii. calculating a gradient distribution measure, and
iv. setting said fitness measure being to be equal to said saliency ratio if said saliency ratio is larger than or equal to said threshold, and setting said fitness measure to be equal to the product of said saliency ratio and said gradient distribution measure if said saliency ratio is smaller than said threshold.
9. The method of
10. The method of
11. A method for change detection in images, comprising:
a. providing a pair of first and second registered gray level images, wherein said step of providing further includes providing respective negative first and second images;
b. extracting by a computer respective first and second pluralities of blobs from each of said images using a modified connectivity along gray levels (CAG) analysis, wherein said step of extracting further includes extracting a third plurality of blobs from said negative first image and a fourth plurality of blobs from said second negative image;
c. locating by a computer at least one unmatched blob in at least one of said images, wherein said step of locating at least one unmatched blob further includes:
i. comparing each blob from said first and third pluralities of blobs with each blob from said second and fourth pluralities of blobs,
ii. locating corresponding blobs based on said comparison, and
iii. identifying said at least one unmatched blob based on a pixel overlap between each pair of said corresponding blobs; and
d. identifying by a computer at least one blob of change related to said images by applying a test on said at least one unmatched blob.
12. The method of
13. The method of
i. calculating a first and a second saliency in each of said first and second gray level images respectively,
ii. dividing said first saliency by said second saliency to obtain a saliency ratio,
iii. calculating a gradient distribution measure, and
iv. setting said fitness measure being to be equal to said saliency ratio if said saliency ratio is larger than or equal to a predetermined threshold, and setting said fitness measure to be equal to the product of said saliency ratio and said gradient distribution measure if said saliency ratio is smaller than said threshold.
14. The method of
15. The method of
16. A method for change detection between a pair of images, the method comprising:
extracting by a computer respective pluralities of blobs from each of a first gray-level image, a second gray-level image, a first negative image related to the first gray-level image, and a second negative image related to the second gray-level image, comparing by a computer extracted blobs in the first gray-level image and first negative image with extracted blobs in the second gray-level image and second negative image to obtain matched blobs and unmatched blobs, and testing by a computer a plurality of the unmatched blobs to identify blobs of change. 17. The method of claim 16, further comprising generating each of the first and second negative images.
18. The method of claim 16, further comprising outputting a list of the blobs of change.
19. The method of claim 16, wherein said testing further comprises calculating a fitness function of a first unmatched blob of the first gray-level and first negative images in a unified list of the extracted blobs from the second gray-level and second negative images.
20. The method of claim 16, wherein the first grey-level image and the second grey-level image comprise different illumination.
21. A method for change detection between a pair of images, the method comprising:
accessing by a computer a first gray-level image, a second gray-level image, a first negative image related to the first gray-level image and a second negative image related to the second gray-level image, and extracting by a computer respective pluralities of blobs from each of the first and second gray-level images and each of the first and second negative images. 22. The method of claim 21, wherein said extracting respective pluralities of blobs further comprises:
extracting a first plurality of blobs from the first gray-level image, a second plurality of blobs from the second gray-level image, a third plurality of blobs from the first negative image and a fourth plurality of blobs from the second negative image; generating a first unified blob list by unifying the first plurality of blobs with the third plurality of blobs; and generating a second unified blob list by unifying the second plurality of blobs with the fourth plurality of blobs. 23. The method of claim 22, further comprising identifying each blob in the first unified blob list that has a corresponding blob in the second unified blob list, and identifying each blob in the second unified blob list that has a corresponding blob in the first unified blob list.
24. The method of claim 21, wherein said extracting respective pluralities of blobs comprises using a modified connectivity along gray levels (CAG) analysis.
25. The method of claim 21, wherein said extracting respective pluralities of blobs comprises excluding from the first, second, third and fourth pluralities of blobs blobs having a pixel size less than a minimum pixel threshold or greater than a maximum pixel threshold.
26. A method for change detection between a pair of images, the method comprising:
comparing by a computer first extracted blobs in a first image and a first negative image related to the first image with second extracted blobs in a second image and a second negative image related to the second image to obtain matched blobs and unmatched blobs, and testing by a computer a plurality of the unmatched blobs to identify blobs of change. 27. The method of claim 26, wherein said comparing comprises determining if each of the first extracted blobs has a corresponding blob in the second extracted blobs and if each of the second extracted blobs has a corresponding blob in the first extracted blobs.
28. The method of claim 27, wherein said determining comprises determining if an overlap of pixels of each of the first extracted blobs with the corresponding blob exceeds a threshold amount.
29. The method of claim 28, wherein the threshold amount is approximately 75%.
30. The method of claim 28, wherein the threshold amount is approximately 90%.
31. The method of claim 28, wherein the plurality of unmatched blobs comprises certain first and second extracted blobs that do not have an overlap of pixels with any corresponding blob that exceeds the threshold amount.
32. The method of claim 31, wherein said testing the plurality of unmatched blobs comprises for each of the plurality of unmatched blobs:
calculating a fitness measure; and comparing the fitness measure with a second threshold amount. 33. The method of claim 32, wherein the fitness measure is based at least on a ratio of first and second saliency gradients associated with, respectively, the first and second images.
34. The method of claim 33, wherein the fitness measure is further based at least on a product of the ratio and a gradient distribution measure.
35. The method of claim 33, further comprising calculating the first and second saliency gradients using a contour-following algorithm.
36. The method of claim 35, further comprising assigning a weight to each pixel in the first and second images indicative of an edge saliency.
37. A system for detecting changes between a pair of images, the system comprising:
a processor configured to
extract a first plurality of blobs from a first gray-level image and a first negative image related to the first gray-level image,
extract a second plurality of blobs from the second gray-level image and a second negative image related to the second gray-level image,
compare each of the first plurality of blobs with each blob in the second plurality of blobs to identify matched blobs and unmatched blobs, and
test the unmatched blobs to identify blobs of change; and
an output configured to generate a list of the blobs of change. 38. The system of claim 37, wherein the matched blobs comprise:
identified ones of the first plurality of blobs that have a corresponding blob in the second plurality of blobs; and identified ones of the second plurality of blobs that have a corresponding blob in the first plurality of blobs. 39. The system of claim 38, wherein each of the matched blobs and its corresponding blob comprise a particular number of pixels that have the same coordinates.
40. The system of claim 38, wherein the processor is further configured to test the unmatched blobs by computing a fitness measure of each of:
the unmatched blobs of the first plurality of blobs in the second graylevel image; and the unmatched blobs of the second plurality of blobs in the first gray-level image. Description Change detection between a pair of digital images is used for automatic detection of new blobs appearing in a given scene. A blob is a connected component that has visible local saliency. An object includes one or more blobs. Therefore, in the context of the present invention blob extraction includes extraction of parts or the entire object. In other words, the term object extraction is also covered by the term “blob extraction” as used herein. The problem of change detection that is addressed herein is defined as follows: a pair of registered gray-level images of the same scene, with different illumination, where each image may contain blobs that are not contained in the other, is given as input. The set of all blobs that do not exist in both images is defined as the change. Prior knowledge about the image, its statistics, and the changes is not given. A typical example is demonstrated in It is important to emphasize that since non-constant (i.e. different) illumination is assumed between images, methods that are based on image subtraction will yield many false alarms. More sophisticated methods that are based on gray-level surface approximation may fail to detect changes that are caused because of small blobs, since surface approximations tend to smooth the boundaries of the blobs and reduce the ability to detect small blobs 1. Prior Art Methods Most of the prior art dealing with the problem of identification of change detection does not address the situation of existing non-constant (different) illumination between two images. Many of the prior art works focus on situations of change detection in video. Usually, the illumination in the two images is assumed to be identical. When the change detection in moving video is motivated by compression, the goal is to detect areas of change, and not blobs of change. An area of a change can include parts of a complete blob. The known techniques for change detection can be classified into the following categories:
All these methods are region based, in contrast with the method of the present invention, which is blob based. The regions are independent of the image content. It is important to note that none of the reviewed prior art methods use segmentation, since exact image segmentation of noisy scenes is very complicated, and it is still considered a difficult problem. Many of the reviewed prior art methods use polynomial approximation on the image surface. This is another drawback, since this approximation smoothes the image, which leads to less accurate results than a blob-based approach. 1.1. Pixel-Level Change Detection Change between two images or frames can be detected by comparing the differences in intensity values of corresponding pixels in the two frames. An algorithm counts the number of the changed pixels, and a camera break is declared if the percentage of the total number of pixels changed exceeds a certain threshold [R. Kasturi and R. Jain, “Dynamic Vision”, Computer Vision: Principles, Eds. R. Kasturi, R. Jain, IEEE Computer Society Press, Washington, pp. 469-480, 1991 (hereinafter KAS91); A. Nagasaka, and Y. Tanaka, “Automatic Video Indexing and Full-Video Search for Blob Appearances”, Visual Database Systems, II, Eds. E. Knuth, and L. M. Wegner, Elsevier Science Publishers B. V., IFIP, pp. 113-127, 1992 (hereinafter NAG92); H. J. Zhang, A. Kankanhalli, and S. W. Smoliar, “Automatic Partitioning of Full-Motion Video”, ACM/Springer Multi-media Systems, Vol. 1, No. 1, pp. 10-28, 1993 (hereinafter ZHA93)]. Mathematically, the difference in pixels and the threshold calculation can be represented by Eqs. 1 and 2.
Camera movement, e.g., pan or zoom, can have an effect of a large number of pixel changes, and hence a segment will be detected. Fast moving blobs also have the same effect. If the mean intensity values of the pixels and their connected pixels are compared [ZHA93], then the effects of the camera and blob motion are reduced. 1.2. Surface Modeling Here the idea is to model the gray-level surface of a pair of images such that the surface of the errors between the images is negligible. Hsu Y. Z., Nagel H. H, and Rekers G., “New likelihood test methods for change detection in image sequences”, Computer Vision Graphics Image Processing, vol. 26, pp. 73-106, 1984 (hereinafter HSU84)] model the gray-level surface by patches of a second order bivariate polynomial in the pixel coordinates. Given two corresponding regions, R_{1}(x_{0},y_{0}) in the image I^{(1) }and R_{2}(x_{0},y_{0}) in I^{(2)}, they represent each region by a set of seven parameters—the six coefficients of the quadratic polynomial patch, and the sum of square differences between the polynomial patch and the gray-levels. Under the assumption that the approximating patch represents the gray-level surface up to uncorrected noise errors, a likelihood test to the two hypotheses is made: H_{0}: R_{1}(x_{0},y_{0}) and R_{2}(x_{0},y_{0}) come from the same gray-value distribution. H_{0}: R_{1}(x_{0},y_{0}) and R_{2}(x_{0},y_{0}) come from different gray-value distributions. This method is not adequate to handle changes in illumination in the pair of images, as shown in Skifstad Kurt and Jain Ramesh, “Illumination Independent Change Detection for Real World Image Sequences”, Computer Vision, Graphics, and Image Processing, Vol. 46, pp. 387-399, 1989 (hereinafter SKI89).
These methods are based on the derivative images instead of working on the original gray-level images. A version of this concept is used by [SKI89]. They partition the image into regions, and each surface in each region is approximated by polynomials. Then, the derivatives of each patch are computed. If the images of the derivatives are denoted by I^{(D1) }and I^{(D2)}, then a threshold is used in order to create a binary image from the image of differences, I^{(D1)−I(D2)}. Areas of change are supposed to be white regions in this binary image. This method is inadequate for noisy inputs. 1.4. Contrast—Invariant Representation Another method that can be used to perform change detection is described in P. Monasse, F. Guichard, “Fast Computation of a Contrast-Invariant Image Representation”, IEEE Trans. on Image Processing, Vol. 9, No. 5, 860-872, 2000 (hereinafter MON00). This paper sets out a new representation of an image, which is contrast-independent. The image is decomposed into a tree of “shapes” based on connected components of level sets, which provides a full and un-redundant representation of the image. This method, which is based on invariance under change of contrast, can be used to perform some kind of change detection between images that have different illumination. However, the formulation of the solution using a level set method cannot handle efficiently many blobs at the same time. 1.5. Region Based Comparison of First or Higher Order Statistics The input images are divided into regions, usually squares of m×m pixels. Denote by R_{1}(x_{0},y_{0}) the square in the image I^{(1) }that its center is the pixel with coordinates (x_{0},y_{0}), and similarly denote by R_{2}(x_{0},y_{0}) the corresponding square in the image I^{(2)}. The gray-levels in the region R_{1}(x_{0},y_{0}) are normalized such that the mean gray-level and the variance of the gray-levels of R_{1}(x_{0},y_{0}) are the same as the mean and variance gray-level of R_{2}(x_{0},y_{0}). Then, the image I^{(2) }is compare to the image I^{(1)}. The normalization process of this statistical method is supposed to be a variation of illumination correction. The shading model method was suggested by SKI89. Each gray-level is basically the product of two components: (1) the amount of source light incident on the scene and (2) the amount of light reflected by the blobs in the scene. The amount of source light incident on a small region of the scene is approximately uniform, but the reflected light of two adjacent blobs may be different. Denote by i(x_{0},y_{0}) the amount of source light incident on point (x_{0},y_{0}) in the scene, and by r(x_{0},y_{0}) the amount of reflected light from the point (x_{0},y_{0}) in the scene. Let I^{(1) }and I^{(2) }be two images with corresponding functions, i_{1}(x,y), i_{2}(x,y), r_{1}(x,y) and r_{2}(x,y). If at pixel (x_{0},y_{0}) both images contain the same blob, then the following is satisfied: Another method of statistical model-based change detection was proposed by Til Aach, Andre Kaup and Rudolf Mester, “Statistical model-based change detection in moving video”, Signal Processing, Vol. 31, pp. 165-180, 1993 (hereinafter AKM93). Given two successive frames I_{(k) }and I_{(k+1)}, let
The likelihood ratio approach is suggested based on the assumption of uniform second-order statistics over a region [KAS91; N. H. Nagel, “Formulation of a blob concept by analysis of systematic time variation in the optically perceptible environment”, Computer Graphics and Image Processing, Vol. 7. pp. 149-194, 1978 (hereinafter NAG78); ZHA93]. The frames can be subdivided into blocks, and then the blocks are compared on the basis of the statistical characteristics of their intensity levels. Eq. (10) represents the formula that calculates the likelihood function. Let μ_{i }and μ_{i+1 }be the mean intensity values for a given region in two consecutive frames, and σ_{i }and σ_{i+1 }be the corresponding variances. The number of the blocks that exceed a certain threshold t are counted. If the number of blocks exceeds a certain value (dependent on the number of blocks), a segment is declared. A subset of the blocks can be used to detect the difference between the images so as to expedite the process of block matching. Another method based on statistical computations was suggested by Sze-Chu Liu, Chang-Wu Fu, and Shyang Chang, “Statistical change detection with moments under time-varying illumination”, IEEE Transactions On Image Processing, Vol. 7, No. 9, September 1998 (hereinafter SZE98). The proposed method consists of two parts. First, based on the defined circular shift moments, this method tries to distinguish the structural changes from those caused by illumination in a noise-free case, which is mentioned by [SKI89]. Moreover, the amount of computation in calculating higher-order circular shift moments can be reduced via a set of iterative formulae. Therefore, the time required for the computation is less than that of the shading model [SKI89]. Second, in accordance with the characteristics of the defined moments, SZE98 also propose a statistical decision rule to cope with the effects of noise. The change detection problem can be treated as one of hypothesis testing. Critical values are determined according to the desired level of significance. This does not perform change detection well, and there are many “false alarms”. There is thus a widely recognized need for, and it would be highly advantageous to have, a fast and robust illumination-insensitive method for extracting blobs that appear in only one of a pair of images, a method that has a low time complexity, and is exact, robust and fast. According to the present invention there is provided a method for illumination-independent change detection between a pair of registered images, comprising: a) providing a first original gray-level image, a second original gray-level image, a first negative image related to the first original image and a second negative images related to the second original image, b) extracting respective pluralities of blobs from each of the first and second original images and each of the first and second negative images, c) matching each extracted blob in the first original and negative images with each extracted blob in the second original and negative images to obtain matched and unmatched blobs, and d) testing all the unmatched blobs to identify blobs of change, whereby the blobs of change indicate a change detected by a method that is exact, fast, robust, illumination-insensitive and has low time-complexity. According to the present invention there is provided a method for change detection in images, comprising: a) providing a pair of first and second registered gray level images, b) extracting respective first and second pluralities of blobs from each of the images using a modified connectivity along gray levels (CAG) analysis, c) locating at least one unmatched blob in at least one of the images, and d) identifying at least one blob of change related to the images by applying a test on the at least one unmatched blob. The invention is herein described, by way of example only, with reference to the accompanying drawings, wherein: The present invention is of a method to extract blobs that appear in only one of two images of any registered pair of images. Specifically, the present invention can be used for illumination-independent change detection in a pair of gray images based on connectivity analysis along gray-levels. The principles and operation of a method to extract blobs that appear in only one of two images according to the present invention may be better understood with reference to the drawings and the accompanying description. The first step (the blob extraction step) of the present invention is preferably based on algorithm for blobs extraction based on connectivity analysis along gray-levels (CAG) as shown in Pikaz Arie, “Connectivity Analysis in Digital Gray-Levels Images and Its Applications”, Ph.D. thesis, Tel-Aviv University, Israel, February 1998 (hereinafter PIK98). The original CAG algorithm is preferably used herein as a starting point for detecting, with an appropriate set of parameters, all visually conspicuous blobs. It is worthwhile pointing out that the CAG algorithm does not deal with change detection. Thus, all the steps of the present method beyond the blob extraction step are novel. The change detection method of the present invention works in general as follows: Let I^{(1) }and I^{(2) }be the input gray-level images. Denote by {C_{i} ^{(1)}}_{i=1} ^{N} ^{ 1 }. the set of N_{1 }blobs that where detected and extracted by the blobs extracting algorithm in image I^{(1)}, and similarly by {C_{j} ^{(2)}}_{j=1} ^{N} ^{ 2 }the set of N_{2 }blobs that were detected and extracted in I^{(2)}. The blobs that define the change belong to either {C_{i} ^{(1)}}_{i=1} ^{N} ^{ 1 }or {C_{j} ^{(2)}}_{j=1} ^{N} ^{ 2 }. The original problem of change detection is reduced to the problem of detecting blobs in the set {C_{j} ^{(2)}}_{j=1} ^{N} ^{ 2 }that are not included in the set {C_{i} ^{(1)}}_{i=1} ^{N} ^{ 1 }and vice versa. A perfect match between corresponding blobs is not expected. More than that, several blobs from one image may be connected or united into a big blob in the other image (thus, even if no change occurred, the values N_{1 }and N_{2 }might be different). For each blob in {C_{i} ^{(1)}}_{i=1} ^{N} ^{ 1 }the change detection method searches for a corresponding blob in {C_{j} ^{(2)}}_{j=1} ^{N} ^{ 2 }, and vice versa. Recall that the pair of images is registered, thus the coordinates of the corresponding blobs should be the same in both images. If sufficient number of pixels of both blobs has the same coordinates (“sufficient” is determined by a pre-defined parameter), the examined pair of blobs is considered as a match. If not, then the blob from one image is “marked” in the second image. The image gradients along its boundary are examined. A measure of saliency is defined according to the distribution of the magnitudes of these gradients. If this saliency measure is sufficiently high, then the blob is classified as existing in both images. Otherwise, it represents a change. The proposed change detection method is very efficient and robust, and it is adequate for real-time applications. 2. The Original CAG Blob Extraction Algorithm Given a digital gray-level image I, I_{t }denotes a binary image that is the result of thresholding image I with a threshold t. In a binary image, a segment is defined as a set of black pixels in which there exists a 4-connected path of black pixels between each two pixels of the set. It is clear that a binary image can be represented by the set of all segments that it contains. A gray-level image I with G gray-levels can be represented by the set {I_{t}}_{t=0}, . . . , of G binary images. Generally, a blob in a gray-level image is visually conspicuous if one or more of the following exist:
Let C_{i} ^{(t) }be the i^{th }connected-segment in the binary image I_{t}. For a value {tilde over (t)}>t there exists a (single) connected segment C_{j} ^{({tilde over (t)}) }in the binary image I_{{tilde over (t)} that contains Ci} ^{(t)}. If we consider t as a time parameter, connected segments spread out as the time advances. As t increases, connected segments are united into larger connected segments until finally, for t=G−1, the whole image becomes one connected segment. For any given significant blob O there exists a sequence of segments (C^{(t)})_{t=t} _{ 0 } ^{G−1}, where C^{(t) }is a connected segment in I_{t}, that satisfies the following:
There is a value of t (or sub-interval of values) for which C^{(t) }represents best the significant blob. If t is too small, the corresponding segment C^{(t) }is only an inner part of the blob. If t is too large, the blob is an inner part of C^{(t)}. The spread-out of connected segments of I_{t }as t is increased is demonstrated in As part of the blob extraction process, the threshold value t that corresponds to each significant blob has to be detected automatically. For this purpose, we define a weight function that is attached to each segment. The weight is a function of the threshold parameter t. It is denoted by w_{c}(t), where C is the relevant segment. The function w_{c}(t) is defined as follows: in the binary image I_{t }there exists at most a single segment C′ that satisfies C′∩C≠Ø. The value of w_{c}(t) is defined as the average value of the gradient magnitudes along the boundary of the segment C′. We choose the gradient magnitude as the weight of each pixel. The weight of a blob is defined as the average weight of the pixels along the blob boundary. This weight is expected to be proportional to the blob saliency, defined hereinbelow. Let C^{(t) }be a segment that corresponds to a “real blob”. Its weight is expected to be the maximal weight among the weights of all the clusters that are not disjoint to C^{({tilde over (t)})}. Any connected cluster C^{(t) }satisfies one of the following: C^{(t)}∩C^{({tilde over (t)})}=Ø, or C^{(t)} ⊂C^{({tilde over (t)}) }and then t≦{tilde over (t)} or C^{({tilde over (t)})} ⊂C^{(t) }and then {tilde over (t)}≦t. The weight function w_{c}(t) is expected to have local maxima at values of t that correspond to the binary image I_{t }that contains the significant blob. For illustration, the weight function that corresponds to blob 304 in A weight is attached to each pixel in the input gray-level image I. The weight that is attached to a pixel is a measure of edge saliency. In particular, a pixel that resides on an edge gets assigned a higher weight than a non-edge pixel. A reasonable choice for the image of weights is the magnitudes of the gradients of I. There are many ways to compute an approximation of the gradient at a pixel. We preferably approximate the gradient vector at a pixel (u,v) according to If an examined blob O corresponds to a connected segment C in a binary image I_{{tilde over (t)}}, then its gray-levels must differ from the gray-levels of its local background. From the definition of the function w_{c}(t) we conclude that if a local maximum of w_{c}(t) exists at point {tilde over (t)}, then the blob O is salient related to its local background. Thus, the combination of connectivity and gradients along the boundary of the connected segments is a powerful measure of the significance of a given blob. 3. The Change Detection Algorithm 3.1 Modifications of the CAG Blob Extraction Algorithm The change detection starts with a novel and much more efficient method and algorithm of the present invention disclose a much more efficient way to extract blobs than the original CAG algorithm in PIK98. The present invention significantly adds to, and substantially enhances the capabilities of the original CAG algorithm with the following features: 1) the CAG of the present invention selects the conspicuous blobs based on local considerations; 2) local considerations are added to the original algorithm so that the analysis of the image is more reliable; 3) the present algorithm is more robust, this being achieved by changing the formulae and computations of local weights. 4) the computation of connected components is based herein on lower and upper bounds, which increases the accuracy of the detected blobs. In the original CAG algorithm, this computation was based only on a lower bound, which could not guarantee the accuracy of the detected blobs. The use of lower and upper bounds yields an enhanced detection capability for very small blobs, on the order of ca. 30 pixels. As suggested in the original CAG algorithm, the lower bound is computed by increasing the value of the threshold t from 0. This yields a binary image according to the threshold t. Then a weight is computed according to Eqs. 11 and 12. As t increases, one gets a monotonically increasing function until t reaches the maximum. This is illustrated in 3.2—Steps of the Change Detection Method Within the change detection method of the present invention, the modified blob extraction algorithm is first applied on two input images I_{1 }and I_{2}(herein “original” images), and on their negatives A blob extraction step 22 is separately applied on each of the input images I_{1}, I_{2 }and their negatives Next, in a first unification step 26, lists SOL_{1 }and In a matching step 30, unique to the present invention, each blob O_{i} ^{(1) }in SOL^{1 }is trial matched with a blob O_{j} ^{(2) }in SOL^{2}. If blobs O_{i} ^{(1) }and O_{j} ^{(2) }overlap (have the same coordinates) by at least α % of pixels (where α ranges typically from 75% to 100%, and preferably between 90-100%, i.e. where preferably the lower bound of α is about 90%) then a logical check step 32 checks that blob O_{i} ^{(1) }also exists in SOL^{2}. A logical yes answer in checking step 32 leads to a “no change” step 34 that determines that this is not a blob of change. This leads back (loop) to step 30 where another blob is taken from the list. If the answer in step 32 is a logical no, the algorithm proceeds to another novel step, unique to the present invention: a fitness-measuring step 36, which computes a fitness measure fm_{o} _{ 1 } ^{(I} ^{ 1 } ^{,I} ^{ 2 } ^{) }of blob O_{i} ^{(1) }in SOL^{2}. The fitness measure is then compared to a pre-defined parameter (threshold) γ in a comparison step 38. If fm_{o} _{ 1 } ^{(I} ^{ 1 } ^{,I} ^{ 2 } ^{)}<γ, then blob O_{i} ^{(1) }is determined as existing in both images (yes), i.e. this is not a blob of change, and the algorithm proceeds back to step 30 to process another blob. If fm_{o} _{ 1 } ^{(i} ^{ t } ^{I} ^{ 2 } ^{)}>γ (no) then the blob is declared in a step 40 as a blob that exists in one list and not in the other. Therefore, this is a blob of change, and the algorithm returns to process another blob in step 30. The parameter γ is preferably between 0 and 1, and most preferably about 0.6. 3.3. Determination of Match Between Corresponding Blobs In the following there is a detailed description of a preferred embodiment of a process of matching blobs in SOL^{1 }with blobs in SOL^{2}. The same procedure is valid for the match among blobs in SOL^{2 }and blobs in SOL^{1}. Let O_{i} ^{(1) }be a blob from SOL^{1}. The matching procedure is composed of two phases. We look for a matched blob O_{j} ^{(2) }in SOL^{2}. In an ideal situation, each pixel of O_{j} ^{(2) }has a corresponding pixel in O^{i} ^{ (1) }with identical coordinates, but usually this is not the case. In our implementation we claim that the two blobs O^{i} ^{ (2) }and O_{i} ^{(1) }match if the coordinates of at least α % of the pixels from both blobs are identical. In real experiments, a is typically chosen to be 90. It is possible that the blob O_{i} ^{(1) }has no matching blob in SOL^{2 }but it has a corresponding blob in the image I^{(2)}. An example for such a case is shown in Next, we deal with all the blobs that remain unmatched after the first step. If a blob in one image exists also in the other image, then its saliency measure must be sufficiently high also in the second image. The saliency measure of a blob is a function of the magnitudes of the gradients of its boundary pixels. In the original CAG algorithm, the saliency measure of a blob O with boundary ∂O was defined as: For each blob O_{i} ^{(1) }of SOL^{1 }that remains unmatched after the first step, we perform the following: assume, without loss of generality, that O_{i} ^{(1) }is contained in SOL^{1}. Compute the values sal(O_{i} ^{(1)};I^{(1)}) and sal(O_{i} ^{(1)};I^{(2)}). If the ratio Assume |∂O|=n. Let {i_{1 }. . . , i_{m}} be the indices of the pixels in ∂O with gradient magnitude greater than the sal(O;I). Then, the indices of the m pixels, i_{1}, . . . , i_{m}, should be uniformly distributed. Let: The fitness measure, denoted by fm, constitutes the foundation for the second phase of the matching phase, and contains the functions in Eqns. 15 and 17 above:
Input: Let I^{(1) }and I^{(2) }be the inputs pair of registered gray-levels images. Let Output: SOL^{(out) }are the list of the output blobs, that is, the final list of “blobs of change”. SOL^{(out) }is initialized to be an empty list. Process:
The worst-case complexity of this pass is linear in the number of pixels of the blob. 4.2. Complexity Analysis The overall time complexity of the algorithm is almost linear in the image size, n. Specifically, it is O(n·α(n,n)), where α(n,n) is the inverse of the Ackermman function [COR90, chapter 22.4], which is almost a constant. Following is the time complexity analysis of each of the three steps listed above in 5.1. The numbers correspond to the step numbers that appeared in the pseudo-code description. The construction of each of the four SOL lists takes O(n·α(n,n)) operations in the worst-case, where α(n,n) is the inverse of the Ackermman function. Therefore, the worst-case time complexity for the creation of SOL^{1 }and SOL^{2 }is O(n·α(n,n)). 1. In the following we compute the complexity analysis of steps 30 and 32 in
In this section we present four examples, and analyze the performance of the change detection algorithm. The first example, in section 5.1, presents the complete process of the change detection algorithm. In section 5.2 we demonstrate the robustness of the algorithm, and its insensitivity to change in the illumination. In section 5.3, we focus on step 36, 5.1 Example I Blobs in SOL^{1 }that have no corresponding blobs in SOL^{2 }(e.g. blob 1002) with α % (α=90) of overlap (the first step in the matching phase) are passed to step 36 in In view of the test above (step 36), the blobs that correspond to the two vehicles in I^{(1) }(802 and 804 in 5.2. Example II This example demonstrates the robustness and the insensitivity of the algorithm of the present invention to change in illumination. Two generated images I^{(1) }(a) and I^{(2) }(b) having extreme differences between their illuminations are shown in Step 24 ( In matching step 30, each blob in SOL^{1 }is searched for a corresponding blob in SOL^{2}, and vice versa. Seven blobs that correspond to the letters “DISK 2/2”, and the blob for capital letter “A” in SOL^{2 }( 5.3. Example III In this example we focus on the “fitness measure” as described above. 5.4 Example IV: Comparison with the “Shading Model” Method This example demonstrates the difference between the method of the present invention and the prior art “Shading Model” described above. In summary, the present invention introduces an efficient and robust method that provides a novel algorithm for performing change detection between a pair of registered gray-level images, under severe differences in the illumination of the images. The output of the algorithm is a set of connected components, where each component describes a “blob of change”, which is a blob that exists in only one of the two images. The time complexity of the change detection algorithm is almost linear in the image size. Therefore, it is suitable for real-time applications. The examples detailed above demonstrate its robustness even when extreme changes in the illumination exist. The main advantages of the method disclosed herein include:
All publications, patents and patent applications mentioned in this specification are herein incorporated in their entirety by reference into the specification, to the same extent as if each individual publication, patent or patent application was specifically and individually indicated to be incorporated herein by reference. In addition, citation or identification of any reference in this application shall not be construed as an admission that such reference is available as prior art to the present invention. While the invention has been described with respect to a limited number of embodiments, it will be appreciated that many variations, modifications and other applications of the invention may be made. Patent Citations
Non-Patent Citations
Referenced by
Classifications
Legal Events
Rotate |