|Publication number||US6912309 B2|
|Application number||US 10/379,909|
|Publication date||Jun 28, 2005|
|Filing date||Mar 6, 2003|
|Priority date||Mar 6, 2003|
|Also published as||EP1462994A2, EP1462994A3, EP1462994B1, US7817859, US20040175043, US20060062458|
|Publication number||10379909, 379909, US 6912309 B2, US 6912309B2, US-B2-6912309, US6912309 B2, US6912309B2|
|Inventors||Harry C. Lee|
|Original Assignee||Lockheed Martin Corporation|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (7), Referenced by (19), Classifications (14), Legal Events (5)|
|External Links: USPTO, USPTO Assignment, Espacenet|
1. Field of Invention
The present invention relates to processing of image data. More particularly, the present invention relates to a method and apparatus for identifying objects in an image.
2. Background Information
Historically, reconnaissance information has provided important information used in planning military operations. For example, prior to the advent of photography, scouts would be sent out to collect information regarding natural resources such as lakes and rivers, enemy troop information and the like. With the advent of photography, these scouts would provide reconnaissance information by capturing a scene of enemy installations, battlefields, and the like, using photographs. As technology advances, new methods are provided for collecting reconnaissance information. For example, it is quite common today to have reconnaissance planes, manned or remotely controlled, or satellites capture a scene for reconnaissance purposes. In addition to conventional photographic techniques, a scene can be captured using infrared detectors and the like.
Typically scenes captured by reconnaissance techniques have been analyzed by humans in order to determine the content of the captured scene. For example, a human would analyze a photograph to determine the location of bodies of water, the location of enemy troops and the location of man-made objects such as buildings and lines of communication. The human who analyzed the photograph would then have to relay the determined information to people in the field, for example, to an airplane pilot in order to identify targets. However, using humans to analyze photographs is very labor intensive. Further, there can be a considerable delay between the time when a scene is captured and the time in which the information in the captured scene is relayed to persons in the field.
In accordance with a first exemplary aspect of the present invention a method and apparatus for identifying objects in an image is provided. In accordance with the aspect the image is processed with a gradient operator to produce a gradient magnitude and direction for each pixel. A number of different gradient directions in a portion of the processed image are determined. The portion of the processed image is identified as an object if the number of different gradient directions exceeds a threshold number of gradient directions.
In accordance with another aspect of the present invention a method and apparatus for identifying objects in an image are provided. In accordance with this aspect, a gradient magnitude is determined for each pixel in the image. A gradient direction for each pixel in the image is determined, the gradient direction being determined using a look up table.
Other objects and advantages of the invention will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments, in conjunction with the accompanying drawings, wherein like reference numerals have been used to designate like elements, and wherein:
In accordance with exemplary embodiments of the present invention, portions of an image are processed to determine the number of different gradient directions present in the portion of the image. Through empirical analysis, it has been determined that closed objects, or nearly closed objects, in an image typically includes a predetermined number of different edge directions. For example, if the directions in an edge direction image are quantitized to one of eight unique directions, an object will normally comprise six, seven or eight different edge directions. It should be recognized that a quantitization of eight edge directions is merely exemplary, and that the present invention is equally applicable to other quantitizations, e.g., 16 or 32 edge directions. If other quantitizations are employed, the number of different edge directions used for identifying objects in an image can be determined by one of ordinary skill in the art through routine empirical analysis.
For each portion of the image a horizontal binary OR operation 142 is performed, followed by a vertical binary OR operation 144. The result of these operations are input to an edge count lookup table in processing block 146, which outputs a value indicating the number of different edge directions present in the portion of the image processed by processing block 140. Specifically, the output can include the thresholded gradient magnitude and gradient direction image with an indication of the number of different directions present in each portion of the image or an indication of which portions of the image contain objects. The output can be provided on a display or in printed form. If this processing is part of an automated system, the output can be in the form of coordinates of where objects are located in the images.
D x =a+2*d+g−c−2*f−i (1)
A gradient y vector is calculated in accordance with the following equation:
D y =a+2*b+c−g−2*h−i (2)
Using the gradient x and y vectors, the gradient magnitude and gradient direction are calculated as follows:
Returning now to
Once the number of different gradient directions are determined, a confidence value of the likelihood that the portion of the image identified is generated as containing an object actually contains an object is generated.
As discussed above, the present invention employs the conventional Sobel operator to determine the gradient directions. However, the conventional Sobel operator described in accordance with equations 1 through 4 above, requires 11 additions, 6 multiplications, 1 division, 1 square root, and 1 inverse tangent. Conventionally, the number of operations are decreased by performing the Sobel operation in accordance with equations 5 through 7 as follows:
D x =a+2*(d−f)+g−c−i (5)
D y =a+2*(b−h)+c−g+i (6)
Magnitude=abs(D x)+abs(D y) (7)
As illustrated in
It can be desirable to further reduce the number of operations required to determine the gradient direction. Prior to describing the exemplary technique for reducing the number of operations in accordance with the present invention, a review of the gradient directions of the conventional Sobel operation will be described in connection with
By rotating the boundaries of the gradient directions 22.5°, the calculations for the Sobel operation can be simplified.
D x =l+k+[(d−f)<<1] (8)
D y =l−k+[(b−h)<<1] (9)
wherein 1=a+i and k=g−c, and the double < represents a one bit binary shift to the right.
Using the x vector and the y vector, a lookup table in
Using the 22.5° rotation described above provides an adequate approximation of the gradient direction, this approximation can be improved. Specifically, using equations 10 through 15 below, takes advantage of the decrease in operations achieved by the 22.5° rotation, while compensating for this rotation.
D′ x =D x*15137−D y*6270 (10)
D′ y =D x*6270+D y*15137 (11)
d1=[CMP(D′ x,0)]>>1 (12)
d2=CMP(D′ y,0) (13)
d3=[CMP(abs(D′ x), abs(D′ y))]>>2 (14)
where CMP represents a comparison operation, LUT represents a lookup table operation, and an exclamation point represents a binary OR operation.
In equations 10 and 11 the values 15,137 and 6,270 are employed to compensate for the 22.5° binary shift. Specifically, the value of 15,137 represents the cosine of 22.5° times a scaling factor, and the value 6,270 represents the sine of 22.5° times a scale factor.
For ease of understanding, the present invention has been generally described as performing processing and logical operations. The processing and logical operations can be implemented using a variety of mechanisms including, but not limited to, Application Specific Integrated Circuits (ASICs), a microprocessor which executes software code, and hard-wired logic circuits. Moreover, the tables described herein can be stored in a variety of devices including buffers, caches, Random Access Memory (RAM), Read Only Memory (ROM), and the like.
The present invention has been described with reference to several exemplary embodiments. However, it will be readily apparent to those skilled in the art that it is possible to embody the invention in specific forms other than those of the exemplary embodiments described above. This may be done without departing from the spirit of the invention. These exemplary embodiments are merely illustrative and should not be considered restrictive in any way. The scope of the invention is given by the appended claims, rather than the preceding description, and all variations and equivalents which fall within the range of the claims are intended to be embraced therein.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US5867592 *||Aug 6, 1997||Feb 2, 1999||Matsushita Electric Works, Ltd.||Method of utilizing edge images of a circular surface for detecting the position, posture, and shape of a three-dimensional objective having the circular surface part|
|US5936674 *||Dec 23, 1996||Aug 10, 1999||Daewoo Electronics Co., Ltd.||Method and apparatus for concealing errors in a transmitted video signal|
|US5940539 *||Dec 27, 1996||Aug 17, 1999||Sony Corporation||Motion vector detecting apparatus and method|
|US6208763 *||Apr 14, 1998||Mar 27, 2001||General Electric Company||Method and apparatus for enhancing discrete pixel images|
|US6658145 *||Dec 22, 2000||Dec 2, 2003||Cognex Corporation||Fast high-accuracy multi-dimensional pattern inspection|
|US6661842 *||Sep 22, 2000||Dec 9, 2003||General Dynamics Decision Systems, Inc.||Methods and apparatus for error-resilient video coding|
|US6697537 *||May 10, 2002||Feb 24, 2004||Fuji Photo Film Co., Ltd.||Image processing method and apparatus|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7689060 *||Oct 27, 2005||Mar 30, 2010||Stmicroelectronics Srl||Digital image processing method transforming a matrix representation of pixels into a vector representation|
|US7720266 *||Dec 23, 2005||May 18, 2010||Shenzhen Mindray Bio-Medical Electronics Co., Ltd.||Ultrasound image enhancement and speckle mitigation method|
|US7817859 *||Jun 27, 2005||Oct 19, 2010||Lockheed Martin Corporation||Method and system for identifying objects in an image|
|US7840054 *||Apr 1, 2010||Nov 23, 2010||Shenzhen Mindray Bio-Medical Electronics Co., Ltd.||Ultrasound image enhancement and speckle mitigation method|
|US7844093 *||Mar 31, 2010||Nov 30, 2010||Shenzhen Mindray Bio-Medical Electronics Co., Ltd.||Ultrasound image enhancement and speckle mitigation method|
|US7961929 *||Nov 23, 2010||Jun 14, 2011||Shenzhen Mindray Bio-Medical Electronics Co., Ltd.||Ultrasound image enhancement and speckle mitigation method|
|US8055065||Sep 22, 2006||Nov 8, 2011||Canon Kabushiki Kaisha||Vectorisation of colour gradients|
|US8417033 *||Apr 27, 2007||Apr 9, 2013||Hewlett-Packard Development Company, L.P.||Gradient based background segmentation and enhancement of images|
|US8553933||Nov 10, 2010||Oct 8, 2013||Raytheon Company||Edge diversity object detection|
|US20060062458 *||Jun 27, 2005||Mar 23, 2006||Lockheed Martin Corporation||Method and system for identifying objects in an image|
|US20060104532 *||Oct 27, 2005||May 18, 2006||Giuseppe Messina||Digital image processing method|
|US20070065009 *||Dec 23, 2005||Mar 22, 2007||Shenzhen Mindray Bio-Medical Electronics Co., Ltd.||Ultrasound image enhancement and speckle mitigation method|
|US20080212873 *||Sep 22, 2006||Sep 4, 2008||Canon Kabushiki Kaisha||Vectorisation of Colour Gradients|
|US20080267497 *||Apr 27, 2007||Oct 30, 2008||Jian Fan||Image segmentation and enhancement|
|US20100189335 *||Apr 1, 2010||Jul 29, 2010||Shenzhen Mindray Bio-Medical Electronics Co., Ltd.||Ultrasound image enhancement and speckle mitigation method|
|US20100189346 *||Mar 31, 2010||Jul 29, 2010||Shenzhen Mindray Bio-Medical Electronics Co., Ltd.||Ultrasound image enhancement and speckle mitigation method|
|US20110096965 *||Nov 23, 2010||Apr 28, 2011||Shenzhen Mindray Bio-Medical Electronics Co., Ltd.||Ultrasound image enhancement and speckle mitigation method|
|CN101937568B||Jun 29, 2009||Jul 25, 2012||中国移动通信集团公司||Stroke direction determining method and device|
|WO2007033429A1 *||Sep 22, 2006||Mar 29, 2007||Canon Kabushiki Kaisha||Vectorisation of colour gradients|
|U.S. Classification||382/197, 382/218, 382/266|
|International Classification||G06K9/48, G06K9/40, G06T7/00, G06K9/32, G06K9/68, G06T5/00|
|Cooperative Classification||G06T2207/10048, G06T7/12, G06K9/3241|
|European Classification||G06K9/32R1, G06T7/00S2|
|Mar 6, 2003||AS||Assignment|
Owner name: LOCKHEED MARTIN CORPORATION, MARYLAND
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:LEE, HARRY C.;REEL/FRAME:013847/0396
Effective date: 20030305
|Nov 29, 2005||CC||Certificate of correction|
|Dec 29, 2008||FPAY||Fee payment|
Year of fee payment: 4
|Oct 2, 2012||FPAY||Fee payment|
Year of fee payment: 8
|Dec 28, 2016||FPAY||Fee payment|
Year of fee payment: 12