« PreviousContinue »
[ DE TERM/N/N6 REG/ON S/ZED BASED UPON ONE OR BOTH REG/ONS ]
[ DETERM/N/NC AGE CLASS FROM D/STANCE AND REG/ON S/ZE ]
1 DETERMINING THE AGE OF A HUMAN SUBJECT IN A DIGITAL IMAGE
The invention relates to digital image processing and analysis and more particularly relates to determining the age of a human subject having redeye in a digital image.
BACKGROUND OF THE INVENTION
A significant and important problem in digital image processing is automatic detection of people in digital images. A further problem is the determination and classification of the age of people in digital images. Reliable automatic age classification of human faces in a digital image Would enable many applications.
U.S. Pat. No. 5,781,650, to Lobo et al. describes a method of classifying the age of a face in a digital image using ratios of certain distances derived from the face and using snakelets to perform a Wrinkle analysis. This approach has some shortcomings. It can be difficult to locate facial features in an image With a high degree of confidence. Wrinkles appear at vastly different ages for different people. The presence of Wrinkles is highly dependent on facial expression. Adding more ambiguity to the problem is that many aging people Who begin to develop Wrinkles opt for plastic surgery and other medical procedures to remove Wrinkles.
Redeye detection is Well knoWn. Examples of techniques for this purpose are disclosed in U.S. Pat. No. 6,292,574 to Schildkraut et al. and PCT Patent publication WO 9917254.
Generally, When an eye exhibits the redeye defect in an image, the pupil is at least partially dilated. The maximum pupil dilation is a function of age. With aging, there is deterioration in vision in loW light conditions. There are neural losses, but the major decline is due to changes in the eye’s optics. The lens and other optical media become more opaque and the lens becomes yelloWer, alloWing less light entering the eye to reach the photoreceptors and making discrimination of blue colors more difficult. The relative size of the pupil shrinks, alloWing less light to enter the eye. The pupil’s response to dim light decreases With age and becomes virtually nil by age 80. Table 1 shoWs hoW the pupil size shrinks With age.
2 Machine Vision Conference, 1998, doWnloaded on the date of Aug. 31, 2004, from the Internet at: http://WWW.bmva.ac.uk/ bmvc/1998/pdf/p 147.pdf. It Would thus be desirable to provide improvements in classifying the age of a human by analyzing an image of the human’s face.
The invention is defined by the claims. The invention, in broader aspects, provides a method, in Which an age class of a human subject is ascertained in a digital image. The subject has a redeye defect pair. Each defect has one or more defect pixels. In the method, tWo regions of pixels corresponding to the defects are identified. The distance betWeen the regions is measured. A region size is determined, based upon the size of at least one of the regions. An age class is determined from the distance and region size.
It is an advantageous effect of the invention that improved Ways of classifying the age of a human by analyzing an image of the human’ s face are provided, Which utilize red eye detection.
BRIEF DESCRIPTION OF THE DRAWINGS
The above-mentioned and other features and objects of this invention and the manner of attaining them Will become more apparent and the invention itself Will be better understood by reference to the folloWing description of an embodiment of the invention taken in conjunction With the accompanying figures Wherein:
FIG. 1 is a block diagram of the digital image processor.
FIG. 2 is a diagrammatical vieW of a system incorporating the digital signal processor of FIG. 1.
FIG. 3 shoWs a graph of the distributions of F for baby/ child and for adult for images of 88 human faces having redeye defect.
FIG. 4 is a graph of P(A:baby/ child|F) vs. F for the images of FIG. 3.
FIG. 5 is a floW chart of a method of the invention.
FIG. 6 is a floW chart of another method of the invention.
DETAILED DESCRIPTION OF THE INVENTION
In the methods herein, the age of a human subject in a digital image is ascertainedusing redeye defects. TWo regions of pixels corresponding to the defects are identified and a distance betWeen the tWo is measured and a region size is determined based on at least one of the defects A range of ages (also referred to herein as an “age class”) is determined from the distance and region size.
In the folloWing description, some embodiments of the present invention Will be described as softWare programs. Those skilled in the art Will readily recognize that the equivalent of such softWare may also be constructed in hardWare. Because image manipulation algorithms and systems are Well knoWn, the present description Will be directed in particular to algorithms and systems forming part of, or cooperating more directly With, the method in accordance With the present invention. Other aspects of such algorithms and systems, and hardWare and/or softWare for producing and otherWise processing the image signals involved thereWith, not specifically shoWn or described herein may be selected from such systems, algorithms, components, and elements knoWn in the art. Given the description as set forth in the folloWing specification, all softWare implementation thereof is conventional and Within the ordinary skill in such arts.
The computer program may be stored in a computer readable storage medium, Which may comprise, for example; magnetic storage media such as a magnetic disk (such as a hard drive or a floppy disk) or magnetic tape; optical storage media such as an optical disc, optical tape, or machine readable bar code; solid state electronic storage devices such as random access memory (RAM), or read only memory (ROM); or any other physical device or medium employed to store a computer program.
The present invention can be implemented in computer hardWare. Referring to FIG. 2, there is illustrated a computer system 10 for implementing the present invention. Although the computer system 10 is shoWn for the purpose of illustrating a preferred embodiment, the present invention is not limited to the computer system 10 shoWn, but may be used on any electronic processing system such as found in home computers, kiosks, retail or Wholesale photofinishing, or any other system for the processing of digital images. The computer system 10 includes a microprocessor-based unit 12 for receiving and processing softWare programs and for performing other processing functions. A display 14 is electrically connected to the microprocessor-based unit 12 for displaying user-related information associated With the softWare, e. g., by means of a graphical user interface. A keyboard 16 is also connected to the microprocessor based unit 12 for permitting a user to input information to the softWare. As an alternative to using the keyboard 16 for input, a mouse 18 may be used for moving a selector 20 on the display 14 and for selecting an item on Which the selector 20 overlays, as is Well knoWn in the art.
A compact disk-read only memory (CD-ROM) 24, Which typically includes softWare programs, is inserted into the microprocessor based unit for providing a means of inputting the softWare programs and other information to the microprocessor based unit 12. In addition, a floppy disk 26 may also include a softWare program, and is inserted into the microprocessor-based unit 12 for inputting the softWare program. The compact disk-read only memory (CD-ROM) 24 or the floppy disk 26 may alternatively be inserted into externally located disk drive unit 22, Which is connected to the microprocessor-based unit 12. Still further, the microprocessorbased unit 12 may be programmed, as is Well knoWn in the art, for storing the softWare program internally. The microprocessor-based unit 12 may also have a netWork connection 27, such as a telephone line, to an external netWork, such as a local area netWork or the Internet. A printer 28 may also be connected to the microprocessor-based unit 12 for printing a hardcopy of the output from the computer system 10.
Images may also be displayed on the display 14 via a personal computer card (PC card) 30, such as, as it Was formerly knoWn, a PCMCIA card (based on the specifications of the Personal Computer Memory Card International Association), Which contains digitized images electronically embodied in the card 3 0. The PC card 3 0 is ultimately inserted into the microprocessor-based unit 12 for permitting visual display of the image on the display 14. Alternatively, the PC card 30 can be inserted into an externally located PC card reader 32 connected to the microprocessor-based unit 12. Images may also be input via the compact disk 24, the floppy disk 26, or the netWork connection 27. Any images stored in the PC card 30, the floppy disk 26 or the compact disk 24, or input through the netWork connection 27, may have been obtained from a variety of sources, such as a digital camera (not shoWn) or a scanner (not shoWn). Images may also be input directly from a digital camera 34 via a camera docking port 36 connected to the microprocessor-based unit 12 or directly from the digital camera 34 via a cable connection 38
to the microprocessor-based unit 12 or via a Wireless connection 40 to the microprocessor-based unit 12.
The output device can be a printer or other output device that provides a paper or other hard copy final image. The output device can also be an output device that provides the final image as a digital file. The output device can also include combinations of output, such as a printed image and a digital file on a memory unit, such as a CD or DVD.
The present invention can be used With multiple capture devices that produce digital images. For example, FIG. 2 can represent a digital photofinishing system Where the imagecapture device is a conventional photographic film camera for capturing a scene on color negative or reversal film, and a film scanner device for scanning the developed image on the film and producing a digital image. The capture device can also be an electronic capture unit (not shoWn) having an electronic imager, such as a charge-coupled device or CMOS imager. The electronic capture unit can have an analog-to-digital converter/amplifier that receives the signal from the electronic imager, amplifies and converts the signal to digital form, and transmits the image signal to the microprocessor-based unit 12.
The present invention can be used With a variety of output devices that can include, but are not limited to, a digital photographic printer and soft copy display. The microprocessor-based unit 12 can be used to process digital images to make adjustments for overall brightness, tone scale, image structure, etc. of digital images in a manner such that a pleasing looking image is produced by an image output device.
The general control computer shoWn in FIG. 2 can store the present invention as a computer program product having a program stored in a computer readable storage medium, Which may include, for example: magnetic storage media such as a magnetic disk (such as a floppy disk) or magnetic tape; optical storage media such as an optical disc, optical tape, or machine readable bar code; solid state electronic storage devices such as random access memory (RAM), or read only memory (ROM). The associated computer program implementation of the present invention may also be stored on any other physical device or medium employed to store a computer program indicated by ofl1ine memory device. Before describing the present invention, it facilitates understanding to note that the present invention is preferably utilized on any Well-knoWn computer system, such as a personal computer.
It should also be noted that the present invention can be implemented in a combination of softWare and/or hardWare and is not limited to devices, Which are physically connected and/ or located Within the same physical location. One or more of the devices illustrated in FIG. 2 may be located remotely and can be connected via a netWork. One or more of the devices can be connected Wirelessly, such as by a radiofrequency link, either directly or via a netWork.
The present invention may be employed in a variety of user contexts and environments. Exemplary contexts and environments include, Without limitation, Wholesale digital photofinishing (Which involves exemplary process steps or stages such as film in, digital processing, prints out), retail digital photofinishing (film in, digital processing, prints out), home printing (home scanned film or digital images, digital processing, prints out), desktop softWare (softWare that applies algorithms to digital prints to make them better4or even just to change them), digital fulfillment (digital images inifrom media or over the Web, digital processing, With images outi in digital form on media, digital form over the Web, or printed on hard-copy prints), kiosks (digital or scanned input, digital processing, digital or hard copy output), mobile devices (e.g.,
PDA or cell phone that can be used as a processing unit, a display unit, or a unit to give processing instructions), and as a service offered via the World Wide Web.
In each case, the invention may stand alone or may be a component of a larger system solution. Furthermore, human interfaces, e.g., the scanning or input, the digital processing, the display to a user (if needed), the input of user requests or processing instructions (if needed), the output, can each be on the same or different devices and physical locations, and communication betWeen the devices and locations can be via public or private netWork connections, or media based communication. Where consistent With the foregoing disclosure of the present invention, the method of the invention can be fully automatic, may have user input (be fully or partially manual), may have user or operator revieW to accept/rej ect the result, or may be assisted by metadata (metadata that may be user supplied, supplied by a measuring device (e.g. in a camera), or determined by an algorithm). Moreover, the algorithm (s) may interface With a variety of WorkfloW user interface schemes.
The invention is inclusive of combinations of the embodiments described herein. References to “a particular embodiment” and the like refer to features that are present in at least one embodiment of the invention. Separate references to “an embodiment” or “particular embodiments” or the like do not necessarily refer to the same embodiment or embodiments; hoWever, such embodiments are not mutually exclusive, unless so indicated or as are readily apparent to one of skill in the art.
Referring noW to FIG. 1, the digital image 102 is input to the microprocessor-based unit 12 for redeye detection and age classification. The digital image 102 is input to the redeye defect corrector 11 0 for detection of redeye defect. The output of the redeye defect detector 110 is a defect pair 112, that is, a pair of human left and right eye defects, in the digital image 102. Each defect of the defect pair 112 includes at least one pixel effected by the redeye defect. One defect of the redeye defect pair corresponds to the left eye of a human affected by the redeye defect, and the other defect corresponds to the right eye.
For a given image, the redeye defect detector may be used to detect 0, 1, or multiple defect pairs 112. The redeye defect detector 110 can internally scale the size of the digital image 102 by interpolation to normalize the analysis image size or to normalize the size of faces or skin regions in the image.
The output of the redeye defect detector 110 can take a variety of forms. A defect of the defect pair 112 can be in the form of an image map Where pixels determined by the redeye defect detector 110 to be affected by redeye defect are assigned a different value than other pixels. A defect of the defect pair 112 can also be a list of pixels ((x,y) coordinate locations and possibly pixel color values) affected by the redeye defect.
The redeye defect detector 110 can be any method knoWn in the art. The preferred redeye defect detector is described in U.S. Pat. No. 6,292,574 B1, to Schildkraut et al. Briefly summarized, first, the image is searched for skin colored regions. Next, an ellipse is fit to each skin region. Next, a resize factor is calculated and applied to each skin region. The goal is to resize a face skin colored region so that the eyes are separated by 75 pixels and fixed sized templates can be applied, When searching for redeye defects. Each skin region is searched for pairs of small red candidate defects. Various scores are analyzed (e.g. symmetry, score With respect to matching an eye template, etc.) and a final classification is performed indicating the position of likely redeye pairs in the Image.
As an optional feature, the defect pair 112 is input to a defect corrector 120 along With the digital image 102 for correction of the defect pixels to produce an output image that is an improved digital image 130 having naturally-appearing pupils rather than pupils that appear to be affected by redeye. The correction is accomplished by modifying the color of pixels belonging to a defect of the defect pair 112 by any method knoWn in the art for correcting redeye defect. For example, in WO 9917254 the color of a redeye-affected pixel is replaced With a value based on a Weighted function of the minimum of the R, G, and B color components.
The defect pair 112 is input to the feature extractor 114 for producing one or more features 116 useful for determining the approximate age or age classification of the human subject associated With the defect pair 112. The feature extractor 114 calculates the geometric distance RD betWeen the tWo redeye defects of the defect pair 112.
The distance RD betWeen defects can be measured various Ways. Measuring from centroid to centroid is preferred, since this approach is relatively tolerant of differences in apparent sizes of eyes due to partial closure and the like. In this approach, the geometric distance RD betWeen the tWo defects of the pair is calculated as folloWs. For each of the tWo defects (D 1 and D2) of the defect pair, the centroid location is calculated. The centroid of the first defect D1 is (xl, yl) and the centroid of the second defect D2 is (x2, y2). The calculation of the centroid (xc, yc) of the Cth defect (DC) is accomplished using the equation:
RD is preferably in units of pixels.
Alternatively, the geometric distance RD betWeen the tWo defects of the defect pair can be calculated With any number of variations based upon differently determined corresponding points on each of the members of the defect pair or points on the members determined by different types of separations. For example, the distance RD can be from rightmost pixel of the first defect to rightmost pixel of the second defect or can be the minimum value of all distances calculated betWeen one pixel of the first defect and one pixel from the second defect.
Next, the region size SC is determined. The region size is a function of the size of one or both of the defects of a defect pair. The region size can be an average of the sizes S1 and S2 of the tWo defects or can be the size S 1 or S2 of one of the tWo defects. For example, the region size can be preset at the maximum or minimum of S 1 and S 2. The measurement based on one defect can be used in all cases or can be used as a backup procedure to be folloWed one of the eyes appears to be inaccurately presented.
In the preferred embodiment, the size S C of a defect is the radius (units of pixels) of the smallest circle that inscribes the defect, that is, the smallest circle that When centered on the
defect’s centroid, contains all of the pixels associated With the defect Within or on its boundary. Accordingly, the size of a defect DC is computed as:
SC = rniax[ (X; —Xc)2 + U’; — yC)2]
SC is the size of defect DC of the defect pair;
(x,-, y,-) are the coordinate locations of the ith pixel belong
ing to defect DC;
(xc, yc) is the centroid of the defect DC.
Alternatively, the size S C of a defect DC is simply the number of pixels included in the defect. As another alternative, the size SC of a defect DC is the square root of the product of the number of pixels included in the defect times the inverse of st:
SC is the size of defect DC of the defect pair;
at is the mathematical constant and is approximately equal
to 3.1415926; and
QC is the number of pixels included in the defect DC.
Another alternative is that the size SC is the largest circle centered at the defect’s centroid that inscribes only defect pixels, that is, the circle does not include any pixels that are not associated With the defect Within or on its border.
The size SC and distance RD are utilized in determining the age class. The size SC and distance RD can be used With a look up table (LUT) of predetermined values to provide an age class.
In a particular embodiment, the size SC and distance RD are features that are input to the age classifier 118. The age classifier 118 considers the input features 116 derived from analysis of redeye defects as Well as other features 119 such as a Wrinkle analysis, hair color analysis (gray hair increases likelihood of the subject being an older adult), and the like.
The age classifier 118 can be any classifier that considers features and outputs a classification. For example, the age classifier 118 can be a neural netWork, a maximum likelihood classifier, a k-nearest neighbor classifier, a learning vector quantizer, or any of a number of classifiers. The classifier 118 outputs an age class information 132 indicating the result of the age classification for the human subject associated With the redeye defects. The age class 132 can be a single age category (e.g. baby, child, adult, older adult) or it can be an age range in years. The age class 132 can also have an associated probability. For example, the age class 132 could indicate that a human subject has:
0% probability of being a baby
10% probability of being a child
60% probability of being an adult, and
30% probability of being an older adult.
Having a probability associated With each age category is useful because it alloWs a doWnstream application to choose its operation based on likelihood of error.
If the preferred embodiment, the feature 116 generated by the feature extractor 114 is a function of both the size S of the defect (i.e. the size of the dilated pupil) and distance RD betWeen defects. Preferably, the feature is computed With the equation
B is related to the pixels of blur in the image; and
K is a constant multiplier used to scale the data. Preferably, BI2. Preferably, K:240_l. The term “pixels of blur” refers to the size of the blur circle (i.e. point spread function) of the image capture device used to generate the digital image 102.
FIG. 3 shoWs the distributions of F for baby/child and for adult for images of 88 human faces having redeye defect. (Discussed further beloW.) This plot shoWs P(FIA:baby/ child) and P(FIA:adult), Where A:a is the event that the human subject’s true age category is a, Where a is the age category (either baby/child or adult in this example). It can easily be seen that the distributions are different and therefore useful for determining the age of a human subject based on the feature related to the redeye defects.
In a particular embodiment, the age classifier 118 is a Bayesian classifier that calculates P(A:aIF), the probability of a human subject belonging to a certain age category given the feature values F. The age classifier 118 uses a learning stage, Where many examples of knoWn age are presented. The feature value is computed and the probability of the human subject belonging to a specific age category is determined by the Bayesian classifier 1118. According to Bayes theory,
Assuming that the prior probability of adults and children is equal (P(A:adult):P(A:baby/child):0.5), the P(A:baby/ childIF) is calculated and shoWn in FIG. 4 for the images of FIG. 3.
Preferably, the age classifier 118 operates in a mode that outputs the probability P(A:aIF); hoWever, the age classifier 118 can also operate in a mode Where it simply outputs the age category having the highest probability.
To demonstrate the utility of the invention, images of 88 human faces having redeye defect Were analyzed. The ages of the human faces Were manually classified into one of tWo categories: baby/child (ages up to approximately 9 years) and adult (older than 9 years). Out of the 88 human faces, 50 Were children, and 38 Were adults. For each face, a defect pair 112 Was identified and the size S and distance RD Was computed by analyzing each defect pair 112. Finally, the feature F Was computed based on aforementioned equation. If clas sification into the age category of either baby/ child or adult Was required, then the classification Was performed as folloWs: computing the value of P(A:aIF) for each age category and assigning the human subject to the age category a With the largest value of P(A:a I F). (This is the maximum apriori probability classifier.) Out of the 88 faces, 62 are correctly classified. This is greater than 70 percent. 8 adults Were incorrectly classified and 18 babies/children Were incorrectly classified. The expected error rate is calculated to be 28%, With an expected success rate of 72%. The probability of misclassification of a child is 35% While the probability of misclassifying an adult is 21%.
Alternatively, an even more effective feature F can be calculated, Which takes into account out-of-plane rotation of the subject’s head relative to the subject plane of the image captured by the camera. With head rotation, the measured dis