Search Images Maps Play YouTube News Gmail Drive More »
Advanced Patent Search | Web History | Sign in

Patents

The present invention is useful in taking a visible light image for identification and other purposes without the requirement of a photobooth, regardless of the background of the visible light image. The original background of the visible light image is replaced with a preselected background. Two IR images with different intensities of IR illumination in the foreground and background regions of the scene, respectively, are compared to produce a difference image of light intensity differences between corresponding pixels of the two images. A binarized image is generated by binarizing the difference image with respect to a predetermined threshold value .theta.. A connectivity constraint is used to generate a binary mask from the binarized image, then a gray-scale mask is produced by multiplying the binary mask times a preselected modulation function of the difference image. Warping the gray-scale mask produces a transformed mask. Finally, the background replaced visible light image is...

InventorsYibing Yang, John C. Bowman, Ibrahim Hajjahmad
Original AssigneePolaroid Corporation
Current U.S. Classification348/586; 348/590; 348/591; 348/E05.058
International Classification: H04N 974

View patent at USPTO
Search USPTO Assignment Database

Citations

Cited PatentFiling dateIssue dateOriginal AssigneeTitle
US4751571Jul 29, 1987Jun 14, 1988General Electric CompanyComposite visible/thermal-infrared imaging apparatus
US4967276May 24, 1989Oct 30, 1990Fujitsu LimitedVideo signal mixing device for infrared/visible integrated imaging
US5001558Jun 11, 1985Mar 19, 1991General Motors CorporationNight vision system with color video camera
US5386242Mar 14, 1994Jan 31, 1995The Grass Valley Group, Inc.Self keyer with background gap fill
US5400081Feb 15, 1994Mar 21, 1995The Grass Valley Group, Inc.Chroma keyer with correction for background defects
US5574511Oct 18, 1995Nov 12, 1996Polaroid CorporationBackground replacement for an image

Referenced by

Citing PatentFiling dateIssue dateOriginal AssigneeTitle
US6198503Jul 28, 1997Mar 6, 2001Infra-red video key
US6348946Aug 14, 1997Feb 19, 2002Lockheed Martin CorporationVideo conferencing with video accumulator array VAM memory
US6454415Jan 2, 2002Sep 24, 2002imatte, Inc.Interactive teleconferencing display system
US6642955Jan 10, 2000Nov 4, 2003Extreme CCTV Inc.Surveillance camera system with infrared and visible light bandpass control circuit
US6704447Feb 21, 2001Mar 9, 2004Justsystem CorporationMethod and apparatus for using illumination from a display for computer vision based user interfaces and biometric authentication
US6785402Feb 15, 2001Aug 31, 2004Hewlett-Packard Development Company, L.P.Head tracking and color video acquisition via near infrared luminance keying
US6819797Jan 28, 2000Nov 16, 2004International Business Machines CorporationMethod and apparatus for classifying and querying temporal and spatial information in video
US6885767May 11, 2001Apr 26, 2005Silhouetting apparatus and method
US6912313May 31, 2001Jun 28, 2005Sharp Laboratories of America, Inc.Image background replacement method
US6950130Jan 5, 1999Sep 27, 2005Sharp Laboratories of America, Inc.Method of image background replacement
US6999113Oct 22, 1999Feb 14, 2006Fuji Photo Film Co., Ltd.Portable printer and camera
US7024039Jun 26, 2002Apr 4, 2006Microsoft CorporationBlock retouching
US7043079Jun 26, 2002May 9, 2006Microsoft Corporation“Don't care” pixel interpolation
US7057649Mar 29, 2001Jun 6, 2006Lockheed Martin CorporationSystem and method for generating digital data and processing in a memory
US7084879Dec 19, 2002Aug 1, 2006Autodesk, Inc.Image processing
US7110596Jun 26, 2002Sep 19, 2006Microsoft CorporationSystem and method facilitating document image compression utilizing a mask
US7120297Jun 26, 2002Oct 10, 2006Microsoft CorporationSegmented layered image system
US7164797Apr 25, 2002Jan 16, 2007Microsoft CorporationClustering
US7165844Jul 24, 2006Jan 23, 2007Hitachi, Ltd.Projector type display apparatus
US7259747May 28, 2002Aug 21, 2007Reactrix Systems, Inc.Interactive video display system
US7263227Apr 25, 2002Aug 28, 2007Microsoft CorporationActivity detector
US7284864Jan 3, 2007Oct 23, 2007Hitachi, Ltd.Projector type display apparatus
US7348963Aug 5, 2005Mar 25, 2008Reactrix Systems, Inc.Interactive video display system
US7359547Sep 17, 2003Apr 15, 2008Nippon Telegraph and Telephone CorporationPseudo three dimensional image generating apparatus
US7376266Aug 16, 2006May 20, 2008Microsoft CorporationSegmented layered image system
US7376275Aug 5, 2005May 20, 2008Microsoft CorporationClustering
US7386171Aug 27, 2007Jun 10, 2008Microsoft CorporationActivity detector
US7392472Apr 25, 2002Jun 24, 2008Microsoft CorporationLayout analysis
US7397952Nov 23, 2005Jul 8, 2008Microsoft Corporation“Don't care” pixel interpolation
US7477417Aug 25, 2000Jan 13, 2009Dai Nippon Printing Co., Ltd.Image processing system
US7512274Mar 30, 2006Mar 31, 2009Microsoft CorporationBlock retouching
US7576727Dec 15, 2003Aug 18, 2009Interactive directed light/sound system
US7710391Sep 20, 2004May 4, 2010Processing an image utilizing a spatially varying pattern
US7724952May 15, 2006May 25, 2010Microsoft CorporationObject matting using flash and no-flash images
US7764834Aug 16, 2006Jul 27, 2010Microsoft CorporationSystem and method facilitating document image compression utilizing a mask
US7789311Jun 5, 2007Sep 7, 2010L-1 Secure Credentialing, Inc.Three dimensional data storage
US7804982Nov 26, 2003Sep 28, 2010L-1 Secure Credentialing, Inc.Systems and methods for managing and detecting fraud in image databases used with identification documents
US7808532May 29, 2007Oct 5, 2010Microsoft CorporationStrategies for extracting foreground information using flash and no-flash image pairs
US7809167May 19, 2009Oct 5, 2010Method and system for processing captured image information in an interactive video display system
US7834846Aug 21, 2006Nov 16, 2010Interactive video display system
US7834894Apr 3, 2007Nov 16, 2010Lifetouch Inc.Method and apparatus for background replacement in still photographs
US8035612Sep 20, 2004Oct 11, 2011Intellectual Ventures Holding 67 LLCSelf-contained interactive video display system
US8035614Oct 30, 2007Oct 11, 2011Intellectual Ventures Holding 67 LLCInteractive video window
US8035624Oct 30, 2007Oct 11, 2011Intellectual Ventures Holding 67 LLCComputer vision based touch screen
US8081822May 31, 2005Dec 20, 2011Intellectual Ventures Holding 67 LLCSystem and method for sensing a feature of an object in an interactive video display
US8098277Dec 4, 2006Jan 17, 2012Intellectual Ventures Holding 67 LLCSystems and methods for communication between a reactive video system and a mobile communication device
US8134576Oct 4, 2010Mar 13, 2012Lifetouch Inc.Method and apparatus for background replacement in still photographs
US8159682Nov 12, 2008Apr 17, 2012Intellectual Ventures Holding 67 LLCLens system
US8199108Aug 17, 2009Jun 12, 2012Intellectual Ventures Holding 67 LLCInteractive directed light/sound system
US8230367Sep 15, 2008Jul 24, 2012Intellectual Ventures Holding 67 LLCGesture-based user interactions with status indicators for acceptable inputs in volumetric zones

Claims

1. A digital processing method of replacing an original background of a visible light image of a scene with a predetermined replacement background, said method comprising the steps of:

making a first infrared (IR) image of the scene at a first time while illuminating the original background with first IR radiation having a first intensity;
measuring the intensity of IR radiation at each pixel of said first IR image;
making a second IR image of the scene at a second time after deactivating said first IR radiation, while illuminating a foreground of the scene with second IR radiation having a second intensity less than said first intensity;
measuring the intensity of IR radiation at each pixel of said second IR image;
making the visible light image of the scene at a third time while illuminating the scene with visible lighting;
generating a transformed mask distinguishing said foreground from said original background by producing a difference image (DIFF), a binarized image, a binary mask and a gray-scale mask; and
producing a modified visible light image by blending said visible light image with said predetermined replacement background, using said transformed mask.

2. The method of claim 1, wherein said step of generating a transform mask further comprises the steps of:

producing the difference image by subtracting the second IR image from the first IR image;
producing the binarized image distinguishing said foreground from said original background by comparing pixels of said difference image to a predetermined parameter .theta., then setting a pixel of the binarized image to a logic high when said pixel of the binarized image is greater than .theta., otherwise setting said pixel of the binarized image to a logic low;
producing the binary mask from the binarized image by removing false foreground pixels of the binarized image according to a connectivity constraint;
producing the gray-scale mask specifying how much of each pixel of the binary mask is attributed to the foreground and how much of each pixel of the binary mask is attributed to the original background by applying a predetermined modulation function of the difference image to the binary mask; and
producing the transformed mask by pixel-to-pixel image registration of the gray-scale mask and the visible light image.

3. The method of claim 2, wherein for a given pixel of the gray-scale mask said predetermined modulation function equals: (DIFF-.theta..sub.L)/(.theta..sub.H -.theta..sub.L) if a corresponding pixel in the difference image is less than a predetermined parameter .theta..sub.H and greater than a predetermined parameter .theta..sub.L ; 0, if the corresponding pixel in the difference image is less than or equal to .theta..sub.L ; and 1, if the corresponding pixel in the difference image is greater than or equal to .theta..sub.H.

4. The method of claim 1, wherein said second IR radiation originates from a front light.

5. The method of claim 1, wherein said second IR radiation originates from one or more foreground lights.

6. The method of claim 1, wherein said second IR radiation originates from ambient light.

7. The method of claim 1, wherein said third time equals said second time.

8. The method of claim 1, wherein said first IR radiation originates from one or more background lights.

9. The method of claim 1, wherein said first IR radiation originates from ambient light.

10. The method of claim 1, wherein said first IR radiation and said second IR radiation both have wavelengths ranging from about 700 nanometers to about 1000 nanometers.

11. The method of claim 1, wherein a difference between said first time and said second time approximates a shutter speed of an imaging device for making said first IR image, said second IR image and said visible light image.

12. The method of claim 1, wherein a difference between said first time and said second time is about 1/30th of a second.

13. The method of claim 1, wherein a difference between said first time and said second time ensures negligible movement of objects within said scene while taking said first IR, second IR and visible light images so that said images will be in focus.

14. The method of claim 1, wherein both said first and second IR images are taken with a first digital imaging device, said visible light image is taken with a second digital imaging device, and said first and second digital imaging devices have the same virtual spatial location.