The present invention is useful in taking a visible light image for identification and other purposes without the requirement of a photobooth, regardless of the background of the visible light image. The original background of the visible light image is replaced with a preselected background. Two IR images with different intensities of IR illumination in the foreground and background regions of the scene, respectively, are compared to produce a difference image of light intensity differences between corresponding pixels of the two images. A binarized image is generated by binarizing the difference image with respect to a predetermined threshold value .theta.. A connectivity constraint is used to generate a binary mask from the binarized image, then a gray-scale mask is produced by multiplying the binary mask times a preselected modulation function of the difference image. Warping the gray-scale mask produces a transformed mask. Finally, the background replaced visible light image is... |
Citations|
| US4751571 | Jul 29, 1987 | Jun 14, 1988 | General Electric Company | Composite visible/thermal-infrared imaging apparatus | | US4967276 | May 24, 1989 | Oct 30, 1990 | Fujitsu Limited | Video signal mixing device for infrared/visible integrated imaging | | US5001558 | Jun 11, 1985 | Mar 19, 1991 | General Motors Corporation | Night vision system with color video camera | | US5386242 | Mar 14, 1994 | Jan 31, 1995 | The Grass Valley Group, Inc. | Self keyer with background gap fill | | US5400081 | Feb 15, 1994 | Mar 21, 1995 | The Grass Valley Group, Inc. | Chroma keyer with correction for background defects | | US5574511 | Oct 18, 1995 | Nov 12, 1996 | Polaroid Corporation | Background replacement for an image |
Referenced by|
| US6198503 | Jul 28, 1997 | Mar 6, 2001 | | Infra-red video key | | US6348946 | Aug 14, 1997 | Feb 19, 2002 | Lockheed Martin Corporation | Video conferencing with video accumulator array VAM memory | | US6454415 | Jan 2, 2002 | Sep 24, 2002 | imatte, Inc. | Interactive teleconferencing display system | | US6642955 | Jan 10, 2000 | Nov 4, 2003 | Extreme CCTV Inc. | Surveillance camera system with infrared and visible light bandpass control circuit | | US6704447 | Feb 21, 2001 | Mar 9, 2004 | Justsystem Corporation | Method and apparatus for using illumination from a display for computer vision based user interfaces and biometric authentication | | US6785402 | Feb 15, 2001 | Aug 31, 2004 | Hewlett-Packard Development Company, L.P. | Head tracking and color video acquisition via near infrared luminance keying | | US6819797 | Jan 28, 2000 | Nov 16, 2004 | International Business Machines Corporation | Method and apparatus for classifying and querying temporal and spatial information in video | | US6885767 | May 11, 2001 | Apr 26, 2005 | | Silhouetting apparatus and method | | US6912313 | May 31, 2001 | Jun 28, 2005 | Sharp Laboratories of America, Inc. | Image background replacement method | | US6950130 | Jan 5, 1999 | Sep 27, 2005 | Sharp Laboratories of America, Inc. | Method of image background replacement | | US6999113 | Oct 22, 1999 | Feb 14, 2006 | Fuji Photo Film Co., Ltd. | Portable printer and camera | | US7024039 | Jun 26, 2002 | Apr 4, 2006 | Microsoft Corporation | Block retouching | | US7043079 | Jun 26, 2002 | May 9, 2006 | Microsoft Corporation | “Don't care” pixel interpolation | | US7057649 | Mar 29, 2001 | Jun 6, 2006 | Lockheed Martin Corporation | System and method for generating digital data and processing in a memory | | US7084879 | Dec 19, 2002 | Aug 1, 2006 | Autodesk, Inc. | Image processing | | US7110596 | Jun 26, 2002 | Sep 19, 2006 | Microsoft Corporation | System and method facilitating document image compression utilizing a mask | | US7120297 | Jun 26, 2002 | Oct 10, 2006 | Microsoft Corporation | Segmented layered image system | | US7164797 | Apr 25, 2002 | Jan 16, 2007 | Microsoft Corporation | Clustering | | US7165844 | Jul 24, 2006 | Jan 23, 2007 | Hitachi, Ltd. | Projector type display apparatus | | US7259747 | May 28, 2002 | Aug 21, 2007 | Reactrix Systems, Inc. | Interactive video display system | | US7263227 | Apr 25, 2002 | Aug 28, 2007 | Microsoft Corporation | Activity detector | | US7284864 | Jan 3, 2007 | Oct 23, 2007 | Hitachi, Ltd. | Projector type display apparatus | | US7348963 | Aug 5, 2005 | Mar 25, 2008 | Reactrix Systems, Inc. | Interactive video display system | | US7359547 | Sep 17, 2003 | Apr 15, 2008 | Nippon Telegraph and Telephone Corporation | Pseudo three dimensional image generating apparatus | | US7376266 | Aug 16, 2006 | May 20, 2008 | Microsoft Corporation | Segmented layered image system | | US7376275 | Aug 5, 2005 | May 20, 2008 | Microsoft Corporation | Clustering | | US7386171 | Aug 27, 2007 | Jun 10, 2008 | Microsoft Corporation | Activity detector | | US7392472 | Apr 25, 2002 | Jun 24, 2008 | Microsoft Corporation | Layout analysis | | US7397952 | Nov 23, 2005 | Jul 8, 2008 | Microsoft Corporation | “Don't care” pixel interpolation | | US7477417 | Aug 25, 2000 | Jan 13, 2009 | Dai Nippon Printing Co., Ltd. | Image processing system | | US7512274 | Mar 30, 2006 | Mar 31, 2009 | Microsoft Corporation | Block retouching | | US7576727 | Dec 15, 2003 | Aug 18, 2009 | | Interactive directed light/sound system | | US7710391 | Sep 20, 2004 | May 4, 2010 | | Processing an image utilizing a spatially varying pattern | | US7724952 | May 15, 2006 | May 25, 2010 | Microsoft Corporation | Object matting using flash and no-flash images | | US7764834 | Aug 16, 2006 | Jul 27, 2010 | Microsoft Corporation | System and method facilitating document image compression utilizing a mask | | US7789311 | Jun 5, 2007 | Sep 7, 2010 | L-1 Secure Credentialing, Inc. | Three dimensional data storage | | US7804982 | Nov 26, 2003 | Sep 28, 2010 | L-1 Secure Credentialing, Inc. | Systems and methods for managing and detecting fraud in image databases used with identification documents | | US7808532 | May 29, 2007 | Oct 5, 2010 | Microsoft Corporation | Strategies for extracting foreground information using flash and no-flash image pairs | | US7809167 | May 19, 2009 | Oct 5, 2010 | | Method and system for processing captured image information in an interactive video display system | | US7834846 | Aug 21, 2006 | Nov 16, 2010 | | Interactive video display system | | US7834894 | Apr 3, 2007 | Nov 16, 2010 | Lifetouch Inc. | Method and apparatus for background replacement in still photographs | | US8035612 | Sep 20, 2004 | Oct 11, 2011 | Intellectual Ventures Holding 67 LLC | Self-contained interactive video display system | | US8035614 | Oct 30, 2007 | Oct 11, 2011 | Intellectual Ventures Holding 67 LLC | Interactive video window | | US8035624 | Oct 30, 2007 | Oct 11, 2011 | Intellectual Ventures Holding 67 LLC | Computer vision based touch screen | | US8081822 | May 31, 2005 | Dec 20, 2011 | Intellectual Ventures Holding 67 LLC | System and method for sensing a feature of an object in an interactive video display | | US8098277 | Dec 4, 2006 | Jan 17, 2012 | Intellectual Ventures Holding 67 LLC | Systems and methods for communication between a reactive video system and a mobile communication device | | US8134576 | Oct 4, 2010 | Mar 13, 2012 | Lifetouch Inc. | Method and apparatus for background replacement in still photographs | | US8159682 | Nov 12, 2008 | Apr 17, 2012 | Intellectual Ventures Holding 67 LLC | Lens system | | US8199108 | Aug 17, 2009 | Jun 12, 2012 | Intellectual Ventures Holding 67 LLC | Interactive directed light/sound system | | US8230367 | Sep 15, 2008 | Jul 24, 2012 | Intellectual Ventures Holding 67 LLC | Gesture-based user interactions with status indicators for acceptable inputs in volumetric zones |
Claims1. A digital processing method of replacing an original background of a visible light image of a scene with a predetermined replacement background, said method comprising the steps of: - making a first infrared (IR) image of the scene at a first time while illuminating the original background with first IR radiation having a first intensity;
- measuring the intensity of IR radiation at each pixel of said first IR image;
- making a second IR image of the scene at a second time after deactivating said first IR radiation, while illuminating a foreground of the scene with second IR radiation having a second intensity less than said first intensity;
- measuring the intensity of IR radiation at each pixel of said second IR image;
- making the visible light image of the scene at a third time while illuminating the scene with visible lighting;
- generating a transformed mask distinguishing said foreground from said original background by producing a difference image (DIFF), a binarized image, a binary mask and a gray-scale mask; and
- producing a modified visible light image by blending said visible light image with said predetermined replacement background, using said transformed mask.
2. The method of claim 1, wherein said step of generating a transform mask further comprises the steps of: - producing the difference image by subtracting the second IR image from the first IR image;
- producing the binarized image distinguishing said foreground from said original background by comparing pixels of said difference image to a predetermined parameter .theta., then setting a pixel of the binarized image to a logic high when said pixel of the binarized image is greater than .theta., otherwise setting said pixel of the binarized image to a logic low;
- producing the binary mask from the binarized image by removing false foreground pixels of the binarized image according to a connectivity constraint;
- producing the gray-scale mask specifying how much of each pixel of the binary mask is attributed to the foreground and how much of each pixel of the binary mask is attributed to the original background by applying a predetermined modulation function of the difference image to the binary mask; and
- producing the transformed mask by pixel-to-pixel image registration of the gray-scale mask and the visible light image.
3. The method of claim 2, wherein for a given pixel of the gray-scale mask said predetermined modulation function equals: (DIFF-.theta..sub.L)/(.theta..sub.H -.theta..sub.L) if a corresponding pixel in the difference image is less than a predetermined parameter .theta..sub.H and greater than a predetermined parameter .theta..sub.L ; 0, if the corresponding pixel in the difference image is less than or equal to .theta..sub.L ; and 1, if the corresponding pixel in the difference image is greater than or equal to .theta..sub.H. 4. The method of claim 1, wherein said second IR radiation originates from a front light. 5. The method of claim 1, wherein said second IR radiation originates from one or more foreground lights. 6. The method of claim 1, wherein said second IR radiation originates from ambient light. 7. The method of claim 1, wherein said third time equals said second time. 8. The method of claim 1, wherein said first IR radiation originates from one or more background lights. 9. The method of claim 1, wherein said first IR radiation originates from ambient light. 10. The method of claim 1, wherein said first IR radiation and said second IR radiation both have wavelengths ranging from about 700 nanometers to about 1000 nanometers. 11. The method of claim 1, wherein a difference between said first time and said second time approximates a shutter speed of an imaging device for making said first IR image, said second IR image and said visible light image. 12. The method of claim 1, wherein a difference between said first time and said second time is about 1/30th of a second. 13. The method of claim 1, wherein a difference between said first time and said second time ensures negligible movement of objects within said scene while taking said first IR, second IR and visible light images so that said images will be in focus. 14. The method of claim 1, wherein both said first and second IR images are taken with a first digital imaging device, said visible light image is taken with a second digital imaging device, and said first and second digital imaging devices have the same virtual spatial location. |