Search Images Maps Play YouTube News Gmail Drive More »
Advanced Patent Search | Web History | Sign in

Patents

The invention relates to a technique for generating an animated three-dimensional video head based on sensed locations of facial features and texture mapping of corresponding two dimensional video image frames onto a shaped head mesh generated using the sensed locations.

Inventors: Randall Ho, David Westwood, James Stewartson, Luciano Pasquale Agostino Nocera, Ulrich F. Buddemeier, Gregory Patrick Lane Lutter, Hartmut Neven
Original Assignee: Nevengineering, Inc.
Primary Examiner: Samir Ahmed
Secondary Examiner: Yosef Kassa
Attorney: Robroy R. Fawcett
Current U.S. Classification: 382/285; 345/473; 348/42; 382/154; 382/294

View patent at USPTO
Search USPTO Assignment Database
Download USPTO Public PAIR data

Citations

Cited PatentFiling dateIssue dateOriginal AssigneeTitle
US4725824Nov 24, 1986Feb 16, 1988Mitsubishi Denki Kabushiki KaishaDoze prevention system
US4805224Feb 27, 1987Feb 14, 1989Fujitsu LimitedPattern matching method and apparatus
US4827413Jun 16, 1987May 2, 1989Kabushiki Kaisha ToshibaModified back-to-front three dimensional reconstruction algorithm
US5159647Mar 4, 1991Oct 27, 1992David Sarnoff Research Center, Inc.Fast and efficient search method for graphical data
US5168529Aug 29, 1988Dec 1, 1992Rayethon CompanyConfirmed boundary pattern matching
US5187574Aug 21, 1991Feb 16, 1993Kanda Tsushin Kogyo Co., Ltd.
Nippon Telegraph and Telephone Corporation		Method for automatically adjusting field of view of television monitor system and apparatus for carrying out the same
US5220441Sep 28, 1990Jun 15, 1993Eastman Kodak CompanyMechanism for determining parallax between digital images
US5280530Sep 5, 1991Jan 18, 1994U.S. Philips CorporationMethod and apparatus for tracking a moving object
US5333165Feb 27, 1992Jul 26, 1994John K. GradyMethod and apparatus for three-dimensional video signals
US5383013Sep 18, 1992Jan 17, 1995NEC Research Institute, Inc.Stereoscopic computer vision system
US5430809Jul 7, 1993Jul 4, 1995Sony CorporationHuman face tracking system
US5432712Jan 29, 1993Jul 11, 1995Axiom Innovation LimitedMachine vision stereo matching
US5511153Jan 18, 1994Apr 23, 1996Massachusetts Institute of TechnologyMethod and apparatus for three-dimensional, textured models from plural video images
US5533177Apr 22, 1993Jul 2, 1996Siemens AktiengesellschaftMethod for detecting and estimating the spatial position of objects from a two-dimensional image
US5550928Dec 15, 1992Aug 27, 1996A.C. Nielsen CompanyAudience measurement system and method
US5581625Jan 31, 1994Dec 3, 1996International Business Machines CorporationStereo vision system for counting items in a queue
US5588033Jun 6, 1995Dec 24, 1996St. Jude Children's Research HospitalMethod and apparatus for three dimensional image reconstruction from multiple stereotactic or isocentric backprojections
US5680487May 2, 1994Oct 21, 1997Texas Instruments IncorporatedSystem and method for determining optical flow
US5699449Nov 14, 1994Dec 16, 1997The University of ConnecticutMethod and apparatus for implementation of neural networks for face recognition
US5714997Jan 6, 1995Feb 3, 1998Virtual reality television system
US5715325Aug 30, 1995Feb 3, 1998Siemens Corporate Research, Inc.
The Trustees of Princeton University		Apparatus and method for detecting a face in a video image
US5719954Jun 2, 1995Feb 17, 1998Matsushita Electric Industrial Co., Ltd.Stereo matching method and disparity measuring method
US5736982Aug 1, 1995Apr 7, 1998Nippon Telegraph and Telephone CorporationVirtual space apparatus with avatars and speech
US5764803Apr 3, 1996Jun 9, 1998Lucent Technologies Inc.Motion-adaptive modelling of scene content for very low bit rate model-assisted coding of video sequences
US5774591Dec 15, 1995Jun 30, 1998Xerox CorporationApparatus and method for recognizing facial expressions and facial gestures in a sequence of images
US5802220Dec 15, 1995Sep 1, 1998Xerox CorporationApparatus and method for tracking facial motion through a sequence of images
US5809171Jan 5, 1996Sep 15, 1998McDonnell Douglas CorporationImage processing method and apparatus for correlating a test image with a template
US5828769Oct 23, 1996Oct 27, 1998Autodesk, Inc.Method and apparatus for recognition of objects via position and orientation consensus of local image encoding
US5917937Apr 15, 1997Jun 29, 1999Microsoft CorporationMethod for performing stereo matching to recover depths, colors and opacities of surface elements
US5982853May 23, 1996Nov 9, 1999Telephone for the deaf and method of using same
US5995119Jun 6, 1997Nov 30, 1999AT&T Corp.Method for generating photo-realistic animated characters
US6011562Feb 23, 1998Jan 4, 2000Avid Technology Inc.Method and system employing an NLE to create and modify 3D animations by mixing and compositing animation data
US6044168Nov 14, 1997Mar 28, 2000Texas Instruments IncorporatedModel based faced coding and decoding using feature detection and eigenface coding
US6052123May 14, 1997Apr 18, 2000International Business Machines CorporationAnimation reuse in three dimensional virtual reality
US6072496Jun 8, 1998Jun 6, 2000Microsoft CorporationMethod and system for capturing and representing 3D geometry, color and shading of facial expressions and other animated objects
US6430589Jun 19, 1998Aug 6, 2002Hynix Semiconductor, Inc.Single precision array processor
US6504546Feb 8, 2000Jan 7, 2003AT&T Corp.Method of modeling objects to synthesize three-dimensional, photo-realistic animations
US6556196Mar 17, 2000Apr 29, 2003Max-Planck-Gesellschaft zur Forderung der Wissenschaften e.V.Method and apparatus for the processing of images
US6580811May 31, 2001Jun 17, 2003Eyematic Interfaces, Inc.Wavelet-based facial motion capture for avatar animation
US6593936Feb 1, 2000Jul 15, 2003AT&T Corp.Synthetic audiovisual description scheme, method and system for MPEG-7
US6597736Mar 29, 2000Jul 22, 2003Cisco Technology, Inc.Throughput enhanced video communication
US6611268Aug 22, 2000Aug 26, 2003Microsoft CorporationSystem and process for generating 3D video textures using video-based rendering techniques

Referenced by

Citing PatentFiling dateIssue dateOriginal AssigneeTitle
US7409091Dec 5, 2003Aug 5, 2008Samsung Electronics Co., Ltd.Human detection method and apparatus
US8150205Dec 1, 2006Apr 3, 2012Sony CorporationImage processing apparatus, image processing method, program, and data configuration
US8189964Dec 7, 2009May 29, 2012Google Inc.Matching an approximately located query image against a reference image set
US8238671Dec 7, 2009Aug 7, 2012Google Inc.Scene classification for place recognition

Claims

1. A method for generating a three-dimensional animated video head, comprising steps for:

capturing a sequence of two-dimensional video image frames of the face of an actor;

for each two-dimensional video image frame:
sensing the locations of the actor's facial features at predetermined node locations in the two-dimensional video image frame, wherein each node location is associated with a particular facial feature;
driving control points on a three-dimensional head mesh based on the sensed node locations in the two-dimensional video image frame to generate a shaped three-dimensional head mesh;
warping the two-dimensional video image frame used to generate the sensed node locations for projection onto the shaped head mesh; and
texture mapping the warped two-dimensional video image frame onto the shaped head mesh to generate a three-dimensional frame head associated with the respective two-dimensional video image frame; and
animating the three-dimensional video head by displaying a sequence of the three-dimensional frame heads associated with the sequence of two-dimensional video image frames.

2. Method for generating a three-dimensional animated video head as defined in claim 1, wherein the step of sensing the locations of the actor's facial features at predetermined node locations in the two-dimensional video image frame is performed using a transformed facial image frame generated based on wavelet transformations.

3. Method for generating a three-dimensional animated video head as defined in claim 1, wherein the step of sensing the locations of the actor's facial features at predetermined node locations in the two-dimensional video image frame is performed using a transformed facial image frame generated based on Gabor wavelet transformations.

4. Method for generating a three-dimensional animated video head as defined in claim 1, wherein each two-dimensional image frame, in the sequence of two-dimensional video image frames, has only one view of the actor's face.

Drawings