SYSTEM AND METHOD FOR CAPTURING
FACIAL AND BODY MOTION
CROSS-REFERENCE TO RELATED
This application claims priority pursuant to 35 U.S.C. § 119(e) to U.S. provisional patent application Ser. No. 60/454,872, filed Mar. 13, 2003, entitled "Motion Capture System." 10
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to three-dimensional graph- 15 ics and animation, and more particularly, to a motion capture system that enables both facial and body motion to be captured simultaneously within a volume that can accommodate plural actors.
2. Description of Related Art 20 Motion capture systems are used to capture the movement
of a real object and map it onto a computer generated object. Such systems are often used in the production of motion pictures and video games for creating a digital representation of a person that is used as source data to create a 25 computer graphics (CG) animation. In a typical system, an actor wears a suit having markers attached at various locations (e.g., having small reflective markers attached to the body and limbs) and digital cameras record the movement of the actor from different angles while illuminating the mark- 30 ers. The system then analyzes the images to determine the locations (e.g., as spatial coordinates) and orientation of the markers on the actor's suit in each frame. By tracking the locations of the markers, the system creates a spatial representation of the markers over time and builds a digital 35 representation of the actor in motion. The motion is then applied to a digital model, which may then be textured and rendered to produce a complete CG representation of the actor and/or performance. This technique has been used by special effects companies to produce incredibly realistic 40 animations in many popular movies.
Motion capture systems are also used to track the motion of facial features of an actor to create a representation of the actor's facial motion and expression (e.g., laughing, crying, smiling, etc.). As with body motion capture, markers are 45 attached to the actor's face and cameras record the actor's expressions. Since facial movement involves relatively small muscles in comparison to the larger muscles involved in body movement, the facial markers are typically much smaller than the corresponding body markers, and the cam- 50 eras typically have higher resolution than cameras usually used for body motion capture. The cameras are typically aligned in a common plane with physical movement of the actor restricted to keep the cameras focused on the actor's face. The facial motion capture system may be incorporated 55 into a helmet or other implement that is physically attached to the actor so as to uniformly illuminate the facial markers and minimize the degree of relative movement between the camera and face. For this reason, facial motion and body motion are usually captured in separate steps. The captured 60 facial motion data is then combined with captured body motion data later as part of the subsequent animation process.
An advantage of motion capture systems over traditional animation techniques, such as keyframing, is the capability 65 of real-time visualization. The production team can review the spatial representation of the actor's motion in real-time
or near real-time, enabling the actor to alter the physical performance in order to capture optimal data. Moreover, motion capture systems detect subtle nuances of physical movement that cannot be easily reproduced using other animation techniques, thereby yielding data that more accurately reflects natural movement. As a result, animation created using source material that was collected using a motion capture system will exhibit a more lifelike appearance.
Notwithstanding these advantages of motion capture systems, the separate capture of facial and body motion often results in animation data that is not truly lifelike. Facial motion and body motion are inextricably linked, such that a facial expression is often enhanced by corresponding body motion. For example, an actor may utilize certain body motion (i.e., body language) to communicate emotions and emphasize corresponding facial expressions, such as using arm flapping when talking excitedly or shoulder shrugging when frowning. This linkage between facial motion and body motion is lost when the motions are captured separately, and it is difficult to synchronize these separately captured motions together. When the facial motion and body motion are combined, the resulting animation will often appear noticeably abnormal. Since it is an objective of motion capture to enable the creation of increasingly realistic animation, the decoupling of facial and body motion represents a significant deficiency of conventional motion capture systems.
Another drawback of conventional motion capture systems is that motion data of an actor may be occluded by interference with other objects, such as props or other actors. Specifically, if a portion of the body or facial markers is blocked from the field of view of the digital cameras, then data concerning that body or facial portion is not collected. This results in an occlusion or hole in the motion data. While the occlusion can be filled in later during post-production using conventional computer graphics techniques, the fill data lacks the quality of the actual motion data, resulting in a defect of the animation that may be discernable to the viewing audience. To avoid this problem, conventional motion capture systems limit the number of objects that can be captured at one time, e.g., to a single actor. This also tends to make the motion data appear less realistic, since the quality of an actor's performance often depends upon interaction with other actors and objects. Moreover, it is difficult to combine these separate performances together in a manner that appears natural.
Yet another drawback of conventional motion capture systems is that audio is not recorded simultaneously with the motion capture. In animation, it is common to record the audio track first, and then animate the character to match the audio track. During facial motion capture, the actor will lip synch to the recorded audio track. This inevitably results in a further reduction of the visual quality of the motion data, since it is difficult for an actor to perfectly synchronize facial motion to the audio track. Also, body motion often affects the way in which speech is delivered, and the separate capture of body and facial motion increases the difficulty of synchronizing the audio track to produce a cohesive end product.
Accordingly, it would be desirable to provide a motion capture system that overcomes these and other drawbacks of the prior art. More specifically, it would be desirable to provide a motion capture system that enables both body and facial motion to be captured simultaneously within a volume that can accommodate plural actors. It would also be desk