US 6356280 B1 Abstract A 3D surface is defined in terms of a set of vertices. The 3D surface and a pixel array are used to generate a 3D image. Portions of the 3D surface are selectively removed using selected criteria. In one embodiment, the 3D surface is defined in terms of a set of triangles, and the triangles are tested for compliance with selected conditions. For example, the triangles can be tested to determine their slope. Based on these tests, the triangles are excised from the geometric surface. In one embodiment, the selected conditions are based on a combination of the slope of the triangles and the color of the pixels bound to the triangles.
Claims(5) 1. A method comprising:
providing a model of a geometric surface, said model comprising a set of portions of said geometric surface, said model being stored in a memory device, said model comprising a plurality of polygons;
excising at least some of said portions of said geometric surface in response to the slope of said some of said portions; and
displaying an image on a display device based on said geometric surface after said portions have been excised,
wherein said polygons are excised in accordance with the formula:
where Max(A0, A1, . . . An) is the largest coordinate value of any of the polygon vertex coordinates along an axis, Min(A0, A1, . . . An) is the smallest coordinate value of any of the polygon vertex coordinates along said axis, and e is a threshold value.
2. Method of
3. A method comprising:
providing a model of a geometric surface, said model comprising a set of portions of said geometric surface, said model being stored in a memory device, said model comprising a plurality of polygons;
excising at least some of said portions of said geometric surface in response to the slope of said some of said portions; and
displaying an image on a display device based on said geometric surface after said portions have been excised,
wherein said polygons are excised in accordance with the formula:
_{i}<Max(A0, A1, . . . An)−Min(A0, A1, . . . An)<b_{i } where Max(A0, A1, . . . An) is the largest coordinate value of any of the polygon vertex coordinates along an axis, Min(A0, A1, . . . An) is the smallest coordinate value of any of the polygon vertex coordinates along said axis, and a
_{i }and b_{i }constitute a range of values.4. Method of
5. Method of
_{i }and b_{i }are functions of the color of pixels bound to the polygon.Description This patent claims priority based on Provisional U.S. patent application Ser. No. 60/118,509, filed Feb. 3, 1999. This invention pertains to the field of 3D image construction. Creating 3D objects in computer graphics is a complex task, which usually requires special equipment and tools. Conventionally, high quality 3D objects are created in two stages: at the first stage, the geometry is created and at the second stage, this geometry is digitally painted or texture mapped. Both stages are time consuming and expensive. This is typically accomplished using a 3D graphics pipeline. U.S. patent application Ser. No. 09/361,470, filed Jul. 27, 1999, incorporated herein by reference in its entirety, describes a method and apparatus for receiving a 2D image, generating a 3D geometry, and digitally painting the 2D image over that 3D geometry. It is possible for the method of our '470 Application to result in certain unnecessary or anomalous geometric features. The present invention pertains to a method for excising those features. In order to facilitate an explanation of the present invention, we will first describe how a 3D graphics pipeline generates a 3D image. We will then describe a method in accordance with the '470 Application for taking a 2D image and generating in response thereto a 3D image. We will then describe a method in accordance with the present invention for removing undesired geometric elements generated during the method in accordance with the '470 Application. The following is a simplified, general description of 3D graphics pipelines. It is not intended to describe any specific product (e.g. products mentioned later in this patent). Rather, the following description is merely a general explanation of 3D graphics pipelines to assist the reader's understanding. Currently, when creating an image of an object with a 3D graphics pipeline, one provides as inputs to the pipeline a set of geometric surfaces and a “texture” that is used to “paint” the geometric surfaces. One way of constructing a geometric surface in a graphics pipeline is to create a “mesh” of “primitives.” A “primitive” is a small geometric surface that can be defined by a set of vertices. For example, the primitive can be a polygon (e.g. a triangle or quadrilateral) defined within the pipeline in terms of the locations (in x, y and z coordinate space) of its comers or vertices. A set of primitives is used to define a larger 3D surface. A 3D graphics pipeline constructs a 3D image of an object from a 2D texel array (typically called a “texture map”). FIG. 1A illustrates a 2D image As mentioned above, FIG. 1A represents a texel array. Physically, the array comprises data loaded into a memory. The texel array is not displayed as such on a CRT. Rather, as explained below, the texel array is used to create an image that is eventually displayed. The next step in the process is to provide or prepare a geometric surface. In this example, the geometric surface is in the form of a mesh The mesh of FIG. 1B is not displayed as such. Rather, the mesh of FIG. 1B is a representation of what is stored in a digital memory. Specifically, the memory stores the locations, in terms of x, y and z coordinates, of each vertex within mesh The next step is to map or “bind” the two-dimensional texture map of FIG. 1A onto mesh This portion of the process is roughly analogous to an upholsterer choosing a piece of fabric, and binding it with a few nails to the comer of a couch being upholstered (the nails are like control points). The upholsterer subsequently asks his apprentice to finish attaching the fabric to the couch. In this case, the 3D graphics pipeline finishes the task instead of an apprentice. FIGS. 1A and 1B describe the process by which one texture map (FIG. 1A) is mapped onto one mesh The next step in the process is to set up a “world coordinate model” of the various objects to be displayed. This requires establishing a position and directional orientation for each object to be displayed. For example, supposing that instead of a house, two objects are to be viewed: a tetrahedron T and a cube C (FIG. The next step is to select a frame of reference. For example, it might be decided that the “viewer” will want to observe the objects from a position corresponding to a corner of the world coordinate model (e.g. position P in FIG. During the above-described of process constructing the pixel array and providing it in the frame buffer, the pipeline a) fetches the portion of texture map Thereafter, the 3D graphics accelerator permits one to manipulate the displayed objects in any desired manner. For example, if one wants to rotate the image of tetrahedron T by 45° (FIG. Similarly, suppose that it is desired to display what would appear to the viewer if he took ten steps forward from his location at position P. The next time the graphics pipeline regenerates the image, it will generate and store another pixel array in the frame buffer corresponding to what would appear to such a viewer, and this pixel array is provided as another image on the computer screen. It is thus seen that the graphics pipeline is extremely useful in applications such as video games, where it is desired to simulate what would appear to a game player if he were wandering past a set of objects. Some graphics pipelines create models of geometric surfaces using an implicit technique. These surfaces are often described as a function of the position coordinates, i.e. f (x,y,z), or can also contain some vertices. Control points and additional formulas associated with such surfaces are used to bind a digital texel array (e.g. an array as shown in FIG. 1A) to the implicitly defined surface, and the process proceeds as described above. The major difference is that instead of defining surface areas in terms of primitives with vertices, the surface areas are defined in terms of mathematical equations. Referring to FIGS. 2A and 2B, a method in accordance with our '470 Application for creating a 3D model based on a 2D image begins with the step of providing a) a 2D image Together with the 2D image (provided in the form of an array of pixels, e.g. pixel array In the '470 method, Z array If a Z array data location bound to a vertex of geometric surface If the Z array location bound to a vertex of geometric surface Thereafter, pixel array Thereafter, the parameters for the 3D graphics pipeline are established. For example, information The information in the Z array can come from any of a number of sources, e.g. an image analyzer, a camera equipped to perceive depth, etc. Unfortunately, when using one of these automated techniques for generating Z array A method in accordance with one embodiment of our invention comprises the step of selectively eliminating certain portions of a geometric surface to improve the appearance of an image being rendered. In one embodiment, we eliminate portions of a geometric surface that have too steep a slope. For example, in an embodiment in which the geometric surface is described in terms of a set of triangles, we perform the step of eliminating those triangles meeting the following condition:
where the term “max(z0, z1, z2)” means the largest z value for any vertex in the triangle, min(z0, z1, z2) means the smallest z value for any vertex in the triangle, and e is a threshold variable. In another embodiment, we eliminate those triangles in which max(z0, z1, z2)−min(z0, z1, z2) falls within a particular range of values. In one embodiment of our invention, along with a first pixel array that contains a two-dimensional image to be bound to the geometric surface, a second array is provided. In this embodiment, the decision as to which triangles to eliminate is based at least partially on the data contained in this second array. In one version of this embodiment, the second array is an image containing depth information. This depth information describes 3D characteristics of the object depicted in the first pixel array. In yet another embodiment of our invention, the decision as to which portions of the geometric surface are to be cut is based on color information in the pixel array. For example, we can eliminate certain “color key” based areas. The decision to eliminate such areas is made based on a combination of two or more conditions. For example, in one version of this embodiment we eliminate those triangles in which max(z0, z1, z2)−min(z0, z1, z2)<e and min(z0, z1, z2)<L, were L is a color key threshold. L is calculated as a function of the color of the pixels that are bound to those particular triangles. Triangle elimination can also be based on color information alone. In one embodiment of our invention, we subdivide portions of the geometric surface to enhance the sharpness of the cuts that we make to the geometric surface. For example, for the case in which the geometric surface comprises a mesh of polygons such as triangles, we subdivide the triangles into smaller triangles to enhance the sharpness of the cuts that we make. We also can subdivide each surface (e.g., each triangle) before we selectively eliminate triangles to increase the quality of the image. In one embodiment, the above editing can be done interactively, and the user can decide whether or not to implement a given cut or triangle subdivision, e.g. after viewing the results of the cut or subdivision. FIGS. 1A to FIG. 2A illustrates an image of a book on a table. FIG. 2B illustrates a model of a flat geometric surface to which the image of FIG. 2A is to be bound. FIG. 3 is a flow diagram illustrating a method in accordance with our '470 Application. FIG. 4A illustrates a 2D image used to generate an image of a 3D object during a method in accordance with our '470 Application. FIG. 4B illustrates a flat geometric surface. FIG. 4C symbolically shows a Z array used to modify the flat geometric surface of FIG. 4B to reflect the three-dimensional shape of the object to be rendered. FIG. 4D illustrates the geometric surface of FIG. 4B after it has been modified to reflect the three-dimensional shape of the object to be rendered. FIG. 4E illustrates a rendered image based on the modified geometric surface of FIG. FIG. 4F illustrates the image of FIG. 4E after portions of the geometric surface of a selected steepness have been excised. FIG. 4G illustrates the image of FIG. 4E after different portions of the geometric surface have been excised. FIG. 5 is a block diagram showing a computer system that can be used to perform a method in accordance with the present invention. FIG. 6 is a flow diagram illustrating a method in accordance with the present invention. As explained above, a method in accordance with our '470 Application comprises the steps of: 1. providing a 2D image in a pixel array; 2. providing a “Z array” which contains information concerning 3D characteristics of the object depicted in the 2D image; 3. providing a geometric surface; and 4. modifying the geometric surface in accordance with the information in the Z array so that the geometric surface reflects the shape of the object depicted in the 2D image. Our '470 Application explains: 1. The Z array can be generated from any of a number of sources, e.g. an image analyzer or a camera equipped with depth perception. 2. Examples of image analyzer algorithms are provided. 3. The Z array can be subjected to a filter before being used to modify the geometric surface. An embodiment of our invention can include these features, and as mentioned above, the '470 Application, in its entirety, is incorporated herein by reference. FIGS. 4A to FIG. 4E shows how a graphics pipeline might generate an image based on the Z array, image During a method in accordance with the present invention, after modifying the geometric surface, but before rendering an image, two tests are performed on each triangle: an elimination threshold test and a zone of action test. If a triangle fails to pass either of these tests, it is eliminated from the geometric surface, and will not be used to render an image. (The user can modify the parameters of these tests in real time, thereby editing the image.) Our method and apparatus supports multiple criteria and tests for geometry elimination. These criteria include the following: In one embodiment, we eliminate portions of the geometric surface based on the equation F(Z)<E, were F is a function of the surface z coordinates and E is a threshold number. In a practical case in which the geometry primitives are triangles, F(Z) can be simplified as F(z0,z1,z2), where z0, z1, and z2 are z-coordinates of the three vertices of the triangle. Below we describe several tests that we believe are useful. During the steepness test, each triangle is tested to determine whether the following condition is true:
where z In effect, triangles of a certain steepness are eliminated by this test. FIG. 4F illustrates the effect on the image of FIG. 4E of eliminating triangles using this threshold test for a particular value of e. As can be seen, steep walls A set of intervals is provided, each interval being characterized by a lower value and an upper value. For each interval i, each triangle is tested to determine whether max(z In one embodiment, a In another embodiment, triangles are eliminated as a function solely of image color parameters. For example, triangles can be eliminated when luminance of the pixels corresponding to three vertices is between 0.2 and 0.3 (in a normalized luminance scale). Alternatively, triangles can be eliminated when the pixels are a certain color, hue or brightness. This flexibility in being able to remove triangles permits one to perform many useful image editing operations. For example, one can separate one type of image from the background. Alternatively, one can separate adjacent structures in an image. Typically, an object that one might want isolated from the background or from other objects is distinguishable both by its color (or color-related parameters such as luminance) and the slope of its surfaces. Thus, the method and apparatus of our invention are well-adapted to performing such isolation tasks. Elimination of triangles within geometric surfaces can be done interactively. For example, one might provide a geometry, Z array and 2D image, and then display that image. If the image looks inappropriate, one might input to the system a value e, or interval variables a The excising of triangles can be performed by removing triangles from the list of triangles within a memory array, tagging certain triangles so that they are ignored by a 3D graphics pipeline during rendering, or by other techniques. The triangles can be stored in one of the memories illustrated in FIG. 5, e.g. memory In one embodiment, in lieu of eliminating triangles within the geometric surface, one can subdivide triangles into other triangles to determine whether that improves the image. Of importance, eliminating triangles without subdividing triangles can result in an image having jagged edges, particularly if a large triangle size has been chosen for binding. If one subdivides triangles at those edges, the jaggedness of the resulting image can be reduced. Each triangle, which supposed to be deleted in the testing procedure can be additionally subdivided to minimize aliasing. (Subdividing is discussed by Shirman et al., “Fast and Accurate Texture Placement”, IEEE Computer Graphics and Applications, January-February 199, p. 60-66, incorporated herein by reference.) One embodiment of our invention can be practiced using a PC having the following: 1. A CPU such as a Celeron or Pentium, e.g. as manufactured by Intel, or a K6 processor, e.g. as manufactured by Advanced Micro Devices. 2. 32 MB of memory or greater. 3. A 3D HW adapter. This is a type of graphics card currently available on the market. The 3D HW adapter should have 4 MB of memory (preferably 8 MB) and an advanced graphics port (AGP) interface. (An AGP interface is a type of bus standard that is well-known in the art.) Alternatively, a peripheral connection interface (“PCI”) can be used in lieu of an AGP. The PCI is a type of bus standard that is well known in the art. Examples of appropriate 3D HW adapters include the TNT-2 available from Riva, the ATI Rage 128, the Matrox G400, the Trident Blade 3D and the S3 Savage. 4. The operating system can be Windows 95, Windows 98, Win2000, or any other operating system that supports direct 3D. The Windows operating system includes a standardized platform called Direct X for Windows. In one embodiment, a user sets up the flat geometric surface (for example, a triangle mesh) in the Direct 3D windows environment. The set of instructions is then provided to the graphics pipeline, which finishes the rendering process. However, in another embodiment, the PC comprises a bypass mechanism that permits one to access the hardware accelerator directly using a software interface provided by the graphics card manufacturer. FIG. 5 is a block diagram of a computer system Also included in system System Graphics controller It is emphasized that system The interactive scissors function can be performed by the CPU operating on information stored within memory FIG. 6 is a flow diagram schematically showing an embodiment of one method in accordance with our invention. Referring to FIG. 6, one provides a model of a 3D surface that has been modified in accordance with a Z array, a pixel array that has been bound to the geometric surface as discussed above. In block While the invention has been described with respect to specific embodiments, those skilled in the art will appreciate that changes can be made in form and detail without departing from the spirit and scope of the invention. For example, instead of basing the above calculations on Z axis values, values along other axes or combinations of axes can be used, e.g. the X or Y axes. When generating the Z array, one can use an image analyzer, e.g. as described in our '470 application. One can also digitally filter the Z array data, e.g. as described in our '470 Application. In lieu of defining geometric surfaces in terms of triangles, other polygons, can be used, e.g. quadrilaterals, pentagons, etc. In yet another embodiment, the geometric surfaces can be defined using the implicit technique. Accordingly, all such changes come within our invention. Patent Citations
Non-Patent Citations
Referenced by
Classifications
Legal Events
Rotate |