CA2297916C - Texture mapping in 3-d computer graphics - Google Patents
Texture mapping in 3-d computer graphics Download PDFInfo
- Publication number
- CA2297916C CA2297916C CA002297916A CA2297916A CA2297916C CA 2297916 C CA2297916 C CA 2297916C CA 002297916 A CA002297916 A CA 002297916A CA 2297916 A CA2297916 A CA 2297916A CA 2297916 C CA2297916 C CA 2297916C
- Authority
- CA
- Canada
- Prior art keywords
- transformation
- source image
- filter
- reconstruction filter
- support
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000013507 mapping Methods 0.000 title description 11
- 230000009466 transformation Effects 0.000 claims abstract description 187
- 230000000694 effects Effects 0.000 claims abstract description 42
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims abstract description 40
- 230000006870 function Effects 0.000 claims abstract description 32
- 238000000034 method Methods 0.000 claims description 89
- 238000009877 rendering Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 description 9
- 238000012952 Resampling Methods 0.000 description 7
- 238000005070 sampling Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 239000013598 vector Substances 0.000 description 4
- 239000003607 modifier Substances 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000001131 transforming effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000004575 stone Substances 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G06T3/18—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/04—Texture mapping
Abstract
A source image may be manipulated by a computer to include special graphics effects. The source image may be used for still pictures or motion video applications and may include a single image or a sequence of images. The special graphics effects applied to the source image to create a desired destination image include various non-uniform scaling or arbitrary distortion functions. The effect has a corresponding affine transformation that approximates the non uniform scaling function. This affine transformation is used to define an ellipse in the source image. A linear transformation from a unit circle in the source image to the ellipse in the source image is calculated. The linear transformation is used to modify a reconstruction filter in the source image. A pixel value in the destination image is determined by identifying pixels included in a support of the modified reconstruction filter in the source image in order to reduce artifacts in non-uniform scaling applications.
Description
FIELD OF THE INVENTION
The invention relates to graphics editing, and more particularly to determining a pixel value in an image produced by a special effect which includes a function that performs non-uniform scaling or arbitrary distortion of a source image function.
BACKGROUND OF THE INVENTION
In many computer graphics applications, an image displayed on a two-dimensional display screen is generated by transforming another image, called a source image. For example, the source image may be texture mapped to a three-dimensional object which is projected onto a two-dimensional plane defined by the display. Another application includes special effects in motion video or animation in which a source image is deformed to create a desired image.
The source image can be represented by a rectangular grid of samples of a continuous 2-D function f(x,y). Each sample is the value of the continuous function at a given sample point or pixel, with each pixel representing a point of the image, including the color of the image at that point. An ideal procedure for scaling the image, i.e., an image represented as discrete pixel locations along fixed scan lines in a rectangular array of scan lines, involves reconstructing the original continuous function of the source image from the sampled data and then deforming the continuous image. A low pass filter is applied to eliminate frequencies in the destination image above a Nyquist frequency. The Nyquist frequency determines the lowest sampling rate possible to preserve details in the image and is equal to twice the frequency of the highest frequency component in a source signal. The low pass filter is used since any frequency component that is higher than half the sampling rate will be sampled incorrectly and will be aliased to a lower frequency. After applying the filter, sampling is performed on the continuous function of the deformed image using a different sampling resolution to produce a destination image. According to sampling theory, the sinc functionf(x,y)=[sin(7cx)sin(7cy)J/it2x, is ideal for reconstructing a two-dimensional image. Sarnpling would determine the color value of each point in the 3o destination to create the final destination image. The entire process of reconstructing, deforming, filtering and sampling is known as resampling.
However, in practice, the ideal process as outlined above is impractical, primarily because an ideal reconstruction filter cannot be implemented practically. For instance, since the sinc function is nonzero on points arbitrarily far from the origin, performing computations on all points is impractical. Therefore, a finite reconstruction filter is used, such as bilinear and bicubic filters are often used for filtering the signal representing a source image.
Such a filter is used in "A High-Quality Filtering Using Forward Texture Mapping," by D. Ghazanfarpour et al., in Computers and Graphics, vol. 15, no. 4, pages 569-577, January 1991. In addition, an actual continuous image cannot be represented in a computer environment and a continuous image cannot be easily distorted.
One way to simulate the ideal process includes applying a transformation to regularly spaced pixels from a source image and then applying a low pass filter to the transformed image.
However, after the transformation is applied, the transformed image may no longer include regularly spaced points and applying a low pass filter to the irregularly spaced points is difficult to perform.
In conventional systems, scaling down may be performed using mip-mapping techniques. In mip-mapping, although a standard shape of a square is used in filtering, the area of the filter may vary. A source image is successively averaged down to a lower resolution.
Each image is half the resolution of the previous image, i.e., half the linear dimension and a quarter the number of samples of its previous image. Using bilinear interpolation, an intensity is 2o determined for values of each pixel as it is scaled down. Linear interpolation between levels of filtering further srnooths the values. However, in motion video applications, mip-mapping may produce undesired artifacts.
Standard box filtering may also be used for mapping an image. A rectangular box corresponding to the approximate shape of the destination pixel may be used to compute a filtered pattern value for the pixel by summing all of the values in the source image included in the rectangle. Each destination pixel has a value that is a weighted average of an area of source pixels. However, if the transformed pixel is not approximately an aligned rectangle, then undesired artifacts may result in the destination image.
The problem of undesired artifacts may be particularly troublesome when an image is non-uniformly scaled, i.e., is subject to an arbitrary deformation. These artifacts arise particularly when applying special effects to an image which involves altering at least one portion of the image~differently from other portions of the image. Two examples of a non-AMENDED S HEET, -2/1 - ' == ..
uniform special effect scaling include sphere and ripple. When applying a ripple effect, the resulting image includes a ripple pattern, such as one resulting from throwing a stone in a lake.
S
AMENDED SHEET, Applying the sphere effect creates an enlarged lateral portion of a sphere, i.e., a"bulge", while maintaining the surrounding image in the sphere.
A need exists for a system or process for transforming a source image to a destination image which approximates the ideal resampling process and eliminates the artifacts caused by conventional filtering methods used with non-uniform scaling applications.
SZJNIlKARY OF THE INVENTION
Some embodiments of the present invention approximate an ideal resampling process by using a reconstruction filter which has been modified by a linear transformation. The modified reconstruction filter is applied in the source image in order to determine values for pixels in the destination image. By applying a modified reconstruction filter in the source image, artifacts caused by resampling a filtered source image are reduced.
One aspect of the present invention is a process for obtaining a pixel value in a destination image produced by non-uniform scaling. A reconstruction filter is chosen and a unit circle centered on a pixel in a destination image is determined. An affine transformation function determined by the non-uniform scaling is performed on the unit circle to define at least two orthogonal axes in a source image. A
linear transformation between a unit circle in the source image space and the orthogonal axes is determined. The linear transformation is then applied to the reconstruction filter. The pixels from the source image space which are included in the modified reconstruction filter are then identified. A value for a pixel in a destination image is determined from the pixels included in the support of the selected reconstruction filter.
7*7787-53 - 3a -Other aspects of the present invention include a computer system that implements this process, and other systems and processes as set forth below and in the appended claims.
According to one particular aspect of the invention, there is provided a process for obtaining a value for a pixel in a destination image produced by non-uniform transformation of a source image, comprising the steps of:
selecting a reconstruction filter; applying an affine transformation determined by the non-uniform transformation on an area centered on the pixel in the destination image to define at least two axes in a source image space;
determining a linear transformation between an area in the source image space and the at least two axes, wherein the linear transformation corresponds to the non-uniform transformation and represents the shape of the area in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation, wherein the area is centered around a center of the at least two axes; modifying the selected reconstruction filter using the linear transformation to create a modified reconstruction filter with a support region; identifying which pixels from the source image are included in the support region of the modified reconstruction filter; and determining the value for the pixel in the destination image from pixel centers of the source image included in the support of the modified reconstruction filter.
A further aspect of the invention provides a computer-implemented process for generating a non-uniformly transformed destination image including the steps of:
determining an affine transformation of a source image based on a non-uniform transformation; defining axes in the source image according to the affine transformation; calculating a - 3b -linear transformation from an area in the source image to the axes, wherein the linear transformation corresponds to a non-uniform transformation and represents the shape of an area in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation; modifying a reconstruction filter in the source image according to the linear transformation;
identifying pixels of the source image included in a support of the modified reconstruction filter; calculating the value of a point in the destination image based on the value of each of the identified pixels in the source image; and displaying the non-uniformly transformed destination image.
There is also provided a computer system for generating a representation of a non-uniformly transformed source image including a processor and a memory connected to the processor for storing a program, which when executed by the processor performs the steps of: receiving a source image, a desired special effect, and a reconstruction filter; modifying the reconstruction filter based on the application of the desired special effect to the source image; identifying pixels in the source image included in a support of the modified reconstruction filter; determining the value of a pixel in a destination based on the identified pixels; and rendering the destination image.
In accordance with a still further aspect of the invention, there is provided a process for obtaining a value for a pixel in a destination image produced by non-uniform transformation of a source image, comprising the steps of:
selecting a reconstruction filter; determining a linear transformation between a unit circle in a source image space and at least two orthogonal axes in the source image space, wherein the linear transformation corresponds to the non-uniform transformation and represents the shape of the unit - 3c -circle in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation, wherein the unit circle is centered around a center of the at least two orthogonal axes;
modifying the selected reconstruction filter using the transformation to create a modified reconstruction filter with a support region; identifying which pixels from the source image are included in the support region of the modified reconstruction filter; and determining the value for the pixel in the destination image from pixel centers of the source image included in the support of the modified reconstruction filter.
According to another aspect of the invention, there is provided a process for obtaining a value for a pixel in a destination image produced by non-uniform transformation of a source image, comprising the steps of:
selecting a reconstruction filter; determining a linear transformation between an area in a source image space and at least two axes, wherein the linear transformation corresponds to the non-uniform transformation and represents the shape of the area in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation, wherein the area is centered around a center of the at least two axes, wherein the at least two axes are at least two units in length, wherein each unit is equal to a distance between two adjacent pixels; modifying the selected reconstruction filter using the transformation to create a modified reconstruction filter with a support region; identifying which pixels from the source image are included in the support region of the modified reconstruction filter; and determining the value for the pixel in the destination image from pixel centers of - 3d -the source image included in the support of the modified reconstruction filter.
A further aspect of the invention provides a computer-implemented process for generating a non-uniformly transformed destination image including the steps of:
calculating a linear transformation from a unit circle in a source image to orthogonal axes in the source image, wherein the linear transformation corresponds to a non-uniform transformation and represents the shape of a unit circle in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation; modifying a reconstruction filter in the source image according to the linear transformation;
identifying pixels of the source image included in a support of the modified reconstruction filter; calculating the value of a point in the destination image based on the value of each of the identified pixels in the source image; and displaying the non-uniformly transformed destination image.
There is also provided a computer-implemented process for generating a non-uniformly transformed destination image including the steps of: calculating a linear transformation from an area in a source image to axes in the source image, wherein the linear transformation corresponds to a non-uniform transformation and represents the shape of the area in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation, wherein the axes are at least two units in length, wherein each unit is equal to a distance between two adjacent pixels; modifying a reconstruction filter in the source image according to the linear transformation; identifying pixels of the source image included in a support of the modified reconstruction filter; calculating the value of a point in the destination - 3e -image based on the value of each of the identified pixels in the source image; and displaying the non-uniformly transformed destination image.
BRIEF DESCRIPTION OF THE DRAWING
In the drawings Fig. 1 is a block diagram of a system for processing non-uniform scaling of images in accordance with an embodiment of the present invention;
Fig. 2 is a block diagram of a general purpose computer suitable for an embodiment of the invention;
Fig. 3 is a functional diagram of the process of an embodiment of the present invention;
Fig. 4 is a flowchart of the process of an embodiment of the present invention; and _ = . , -4- . ., , Fig. 5 is a block diagram of a system for processing non-uniform scaling of images in accordance with an embodiment of the present invention.
DETAILED DESCRIPTION
The present invention will be more completely understood through the following detailed description which should be read in conjunction with the attached drawings in which similar reference numbers indicate similar structures.
The present invention approximates an ideal resampling process by using a reconstruction filter which has been modified by, a linear transformation. The modified 1o reconstruction filter is applied in the source image in order to determine values for pixels in the destination image. By applying a modified reconstruction filter in the source image, artifacts caused by resampling a filtered source image are reduced.
This process will now be described generally in connection with Fig. 1 and will be described in more detail below in connection with Figs. 3, 4 and 5. Special effect tools module 125 creates an effect which includes non-uniform scaling. Various tools may be used to define such effects. The effect data 130 may be stored for future use, or may be sent to filter modifier 105. Filter modifier 105 receives a selected filter 120, a selected pixel 115 in a destination image, and data 130 indicating the effect to be applied to a source image 110.
The filter modifier 105 modifies the selected filter 120 according to a relationship, defmed by the effect 130, between the selected pixel 115 and a point in the source image 110 to create a transformed filter 135. The filter application module 140 applies the modified filter 135 to the source image 110 to obtain a pixel value 145 in the destination image. The pixel value 145 is outputted to a rendering device 150 to display the-destination image.
The present invention may be implemented using a general purpose computer that is programmed to generate, with reduced artifacts, a destination image by non-uniformly scaling a source image. An example computer system is shown in Fig. 2., In particular, the apparatus as shown in Fig. 2 includes a general purpose computer 20 having a processor 22, an input device 24, an output device 26, and memory 28 connected together through a computer bus 30. The memory 28 includes primary memory 32 (i.e., fast volatile memory such as dynamic semiconductor memory) and secondary memory 34 (i.e., nonvolatile memory such as magnetic disks). The memory 28 stores one or more programs 36 that, when executed by the processor 22, effect a transformation from a source image to a destination image. The input device 24 receives AMENDED SHEET
The invention relates to graphics editing, and more particularly to determining a pixel value in an image produced by a special effect which includes a function that performs non-uniform scaling or arbitrary distortion of a source image function.
BACKGROUND OF THE INVENTION
In many computer graphics applications, an image displayed on a two-dimensional display screen is generated by transforming another image, called a source image. For example, the source image may be texture mapped to a three-dimensional object which is projected onto a two-dimensional plane defined by the display. Another application includes special effects in motion video or animation in which a source image is deformed to create a desired image.
The source image can be represented by a rectangular grid of samples of a continuous 2-D function f(x,y). Each sample is the value of the continuous function at a given sample point or pixel, with each pixel representing a point of the image, including the color of the image at that point. An ideal procedure for scaling the image, i.e., an image represented as discrete pixel locations along fixed scan lines in a rectangular array of scan lines, involves reconstructing the original continuous function of the source image from the sampled data and then deforming the continuous image. A low pass filter is applied to eliminate frequencies in the destination image above a Nyquist frequency. The Nyquist frequency determines the lowest sampling rate possible to preserve details in the image and is equal to twice the frequency of the highest frequency component in a source signal. The low pass filter is used since any frequency component that is higher than half the sampling rate will be sampled incorrectly and will be aliased to a lower frequency. After applying the filter, sampling is performed on the continuous function of the deformed image using a different sampling resolution to produce a destination image. According to sampling theory, the sinc functionf(x,y)=[sin(7cx)sin(7cy)J/it2x, is ideal for reconstructing a two-dimensional image. Sarnpling would determine the color value of each point in the 3o destination to create the final destination image. The entire process of reconstructing, deforming, filtering and sampling is known as resampling.
However, in practice, the ideal process as outlined above is impractical, primarily because an ideal reconstruction filter cannot be implemented practically. For instance, since the sinc function is nonzero on points arbitrarily far from the origin, performing computations on all points is impractical. Therefore, a finite reconstruction filter is used, such as bilinear and bicubic filters are often used for filtering the signal representing a source image.
Such a filter is used in "A High-Quality Filtering Using Forward Texture Mapping," by D. Ghazanfarpour et al., in Computers and Graphics, vol. 15, no. 4, pages 569-577, January 1991. In addition, an actual continuous image cannot be represented in a computer environment and a continuous image cannot be easily distorted.
One way to simulate the ideal process includes applying a transformation to regularly spaced pixels from a source image and then applying a low pass filter to the transformed image.
However, after the transformation is applied, the transformed image may no longer include regularly spaced points and applying a low pass filter to the irregularly spaced points is difficult to perform.
In conventional systems, scaling down may be performed using mip-mapping techniques. In mip-mapping, although a standard shape of a square is used in filtering, the area of the filter may vary. A source image is successively averaged down to a lower resolution.
Each image is half the resolution of the previous image, i.e., half the linear dimension and a quarter the number of samples of its previous image. Using bilinear interpolation, an intensity is 2o determined for values of each pixel as it is scaled down. Linear interpolation between levels of filtering further srnooths the values. However, in motion video applications, mip-mapping may produce undesired artifacts.
Standard box filtering may also be used for mapping an image. A rectangular box corresponding to the approximate shape of the destination pixel may be used to compute a filtered pattern value for the pixel by summing all of the values in the source image included in the rectangle. Each destination pixel has a value that is a weighted average of an area of source pixels. However, if the transformed pixel is not approximately an aligned rectangle, then undesired artifacts may result in the destination image.
The problem of undesired artifacts may be particularly troublesome when an image is non-uniformly scaled, i.e., is subject to an arbitrary deformation. These artifacts arise particularly when applying special effects to an image which involves altering at least one portion of the image~differently from other portions of the image. Two examples of a non-AMENDED S HEET, -2/1 - ' == ..
uniform special effect scaling include sphere and ripple. When applying a ripple effect, the resulting image includes a ripple pattern, such as one resulting from throwing a stone in a lake.
S
AMENDED SHEET, Applying the sphere effect creates an enlarged lateral portion of a sphere, i.e., a"bulge", while maintaining the surrounding image in the sphere.
A need exists for a system or process for transforming a source image to a destination image which approximates the ideal resampling process and eliminates the artifacts caused by conventional filtering methods used with non-uniform scaling applications.
SZJNIlKARY OF THE INVENTION
Some embodiments of the present invention approximate an ideal resampling process by using a reconstruction filter which has been modified by a linear transformation. The modified reconstruction filter is applied in the source image in order to determine values for pixels in the destination image. By applying a modified reconstruction filter in the source image, artifacts caused by resampling a filtered source image are reduced.
One aspect of the present invention is a process for obtaining a pixel value in a destination image produced by non-uniform scaling. A reconstruction filter is chosen and a unit circle centered on a pixel in a destination image is determined. An affine transformation function determined by the non-uniform scaling is performed on the unit circle to define at least two orthogonal axes in a source image. A
linear transformation between a unit circle in the source image space and the orthogonal axes is determined. The linear transformation is then applied to the reconstruction filter. The pixels from the source image space which are included in the modified reconstruction filter are then identified. A value for a pixel in a destination image is determined from the pixels included in the support of the selected reconstruction filter.
7*7787-53 - 3a -Other aspects of the present invention include a computer system that implements this process, and other systems and processes as set forth below and in the appended claims.
According to one particular aspect of the invention, there is provided a process for obtaining a value for a pixel in a destination image produced by non-uniform transformation of a source image, comprising the steps of:
selecting a reconstruction filter; applying an affine transformation determined by the non-uniform transformation on an area centered on the pixel in the destination image to define at least two axes in a source image space;
determining a linear transformation between an area in the source image space and the at least two axes, wherein the linear transformation corresponds to the non-uniform transformation and represents the shape of the area in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation, wherein the area is centered around a center of the at least two axes; modifying the selected reconstruction filter using the linear transformation to create a modified reconstruction filter with a support region; identifying which pixels from the source image are included in the support region of the modified reconstruction filter; and determining the value for the pixel in the destination image from pixel centers of the source image included in the support of the modified reconstruction filter.
A further aspect of the invention provides a computer-implemented process for generating a non-uniformly transformed destination image including the steps of:
determining an affine transformation of a source image based on a non-uniform transformation; defining axes in the source image according to the affine transformation; calculating a - 3b -linear transformation from an area in the source image to the axes, wherein the linear transformation corresponds to a non-uniform transformation and represents the shape of an area in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation; modifying a reconstruction filter in the source image according to the linear transformation;
identifying pixels of the source image included in a support of the modified reconstruction filter; calculating the value of a point in the destination image based on the value of each of the identified pixels in the source image; and displaying the non-uniformly transformed destination image.
There is also provided a computer system for generating a representation of a non-uniformly transformed source image including a processor and a memory connected to the processor for storing a program, which when executed by the processor performs the steps of: receiving a source image, a desired special effect, and a reconstruction filter; modifying the reconstruction filter based on the application of the desired special effect to the source image; identifying pixels in the source image included in a support of the modified reconstruction filter; determining the value of a pixel in a destination based on the identified pixels; and rendering the destination image.
In accordance with a still further aspect of the invention, there is provided a process for obtaining a value for a pixel in a destination image produced by non-uniform transformation of a source image, comprising the steps of:
selecting a reconstruction filter; determining a linear transformation between a unit circle in a source image space and at least two orthogonal axes in the source image space, wherein the linear transformation corresponds to the non-uniform transformation and represents the shape of the unit - 3c -circle in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation, wherein the unit circle is centered around a center of the at least two orthogonal axes;
modifying the selected reconstruction filter using the transformation to create a modified reconstruction filter with a support region; identifying which pixels from the source image are included in the support region of the modified reconstruction filter; and determining the value for the pixel in the destination image from pixel centers of the source image included in the support of the modified reconstruction filter.
According to another aspect of the invention, there is provided a process for obtaining a value for a pixel in a destination image produced by non-uniform transformation of a source image, comprising the steps of:
selecting a reconstruction filter; determining a linear transformation between an area in a source image space and at least two axes, wherein the linear transformation corresponds to the non-uniform transformation and represents the shape of the area in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation, wherein the area is centered around a center of the at least two axes, wherein the at least two axes are at least two units in length, wherein each unit is equal to a distance between two adjacent pixels; modifying the selected reconstruction filter using the transformation to create a modified reconstruction filter with a support region; identifying which pixels from the source image are included in the support region of the modified reconstruction filter; and determining the value for the pixel in the destination image from pixel centers of - 3d -the source image included in the support of the modified reconstruction filter.
A further aspect of the invention provides a computer-implemented process for generating a non-uniformly transformed destination image including the steps of:
calculating a linear transformation from a unit circle in a source image to orthogonal axes in the source image, wherein the linear transformation corresponds to a non-uniform transformation and represents the shape of a unit circle in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation; modifying a reconstruction filter in the source image according to the linear transformation;
identifying pixels of the source image included in a support of the modified reconstruction filter; calculating the value of a point in the destination image based on the value of each of the identified pixels in the source image; and displaying the non-uniformly transformed destination image.
There is also provided a computer-implemented process for generating a non-uniformly transformed destination image including the steps of: calculating a linear transformation from an area in a source image to axes in the source image, wherein the linear transformation corresponds to a non-uniform transformation and represents the shape of the area in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation, wherein the axes are at least two units in length, wherein each unit is equal to a distance between two adjacent pixels; modifying a reconstruction filter in the source image according to the linear transformation; identifying pixels of the source image included in a support of the modified reconstruction filter; calculating the value of a point in the destination - 3e -image based on the value of each of the identified pixels in the source image; and displaying the non-uniformly transformed destination image.
BRIEF DESCRIPTION OF THE DRAWING
In the drawings Fig. 1 is a block diagram of a system for processing non-uniform scaling of images in accordance with an embodiment of the present invention;
Fig. 2 is a block diagram of a general purpose computer suitable for an embodiment of the invention;
Fig. 3 is a functional diagram of the process of an embodiment of the present invention;
Fig. 4 is a flowchart of the process of an embodiment of the present invention; and _ = . , -4- . ., , Fig. 5 is a block diagram of a system for processing non-uniform scaling of images in accordance with an embodiment of the present invention.
DETAILED DESCRIPTION
The present invention will be more completely understood through the following detailed description which should be read in conjunction with the attached drawings in which similar reference numbers indicate similar structures.
The present invention approximates an ideal resampling process by using a reconstruction filter which has been modified by, a linear transformation. The modified 1o reconstruction filter is applied in the source image in order to determine values for pixels in the destination image. By applying a modified reconstruction filter in the source image, artifacts caused by resampling a filtered source image are reduced.
This process will now be described generally in connection with Fig. 1 and will be described in more detail below in connection with Figs. 3, 4 and 5. Special effect tools module 125 creates an effect which includes non-uniform scaling. Various tools may be used to define such effects. The effect data 130 may be stored for future use, or may be sent to filter modifier 105. Filter modifier 105 receives a selected filter 120, a selected pixel 115 in a destination image, and data 130 indicating the effect to be applied to a source image 110.
The filter modifier 105 modifies the selected filter 120 according to a relationship, defmed by the effect 130, between the selected pixel 115 and a point in the source image 110 to create a transformed filter 135. The filter application module 140 applies the modified filter 135 to the source image 110 to obtain a pixel value 145 in the destination image. The pixel value 145 is outputted to a rendering device 150 to display the-destination image.
The present invention may be implemented using a general purpose computer that is programmed to generate, with reduced artifacts, a destination image by non-uniformly scaling a source image. An example computer system is shown in Fig. 2., In particular, the apparatus as shown in Fig. 2 includes a general purpose computer 20 having a processor 22, an input device 24, an output device 26, and memory 28 connected together through a computer bus 30. The memory 28 includes primary memory 32 (i.e., fast volatile memory such as dynamic semiconductor memory) and secondary memory 34 (i.e., nonvolatile memory such as magnetic disks). The memory 28 stores one or more programs 36 that, when executed by the processor 22, effect a transformation from a source image to a destination image. The input device 24 receives AMENDED SHEET
commands from a graphics editor, and the output device 26 displays results of the transformed image to the graphics editor. The results of the transformed image can be stored in memory 28 for future use.
It should be understood that one or more output devices may be connected to the computer 20. Example output devices 26 include various imaging devices including a cathode ray tube (CRT) display, a liquid crystal display (LCD), printers, and communication devices such as a modem. One or more input devices 24 may be connected to the computer system.
Example input devices include a keyboard, keypad, track ball, mouse, pen and tablet, communication device and scanner. It should be understood the invention is not limited to the 1o particular input or output devices used in combination with the computer system 20 or to those described herein. Also, the invention is not limited to a particular computer platform, particular processor or particular high-level programming language. Additionally, the computer system 20 may be a multiprocessor computer system or may include multiple computers connected over a computer network.
Referring now to Fig. 3, a general example illustrating principles of a process in accordance with the present invention will now be described. In this process, a pixel value in a destination image 205 is determined according to non-uniform scaling of a source image 210.
Non-uniform scaling may include any deformation of the source image. Such scaling may be used for enhancing graphics either in still pictures, e.g., a high quality poster, or in motion videos, e.g., video games, movies, etc., and may include two-dimensional or three-dimensional special effects. Three-dimensional effects may include arbitrary deformation of a source image by mapping a two-dimensional source image into another two-dimensional image.
In particular, this may include arbitrary deformation by mapping onto a three-dimensional surface followed by orthographic or perspective projection. The two-dimensional special effects to be applied using non-uniform scaling of an image may include sphere, ripple, twirl, page turn, warp, tumble, flag wave or pinch effects, or any deformation determined by a function from one plane to another plane.
In Fig. 3, both destination image 205 and source image 210 may be considered as a set of samples, called points or pixels, in a two-dimensional plane, where each sample has coordinates in the plane and a value indicative of its color. It should be understood that the invention also applies to black and white images. The destination image 205 and source image 210 may have an arbitrary shape and size. A point 225 in the source image 210 may contribute to the value of a point, e.g., 215 or 230, in the destination image 205, according to the deformation to be applied to the source image 210.
For any arbitrary deformation of the source image 210, an affine transformation can be defined that maps a point 230 in the destination image 205 to a point 225 in the source image 210. Similarly, this affine transformation maps a unit circle 235 centered about point 230 in the destination image to an ellipse 260 in the source image 210. The length and direction of the orthogonal axes of the ellipse 260 may be determined using this affine transformation. When the length of one of the axes is greater than two units, the deformation to be applied to the source implies low pass filtering should occur in the direction of that axis. If the length of the axis is t o less than two, no low pass filtering should occur in the direction of that axis, and the length of that axis may be assumed to be one, as shown at 220. A unit is the distance between two adjacent pixels in the source image 210. Given the length and direction of the orthogonal axes R, and R2 of the ellipse 220, a linear transformation is determined which maps a unit circle 240 concentric with the ellipse 220 into the ellipse 220. A desired reconstruction filter 245, e.g., a bilinear or bicubic filter, also concentric with the unit circle 240 is modified according to this linear transformation. This modified reconstruction filter is represented at 250. Pixels 255 of the source image 210 that fall within the modified reconstruction filter 250 are used to determine a pixel value for point 230 in destination image 205.
One embodiment of the process described above in connection with Fig. 3 will now be described in more detail in connection with Fig. 4. This process starts in step 305 with selection of a special effect involving non-uniform scaling or arbitrary deformation of an image, which may be input to computer 20 (Fig. 2). The special effect may be a two-dimensional or three-dimensional effect, such as described above, which is used to transform a source image 210 into a destination image 205. Predetermined special effects may be stored in memory 28 ( Fig. 2) and may be selected according to user input into computer 20 (Fig. 2). Example arbitrary deformation functions include mapping a two-dimensional source image to a three-dimensional surface, such as in texture mapping, and then projecting the surface as a two-dimensional destination image. Another kind of arbitrary deformation function is an inverse perspective transformation into three-dimensional space followed by a mapping back into two-dimensional space. Other functions which scale an image non-uniformly also may be used.
A reconstruction filter is chosen in step 310. The filter may be selected from predetermined functions stored in memory 28, which also may have user selectable parameters.
It should be understood that one or more output devices may be connected to the computer 20. Example output devices 26 include various imaging devices including a cathode ray tube (CRT) display, a liquid crystal display (LCD), printers, and communication devices such as a modem. One or more input devices 24 may be connected to the computer system.
Example input devices include a keyboard, keypad, track ball, mouse, pen and tablet, communication device and scanner. It should be understood the invention is not limited to the 1o particular input or output devices used in combination with the computer system 20 or to those described herein. Also, the invention is not limited to a particular computer platform, particular processor or particular high-level programming language. Additionally, the computer system 20 may be a multiprocessor computer system or may include multiple computers connected over a computer network.
Referring now to Fig. 3, a general example illustrating principles of a process in accordance with the present invention will now be described. In this process, a pixel value in a destination image 205 is determined according to non-uniform scaling of a source image 210.
Non-uniform scaling may include any deformation of the source image. Such scaling may be used for enhancing graphics either in still pictures, e.g., a high quality poster, or in motion videos, e.g., video games, movies, etc., and may include two-dimensional or three-dimensional special effects. Three-dimensional effects may include arbitrary deformation of a source image by mapping a two-dimensional source image into another two-dimensional image.
In particular, this may include arbitrary deformation by mapping onto a three-dimensional surface followed by orthographic or perspective projection. The two-dimensional special effects to be applied using non-uniform scaling of an image may include sphere, ripple, twirl, page turn, warp, tumble, flag wave or pinch effects, or any deformation determined by a function from one plane to another plane.
In Fig. 3, both destination image 205 and source image 210 may be considered as a set of samples, called points or pixels, in a two-dimensional plane, where each sample has coordinates in the plane and a value indicative of its color. It should be understood that the invention also applies to black and white images. The destination image 205 and source image 210 may have an arbitrary shape and size. A point 225 in the source image 210 may contribute to the value of a point, e.g., 215 or 230, in the destination image 205, according to the deformation to be applied to the source image 210.
For any arbitrary deformation of the source image 210, an affine transformation can be defined that maps a point 230 in the destination image 205 to a point 225 in the source image 210. Similarly, this affine transformation maps a unit circle 235 centered about point 230 in the destination image to an ellipse 260 in the source image 210. The length and direction of the orthogonal axes of the ellipse 260 may be determined using this affine transformation. When the length of one of the axes is greater than two units, the deformation to be applied to the source implies low pass filtering should occur in the direction of that axis. If the length of the axis is t o less than two, no low pass filtering should occur in the direction of that axis, and the length of that axis may be assumed to be one, as shown at 220. A unit is the distance between two adjacent pixels in the source image 210. Given the length and direction of the orthogonal axes R, and R2 of the ellipse 220, a linear transformation is determined which maps a unit circle 240 concentric with the ellipse 220 into the ellipse 220. A desired reconstruction filter 245, e.g., a bilinear or bicubic filter, also concentric with the unit circle 240 is modified according to this linear transformation. This modified reconstruction filter is represented at 250. Pixels 255 of the source image 210 that fall within the modified reconstruction filter 250 are used to determine a pixel value for point 230 in destination image 205.
One embodiment of the process described above in connection with Fig. 3 will now be described in more detail in connection with Fig. 4. This process starts in step 305 with selection of a special effect involving non-uniform scaling or arbitrary deformation of an image, which may be input to computer 20 (Fig. 2). The special effect may be a two-dimensional or three-dimensional effect, such as described above, which is used to transform a source image 210 into a destination image 205. Predetermined special effects may be stored in memory 28 ( Fig. 2) and may be selected according to user input into computer 20 (Fig. 2). Example arbitrary deformation functions include mapping a two-dimensional source image to a three-dimensional surface, such as in texture mapping, and then projecting the surface as a two-dimensional destination image. Another kind of arbitrary deformation function is an inverse perspective transformation into three-dimensional space followed by a mapping back into two-dimensional space. Other functions which scale an image non-uniformly also may be used.
A reconstruction filter is chosen in step 310. The filter may be selected from predetermined functions stored in memory 28, which also may have user selectable parameters.
Example reconstruction filters include bilinear filters, bicubic filters and box filters. In the example shown in Fig. 3, the reconstruction filter is a bilinear filter. A
value for a pixel in the destination image 205 is then determined through steps 315 through 350 which will now be described. In step 315, a point 230 in the destination image 205 is selected. The point may be the center of a pixel in the destination image, or may be an arbitrary point in the destination image space which is in turn used to compute the value of a particular pixel. The coordinates of the selected point 230 are used to define an affine transformation that maps a unit circle 235 centered around the selected point 230 into an ellipse 260 in the source image. This affine transformation locally approximates and represents the l0 inverse of the arbitrary deformation function that implements the special effect selected in step 305.
In one embodiment, the affine function is represented by coordinate functions uA(x,y) and vA(x,y) which map a point (x,y) in the destination image to a point (u,v) in the source image space. Assuming uA(x,y), vA(x,y) have partial derivatives at (xo,yo), point 230, the affine transformation representing the arbitrary differentiable function from (x,y) to (u,v) is:
uA(x,Y) =u(xo,Yo) * ax (xo,Ya)(x -xo) + d-~y (xp,Yo)(Y -Yo) (Eq. 1) V A(x,Y) =V (xa'Yo) + av (xoeYo)(x -xa) + av (xorYp)(Y -Yo) (Eq. 2) Here X is the partial derivative of u(x,y) with respect to x; and X (xo,Yo) is that partial derivative evaluated at (xo,yo), resulting in a numeric value.
In step 320, the ellipse 260 is defined in the source image 210 by computing, using the affine transformation, the length and direction of its orthogonal axes. Given an arbitrary deformation from source image space to destination image space, a point (xo , yo) in the destination image space, and given a local inverse deformation with coordinate functions u(x,y), v(x,y), it is possible to compute axis directions (u,,v,), (u2,v2) and magnitudes R,,RZ for the ellipse 220 which is the image under the inverse deformation of the unit circle 235 centered at (xo,yo), point 230.
. . ;
, . , -g- , õ ..
The functions u(x,y), v(x,y) may have well-defined partial derivatives at (xo,yo) which may be used to define values a, b, c, d as:
a= X (xo,Yo) (Eq. 3) b= ~ (xo~Yo) (Eq. 4) c= ~ (xo,Yo) (Eq= 5) d= ~ (xo,yo) (Eq. 6) If a=b=c=d=0, then R,=R,=O, and (u,,v,)=(1,0), (uZ,v2)=(0,1). Otherwise, if ad-bc=0, then Rj= a Z+b Z+c 2+d Z, R2=0. (Eq. 7) Since one of a, b, c, d is non-zero, either a Z+b 2 or c z+d 2 is non-zero. If a 2 +b Z is non-zero, then (ui,v1)= a b (Eq.8) 1aZ+b2 VaZ+b2 (u2,V2)= -b a (Eq. 9) Ja2+bz a2+b2 Ifc Z+d 2 is non-zero, then (ul,vi)= c d (Eq. 10) VC_T+_d2 cz+d2 (u2,v2)= -d c (Eq. 11) cz+dz c2+d2 AMENDED SHEEt, Otherwise, the axes are computed as follows:
bz+dz =
A = (Eq.12) (ad -bc)z B = -2 ab +cd E 13 (ad -bc)z ( q ) az+c2 c = (Eq. 14) (ad -bc)2 A -C+(A-CY +B2 2 (Eq. 15) lo If,X=0, R,= F (Eq. 16) RZ= (u1,v1)=(1,0), (uZ,vz)=(0,i) (Eq. 17) Otherwise, the following value is computed:
4)Lz(A +C) .
u = B 2+4).2 , (Eq.18) Then R,= 2V2- X E. 19 ( q ) B 2 +4 ~Lz R2= 2Vf2 ~
(Eq. 20) B I 2+4;L2 -) and (u],vi) = 2;L B (Eq.21) B z+47Lz B z+412 (u2,v2) = -B 2X (Eq. 22) B z+4~.2 B 2 +412 After the ellipse 260 is defined by determining R,, R2, (u,, v,) and (u2, v2), the ellipse 260 is stretched, in step 325, along its axes to make the axes at least two units in length to produce stretched ellipse 220. In particular, if either R, or R2 has a value less than 1, then its value is set to 1. The axes of the ellipse 220 then have a length 2R, and 2R2 in the directions defined by vectors (u,,v,) and (uZ,vZ), respectively, where R, and R2 are the half lengths of first and second orthogonal axes, respectively, of the ellipse 220. Therefore, the axes of the ellipse are defined by vectors (R, u,,R, v,) and (R2 u2,R2 v,).
Next, in step 330, a linear transformation from a unit circle 240 concentric with ellipse 220 to the ellipse 220 then is determined. The simplest linear transformation from the unit circle 240 to the ellipse 220 maps unit vectors (1,0) and (0,1) to vectors (R, u,,R, v) and (R2u2,R2v2), respectively, and has coordinate functions u'(u,v), v'(u,v) as follows:
u'(u,v) = R, u, u +R2 u2 V (Eq. 23) v'(u,v) = R, v, u+RZ v2 v (Eq. 24) where u and v are coordinates in the source image with (u,v) = (0,0) the origin of the ellipse 220.
The affine transformation used to create the ellipse 220 may introduce a rotational or skew component in any transformation between the unit circle 240 and the ellipse 220. The linear transformation shown in Equations 23 and 24 has a rotational component, but no skew component.
A linear transformation that removes both skew and rotational components also may be defined. An example of such a linear transformation maps (u,,v,) to (R,u,,R,v,) and (uz,v,) to (R, u2,R2 v2) using coordinate functions as follows:
u'(u,v) =((R, u, v,-R,u, v,)u -(R,-RZ)u, u2 v) /(u, v, - vl u,) (Eq. 25) v'(u,v) =((Rl-R,)v, v, u - (R, u2 vl-R2 u, vZ)v) /(u, V2 - v, u,) (Eq. 26) These transformations treat the center of the ellipse 220 as the origin in the source image space.
A translation to the actual coordinates may be performed by adding appropriate offsets to Equations 25 and 26 or to the application of its inverse in steps 340 to 345 below.
Next, in step 335, the selected reconstruction filter 245 centered around the origin 225 is modified to obtain a new reconstruction filter 250 centered about the origin 225 by defining a filter that first applies the inverse of the linear transformation to the coordinates of a point and then filters those points with the original selected reconstruction filter 245. Note that the modified filter and the original filter may be the same if the linear transformation is a unity transformation, which simplifies the processing of steps 340 and 345 described below. If the size of the selected reconstruction filter when centered on the unit circle 240 includes the area of the unit circle, then the modified reconstruction filter includes the entire ellipse 220, as shown.
However, the selected reconstruction filter 245 may be included within the unit circle 240, such as a box filter with a radius of one-half. Since the filter 245 in that case is smaller than the unit circle 240, then the modified filter 250 is smaller than the ellipse 220. In step 340, the pixels 255 in the source image 210 included in the area defined by the modified reconstruction filter 250 are identified. This support area of the reconstruction filter includes any pixels where the filter is non-zero. Several methods may be used to identify the pixels with centers included in the support of the transformed filter 250. In one embodiment, a rectangular bounding box which includes the support of the modified filter is defined. This bounding box may be defined by applying the linear transformation determined in step 330 to the coordinates of the four corners of the filter. For each pixel in the source image with a center 1 o inside the bounding box, the inverse of Equations 25 and 26 is applied to its coordinates to obtain transformed coordinates in the source image space. If the transformed coordinates of the pixel lie inside of the support of the original unmodified filter in the source image space, then the pixel is within the support of the modified filter.
In another embodiment, only those pixels with centers in the support of the modified filter are identified, with the support defined by a polygon. Then a standard polygon rasterizing technique, such as using the Bresenham line drawing algorithm along the sides of the polygon may be used. The pixels which lie between the left and right lines are then identified and the inverse of the linear transform of Equations 25 and 26 is applied to the coordinates of each pixel to obtain transformed coordinates in the source image space. If the transformed coordinates of the pixel lie inside of the support of the original unmodified filter in the source image space, then the pixel is within the support of the modified filter.
The values, such as color and intensity, of pixels 255 included in the support of modified reconstruction filter 250 are then used to determine a pixel value for the selected pixel 230 in destination image 205, in step 345. In one embodiment, the color value of each pixel within the support of the reconstruction filter is scaled by a corresponding value from the original unmodified filter. These scaled values are averaged to provide a color value for the selected pixel 230 in the destination image.
If a value for each pixel in destination image 205 has been calculated, as determined in step 350, then the process ends and the destination image 205 is rendered in step 355. Otherwise a next pixel is selected and the process continues with step 315 as described above.
Fig. 5 illustrates a corresponding system that implements the embodiment described by the flowchart of Fig. 4, and illustrates in more detail one embodiment of the system of Fig. 1.
This system may be implemented using a high-level computer programming language on a computer system, such as described above in Fig. 2. In this system, an affine transformation selection module 415 receives user input 405 typically via a graphical user interface. The user input 405 includes an indication of a desired special effect to be achieved that performs non-uniform scaling or arbitrary deformation. Given a specified special effect 405, an affine transformation 420 is generated or retrieved, which represents and approximates the arbitrary deformation of the special effect. How such an affine transformation is generated is well known in the art. The affine transformation 420 may be stored in memory.
The ellipse definer 430 receives an indication of a point 425 identified in the destination l o image and a source image 410 and uses the affine transformation 420 to define an ellipse in the source image 410 which corresponds to a transformed unit circle defined about point 425. The ellipse is defined to have axes of at least two units in length as described above. An indication of the center 435 of the defined ellipse, and the length and direction of its orthogonal axes is output by the ellipse definer 430 to transformation unit 445. The transformation unit 445 defines a linear transformation between a unit circle 440 in the source image around the defined center of the ellipse 435 and the ellipse. Such a linear transformation is described above. The linear transformation 450 required to scale the unit circle 440 may be stored in memory for later use.
A reconstruction filter 455, as described above, is input along with an indication of the linear transformation 450 to a filter transformation unit 460. The unit 460 outputs a modified filter 465 as described above in step 335 of Fig. 4. The modified filter may be stored in memory and applied to other source images. The identification unit 470 determines which pixels in the source image are included in the domain or support of the modified filter 465.
The identification unit 470 may use various methods to determine which pixels are included in the support of the modified filter, as discussed above in reference to step 340 of Fig. 4. The values of the pixels 475 in the source image included in the support of the filter 465 are stored in memory. A
calculator unit 480 receives pixel values 475 to determine the value of a point in a destination image. The calculator unit may use any of variety of functions to make this determination. For example, the values for the pixels in the support of the filter may be filtered then averaged to determine the pixel value for the destination image 485. A display device 490 displays the non-uniformly scaled image.
The process and apparatus of the present invention may be applied to resample an image of three dimensions or higher. In such applications, a unit circle 240 (Fig. 3) in the source image 210 is an N-dimensional sphere and an ellipse 220 in the source image 210 is an N-dimensional ellipsoid. The source image, such as image 210 may be a three-dimensional image including volumetric data such as that used for medical imaging or weather applications. The source image 210 may also be a four-dimensional image, which would include, for example, a three-dimensional image changing over time. In addition, the source image in any dimension may be a single image or it may be a sequence of images.
Another embodiment of the present invention includes optimizing the resampling process shown in Fig. 3. Since the non-uniform scaling of the source image is performed by an arbitrary function, the ellipse computations for each pixel of the destination image may be too lo time consuming and expensive for some applications. Therefore, the process as shown in Figs. 3 and 4 may be optimized to approximate the arbitrary scaling using a piecewise approach. In this embodiment, rather than identifying an individual pixel, a plurality of triangles may be defined in the destination image. Each triangle would represent a collection of points in the destination image. The vertices of each triangle are then used to determine the non-uniform scaling function from the destination image to the source image. An affine transformation representing the non-uniform scaling function of each vertex, and therefore an affine transformation on each individual triangle, based on each vertex, is performed to define an ellipse in a source image.
The affine transformation represents the non-uniform scaling function of the source image and the ellipse calculations are performed one time per triangle, and thus is applied to several pixels at a time. The linear transformation performed to transform a unit circle centered around a center point of an ellipse to the ellipse is then applied to the reconstruction filter.
Having now described a few embodiments of the invention, it should be apparent to those skilled in the art that the foregoing is merely illustrative and not limiting, having been presented by way of example only. Numerous modifications and other embodiments are within the scope of one of ordinary skill in the art and are contemplated as falling within the scope of the invention as defined by the appended claims and equivalents thereto.
value for a pixel in the destination image 205 is then determined through steps 315 through 350 which will now be described. In step 315, a point 230 in the destination image 205 is selected. The point may be the center of a pixel in the destination image, or may be an arbitrary point in the destination image space which is in turn used to compute the value of a particular pixel. The coordinates of the selected point 230 are used to define an affine transformation that maps a unit circle 235 centered around the selected point 230 into an ellipse 260 in the source image. This affine transformation locally approximates and represents the l0 inverse of the arbitrary deformation function that implements the special effect selected in step 305.
In one embodiment, the affine function is represented by coordinate functions uA(x,y) and vA(x,y) which map a point (x,y) in the destination image to a point (u,v) in the source image space. Assuming uA(x,y), vA(x,y) have partial derivatives at (xo,yo), point 230, the affine transformation representing the arbitrary differentiable function from (x,y) to (u,v) is:
uA(x,Y) =u(xo,Yo) * ax (xo,Ya)(x -xo) + d-~y (xp,Yo)(Y -Yo) (Eq. 1) V A(x,Y) =V (xa'Yo) + av (xoeYo)(x -xa) + av (xorYp)(Y -Yo) (Eq. 2) Here X is the partial derivative of u(x,y) with respect to x; and X (xo,Yo) is that partial derivative evaluated at (xo,yo), resulting in a numeric value.
In step 320, the ellipse 260 is defined in the source image 210 by computing, using the affine transformation, the length and direction of its orthogonal axes. Given an arbitrary deformation from source image space to destination image space, a point (xo , yo) in the destination image space, and given a local inverse deformation with coordinate functions u(x,y), v(x,y), it is possible to compute axis directions (u,,v,), (u2,v2) and magnitudes R,,RZ for the ellipse 220 which is the image under the inverse deformation of the unit circle 235 centered at (xo,yo), point 230.
. . ;
, . , -g- , õ ..
The functions u(x,y), v(x,y) may have well-defined partial derivatives at (xo,yo) which may be used to define values a, b, c, d as:
a= X (xo,Yo) (Eq. 3) b= ~ (xo~Yo) (Eq. 4) c= ~ (xo,Yo) (Eq= 5) d= ~ (xo,yo) (Eq. 6) If a=b=c=d=0, then R,=R,=O, and (u,,v,)=(1,0), (uZ,v2)=(0,1). Otherwise, if ad-bc=0, then Rj= a Z+b Z+c 2+d Z, R2=0. (Eq. 7) Since one of a, b, c, d is non-zero, either a Z+b 2 or c z+d 2 is non-zero. If a 2 +b Z is non-zero, then (ui,v1)= a b (Eq.8) 1aZ+b2 VaZ+b2 (u2,V2)= -b a (Eq. 9) Ja2+bz a2+b2 Ifc Z+d 2 is non-zero, then (ul,vi)= c d (Eq. 10) VC_T+_d2 cz+d2 (u2,v2)= -d c (Eq. 11) cz+dz c2+d2 AMENDED SHEEt, Otherwise, the axes are computed as follows:
bz+dz =
A = (Eq.12) (ad -bc)z B = -2 ab +cd E 13 (ad -bc)z ( q ) az+c2 c = (Eq. 14) (ad -bc)2 A -C+(A-CY +B2 2 (Eq. 15) lo If,X=0, R,= F (Eq. 16) RZ= (u1,v1)=(1,0), (uZ,vz)=(0,i) (Eq. 17) Otherwise, the following value is computed:
4)Lz(A +C) .
u = B 2+4).2 , (Eq.18) Then R,= 2V2- X E. 19 ( q ) B 2 +4 ~Lz R2= 2Vf2 ~
(Eq. 20) B I 2+4;L2 -) and (u],vi) = 2;L B (Eq.21) B z+47Lz B z+412 (u2,v2) = -B 2X (Eq. 22) B z+4~.2 B 2 +412 After the ellipse 260 is defined by determining R,, R2, (u,, v,) and (u2, v2), the ellipse 260 is stretched, in step 325, along its axes to make the axes at least two units in length to produce stretched ellipse 220. In particular, if either R, or R2 has a value less than 1, then its value is set to 1. The axes of the ellipse 220 then have a length 2R, and 2R2 in the directions defined by vectors (u,,v,) and (uZ,vZ), respectively, where R, and R2 are the half lengths of first and second orthogonal axes, respectively, of the ellipse 220. Therefore, the axes of the ellipse are defined by vectors (R, u,,R, v,) and (R2 u2,R2 v,).
Next, in step 330, a linear transformation from a unit circle 240 concentric with ellipse 220 to the ellipse 220 then is determined. The simplest linear transformation from the unit circle 240 to the ellipse 220 maps unit vectors (1,0) and (0,1) to vectors (R, u,,R, v) and (R2u2,R2v2), respectively, and has coordinate functions u'(u,v), v'(u,v) as follows:
u'(u,v) = R, u, u +R2 u2 V (Eq. 23) v'(u,v) = R, v, u+RZ v2 v (Eq. 24) where u and v are coordinates in the source image with (u,v) = (0,0) the origin of the ellipse 220.
The affine transformation used to create the ellipse 220 may introduce a rotational or skew component in any transformation between the unit circle 240 and the ellipse 220. The linear transformation shown in Equations 23 and 24 has a rotational component, but no skew component.
A linear transformation that removes both skew and rotational components also may be defined. An example of such a linear transformation maps (u,,v,) to (R,u,,R,v,) and (uz,v,) to (R, u2,R2 v2) using coordinate functions as follows:
u'(u,v) =((R, u, v,-R,u, v,)u -(R,-RZ)u, u2 v) /(u, v, - vl u,) (Eq. 25) v'(u,v) =((Rl-R,)v, v, u - (R, u2 vl-R2 u, vZ)v) /(u, V2 - v, u,) (Eq. 26) These transformations treat the center of the ellipse 220 as the origin in the source image space.
A translation to the actual coordinates may be performed by adding appropriate offsets to Equations 25 and 26 or to the application of its inverse in steps 340 to 345 below.
Next, in step 335, the selected reconstruction filter 245 centered around the origin 225 is modified to obtain a new reconstruction filter 250 centered about the origin 225 by defining a filter that first applies the inverse of the linear transformation to the coordinates of a point and then filters those points with the original selected reconstruction filter 245. Note that the modified filter and the original filter may be the same if the linear transformation is a unity transformation, which simplifies the processing of steps 340 and 345 described below. If the size of the selected reconstruction filter when centered on the unit circle 240 includes the area of the unit circle, then the modified reconstruction filter includes the entire ellipse 220, as shown.
However, the selected reconstruction filter 245 may be included within the unit circle 240, such as a box filter with a radius of one-half. Since the filter 245 in that case is smaller than the unit circle 240, then the modified filter 250 is smaller than the ellipse 220. In step 340, the pixels 255 in the source image 210 included in the area defined by the modified reconstruction filter 250 are identified. This support area of the reconstruction filter includes any pixels where the filter is non-zero. Several methods may be used to identify the pixels with centers included in the support of the transformed filter 250. In one embodiment, a rectangular bounding box which includes the support of the modified filter is defined. This bounding box may be defined by applying the linear transformation determined in step 330 to the coordinates of the four corners of the filter. For each pixel in the source image with a center 1 o inside the bounding box, the inverse of Equations 25 and 26 is applied to its coordinates to obtain transformed coordinates in the source image space. If the transformed coordinates of the pixel lie inside of the support of the original unmodified filter in the source image space, then the pixel is within the support of the modified filter.
In another embodiment, only those pixels with centers in the support of the modified filter are identified, with the support defined by a polygon. Then a standard polygon rasterizing technique, such as using the Bresenham line drawing algorithm along the sides of the polygon may be used. The pixels which lie between the left and right lines are then identified and the inverse of the linear transform of Equations 25 and 26 is applied to the coordinates of each pixel to obtain transformed coordinates in the source image space. If the transformed coordinates of the pixel lie inside of the support of the original unmodified filter in the source image space, then the pixel is within the support of the modified filter.
The values, such as color and intensity, of pixels 255 included in the support of modified reconstruction filter 250 are then used to determine a pixel value for the selected pixel 230 in destination image 205, in step 345. In one embodiment, the color value of each pixel within the support of the reconstruction filter is scaled by a corresponding value from the original unmodified filter. These scaled values are averaged to provide a color value for the selected pixel 230 in the destination image.
If a value for each pixel in destination image 205 has been calculated, as determined in step 350, then the process ends and the destination image 205 is rendered in step 355. Otherwise a next pixel is selected and the process continues with step 315 as described above.
Fig. 5 illustrates a corresponding system that implements the embodiment described by the flowchart of Fig. 4, and illustrates in more detail one embodiment of the system of Fig. 1.
This system may be implemented using a high-level computer programming language on a computer system, such as described above in Fig. 2. In this system, an affine transformation selection module 415 receives user input 405 typically via a graphical user interface. The user input 405 includes an indication of a desired special effect to be achieved that performs non-uniform scaling or arbitrary deformation. Given a specified special effect 405, an affine transformation 420 is generated or retrieved, which represents and approximates the arbitrary deformation of the special effect. How such an affine transformation is generated is well known in the art. The affine transformation 420 may be stored in memory.
The ellipse definer 430 receives an indication of a point 425 identified in the destination l o image and a source image 410 and uses the affine transformation 420 to define an ellipse in the source image 410 which corresponds to a transformed unit circle defined about point 425. The ellipse is defined to have axes of at least two units in length as described above. An indication of the center 435 of the defined ellipse, and the length and direction of its orthogonal axes is output by the ellipse definer 430 to transformation unit 445. The transformation unit 445 defines a linear transformation between a unit circle 440 in the source image around the defined center of the ellipse 435 and the ellipse. Such a linear transformation is described above. The linear transformation 450 required to scale the unit circle 440 may be stored in memory for later use.
A reconstruction filter 455, as described above, is input along with an indication of the linear transformation 450 to a filter transformation unit 460. The unit 460 outputs a modified filter 465 as described above in step 335 of Fig. 4. The modified filter may be stored in memory and applied to other source images. The identification unit 470 determines which pixels in the source image are included in the domain or support of the modified filter 465.
The identification unit 470 may use various methods to determine which pixels are included in the support of the modified filter, as discussed above in reference to step 340 of Fig. 4. The values of the pixels 475 in the source image included in the support of the filter 465 are stored in memory. A
calculator unit 480 receives pixel values 475 to determine the value of a point in a destination image. The calculator unit may use any of variety of functions to make this determination. For example, the values for the pixels in the support of the filter may be filtered then averaged to determine the pixel value for the destination image 485. A display device 490 displays the non-uniformly scaled image.
The process and apparatus of the present invention may be applied to resample an image of three dimensions or higher. In such applications, a unit circle 240 (Fig. 3) in the source image 210 is an N-dimensional sphere and an ellipse 220 in the source image 210 is an N-dimensional ellipsoid. The source image, such as image 210 may be a three-dimensional image including volumetric data such as that used for medical imaging or weather applications. The source image 210 may also be a four-dimensional image, which would include, for example, a three-dimensional image changing over time. In addition, the source image in any dimension may be a single image or it may be a sequence of images.
Another embodiment of the present invention includes optimizing the resampling process shown in Fig. 3. Since the non-uniform scaling of the source image is performed by an arbitrary function, the ellipse computations for each pixel of the destination image may be too lo time consuming and expensive for some applications. Therefore, the process as shown in Figs. 3 and 4 may be optimized to approximate the arbitrary scaling using a piecewise approach. In this embodiment, rather than identifying an individual pixel, a plurality of triangles may be defined in the destination image. Each triangle would represent a collection of points in the destination image. The vertices of each triangle are then used to determine the non-uniform scaling function from the destination image to the source image. An affine transformation representing the non-uniform scaling function of each vertex, and therefore an affine transformation on each individual triangle, based on each vertex, is performed to define an ellipse in a source image.
The affine transformation represents the non-uniform scaling function of the source image and the ellipse calculations are performed one time per triangle, and thus is applied to several pixels at a time. The linear transformation performed to transform a unit circle centered around a center point of an ellipse to the ellipse is then applied to the reconstruction filter.
Having now described a few embodiments of the invention, it should be apparent to those skilled in the art that the foregoing is merely illustrative and not limiting, having been presented by way of example only. Numerous modifications and other embodiments are within the scope of one of ordinary skill in the art and are contemplated as falling within the scope of the invention as defined by the appended claims and equivalents thereto.
Claims (59)
1. A process for obtaining a value for a pixel in a destination image produced by non-uniform transformation of a source image, comprising the steps of:
selecting a reconstruction filter;
applying an affine transformation determined by the non-uniform transformation on an area centered on the pixel in the destination image to define at least two axes in a source image space;
determining a linear transformation between an area in the source image space and the at least two axes, wherein the linear transformation corresponds to the non-uniform transformation and represents the shape of the area in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation, wherein the area is centered around a center of the at least two axes;
modifying the selected reconstruction filter using the linear transformation to create a modified reconstruction filter with a support region;
identifying which pixels from the source image are included in the support region of the modified reconstruction filter; and determining the value for the pixel in the destination image from pixel centers of the source image included in the support of the modified reconstruction filter.
selecting a reconstruction filter;
applying an affine transformation determined by the non-uniform transformation on an area centered on the pixel in the destination image to define at least two axes in a source image space;
determining a linear transformation between an area in the source image space and the at least two axes, wherein the linear transformation corresponds to the non-uniform transformation and represents the shape of the area in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation, wherein the area is centered around a center of the at least two axes;
modifying the selected reconstruction filter using the linear transformation to create a modified reconstruction filter with a support region;
identifying which pixels from the source image are included in the support region of the modified reconstruction filter; and determining the value for the pixel in the destination image from pixel centers of the source image included in the support of the modified reconstruction filter.
2. A process as claimed in claim 1, wherein the identifying step further includes:
creating a rectangular bounding box which includes the support of the modified filter;
applying an inverse transformation to coordinates of each pixel with a center inside the bounding box; and determining whether the coordinates are included in the support of the unmodified selected reconstruction filter.
creating a rectangular bounding box which includes the support of the modified filter;
applying an inverse transformation to coordinates of each pixel with a center inside the bounding box; and determining whether the coordinates are included in the support of the unmodified selected reconstruction filter.
3. A process as claimed in claim 1, wherein the determining a value step includes:
multiplying a color value of each pixel included in the support of the modified filter by the value from the unmodified selected reconstruction filter;
determining a weighted average of the pixel values included in the modified filter.
multiplying a color value of each pixel included in the support of the modified filter by the value from the unmodified selected reconstruction filter;
determining a weighted average of the pixel values included in the modified filter.
4. A process as claimed in claim 1, wherein the identifying step further includes:
determining pixels with centers in the support of the modified filter by treating the support of the transformed filter as a polygon; and using a polygon rasterizing technique.
determining pixels with centers in the support of the modified filter by treating the support of the transformed filter as a polygon; and using a polygon rasterizing technique.
5. A process as claimed in claim 1, wherein the affine transformation step is performed for a plurality of pixels.
6. A process as claimed in claim 1, wherein the performing an affine transformation step includes approximating the non-uniform transformation for implementing two-dimensional special effects by a function.
7. A process as claimed in claim 1, wherein the performing an affine transformation step includes approximating the non-uniform transformation for implementing three-dimensional effects.
8. A process as claimed in claim 1, wherein the determining a linear transformation step includes removing a skew component.
9. A process as claimed in claim 1, wherein the determining a linear transformation step includes removing a skew and a rotational component.
10. A process as claimed in claim 1, wherein the performing an affine transformation step is followed by performing a perspective projection.
11. A process as claimed in claim 1, wherein the selected reconstruction filter is a quadrilateral.
12. A process as claimed in claim 1, where the at least two axes determined by the affine transformation step are eigenvectors.
13. A computer-implemented process for generating a non-uniformly transformed destination image including the steps of:
determining an affine transformation of a source image based on a non-uniform transformation;
defining axes in the source image according to the affine transformation;
calculating a linear transformation from an area in the source image to the axes, wherein the linear transformation corresponds to a non-uniform transformation and represents the shape of an area in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation;
modifying a reconstruction filter in the source image according to the linear transformation;
identifying pixels of the source image included in a support of the modified reconstruction filter;
calculating the value of a point in the destination image based on the value of each of the identified pixels in the source image; and displaying the non-uniformly transformed destination image.
determining an affine transformation of a source image based on a non-uniform transformation;
defining axes in the source image according to the affine transformation;
calculating a linear transformation from an area in the source image to the axes, wherein the linear transformation corresponds to a non-uniform transformation and represents the shape of an area in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation;
modifying a reconstruction filter in the source image according to the linear transformation;
identifying pixels of the source image included in a support of the modified reconstruction filter;
calculating the value of a point in the destination image based on the value of each of the identified pixels in the source image; and displaying the non-uniformly transformed destination image.
14. A computer-implemented process as claimed in claim 13, wherein the reconstruction filter is a quadrilateral.
15. A computer-implemented process as claimed in claim 13, wherein the identifying step further includes:
creating a rectangular bounding box which includes the support of the modified filter;
applying an inverse transformation to coordinates of each pixel with a center inside the bounding box; and determining whether the coordinates are included in the support of the unmodified reconstruction filter.
creating a rectangular bounding box which includes the support of the modified filter;
applying an inverse transformation to coordinates of each pixel with a center inside the bounding box; and determining whether the coordinates are included in the support of the unmodified reconstruction filter.
16. A computer-implemented process as claimed in claim 13, where the axes defined according to the affine transformation are eigenvectors.
17. A computer system for generating a representation of a non-uniformly transformed source image including a processor and a memory connected to the processor for storing a program, which when executed by the processor performs the steps of:
receiving a source image, a desired special effect, and a reconstruction filter;
modifying the reconstruction filter based on the application of the desired special effect to the source image;
identifying pixels in the source image included in a support of the modified reconstruction filter;
determining the value of a pixel in a destination based on the identified pixels; and rendering the destination image.
receiving a source image, a desired special effect, and a reconstruction filter;
modifying the reconstruction filter based on the application of the desired special effect to the source image;
identifying pixels in the source image included in a support of the modified reconstruction filter;
determining the value of a pixel in a destination based on the identified pixels; and rendering the destination image.
18. A computer system as claimed in claim 17, wherein the source image is a single image.
19. A computer system as claimed in claim 17, wherein the source image is a sequence of images.
20. A computer system as claimed in claim 17, wherein the modified reconstruction filter does not include at least one of a skew or a rotational component.
21. A computer system as claimed in claim 17, wherein the special effect includes two-dimensional or higher dimensional effects.
22. A computer system as claimed in claim 17, wherein the reconstruction filter is a quadrilateral.
23. A computer system as claimed in claim 17, wherein the identifying step further includes:
creating a rectangular bounding box which includes the support of the modified filter;
applying an inverse transformation to coordinates of each pixel with a center inside the bounding box; and determining whether the coordinates are included in the support of the unmodified reconstruction filter.
creating a rectangular bounding box which includes the support of the modified filter;
applying an inverse transformation to coordinates of each pixel with a center inside the bounding box; and determining whether the coordinates are included in the support of the unmodified reconstruction filter.
24. A process for obtaining a value for a pixel in a destination image produced by non-uniform transformation of a source image, comprising the steps of:
selecting a reconstruction filter;
determining a linear transformation between a unit circle in a source image space and at least two orthogonal axes in the source image space, wherein the linear transformation corresponds to the non-uniform transformation and represents the shape of the unit circle in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation, wherein the unit circle is centered around a center of the at least two orthogonal axes;
modifying the selected reconstruction filter using the transformation to create a modified reconstruction filter with a support region;
identifying which pixels from the source image are included in the support region of the modified reconstruction filter; and determining the value for the pixel in the destination image from pixel centers of the source image included in the support of the modified reconstruction filter.
selecting a reconstruction filter;
determining a linear transformation between a unit circle in a source image space and at least two orthogonal axes in the source image space, wherein the linear transformation corresponds to the non-uniform transformation and represents the shape of the unit circle in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation, wherein the unit circle is centered around a center of the at least two orthogonal axes;
modifying the selected reconstruction filter using the transformation to create a modified reconstruction filter with a support region;
identifying which pixels from the source image are included in the support region of the modified reconstruction filter; and determining the value for the pixel in the destination image from pixel centers of the source image included in the support of the modified reconstruction filter.
25. A process as claimed in claim 24, wherein the determining a linear transformation step further includes:
applying an affine transformation determined by the non-uniform transformation on a unit circle centered on the pixel in the destination image to define the at least two orthogonal axes.
applying an affine transformation determined by the non-uniform transformation on a unit circle centered on the pixel in the destination image to define the at least two orthogonal axes.
26. A process as claimed in claim 24, wherein the identifying step further includes:
creating a rectangular bounding box which includes the support of the modified filter;
applying an inverse transformation to coordinates of each pixel with a center inside the bounding box; and determining whether the coordinates are included in the support of the unmodified selected reconstruction filter.
creating a rectangular bounding box which includes the support of the modified filter;
applying an inverse transformation to coordinates of each pixel with a center inside the bounding box; and determining whether the coordinates are included in the support of the unmodified selected reconstruction filter.
27. A process as claimed in claim24, wherein the determining a value step includes:
multiplying a color value of each pixel included in the support of the modified filter by the value from the unmodified selected reconstruction filter;
determining a weighted average of the pixel values included in the modified filter.
multiplying a color value of each pixel included in the support of the modified filter by the value from the unmodified selected reconstruction filter;
determining a weighted average of the pixel values included in the modified filter.
28. A process as claimed in claim 24, wherein the identifying step further includes:
determining pixels with centers in the support of the modified filter by treating the support of the transformed filter as a polygon; and using a polygon rasterizing technique.
determining pixels with centers in the support of the modified filter by treating the support of the transformed filter as a polygon; and using a polygon rasterizing technique.
29. A process as claimed in claim 24, wherein the linear transformation step is performed for a plurality of pixels.
30. A process as claimed in claim 24, wherein the determining a linear transformation step includes approximating the non-uniform transformation for implementing two-dimensional special effects by a function.
31. A process as claimed in claim 24, wherein the determining a linear transformation step includes approximating the non-uniform transformation for implementing three-dimensional effects by a function.
32. A process as claimed in claim 24, wherein the determining a linear transformation step includes removing a skew component.
33. A process as claimed in claim 24, wherein the determining a linear transformation step includes removing a skew and a rotational component.
34. A process as claimed in claim 25, wherein the applying an affine transformation step is followed by performing a perspective projection.
35 A process as claimed in claim 24, wherein the selected reconstruction filter is a quadrilateral
36. A process as claimed in claim 24, where the at least two orthogonal axes determined by the linear transformation step are orthogonal eigenvectors.
37 A process f6r obtaining a value for a pixel in a destination image produced by non-uniform transformation of a source image, comprising the steps of:
selecting a reconstruction filter;
determining a linear transformation between an area in a source image space and at least two axes, wherein the linear transformation corresponds to the non-uniform transformation and represents the shape of the area in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation, wherein the area is centered around a center of the at least two axes, wherein the at least two axes are at least two units in length, wherein each unit is equal to a distance between two adjacent pixels;
modifying the selected reconstruction filter using the transformation to create a modified reconstruction filter with a support region;
identifying which pixels from the source image are included in the support region of the modified reconstruction filter; and determining the value for the pixel in the destination image from pixel centers of the source image included in the support of the modified reconstruction filter.
selecting a reconstruction filter;
determining a linear transformation between an area in a source image space and at least two axes, wherein the linear transformation corresponds to the non-uniform transformation and represents the shape of the area in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation, wherein the area is centered around a center of the at least two axes, wherein the at least two axes are at least two units in length, wherein each unit is equal to a distance between two adjacent pixels;
modifying the selected reconstruction filter using the transformation to create a modified reconstruction filter with a support region;
identifying which pixels from the source image are included in the support region of the modified reconstruction filter; and determining the value for the pixel in the destination image from pixel centers of the source image included in the support of the modified reconstruction filter.
38. A process as claimed in claim 37, wherein the determining a linear transformation step further includes:
applying an affine transformation determined by the non-uniform transformation on an area centered on the pixel in the destination image to define the at least two axes.
applying an affine transformation determined by the non-uniform transformation on an area centered on the pixel in the destination image to define the at least two axes.
39. A process as claimed in claim 37, wherein the identifying step further,includes:
creating a rectangular bounding box which includes the support of the modified filter;
applying an inverse transformation to coordinates of each pixel with a center inside the bounding box; and determining whether the coordinates are included in the support of the unmodified selected reconstruction filter.
creating a rectangular bounding box which includes the support of the modified filter;
applying an inverse transformation to coordinates of each pixel with a center inside the bounding box; and determining whether the coordinates are included in the support of the unmodified selected reconstruction filter.
40. A process as claimed in claim 37, wherein the determining a value step includes:
multiplying a color value of each pixel included in the support of the modified filter by the value from the unmodified selected reconstruction filter;
determining a weighted average of the pixel values included in the modified filter.
multiplying a color value of each pixel included in the support of the modified filter by the value from the unmodified selected reconstruction filter;
determining a weighted average of the pixel values included in the modified filter.
41. A process as claimed in claim 37, wherein the identifying step further includes:
determining pixels with centers in the support of the modified filter by treating the support of the transformed filter as a polygon; and using a polygon rasterizing technique.
determining pixels with centers in the support of the modified filter by treating the support of the transformed filter as a polygon; and using a polygon rasterizing technique.
42. A process as claimed in claim 37, wherein the linear transformation step is performed for a plurality of pixels.
43. A process as claimed in claim 37, wherein the determining a linear transformation step includes approximating the non-uniform transformation for implementing two-dimensional special effects by a function.
44. A process as claimed in claim 37, wherein the determining a linear transformation step includes approximating the non-uniform transformation for implementing three-dimensional effects by a function.
45. A process as claimed in claim 37, wherein the determining a linear transformation step includes removing a skew component.
46. A process as claimed in claim 37, wherein the determining a linear transformation step includes removing a skew and a rotational component.
47. A process as claimed in claim 38, wherein the applying an affine transformation step is followed by performing a perspective projection.
48. A process as claimed in claim 37, wherein the selected reconstruction filter is a quadrilateral.
49. A process as claimed in claim 37, where the at least two axes determined by the linear transformation step are eigenvectors.
50. A computer-implemented process for generating a non-uniformly transformed destination image including the steps of:
calculating a linear transformation from a unit circle in a source image to orthogonal axes in the source image, wherein the linear transformation corresponds to a non-uniform transformation and represents the shape of a unit circle in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation;
modifying a reconstruction filter in the source image according to the linear transformation;
identifying pixels of the source image included in a support of the modified reconstruction filter;
calculating the value of a point in the destination image based on the value of each of the identified pixels in the source image; and displaying the non-uniformly transformed destination image.
calculating a linear transformation from a unit circle in a source image to orthogonal axes in the source image, wherein the linear transformation corresponds to a non-uniform transformation and represents the shape of a unit circle in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation;
modifying a reconstruction filter in the source image according to the linear transformation;
identifying pixels of the source image included in a support of the modified reconstruction filter;
calculating the value of a point in the destination image based on the value of each of the identified pixels in the source image; and displaying the non-uniformly transformed destination image.
51. A computer-implemented process as claimed in claim 50, wherein the reconstruction filter is a quadrilateral.
52. A computer-implemented process as claimed in claim 50, wherein the identifying step further includes:
creating a rectangular bounding box which includes the support of the modified filter;
applying an inverse transformation to coordinates of each pixel with a center inside the bounding box; and determining whether the coordinates are included in the support of the unmodified reconstruction filter.
creating a rectangular bounding box which includes the support of the modified filter;
applying an inverse transformation to coordinates of each pixel with a center inside the bounding box; and determining whether the coordinates are included in the support of the unmodified reconstruction filter.
53. A computer-implemented process as claimed in claim 50, where the orthogonal axes defined according to the linear transformation are orthogonal eigenvectors.
54. A computer-implemented process as claimed in claim 50, wherein the calculating a linear transformation step further includes:
determining an affine transformation of the source image based on the non-uniform effect; and defining the orthogonal axes according to the affine transformation.
determining an affine transformation of the source image based on the non-uniform effect; and defining the orthogonal axes according to the affine transformation.
55. A computer-implemented process for generating a non-uniformly transformed destination image including the steps of:
calculating a linear transformation from an area in a source image to axes in the source image, wherein the linear transformation corresponds to a non-uniform transformation and represents the shape of the area in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation, wherein the axes are at least two units in length, wherein each unit is equal to a distance between two adjacent pixels;
modifying a reconstruction filter in the source image according to the linear transformation;
identifying pixels of the source image included in a support of the modified reconstruction filter;
calculating the value of a point in the destination image based on the value of each of the identified pixels in the source image; and displaying the non-uniformly transformed destination image.
calculating a linear transformation from an area in a source image to axes in the source image, wherein the linear transformation corresponds to a non-uniform transformation and represents the shape of the area in the destination image transformed by a transformation defined according to the inverse of the non-uniform transformation, wherein the axes are at least two units in length, wherein each unit is equal to a distance between two adjacent pixels;
modifying a reconstruction filter in the source image according to the linear transformation;
identifying pixels of the source image included in a support of the modified reconstruction filter;
calculating the value of a point in the destination image based on the value of each of the identified pixels in the source image; and displaying the non-uniformly transformed destination image.
56. A computer-implemented process as claimed in claim 55, wherein the reconstruction filter is a quadrilateral.
57. A computer-implemented process as claimed in claim 55, wherein the identifying step further includes:
creating a rectangular bounding box which includes the support of the modified filter;
applying an inverse transformation to coordinates of each pixel with a center inside the bounding box; and determining whether the coordinates are included in the support of the unmodified reconstruction filter.
creating a rectangular bounding box which includes the support of the modified filter;
applying an inverse transformation to coordinates of each pixel with a center inside the bounding box; and determining whether the coordinates are included in the support of the unmodified reconstruction filter.
58. A computer-implemented process as claimed in claim 55 , where the axes defined according to the transformation are eigenvectors.
59. A computer-implemented process as claimed in claim 55 , wherein the calculating a linear transformation step further includes:
determining an affine transformation of the source image based on the non-uniform effect; and defining the axes according to the affine transformation.
determining an affine transformation of the source image based on the non-uniform effect; and defining the axes according to the affine transformation.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US08/904,470 | 1997-08-01 | ||
| US08/904,470 US6016152A (en) | 1997-08-01 | 1997-08-01 | Apparatus and method for non-uniform image scaling |
| PCT/US1998/010763 WO1999006952A1 (en) | 1997-08-01 | 1998-05-27 | Texture mapping in 3-d computer graphics |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2297916A1 CA2297916A1 (en) | 1999-02-11 |
| CA2297916C true CA2297916C (en) | 2009-01-27 |
Family
ID=25419222
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002297916A Expired - Fee Related CA2297916C (en) | 1997-08-01 | 1998-05-27 | Texture mapping in 3-d computer graphics |
Country Status (7)
| Country | Link |
|---|---|
| US (1) | US6016152A (en) |
| EP (1) | EP0998727B1 (en) |
| JP (1) | JP2001512265A (en) |
| AU (1) | AU7697498A (en) |
| CA (1) | CA2297916C (en) |
| DE (1) | DE69816824T2 (en) |
| WO (1) | WO1999006952A1 (en) |
Families Citing this family (38)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6259827B1 (en) | 1996-03-21 | 2001-07-10 | Cognex Corporation | Machine vision methods for enhancing the contrast between an object and its background using multiple on-axis images |
| US6381375B1 (en) * | 1998-02-20 | 2002-04-30 | Cognex Corporation | Methods and apparatus for generating a projection of an image |
| US6191795B1 (en) * | 1998-09-30 | 2001-02-20 | Silicon Graphics, Inc. | Ellipsoidal projection mapping |
| US6687402B1 (en) | 1998-12-18 | 2004-02-03 | Cognex Corporation | Machine vision methods and systems for boundary feature comparison of patterns and images |
| US6577776B1 (en) | 1999-02-24 | 2003-06-10 | Media 100, Inc. | Transforming video images |
| US6388673B1 (en) * | 1999-09-30 | 2002-05-14 | International Business Machines Corporation | Method and system for filter type approximation in computer graphics processing |
| US6922483B2 (en) * | 1999-12-28 | 2005-07-26 | Texas Instruments Incorporated | Methods for measuring DMD low frequency spatial uniformity |
| US6771841B1 (en) * | 1999-12-29 | 2004-08-03 | Intel Corporation | Determining a bounding shape for a collection of points |
| US7343617B1 (en) | 2000-02-29 | 2008-03-11 | Goldpocket Interactive, Inc. | Method and apparatus for interaction with hyperlinks in a television broadcast |
| US7367042B1 (en) * | 2000-02-29 | 2008-04-29 | Goldpocket Interactive, Inc. | Method and apparatus for hyperlinking in a television broadcast |
| US6748104B1 (en) | 2000-03-24 | 2004-06-08 | Cognex Corporation | Methods and apparatus for machine vision inspection using single and multiple templates or patterns |
| US6828983B1 (en) | 2000-05-12 | 2004-12-07 | S3 Graphics Co., Ltd. | Selective super-sampling/adaptive anti-aliasing of complex 3D data |
| US6766281B1 (en) * | 2000-05-12 | 2004-07-20 | S3 Graphics Co., Ltd. | Matched texture filter design for rendering multi-rate data samples |
| AU2001288552A1 (en) * | 2000-08-30 | 2002-03-13 | Watchpoint Media, Inc. | A method and apparatus for hyperlinking in a television broadcast |
| US6665450B1 (en) * | 2000-09-08 | 2003-12-16 | Avid Technology, Inc. | Interpolation of a sequence of images using motion analysis |
| US7043058B2 (en) * | 2001-04-20 | 2006-05-09 | Avid Technology, Inc. | Correcting motion vector maps for image processing |
| US7545957B2 (en) * | 2001-04-20 | 2009-06-09 | Avid Technology, Inc. | Analyzing motion of characteristics in images |
| KR100423503B1 (en) * | 2001-09-14 | 2004-03-18 | 삼성전자주식회사 | Apparatus for disposing digital image and a method using the same |
| US7194676B2 (en) | 2002-03-01 | 2007-03-20 | Avid Technology, Inc. | Performance retiming effects on synchronized data in an editing system |
| US6940511B2 (en) * | 2002-06-07 | 2005-09-06 | Telefonaktiebolaget L M Ericsson (Publ) | Graphics texture processing methods, apparatus and computer program products using texture compression, block overlapping and/or texture filtering |
| US20070003167A1 (en) * | 2003-06-04 | 2007-01-04 | Koninklijke Philips Electronics N.V. | Interpolation of images |
| JP2008512766A (en) * | 2004-09-09 | 2008-04-24 | シリコン オプティックス インコーポレイテッド | Single-pass image warping system and method using anisotropic filtering |
| US7064770B2 (en) * | 2004-09-09 | 2006-06-20 | Silicon Optix Inc. | Single-pass image resampling system and method with anisotropic filtering |
| US8111904B2 (en) * | 2005-10-07 | 2012-02-07 | Cognex Technology And Investment Corp. | Methods and apparatus for practical 3D vision system |
| US8162584B2 (en) * | 2006-08-23 | 2012-04-24 | Cognex Corporation | Method and apparatus for semiconductor wafer alignment |
| US8442316B2 (en) | 2007-01-05 | 2013-05-14 | Geo Semiconductor Inc. | System and method for improving color and brightness uniformity of backlit LCD displays |
| US8055070B2 (en) * | 2007-01-05 | 2011-11-08 | Geo Semiconductor Inc. | Color and geometry distortion correction system and method |
| US8073291B2 (en) * | 2007-10-17 | 2011-12-06 | Seiko Epson Corporation | Center based image resizer |
| US8200037B2 (en) * | 2008-01-28 | 2012-06-12 | Microsoft Corporation | Importance guided image transformation |
| US8086073B2 (en) * | 2008-02-07 | 2011-12-27 | Seiko Epson Corporation | Non-uniform image resizer |
| GB2470942B (en) * | 2009-06-11 | 2014-07-16 | Snell Ltd | Detection of non-uniform spatial scaling of an image |
| EP2860195A1 (en) * | 2009-12-01 | 2015-04-15 | Boston Medical Center Corporation | Treatment of IgE-mediated disease |
| US8803902B2 (en) * | 2009-12-03 | 2014-08-12 | Intel Corporation | Computing level of detail for anisotropic filtering |
| JP5559222B2 (en) * | 2012-03-01 | 2014-07-23 | クゥアルコム・インコーポレイテッド | Single-pass image warping system and method using anisotropic filtering |
| US9865073B2 (en) | 2015-10-05 | 2018-01-09 | Adobe Systems Incorporated | Computer animation of artwork using fast 2-dimensional deformation solving |
| US9858701B2 (en) | 2015-10-05 | 2018-01-02 | Adobe Systems Incorporated | Computer animation of artwork using adaptive meshing |
| US10672155B2 (en) | 2016-08-17 | 2020-06-02 | International Business Machines Corporation | Non-linear, multi-resolution visualization of a graph |
| US10552939B1 (en) | 2019-02-12 | 2020-02-04 | Google Llc | Image processor complex transfer functions |
Family Cites Families (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4602286A (en) * | 1982-01-15 | 1986-07-22 | Quantel Limited | Video processing for composite images |
| FR2546017B1 (en) * | 1983-05-11 | 1986-01-17 | Thomson Csf | METHOD FOR PERFORMING A GEOMETRIC TRANSFORMATION ON A VIDEO IMAGE, AND DEVICES IMPLEMENTING THIS METHOD |
| US4698666A (en) * | 1985-07-12 | 1987-10-06 | The Grass Valley Group, Inc. | Video key glow and border generator |
| CA1309198C (en) * | 1987-12-10 | 1992-10-20 | Carlo J. Evangelisti | Parallel rendering of smoothly shaded color triangles with anti-aliased edges for a three dimensional color display |
| US5179641A (en) * | 1989-06-23 | 1993-01-12 | Digital Equipment Corporation | Rendering shaded areas with boundary-localized pseudo-random noise |
| US5204944A (en) * | 1989-07-28 | 1993-04-20 | The Trustees Of Columbia University In The City Of New York | Separable image warping methods and systems using spatial lookup tables |
| US5153937A (en) * | 1989-09-22 | 1992-10-06 | Ampex Corporation | System for generating anti-aliased video signal |
| US5224208A (en) * | 1990-03-16 | 1993-06-29 | Hewlett-Packard Company | Gradient calculation for texture mapping |
| US5333245A (en) * | 1990-09-07 | 1994-07-26 | Modacad, Inc. | Method and apparatus for mapping surface texture |
| JP2956213B2 (en) * | 1990-11-30 | 1999-10-04 | ソニー株式会社 | Image processing device |
| DE69114465T2 (en) * | 1991-01-07 | 1996-03-21 | Shima Seiki Mfg | Process and device for the preparation and generation of video images. |
| GB9109999D0 (en) * | 1991-05-09 | 1991-07-03 | Quantel Ltd | Improvements in or relating to keying systems and methods for television image processing |
| US5185808A (en) * | 1991-06-06 | 1993-02-09 | Eastman Kodak Company | Method for merging images |
| US5339387A (en) * | 1991-10-24 | 1994-08-16 | Abekas Video Systems, Inc. | Planar color gradients constructed as an arbitrary function of a distance function from an arbitrary 2-D curvilinear function |
| EP0540313B1 (en) * | 1991-10-30 | 1996-12-27 | Fujitsu Limited | Color adjustment for smoothing a boundary between color images |
| US5592599A (en) * | 1991-12-18 | 1997-01-07 | Ampex Corporation | Video special effects system with graphical operator interface |
| WO1993013489A1 (en) * | 1991-12-24 | 1993-07-08 | Sierra Semiconductor Corporation | An anti-aliasing method for polynomial curves using integer arithmetics |
| US5715385A (en) * | 1992-07-10 | 1998-02-03 | Lsi Logic Corporation | Apparatus for 2-D affine transformation of images |
| US5581665A (en) * | 1992-10-27 | 1996-12-03 | Matsushita Electric Industrial Co., Ltd. | Three-dimensional object movement and transformation processing apparatus for performing movement and transformation of an object in a three-diamensional space |
| DE4492446B4 (en) * | 1993-04-15 | 2005-10-06 | Ultimatte Corp., Chatsworth | Screen filtering boundary detection for an image composition |
| US5644364A (en) * | 1993-04-16 | 1997-07-01 | Avid Technology, Inc. | Media pipeline with multichannel video processing and playback |
| US5420971A (en) * | 1994-01-07 | 1995-05-30 | Panasonic Technologies, Inc. | Image edge finder which operates over multiple picture element ranges |
| US5596687A (en) * | 1994-07-29 | 1997-01-21 | David Sarnoff Research Center, Inc. | Apparatus and method for addressing pixel values within an image pyramid using a recursive technique |
| JPH08129647A (en) * | 1994-10-28 | 1996-05-21 | Yamaha Corp | Graphics device |
| US5710876A (en) * | 1995-05-25 | 1998-01-20 | Silicon Graphics, Inc. | Computer graphics system for rendering images using full spectral illumination data |
| US5867166A (en) * | 1995-08-04 | 1999-02-02 | Microsoft Corporation | Method and system for generating images using Gsprites |
| US5754180A (en) * | 1996-03-11 | 1998-05-19 | Avid Technology, Inc. | Computer system and process for defining and manipulating images using structured objects with variable edge characteristics |
| US5835099A (en) * | 1996-06-26 | 1998-11-10 | Xerox Corporation | Representing a region of a color image using a space-color separable model |
| US5808623A (en) * | 1996-10-07 | 1998-09-15 | Adobe Systems Incorporated | System and method for perspective transform in computer using multi-pass algorithm |
-
1997
- 1997-08-01 US US08/904,470 patent/US6016152A/en not_active Expired - Fee Related
-
1998
- 1998-05-27 EP EP98924914A patent/EP0998727B1/en not_active Expired - Lifetime
- 1998-05-27 WO PCT/US1998/010763 patent/WO1999006952A1/en active IP Right Grant
- 1998-05-27 CA CA002297916A patent/CA2297916C/en not_active Expired - Fee Related
- 1998-05-27 DE DE69816824T patent/DE69816824T2/en not_active Expired - Fee Related
- 1998-05-27 AU AU76974/98A patent/AU7697498A/en not_active Abandoned
- 1998-05-27 JP JP2000505604A patent/JP2001512265A/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| WO1999006952A8 (en) | 1999-03-25 |
| CA2297916A1 (en) | 1999-02-11 |
| EP0998727A1 (en) | 2000-05-10 |
| AU7697498A (en) | 1999-02-22 |
| DE69816824T2 (en) | 2004-05-19 |
| JP2001512265A (en) | 2001-08-21 |
| DE69816824D1 (en) | 2003-09-04 |
| US6016152A (en) | 2000-01-18 |
| WO1999006952A1 (en) | 1999-02-11 |
| EP0998727B1 (en) | 2003-07-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2297916C (en) | Texture mapping in 3-d computer graphics | |
| US6782130B2 (en) | Rendering of photorealistic computer graphics images | |
| US5224208A (en) | Gradient calculation for texture mapping | |
| US5754180A (en) | Computer system and process for defining and manipulating images using structured objects with variable edge characteristics | |
| US6292193B1 (en) | Techniques for anisotropic texture mapping using multiple space-invariant filtering operations per pixel | |
| US6232981B1 (en) | Method for improving texture locality for pixel quads by diagonal level-of-detail calculation | |
| JP5149288B2 (en) | Composition of ANISOMETRIC texture (TEXTURE) (SYNTHESIS) | |
| Cohen et al. | Photo‐Realistic Imaging of Digital Terrains | |
| US4827413A (en) | Modified back-to-front three dimensional reconstruction algorithm | |
| US7692661B2 (en) | Method of creating and evaluating bandlimited noise for computer graphics | |
| US5369739A (en) | Apparatus and method for generating point sample masks in a graphics display system | |
| US7038678B2 (en) | Dependent texture shadow antialiasing | |
| US7672476B2 (en) | Bandlimited noise for computer graphics | |
| US8040352B2 (en) | Adaptive image interpolation for volume rendering | |
| US6400370B1 (en) | Stochastic sampling with constant density in object space for anisotropic texture mapping | |
| JPH11126261A (en) | Texture mapping method and device thereof | |
| Wolberg | Geometric transformation techniques for digital images: a survey | |
| Räsänen | Surface splatting: Theory, extensions and implementation | |
| Wan et al. | Isocube: Exploiting the cubemap hardware | |
| EP1058912A1 (en) | Subsampled texture edge antialiasing | |
| Popescu et al. | Forward rasterization | |
| US7689057B2 (en) | Method of bandlimiting data for computer graphics | |
| Willis | High fidelity pictures from digitizing tablets | |
| Evemy | Real-time computer generated imagery using stream processing techniques | |
| CA2307356A1 (en) | Texturing apparatus and method therefor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKLA | Lapsed | ||
| MKLA | Lapsed |
Effective date: 20100527 |