Search Images Maps Play YouTube News Gmail Drive More »
Sign in
Screen reader users: click this link for accessible mode. Accessible mode has the same essential features but works better with your reader.

Patents

  1. Advanced Patent Search
Publication numberUS6456745 B1
Publication typeGrant
Application numberUS 09/154,227
Publication dateSep 24, 2002
Filing dateSep 16, 1998
Priority dateSep 16, 1998
Fee statusPaid
Also published asCA2347364A1, CN1326574A, EP1114399A1, WO2000016263A1
Publication number09154227, 154227, US 6456745 B1, US 6456745B1, US-B1-6456745, US6456745 B1, US6456745B1
InventorsLeonard Thomas Bruton, Todd Simpson, Barry Yee
Original AssigneePush Entertaiment Inc.
Export CitationBiBTeX, EndNote, RefMan
External Links: USPTO, USPTO Assignment, Espacenet
Method and apparatus for re-sizing and zooming images by operating directly on their digital transforms
US 6456745 B1
Abstract
Digital images are represented in the spatial domain by numbers that correspond to pixels and may be transformed to a transform domain image by means of a Transform Method. Other Transform Methods are used to transform the transform domain image back to the original spatial domain image. In this invention, these other Transform Methods are modified in such a way that the Modified Transform Method yields spatial domain images that are re-sized versions of the original spatial domain image. Methods are disclosed for employing the Modified Transform Method to implement zoomed and panned versions of the re-sized images. The re-sized images do not require pixel-level smoothing filters and other methods for removing image distortions due to re-sizing.
Images(8)
Previous page
Next page
Claims(15)
What is claimed is:
1. A system for displaying multiple images comprising:
means for displaying an image;
means for selecting a portion of said displayed image; and
means for concurrently displaying, on said display means, the image and the selected portion, wherein said selected portion is modified using a Modified Transform Method, wherein said Modified Transform Method is associated with transforming a transform domain image to a spatial domain image and includes a plurality of synthetic data points.
2. The system of claim 1, wherein said means for selecting a portion of the image comprises:
a television remote control.
3. The system of claim 1, wherein said means for selecting a portion of the image comprises:
a computer pointing device.
4. The system of claim 1, wherein said modified selected portion has been resized using said Modified Transform Method.
5. The system of claim 1, wherein said displayed selected portion has been panned using said Modified Transform Method.
6. The system of claim 1, wherein said displayed selected portion has been zoomed-in using said Modified Transform Method.
7. The system of claim 3, wherein said displayed selected portion has been zoomed-out using said Modified Transform Method.
8. A method of modifying a digital image by array operations, wherein the digital image is represented by data stored in a first array having N columns and M rows as spatial domain information, wherein said digital image is to be scaled by a scaling factor s1 in the horizontal direction and by a scaling factor s2 in the vertical direction, comprising the steps of:
transforming data in the first array to a frequency domain via a first array operation to produce a second array, wherein said first array operation includes utilizing basis functions of the form b(k1, k2, n1, n2), wherein k1 and k2 are elements of the frequency domain, and wherein n1 and n2 are elements of the spatial domain;
modifying the second array to produce a third array having X columns and Y rows as frequency domain information, wherein at least one of the following conditions are met: X≠N and Y≠M, and wherein scaling of the digital image does not occur in the modifying step; and
transforming data in the third array into spatial domain information via a modified array operation, said modified array operation being associated with basis functions of the form b(k1/s1, k2/s2, n1, n2), and wherein the modified array operation is not the inverse of the first array operation.
9. The method of claim 8 wherein said step of modifying includes adding synthetic data points to the second array.
10. The method of claim 8 wherein said step of transforming data in the third array produces a fourth array, said method further comprising the step of:
cropping data in the fourth array when at least one of s1 and s2 is less than one.
11. The method of claim 8 wherein at least one of the following conditions need not be met by said modifying: X=s1N and Y=s2M.
12. A system for modifying a digital image by array operations, wherein the digital image is represented by data stored in a first array having N columns and M rows as spatial domain information, wherein said digital image is to be scaled by a scaling factor s1 in the horizontal direction and by a scaling factor s2 in the vertical direction, comprising the steps of:
means for transforming data in the first array to a frequency domain via a first array operation to produce a second array, wherein said first array operation includes utilizing basis functions of the form b(k1, k2, n1, n2), wherein k1 and k2 are elements of the frequency domain, and wherein n1 and n2 are elements of the spatial domain;
means for modifying the second array to produce a third array having X columns and Y rows as frequency domain information, wherein at least one of the following conditions are met: X≠N and Y≠M, and wherein scaling does not occur in the modifying step; and
means for transforming data in the third array into spatial domain information via a modified array operation, said modified array operation being associated with basis functions of the form b(k1/s1, k2/s2, n1, n2), and wherein the modified array operation is not the inverse of the first array operation.
13. The system of claim 12 wherein said means for modifying includes adding synthetic data points to the second array.
14. The system of claim 12 wherein said means for transforming data in the third array produces a fourth array, said system further comprising:
means for cropping data in the fourth array, wherein said means for cropping operates when at least one of s1 and s2 is less than one.
15. The system of claim 12 wherein at least one of the following conditions is met: the value of X is greater than s1N and the value of Y is greater than s2M.
Description
TECHNICAL FIELD OF THE INVENTION

This invention relates to video signal processing and more particularly to a method and system for re-sizing images and zooming into and out of images in a way that makes use of non-spatial interpretations of the images.

BACKGROUND OF THE INVENTION

Images can be imported into digital systems using opto-electronic transducing devices, such as video cameras and computer scanning devices. Also, images can be created within digital systems using a variety of computer software programs, such as drawing and animation programs. Images are commonly represented and stored in digital systems as arrays of digital numbers.

Images are typically displayed on a suitable display device, such as the cathode ray tube (CRT) of a television or computer monitor, as a two-dimensional spatial representation on the surface of the display device. A series of slightly different images can be displayed in rapid temporal sequence in order to create the perception of smooth motion as, for example, in the case of television image sequences.

Pixels are relatively small localized fixed regions on the surface of the image display device. Displayed images are often composed of many thousands of pixels, wherein each pixel has attributes of size, shape, intensity and color. Objects within displayed images are represented by groups of pixels. Generally, the pixels are sufficiently close to each other on the surface of the display device to ensure that when the displayed image is viewed from a sufficient distance objects are perceived to have the same characteristics of shape, texture, edges and shading as similar objects in the real world.

Pixels are often, but not always, arranged on the surface of the display in the form of a rectangular grid consisting of uniform rows and columns of pixels. The horizontal and vertical spatial resolutions of an image are determined by the average number of pixels per unit of distance in the horizontal direction along the rows of pixels and in the vertical direction along the columns of pixels. The spatial resolution determines the minimum distance at which an image must be viewed so that the human vision system (HVS) perceives objects within the image and not the individual pixels.

The spatial resolution of a displayed image can be increased by increasing the horizontal resolution and the vertical resolution. A high resolution image will yield a more natural looking approximation of an original real world image than a low resolution image. The quality of the image results from the HVS's perception of the lines and edges within displayed images. In low resolution images, the lines and edges are often perceived to be jagged because of the staircase effect caused by rectangular-shaped pixels in the displayed image.

The perception of jagged edges and lines, due to the staircase effect, is less discernible in high resolution images because of the reduced size of the rectangular-shaped pixels in the displayed image.

The finite resolution of the displayed image is a major factor limiting the faithfulness by which objects may be represented.

A digital image is a set of numbers where each number corresponds to a pixel of the displayed image. For example, a displayed image might consist of 512 by 640 pixels where each pixel is characterized by a range of possible luminous intensities and colors. If we decompose the color of each pixel into its primary colors of red (R), blue (B) and green (G), then the displayed image may be numerically represented as the combination of the R, B and G component images. Each of the R, B and G component images is a monochromatic image in which each pixel is characterized only by a digital number representing the luminous intensity of the pixel. A displayed image, consisting of a rectangular array of pixels, may therefore be represented as three monochromatic component images, each of which may be represented in digital form as a corresponding rectangular array of numbers. We refer to such a rectangular array of numbers as a digital image where it is understood that it may represent a monochromatic component of a displayed color image or some suitable combination of the monochromatic components.

An image display device has a fixed viewing area which is usually a rectangular viewing screen. Images displayed on the viewing screen often completely fill the viewing screen. Re-sizing of the image means that the image is re-displayed on the viewing screen in such a way that the image is perceived by the HVS to have been horizontally or vertically stretched or compressed. The ratio of the horizontal size of the image after re-sizing to the horizontal size before re-sizing is the horizontal scaling factor. Similarly, the vertical scaling factor is the ratio of the vertical size of the image after re-sizing to the vertical size before re-sizing. Scaling factors greater than unity represent stretching and scaling factors less than unity represent shrinking of a displayed image.

The aspect ratio of a displayed image is the ratio of its width to its height. If the image is resized using equal horizontal and vertical scaling factors, then the resized image will have the same aspect ratio as the original. By using equal horizontal and vertical scaling factors, objects will grow or shrink without changing their shapes and the re-sizing operation will be perceived by the HVS as having caused all objects within the image to have moved closer to or farther away from the viewer.

Resized images may exceed the dimensions of the viewing screen. Thus, some parts of the resized image may lie outside of the viewing screen and may not be visible. A cropped image is that portion of the image that is displayed in the viewing screen.

Zooming is the re-sizing of a set of digitized images or displayed images to create the perception in the HVS that objects within the displayed image are growing or shrinking with time. Zooming creates the illusion that the distances between the viewer and the objects in the displayed image are growing or shrinking.

Smooth zooming is zooming such that the scaling factors increase or decrease sufficiently slowly over the set of displayed images to create the perception in the HVS that objects within the displayed image are growing or shrinking continuously with time during the process of zooming.

If the scaling factors are increasing with time the zooming process is in the zoom-in mode and if the scaling factors are decreasing with time the zooming process is in the zoom-out mode. For example, a smooth zoom in the zoom-in mode can be achieved in 11 successive frames having scale factors, relative to the first frame, of 1.0, 1.1, 1.2 . . . 1.8, 1.9 and 2.0, resulting in the final scaling factor of 2.

Prior art methods for re-sizing and zooming digitized images operate directly on the numbers which represent each of the pixels in the displayed image. That is, they operate in the spatial domain.

One prior art method of re-sizing images is the pixel replication method which uses integer scale factors. The pixel replication method simply copies each pixel some integer number of times in both the horizontal and vertical directions. For example, with a scaling factor of 3, each pixel of the original image is replicated to form a corresponding 33 square of pixels in the resized image. A primary disadvantage of pixel replication methods is that the jagged staircase effects of the pixelization increase in proportion to the scaling factor. Thus, pixel replication introduces significant and often unacceptable distortion of the image after re-sizing. A further disadvantage of the pixel replication method is that it is limited to enlarging images by integer scaling factors. Therefore, it cannot be used to reduce the size of the image, nor can it be used to re-size an image by non-integer scale factors, such as 1.1, 1.2, etc., as required for smooth zooming operations.

Another prior art method is the pixel sub-sampling method, which is used to reduce the size of images using scaling factors that are the reciprocals of integers, such as , ⅓ or . This method implements the scaling factor 1/Q in the horizontal or vertical directions, where Q is an integer, by retaining only one of every Q numbers, or pixels, in the digital image in the respective directions. The pixel sub-sampling method creates undesirable distortions in displayed images. Such distortions lead to the loss of edge definition in objects and are noticeable by the HVS in displayed images that are resized using this method, especially for large values of Q.

Abrupt spatial variations of the intensity of the resized image are especially susceptible to distortion and are therefore objectionable to the HVS. This distortion can be alleviated if the intensity variation between pixels is made to vary smoothly in all directions after pixel replication or before pixel sub-sampling. The prior art includes a variety of methods for achieving the required spatial domain smoothing. These prior art spatial domain methods have a number of disadvantages, including the need for operating directly on each of the pixels in the unscaled spatial domain version of the digital image. Other versions of the image, such as encoded or compressed versions, in which the image is not spatially represented pixel by pixel, cannot be used with the prior art spatial domain smoothing methods.

In the case of pixel replication, the required smoothed image is often obtained by means of a two-dimensional spatial domain low-pass filtering operation that effectively smooths out the spatial intensity variations in the digitized image. Two-dimensional spatial domain low-pass filtering methods vary in sophistication depending upon the level of distortion that can be tolerated in the displayed image. Simple spatial domain weighted averaging of regional pixels can be used to smooth the intensity variations

in the pixel-replicated version of the image, thereby reducing the jagged edge effects. More complicated spatial domain filtering methods smooth each number in each row and column of the resized digital image. This may require hundreds of multiplication and addition operations to be performed on each pixel, which may lead to millions of operations for each resized image. For example, if an image of size 512512 is resized to 40964096 (scaling factor of 8) by pixel replication and a smoothing operation requiring 100 multiplication operations per pixel is employed on the resized image, then the total number of multiplication operations that is required for smoothing each resized image is 40964096100. This results in over 1,600,000,000 multiplication operations per resized image.

If the image is cropped to a 512512 size, then the minimum number of multiplication operations is reduced to 512512100, which is over 25,000,000 per cropped image. Accordingly, the prior art methods for the spatial domain reduction of distortion due to pixel replication involves the use of extensive hardware and software resources that operate on a pixel by pixel basis.

In the case of pixel sub-sampling, a two-dimensional low-pass filtering operation is performed on the image prior to sub-sampling to smooth out the intensity variations. This spatial domain smoothing method has the effect of pre-distorting the image to minimize the distortion effects in the resized image following pixel sub-sampling. The effect of this smoothing operation is to blur many objects in the image so that their sharp edges are no longer distinct after pixel sub-sampling. The required spatial domain filtering operations, like those used in pixel replication, vary in sophistication and complexity. When simple weighted averaging is used to achieve this filtering, it is often necessary to use hundreds of multiplication and addition operations per pixel. For example, to re-size a 512512 image to a 6464 image (scaling factor of ⅛) by pixel sub-sampling using a smoothing operation that requires 100 multiplication operations per pixel of the resized image, the total number of multiplication operations required for smoothing each resized image is 6464100. This results in over 360,000 multiplication operations per resized image of size 6464. Pre-distorting the original image using this smoothing operation requires 512512100 multiplication operations on the original digital image. This method, like the pixel replication method, also requires extensive hardware and software resources that operate on a pixel by pixel basis.

There are prior art methods for re-sizing images in the spatial domain by fractional scaling factors, such as {fraction (22/73)} or {fraction (101/100)}. This capability is required for smooth zooming and other re-sizing effects. Almost all of the prior art methods involve a combination of the pixel replication and pixel sub-sampling methods described above, along with the appropriate smoothing operations. The prior art methods for fractional scaling factors essentially combine pixel replication and pixel sub-sampling in a single system, which are referred to in the digital signal processing literature as multirate digital systems. Such methods operate on the image in the spatial domain, pixel by pixel, and they employ complicated, computationally intensive digital filtering techniques to smooth out the jagged edges and other distortions in the resized image.

Panning is the cropping of a set of digitized images or displayed images to create the perception in the HVS that all objects within the displayed image are uniformly translated between any two adjacent images in the displayed sequence of images.

Smooth panning is panning such that displacements of objects in successive displayed images are sufficiently small to create the perception in the HVS that all objects within the displayed image are moving continuously with time in the displayed sequence of images.

When objects in displayed images are moving during the zooming or panning processes, the perceptions of zooming and panning are retained by the HVS provided that the movements over time of objects in images are sufficiently slow.

Image compression is the process by which digital images are represented by a fewer number of bits of information. Image compression allows images to be stored in fewer bits of digital memory and thereby reduces the size of computer files that are required for storing images. Image compression also allows the compressed version of digital images to be transmitted over communication channels at a faster rate than uncompressed images.

Compressed versions of spatial domain images must be decompressed prior to display.

Most image compression methods employ mathematical techniques that transform the spatial domain representation of the image to a corresponding transform domain version of the image that is more suitable for compression. For example, the transform domain version typically employs data that has the mathematical property that it is significantly less correlated than the data in the spatial domain which is a desirable property for subsequent compression. A wide variety of compression methods, including quantization and coding, are routinely employed to compress the transform domain version of the image.

The decompression process attempts to recover the original transform domain version in which case the compression-decompression process is said to be lossless. In practice the decompression algorithms often recovers only an approximation to the original spatial domain image, and the process is said to be lossy.

The transform domain versions of images do not necessarily contain data that corresponds to spatial domain pixels and therefore may not be operated upon by the above mentioned prior art re-sizing and zooming methods to achieve re-sizing and zooming of displayed images.

International standards have been developed for accomplishing compression of digital images. For example, the MPEG 1 and 2 compression standards are the most widely used for digital image sequences while JPEG is commonly used for still images.

These standards use the discrete cosine transform (DCT) to convert spatial domain images to the transform domain and the inverse discrete cosine transform (IDCT) is used to convert the transform domain version to the corresponding decompressed spatial domain image. Prior art methods of re-sizing, zooming or panning displayed images, that have been compressed using the MPEG standards, operate on the spatial domain image.

SUMMARY OF THE INVENTION

The present invention provides a method of re-sizing a spatial domain image by modifying the transform domain method that is used to transform the transform domain version of the image to the spatial domain version of the image. By modifying the transform domain method, there is no need to perform further pixel-by-pixel image re-sizing, zooming or panning operations on the image in the spatial domain. The detrimental effects caused by re-sizing an image in the spatial domain, such as jagged edges, blurring, aliasing or other image degradations, are reduced in the present invention.

The standard transform method, which transforms the image from the transform domain to the spatial domain, is modified. Transformation of the transform domain image, using the Modified Transform Method, produces an altered spatial domain image. These modifications to the transform operation are performed by the Modified Transform Method in such a way as to achieve desirable re-sizing or zooming effects in the altered spatial domain image after transformation.

It is a technical advantage of the present invention to provide a method of modifying the appearance of the spatial domain representation of a digital image by applying a Modified Transform Method to the transform domain version of the image. The re-sized spatial domain image can be achieved by modifying a large number of transformation methods, such as the Inverse Discrete Cosine Transform method, the Inverse Discrete Fourier Transform method and the Inverse Discrete Wavelet Transform method. Furthermore, the invention can be applied to the block images (that is, blocks) used in standard image compression techniques, such as MPEG or JPEG.

It is a further technical advantage of the invention that no further modification of the spatial domain image is required after performing the Modified Transform Domain Method. This precludes the need for extensive pixel-by-pixel modification of the spatial domain image in order to re-size or zoom.

An additional technical advantage of the invention is to provide for modification of digital images without using extensive hardware and software resources. Re-sizing and zooming modification of the spatial domain representation of an image requires expensive and time consuming pixel-by-pixel processing. In the present invention, the spatial domain images created by the Modified Transformation Method are in their re-sized or zoomed form and are capable of being displayed without further modification or processing.

The foregoing has rather broadly outlined the features and technical advantages of the present invention in order that the detailed description of the invention that follows may be better understood. Additional features and advantages of the invention will be described hereinafter which form the subject of the claims of the invention. It should be appreciated by those skilled in the art that the concept and the specific embodiment disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purses of the present invention. It should also be realized by those skilled in the art that such equivalent constructions do not depart from the spirit and scope of the invention as set forth in the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the present invention, and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:

FIG. 1A is a representation of the transformation of an image from the spatial domain to a transform domain and is referred to herein as a Φ-Transform Method;

FIG. 1B is a representation of the transformation of an image from a transform domain to the spatial domain and is referred to herein as a Ψ-Transform Method;

FIG. 2 shows the results of performing a Φ-Transform Method, a Ψ-Transform Method and a Modified Ψ-Transform Method;

FIG. 3 shows a Modified Ψ-Transform Method using an augmented transform domain image;

FIG. 4A shows a resized image which is too large for the display window;

FIG. 4B shows the resized image of FIG. 4A cropped to fit the display window;

FIG. 4C shows the resized image of FIG. 4A cropped to fit some sub-portion of the display window;

FIG. 5A shows the resized image divided into discrete blocks;

FIG. 5B shows a display window moving with respect to the blocks of the resized image of FIG. 5A;

FIG. 6 illustrates enlarging a digital image using the method of the present invention;

FIG. 7 illustrates shrinking a digital image using the method of the present invention;

FIG. 8 shows a television viewing system having viewer control.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

A digital image is an array of digital numbers wherein each digital number in the array represents a corresponding pixel of the displayed image. Both the digital image and its displayed version are spatial-domain images and may be represented mathematically by a real-valued function x(n1,n2) where n1 and n2 are non-negative integers that point to the number in the n1 th column and n2 th row of the digital image array, as shown in FIG. 1A. In the displayed image, n1 and n2 point to the corresponding displayed pixel, for a rectangular digital image of size N1 columns by N2 rows, 0≦n1≦(N1−1) and 0≦n2≦(N2−1). The spatial-domain is referred to as D 10 in FIG. 1A and contains the digital image 14. We say that n1,n2εD.

The present invention provides a method of re-sizing a digital image x(n1,n2) to create a re-sized version xr(n1,n2) of the digital image by modifying a Transform Domain Method. By means of the re-sizing method, this invention further provides methods for zooming and panning of digital images and thereby of displayed images. The Method may be used to achieve re-sizing, panning and zooming without operating directly on the spatial domain image.

A set of mathematical operations that transforms a spatial-domain digital image x(n1,n2) to a transform domain version Φ[(n1,n2)] of the image is referred to as a Φ-Transform Method if the transform domain data Φ[x(n1,n2)] does not have a one-to-one correspondence with spatial pixels in the spatial domain image x(n1,n2). Conversely, a set of mathematical operations that transforms a transform-domain version of an image to a spatial domain version of the image is referred to as a Ψ-Transform Method.

The Ψ-Transform Method may be applied to the transform domain version of an image Φ[x(n1,n2)] to obtain the spatial-domain image Ψ[Φ[x(n1,n2)]].

If the application of the Φ-Transform Method, followed by the application of the Ψ-Transform Method, leads to the original spatial domain digital image x(n1,n2), then the k-Transform Method is the Inverse Transform of the Φ-Transform Method and we say that Ψ=Φ−1.

There exists a wide variety of operations on spatial-domain images that transform them from the spatial-domain to a transform-domain. An important class of Φ-Transform Methods yields a transform domain image of the form X(k1,k2)=Φ[x(n1,n2)] where k1 and k2 are the so-called transform-domain variables in the transform-domain ΩD. Such transforms may be written in terms of their basis functions b(k1,k2,n1,n2), in the form X ( k 1 k 2 ) = n 1 n 2 x ( n 1 , n 2 ) b ( k 1 , k 2 , n 1 , n 2 ) ,

and is summarized in FIG. 1A where the transform operation 16 is shown yielding a transform-domain representation X(k1,k2) of the digital image x(n1,n2). In FIG. 1A, X(k1,k2) 18 is of size K1K2 and the transform-domain ΩD 12 is shown.

The basis function b(k1,k2,n1,n2) is independent of the digital image x(n1,n2), but it does depend on the type of transform that is selected. For example, the Discrete Fourier Transform and the family of Discrete Cosine Transforms have different and particular basis functions. Also, the basis functions of the Discrete Fourier Transform and the Inverse Fourier Transform are different.

The transform-domain image X(k1,k2) is an image that is not usually viewed or displayed because the values of a transform-domain image X(k1,k2) do not correspond to pixels in the displayed version of the digital image.

There exists a corresponding set of Ψ-Transform Methods that operate on transform domain images to transform them from the transform domain to the spatial-domain. An important class of Ψ-Transform Methods yields a spatial domain image of the form x(n1,n2)=Ψ[X(k1,k2)] where n1 and n2 are the so-called spatial-domain variables in the spatial-domain D. Such transforms may be written in terms of their basis functions b(k1,k2,n1,n2) in the form x ( n 1 n 2 ) = k 1 k 2 X ( k 1 , k 2 ) b ( k 1 , k 2 , n 1 , n 2 ) ,

As summarized in FIG. 1B, the Ψ-Transform Method Ψ[X(k1,k2)] 20 is shown yielding a spatial-domain representation of the digital image x(n1,n2) 14′. If the operations in FIGS. 1A and 1B are applied in sequence and images 14 and 14′ are identical, then Ψ[X(k1,k2)] is the inverse transformation of Φ[x(n1,n2)] and therefore Ψ=−1.

In the prior art, a re-sized displayed image is achieved by operating on the spatial-domain digital image, employing methods that operate on each number in the spatial-domain array of the digital image 14′ in FIG. 1B. As further described above, the result of these prior art methods is that a very large number of arithmetic operations are required to avoid undesirable jagged edges and other image distortions that result from the down-sampling and up-sampling operations which are required by spatial domain re-sizing.

The present re-sizing ,invention avoids the requirements for spatial-domain operations on each number in the spatial-domain array x(n1,n2) 14′ and also avoids the need for the associated spatial-domain smoothing operations. The present invention is to modify the Ψ-Transform Method and is therefore referred to as the Modified Ψ-Transform Method. Upon applying the Modified Ψ-Transform Method to the transform-domain image X(k1,k2) the required re-sized spatial-domain digital image xr(n1,n2) is created and may be displayed.

FIG. 2 illustrates the operation of the preferred embodiment of the re-sizing method of the present invention. Image x(n1,n2) 22 represents the spatial-domain digital image prior to re-sizing. Images 22, 24 and 34 may be entire displayed images or sub-image regions of the displayed image. If the sub-image region is spatially-rectangular, it is an image-block or block. The transform-domain image X(k1,k2) 24 is in the transform domain ΩD. The unmodified Ψ-Transform Method may be employed as an inverse transform Ψ=Φ−1 to convert the transform-domain image X(k1,k2) 24 back to the original spatial-domain digital image 22.

By the preferred re-sizing embodiment of this invention, a Modified Ψ-Transform Method 36 is applied to the transform domain image X(k1,k2) 24 to create the re-sized spatial-domain digital image xr(n1,n2) 34 where the Modified Ψ-Transform Method is written

ΨMOD [X(k 1 ,k 2)]

and where ΨMOD[X(k1,k2)] 36 is shown in FIG. 2.

The Modified Ψ-Transform Method is any set of modifying operations ΨMOD[X(k1,k2)] on any data or algorithms of the unmodified Ψ-Transform Method Ψ[X(k1,k2)] that causes the image xr(n1,n2) 34 to be a predetermined re-sized version of the digital image x(n1,n2) 22. Such operations include, but are not limited to, changing the size of the data array X(k1,k2) 24 by augmenting the array with additional data and also include, but are not limited to, modifying the parameters of the transformation, such as its basis functions.

The data points of the transform-domain image X(k1,k2) 24 may be complex numbers, depending on the selected Φ-Transform Method.

A preferred embodiment of this invention is to modify the Ψ-Transform Method by modifying the basis functions b(k1,k2,n1,n2) to create a modified set of basis functions bMOD(k1,k2,n1,n2) and, in particular, such that b MOD ( k 1 , k 2 , n 1 , n 2 ) = b ( k 1 s 1 , k 2 s 2 , n 1 , n 2 )

where bMOD(k1,k2,n1,n2) are the modified basis functions and s1 and s2 are real numbers that are called the horizontal and vertical scaling factors, respectively. The basis functions are modified as part of the Modified Ψ-Transform Method.

For example, if the basis functions are of the form b ( k 1 , k 2 , n 1 , n 2 ) = cos ( ( 2 n 1 + 1 ) k 1 π 2 N 1 ) cos ( ( 2 n 2 + 1 ) k 2 π 2 N 2 )

then the modified basis functions are of the form b MOD ( k 1 , k 2 , n 1 , n 2 ) = cos ( ( 2 n 1 + 1 ) ( k 1 s 1 ) π 2 N 1 ) cos ( ( 2 n 2 + 1 ) ( k 2 s 2 ) π 2 N 2 )

When the value s1 is greater than unity, the above scaling of the basis functions causes the re-sized image xr(n1,n2) 34 to be stretched in the horizontal direction relative to x(n1,n2) 22. When the value of s1 is less than unity, the re-sized image xr(n1,n2) 34 is shrunk in the horizontal direction relative to x(n1,n2) 22. Similarly, s2 achieves stretching or shrinking of xr(n1,n2) 34 in the vertical direction, depending on whether s2 is greater or less than unity.

If s1=s2 then the stretching or shrinking of xr(n1,n2) 34 relative to x(n1,n2) 22 occurs without altering the aspect ratio of the re-sized image relative to the original and without altering the perceived shapes of objects in displayed versions of those images.

A further modification to the Ψ-Transform Method related to re-sizing images, for the case when s1 and s2 are greater than unity, is to increase the size of the transform-domain image X(k1,k2) 24 by generating additional synthetic data points so that the overall size of the augmented transform-domain image X(k1,k2) 24 and the re-sized spatial domain image xr(n1,n2) 34 are predetermined. In FIG. 3, the original array of data points C11 30A of the Φ-Transform Method is augmented by additional generated arrays C12 30B, C21 30C and C22 30D of synthetic data points that serve to modify and increase the size of the transform-domain image X(k1,k2) 24 and create the modified transform-domain image XAUG(k1,k2) 32. The synthetic data in the additional generated arrays C12 30B, C21 30C, and C22 30D may be generated by means of operations on the data points in C11 30A or by some other means or they may be set to zero value.

The sizes of rectangular arrays C12 30B, C21 30C, and C22 30D may be chosen so that the number of rows P1 of 32 satisfies

P 1 ≧s 1 N 1

and the number of columns P2 of 32 satisfies

P 2 ≧s 2 N 2

For example, if N1=N2=8 for an image-block and the desired scaling factors are s1=s2=1.5, then we choose P1,2≧12 for the size of the image-block xr(n1,n2)34. If s1N1 is an integer, then the number of columns P1 of image 32 may be chosen equal to s1N1 and if s2N2 is an integer then the number of rows P2 of image 32 may be chosen equal to s2N2. The transform-domain image 32 is referred to as the Augmented Transform Domain Image and denoted XAUG(k1,k2).

To obtain the re-sized spatial domain image xr(n1,n2) 34, the Augmented Transform Domain image XAUG(k1,k2) 32 of transform domain data is transformed using a Modified Ψ-Transform which employs the modified basis functions bMOD(k1,k2,nl,n2). The Modified Ψ-Transform can then be expressed mathematically in the form xr ( n 1 n 2 ) = k 1 k 2 b MOD ( k 1 , k 2 , n 1 , n 2 ) X AUG ( k 1 , k 2 )

where k1,k2εΩD, implying that the summation over the transform domain variables k1,k2 is over a suitably chosen region of the transform domain ΩD. By using the above Modified Ψ-Transform Method to obtain re-sized spatial domain image xr(n1,n2) 34, jagged edges and other undesirable aliasing effects are avoided and there is no need for extensive smoothing operations on the spatial domain image xr(n1,n2) 34 to reduce jagged edges and other distortion effects in tie re-sized image.

For example, a spatial-domain image x(n1,n2) 14 of size N1=N2=128 may be re-sized by scaling factors s1=s2=2 to obtain a re-sized image xr(n1,n2) 34 of size P1=P2=256 by operating on the transform domain image X(k1,k2) 24. The data array C11 30A represents the array of X(k1,k2) 24 and is of size 128128 for the unmodified Inverse Discrete Cosine Transform. The augmented arrays C12 30B, C21 30C and C22 30D may be chosen as zero-valued arrays of size 128128 (represented as 0 below) so that the augmented array XAUG(k1,k2) 32 is given by X AUG ( k 1 , k 2 ) = [ C 11 C 12 C 21 C 22 ] = [ X ( k 1 , k 2 ) 0 0 0 ]

which is of size 256256. The Modified Ψ-Transform Method is selected as the Inverse Discrete Cosine Transform operation on this Augmented Transform Domain Image XAUG(k1,k2) using the following modified basis functions with s1=s2=2 b MOD ( k 1 , k 2 , n 1 , n 2 ) = cos ( ( 2 n 1 + 1 ) ( k 1 s 1 ) π 2 N 1 ) cos ( ( 2 n 2 + 1 ) ( k 2 s 2 ) π 2 N 2 )

so that the Modified Ψ-Transform Method is xr ( n 1 , n 2 ) = k 1 = 0 P 1 = 255 k 2 = 0 P 2 = 255 X AUG ( k 1 , k 2 ) cos ( ( 2 n 1 + 1 ) ( k 1 s 1 ) π 2 N 1 ) cos ( ( 2 n 2 + 1 ) ( k 2 s 2 ) π 2 N 2 )

It is an advantage of this invention that objects in the re-sized image xr(n1,n2) 34 do not exhibit the distortion effects of pixel replication and the edges of objects are visually smooth. This is achieved without spatial-domain smoothing filters.

The above example can be implemented with small block images. For example, the original image x(n1,n2) could be a block image of size 88 and, with s1=s2=2, the re-sized image block would be of size 1616. The total re-sized and displayed image could be composed of many such re-sized image blocks.

FIG. 4A illustrates the case when image 42 is re-sized such that the re-sized image xr(n1,n2) 44 is too large for the display window 46. A similar problem occurs if the image is to be used in a desired sub-area of the display window 46. In some cases, the re-sized image may fit within the overall display area but may be too large for the desired sub-area of the display. In FIG. 4A, the original image x(n1,n2) 42 of size N1N2 fits within the display window 46 which is of size W1W2. Display window 46 can represent either the entire area of a display or a sub-area of the display. Re-sized image xr(n1,n2) 44 is an enlarged version of image 42 produced using the Modified Ψ-Transform Method with scaling factors s1 and s2 as disclosed above. Re-sized image 44 is of size R1R2, where R1=s1N1 and R2s2N2. In this example, re-sized image 44 is larger than display window 46, causing portions of the enlarged image to fall outside of display window 46. FIG. 4B shows cropped image xcr(n1,n2) 48 which comprises a portion of re-sized image 44, such that the cropped image 48 fits within the display window 46. The cropped image 48 is of size CR1CR2 where CR1≈N1 and CR2≈N2 so that cropped image 48 approximates the dimensions of original image 42. As shown in FIG. 4C, re-sized image 44 also may be cropped so that the dimensions, CR1 1 and CR2 1, of cropped image 50 are not equivalent to those of original image 42. In this case, cropped image 50 can be of any suitable size for viewing on a sub-portion of the display, such as display window 86 of FIG. 8, which shows a zoomed-in version 84′ of image 84.

In cases where the resized image is larger than the display window, instead of cropping the resized image, it may be desirable to pan the resized image so that sequential portions of the entire resized image may be viewed in the display window.

As shown in FIG. 5A, resized image 44 can consists of blocks 52 which correspond to discrete spatial regions of resized image 44. Blocks 52 are of dimension B1B2 and may be of various sizes, from single pixels to multiple pixel rows and columns.

FIG. 5B shows display window 46 panning over resized image 44.

Display window 46 covers a portion of the blocks 52 of the resized image 44.

Blocks 52 which are within display window 46 can be viewed by a user. The resized image 44 is panned by sequentially moving blocks 52 across the display window 46. For pixel-level panning, blocks 52 correspond to individual pixels of resized image 44. The image is panned by sequentially adding one row or column of pixels to an edge 54 of display window 46 and removing one row or column of pixels from the opposite edge 56 of display window 46. A shift vector 58, represented as dpan=[dhorizdvert] pixels/frame, defmes the movement of re-sized image xr(n1,n2) 44 with respect to display window 46, where dhoriz is the number of blocks in the horizontal dimension and dvert is the number of pixels in the vertical dimension.

For pixel-level panning, dhoriz=−1,0 or +1 and dvert=−1,0 or +1, so that the display window moves one row and column per frame. Resized image 44 can also be panned using multiple pixels. In multiple-pixel-level panning, multiple rows and columns of pixels may be added and removed at the edges1 so that the components of the shift vector are:

d horiz =−m horiz, 0 or +m horiz and d vert =−m vert, 0 or +m vert,

where mhoriz and mvert represent multiple columns and rows of pixels. In block-level panning mhoriz=M1, the horizontal block size, and mvert=M2, the vertical block size.

Panning can be accomplished using the Modified Ψ-Transform Method disclosed above. By identifying groups of blocks in the transform domain image which are within or partially within the display window, the Modified Ψ-Transform Method is applied to each of those blocks to produce that portion of the resized image 44. A second group of blocks is then identified which corresponds to those blocks which will overlap the display window in the next frame after the display window is moved by the appropriate shift vector. The Modified Ψ-Transform Method is then applied to the second group of blocks to produce a shifted portion of the resized image 44.

The cropping and panning functions described above will be understood to apply to images which have been resized using scale factors that are greater than, less than or equal to unity. Additionally, the re-sizing, panning and cropping functions may be applied to images in real-time. For example, an image may be received by a digital system via an opto-electronic transducing device, such as a video camera, the image may then be compressed using an appropriate MPEG standard. The compressed image can then be inverse transformed using a Modified Ψ-Transform Method to achieve a displayed image which has been resized, zoomed, panned or cropped according to the present invention.

FIG. 6 illustrates the re-sizing method of the present invention for the case in which the original spatial domain image x(n1,n2) 600 is enlarged by a non-integer scale factor. In this example, the scale factor is s1,=s2=1.5. The re-sizing shown in FIG. 6 also includes an example of a re-sizing operation that is combined with a cropping operation to produce the final resized image xcr(n1,n2) 602. Original image 600 is transformed to the transform domain using the Discrete Cosine Transform Φ[x(n1,n2)] 604 to produce transform domain image X(k1,k2) 606. This transformation is represented mathematically as: X ( k 1 , k 2 ) = n 1 = 0 7 n 2 = 0 7 x ( n 1 , n 2 ) cos ( ( 2 n 1 + 1 ) π k 1 ( 2 ) ( 8 ) ) cos ( ( 2 n 2 + 1 ) π k 2 ( 2 ) ( 8 ) )

where the original image size corresponds to N1,2=8 in this example.

Transform-domain image 606 is then augmented with synthetic data points DP 608 to create augmented transform image XAUG(k1,k2) 610. Augmented transform image 610 is of size P1P2, where P1,2≧s1,2N1,2. For a scale factor of 1.5 and an original spatial domain of size 88, P1,2 must be greater than or equal to (1.5) (8)=12. In this example, instead of using an augmented transform image of size 1212, the augmented transform image 610 is of size 1616. This satisfies the requirement P1,2≧12 and may be chosen because fast transforms exist for this size. A value of P1,2≧12 may also be selected due to consideration of such factors as the hardware or software used by the system.

Augmented transform image 610 is then transformed using a Modified Ψ-Transform Method ΨMOD[XAUG(k1,k2)] 612. Modified Ψ-Transform Method 612 is derived by scaling the basis functions b(k1,k2,n1,n2) of the unmodified Ψ-Transform Method Ψ[X(k1,k2)] by scale factors s1,2. This creates modified basis function b MOD ( k 1 , k 2 , n 1 , n 2 ) = b ( k 1 s 1 , k 2 s 2 , n 1 , n 2 ) .

Applying the Modified Ψ-Transform Method 612 to the Augmented Transform Domain Image 610 directly yields the re-sized image xr(n1,n2) 614 in the spatial domain. For this example, using a modified Inverse Discrete Cosine Transform operation as the Modified Ψ-Transform Method, the inverse transformation of the augmented image 610 is represented mathematically as: xr ( n 1 , n 2 ) = k 1 = 0 15 k 2 = 0 15 X AUG ( k 1 , k 2 ) cos ( ( 2 n 1 + 1 ) π ( k 1 / 1.5 ) ( 2 ) ( 8 ) ) cos ( ( 2 n 2 + 1 ) π ( k 2 / 1.5 ) ( 2 ) ( 8 ) )

Resized image 614 is of size 1616 and includes replicated borders 616 and 618 which result from augmented image 610 being larger than size 1212. A 1212 augmented transform image would correspond to the scale factor s1,2=1.5. However, by choosing an augmented image 610 of a larger size, an excess amount of resized image is created following the Modified Ψ-Transform Method. This excess image that is contained in borders 616 and 618 can be cropped to give cropped resized image 602 of size 1212, corresponding to a 1.5 scaling of the original 88 image 600.

Similarly, FIG. 7 illustrates the re-sizing method of the present invention for the case in which the original spatial domain image x(n1,n2) 700 is reduced by a non-integer scale factor to create the final cropped and resized image xcr(n1,n2) 702. In this example, the scale factor is . Original image 700 is transformed using transform operation Φ[x(n1,n2)] 704 to give transform image X(k1,k2) 706. Since this re-sizing is not an enlargement, transform image 706 does not have to be augmented. Instead, the Modified Ψ-Transform Method ΨMOD[X(k1,k2)] 708 is applied directly to transform image 706 to give resized image xr(n1,n2) 710.

Resized image 710 contains excess image in borders 712 and 714 due to the scaling of the Modified Ψ-Transform Method 708. By cropping resized image 710 to remove excess border 712 and 714, cropped and resized image xcr(n1,n2) 702 is created. Image 702 corresponds to a scaling of original image 700.

FIG. 8 shows a system 80 in which a viewing device 81, such as a television or personal computer, has a picture 82 with a plurality of images 801, 802 and 803. A user (not shown) using a remote control 83 or other device, such as a computer mouse, can select a particular image for further zooming. For example, portion 84 of image 82 is selected by adjusting outline 85. This selected image can then be displayed on the screen in place of image 82 or, optionally, in a separate window 86. The image displayed in window 86 can then be resized or zoomed in or out. As discussed above, the viewer could also elect to pan the image of screen 82 (or of screen 86) either row-by-row, column-by-column, block-by-block or in a combination of these.

Although the present invention and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims.

Patent Citations
Cited PatentFiling datePublication dateApplicantTitle
US5051927Feb 1, 1990Sep 24, 1991Minolta Camera Kabushiki KaishaImage editing apparatus for transferring a partial image from one locale to another
US5168375Sep 18, 1991Dec 1, 1992Polaroid CorporationImage reconstruction by use of discrete cosine and related transforms
US5227875Aug 20, 1991Jul 13, 1993Kabushiki Kaisha ToshibaSystem for transmitting encoded image data with quick image expansion and contraction
US5477397Jun 8, 1993Dec 19, 1995Matsushita Electric Corporation Of AmericaDigital high definition television receiver with features that facilitate trick-play modes on a digital VCR
US5495292 *Sep 3, 1993Feb 27, 1996Gte Laboratories IncorporatedInter-frame wavelet transform coder for color video compression
US5519452 *Jun 17, 1994May 21, 1996Eastman Kodak CompanyMechanism for improving television display of still images using image motion-dependent filter
US5552824 *Jun 28, 1994Sep 3, 1996Lynx System Developers, Inc.Line object scene generation apparatus
US5574508 *Nov 2, 1994Nov 12, 1996Rca Thomson Licensing CorporationVertical panning for interlaced video
US5596346 *Feb 6, 1996Jan 21, 1997Eastman Kodak CompanyMethod and apparatus for applying a function to a localized area of a digital image using a window
US5604494 *Sep 20, 1994Feb 18, 1997Sony CorporationEfficient encoding/decoding apparatus
US5619738 *May 2, 1995Apr 8, 1997Eastman Kodak CompanyPre-processing image editing
US5649032 *Nov 14, 1994Jul 15, 1997David Sarnoff Research Center, Inc.System for automatically aligning images to form a mosaic image
US5652849 *Mar 16, 1995Jul 29, 1997Regents Of The University Of MichiganApparatus and method for remote control using a visual information stream
US5706216 *Jul 28, 1995Jan 6, 1998Reisch; Michael L.System for data compression of an image using a JPEG compression circuit modified for filtering in the frequency domain
US5729673 *Apr 7, 1995Mar 17, 1998Avid Technology, Inc.Direct manipulation of two-dimensional moving picture streams in three-dimensional space
US5774598 *May 15, 1995Jun 30, 1998Polaroid CorporationSystem and method for sample rate conversion of an image using discrete cosine transforms
US5990890 *Aug 25, 1997Nov 23, 1999Liberate TechnologiesSystem for data entry and navigation in a user interface
US6154762 *Jun 3, 1998Nov 28, 2000Microsoft CorporationFast system and method for computing modulated lapped transforms
US6298166 *Mar 30, 1998Oct 2, 2001Seiko Epson CorporationImage transformations in the compressed domain
EP0740269A2Apr 19, 1996Oct 30, 1996Hewlett-Packard CompanyScaling compressed images
JPH04239294A * Title not available
WO1997034420A1Mar 6, 1997Sep 18, 1997Multimedia Systems CorpInteractive cable network providing fax and voice mail
Non-Patent Citations
Reference
1"Adaptive Motion Vector Resampling for Compressed Video Down-Scaling", by Bo Shen, et al.; IEEE; Jul. 1997; pp. 771-774.
2"Circuits and Systems Letters", by P. Sathyanarayana, et al.; IEEE Transactions on Circuits and Systems, vol. 37, No. 5; May 1990; pp. 623-625.
3"Manipulation and Compositing of MC-DCT Compressed Video", IEEE Journal on Selected Areas in Communications, US, IEEE Inc., New York, vol. 13, No. 1, pp. 1-11 XP000492740.
4"Video Post-Production with Compressed Images", SMPTE Journal, US, SMPTE Inc., Scarsdale, New York, vol. 103, No. 2, pp. 76-84 XP000429905.
5International Search Report, PCT/US99/21298.
Referenced by
Citing PatentFiling datePublication dateApplicantTitle
US6687422 *Feb 16, 2000Feb 3, 2004The Cleveland Clinic FoundationDiscrete image data interpolation using sub-unity shift
US6807310 *May 23, 2000Oct 19, 2004The Board Of Trustees Of The University Of IllinoisTransformation of image parts in different domains to obtain resultant image size different from initial image size
US6970179May 12, 2000Nov 29, 2005International Business Machines CorporationMethod and apparatus for the scaling up of data
US7062098 *May 12, 2000Jun 13, 2006International Business Machines CorporationMethod and apparatus for the scaling down of data
US7085436 *Aug 20, 2002Aug 1, 2006VisioprimeImage enhancement and data loss recovery using wavelet transforms
US7352810 *Nov 16, 2001Apr 1, 2008Thomson LicensingCoding process and device for the displaying of a zoomed MPEG2 coded image
US7373003 *Nov 4, 2004May 13, 2008International Business Machines CorporationMethod for the scaling down of data
US7454707 *Sep 16, 2003Nov 18, 2008Canon Kabushiki KaishaImage editing method, image editing apparatus, program for implementing image editing method, and recording medium recording program
US7463777 *Nov 4, 2004Dec 9, 2008International Business Machines CorporationMethod for the scaling down of data
US7515732 *Jan 12, 2005Apr 7, 2009Xerox CorporationMethod of embedding color information in printed documents using watermarking
US7549127 *Aug 1, 2002Jun 16, 2009Realnetworks, Inc.Method and apparatus for resizing video content displayed within a graphical user interface
US7574071 *Oct 5, 2004Aug 11, 2009Seiko Epson CorporationMethod and apparatus for resizing images
US7720310Feb 22, 2008May 18, 2010International Business Machines CorporationScaling down of data
US7733405 *Feb 10, 2005Jun 8, 2010Seiko Epson CorporationApparatus and method for resizing an image
US7737995 *Feb 28, 2005Jun 15, 2010Microsoft CorporationGraphical user interface system and process for navigating a set of images
US7903889Oct 28, 2008Mar 8, 2011International Business Machines CorporationSystem and computer readable medium for the scaling down of data
US7995219 *Mar 12, 2003Aug 9, 2011Canon Kabushiki KaishaImage processing method and apparatus, and printer driver
US8090141Jan 31, 2006Jan 3, 2012Xerox CorporationSystem and method to automatically establish preferred area for image-wise watermark
US8112712Jul 30, 2008Feb 7, 2012Canon Kabushiki KaishaImage editing method, image editing apparatus, program for implementing image editing method, and recording medium recording program
US8115734 *Jul 28, 2010Feb 14, 2012Promethean Ltd.Moveable desktop
US8208010Jun 2, 2005Jun 26, 2012Sony Ericsson Mobile Communications AbFace image correction using multiple camera angles
US8217931Mar 25, 2011Jul 10, 2012Conversion Works, Inc.System and method for processing video images
US8224124 *Jun 24, 2009Jul 17, 2012Sony CorporationImage processing device and image processing method, and program
US8274530Mar 11, 2008Sep 25, 2012Conversion Works, Inc.Systems and methods for filling occluded information for 2-D to 3-D conversion
US8295645 *Sep 9, 2009Oct 23, 2012Keyence CorporationImage processing apparatus, image processing method, and computer program
US8368965 *Oct 20, 2010Feb 5, 2013Primax Electronics Ltd.Image cropping process
US8655052Jan 26, 2007Feb 18, 2014Intellectual Discovery Co., Ltd.Methodology for 3D scene reconstruction from 2D image sequences
US20090324092 *Jun 24, 2009Dec 31, 2009Koji AoyamaImage processing device and image processing method, and program
US20100080484 *Sep 9, 2009Apr 1, 2010Keyence CorporationImage Processing Apparatus, Image Processing Method, and Computer Program
US20100289742 *Jul 28, 2010Nov 18, 2010Promethean LimitedMoveable desktop
US20120038954 *Oct 20, 2010Feb 16, 2012Primax Electronics Ltd.Image cropping process
Classifications
U.S. Classification382/298, 382/276
International ClassificationG06T3/40
Cooperative ClassificationG06T3/4084
European ClassificationG06T3/40T
Legal Events
DateCodeEventDescription
Feb 7, 2013ASAssignment
Effective date: 20121220
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:CONVERSION WORKS, INC.;REEL/FRAME:029772/0078
Owner name: INTELLECTUAL DISCOVERY CO., LTD., KOREA, REPUBLIC
Nov 20, 2012ASAssignment
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:WORLDPLAY (BARBADOS) INC.;REEL/FRAME:029327/0750
Owner name: CONVERSION WORKS, INC., CANADA
Effective date: 20121119
Oct 13, 2010ASAssignment
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:CONVERSION WORKS, INC.;REEL/FRAME:025126/0605
Owner name: WORLDPLAY (BARBADOS) INC., BARBADOS
Effective date: 20100811
Mar 18, 2010FPAYFee payment
Year of fee payment: 8
Mar 6, 2006FPAYFee payment
Year of fee payment: 4
Jan 21, 2005ASAssignment
Owner name: CONVERSIN WORKS, INC., ALBERTA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HEADPLAY, INC.;REEL/FRAME:016153/0640
Effective date: 20040816
Owner name: HEADPLAY, INC., CANADA
Free format text: CHANGE OF NAME;ASSIGNOR:PUSH ENTERTAINMENT INC.;REEL/FRAME:016153/0587
Effective date: 20031230
Owner name: CONVERSIN WORKS, INC. 1111-11TH AVENUE SW, SUITE 3
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HEADPLAY, INC. /AR;REEL/FRAME:016153/0640
Owner name: HEADPLAY, INC. 1111 - 11TH AVENUE SW, SUITE 301CAL
Free format text: CHANGE OF NAME;ASSIGNOR:PUSH ENTERTAINMENT INC. /AR;REEL/FRAME:016153/0587
Feb 25, 2003CCCertificate of correction
Dec 4, 1998ASAssignment
Owner name: PUSH ENTERTAINMENT INC., CANADA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BRUTON, LEONARD THOMAS;SIMPSON, TODD;YEE, BARRY;REEL/FRAME:009622/0709
Effective date: 19981106