WO1998009445A1 - Method for producing a synthesized stereoscopic image - Google Patents
Method for producing a synthesized stereoscopic image Download PDFInfo
- Publication number
- WO1998009445A1 WO1998009445A1 PCT/US1997/015216 US9715216W WO9809445A1 WO 1998009445 A1 WO1998009445 A1 WO 1998009445A1 US 9715216 W US9715216 W US 9715216W WO 9809445 A1 WO9809445 A1 WO 9809445A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- images
- source image
- different images
- spatially different
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/10—Geometric effects
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00163—Optical arrangements
- A61B1/00194—Optical arrangements adapted for three-dimensional imaging
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B30/00—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images
- G02B30/20—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes
- G02B30/22—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the stereoscopic type
- G02B30/23—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the stereoscopic type using wavelength separation, e.g. using anaglyph techniques
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B30/00—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images
- G02B30/20—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes
- G02B30/22—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the stereoscopic type
- G02B30/24—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the stereoscopic type involving temporal multiplexing, e.g. using sequentially activated left and right shutters
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B30/00—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images
- G02B30/20—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes
- G02B30/22—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the stereoscopic type
- G02B30/25—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the stereoscopic type using polarisation techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/111—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
- H04N13/117—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/261—Image signal generators with monoscopic-to-stereoscopic image conversion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/275—Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
- A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00163—Optical arrangements
- A61B1/00193—Optical arrangements adapted for stereoscopic vision
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/167—Synchronising or controlling image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/189—Recording image signals; Reproducing recorded image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/194—Transmission of image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/207—Image signal generators using stereoscopic image cameras using a single 2D image sensor
- H04N13/218—Image signal generators using stereoscopic image cameras using a single 2D image sensor using spatial multiplexing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/239—Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/257—Colour aspects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/286—Image signal generators having separate monoscopic and stereoscopic modes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
- H04N13/334—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] using spectral multiplexing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
- H04N13/337—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] using polarisation multiplexing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
- H04N13/341—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] using temporal multiplexing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
- H04N13/344—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/398—Synchronisation thereof; Control thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N2013/0074—Stereoscopic image analysis
- H04N2013/0081—Depth or disparity estimation from stereoscopic image signals
Definitions
- the present invention relates to methods, devices and display systems for converting source images to synthesized stereoscopic images and more specifically for forming two or more images from a source image where at least one image of the two or more images has been modified relative to the source image such that synthesized stereoscopic images are formed.
- Stereoscopic imagery for television and computer monitors is performed by presenting a different image to each eye of the observer. Images for each eye may be created and presented electronically, electrooptically and/or purely optically such that the human observer perceives a three dimensional image. Using these methods, it is possible to create complex three dimensional presentations which may be used in a wide variety of applications.
- Patent No. 5,510,832 such conversions have not produced three dimensional imagery having sufficient quality. For example, it is not possible to create a quality three dimensional image of a still image using prior art methods since field delay may be required. Other transformations of such pairs of images from one encoding method to another has been also been difficult and costly because they generally require depth information and computation. Furthermore, it has been necessary to generate stereo pairs of images using two separate cameras or a single camera with special lenses. Such arrangements are costly and difficult to use.
- the device and method should also be able to form the stereoscopic image without substantially degrading the image or color.
- the present invention relates to methods, logic, data signals, recorded data, devices and systems for use in providing a synthesized stereoscopic image from a source image by forming two or more modified stereo images from the source image where at least one of the modified stereo images has been modified relative to the source image such that at least two of the modified stereo images have a different spacial appearance than each other.
- the stereoscopic image formed may be used in a holographic display.
- "different spacial appearance” refers to a difference in the size and/or orientation of two images and/or a difference in the spacial relationship between elements of one image and the corresponding elements of the other image. The different spacial appearance can be perceived when the two images are displayed and compared.
- the images with a different spacial appearance can be expressed as data encoding the images which, when displayed, have a different spacial appearance.
- the images can also be expressed as data encoding a source image in combination with one or more algorithms for modifying the source image and producing two images which have a different spacial appearance.
- modified stereo images refers to the two or more images derived from the source image where at least one image of the two or more images has been modified relative to the source image such that at least two of the two or more images have a different spacial appearance. These modifications to the images do not required knowledge of the relative depts of particular elements of the image. Modified stereo images can be in the form of two or more images being displayed as well as data encoding the two or more images which when displayed would have a different spacial appearance.
- Modified stereo images can be in the form of data encoding a source image in combination with one or more algorithms for modifying the source image and producing two or more images which have a different spacial appearance.
- a wide variety of modifications may be made to the source object in order to form the modified stereo images. These modifications may involve modifying one or more of the images forming the modified stereo images relative to the source image. For example, at least one image of the modified stereo images may be magnified, reduced, or rotated in the X, Y and/or Z plane relative to the source image. Alternatively or in addition, the position of one or more elements of one of the modified stereo images may also be changed relative to the source image.
- At least one image of the modified stereo images may be transformed relative to the source image using a function which alters the position of elements of the image along the X or Y axis relative to the source image.
- the function is a distorting algorithm, for example an elliptical or aspheric distorting algorithm.
- the function distorts at least two regions of the source image such that at least one region appears to have a concave appearance while at least one region appears to have a convex appearance.
- the function distorts at least two regions of the source image such that the at least two regions appear to have a concave appearance, the at least two regions being separated by a region which appears to have an at least partially planar or convex appearance.
- the function may distort at least two regions of the source image such that the at least two regions appear to have a convex appearance, the at least two regions being separated by a region which appears to have an at least partially planar or concave appearance.
- the source image may be any two or three dimensional image, holographic image or set of two or three dimensional images.
- Examples of source images include, but are not limited to motion picture film, photographs, computer images, video and tomographic data sets, such as those derived from
- the source images may be recorded, for example on VHS video tape, betacam SP or DI tape, nonlinear edit system, time base corrector, computer floppy disc, computer hard drive, RAM, CDROM, laserdisc and DVD.
- the source image may also be provided via a live broadcast, video signals, or generated by a software program.
- the source image can also be an analog or digital set of data corresponding to one or more two or three dimensional images.
- a method for producing a synthesized stereoscopic image by displaying modified stereo images having a different spacial appearance, and viewing the modified stereo images through stereo viewing glasses which transmit the spatially different images to left and right eyes of the viewer to form a synthesized stereoscopic image of the source image, such as a VR headset.
- an image display is used to transmit the modified stereo images to the stereo viewing glasses.
- image signals encoding the modified stereo images are transmitted to the image display by a device which converts recorded data encoding the modified stereo images into image signals which are received by the image display.
- image signals encoding the modified stereo images are transmitted to the image display by an image processor which converts a signal encoding the source image into signals encoding the modified stereo images.
- conversion of the signal encoding the source image into the signals encoding the modified stereo images is performed in real time. Conversion may alternatively be delayed. Conversion of the image signals may be an analog to analog, analog to digital, digital to digital, digital to analog, and/or in combination with optical to optical signal conversion.
- the signal encoding the source image may include a signal which provides instructions to the image processor regarding how to convert the signal encoding the source image into signals encoding the modified stereo images. These instructions may be simultaneously broadcast and may include, for example, image element by image element instructions or image frame by image frame instructions.
- a method is provided for producing a synthesized stereoscopic image using stereo viewing glasses which form modified stereo images. In this embodiment, the method includes displaying on an image display a source image, and viewing the source image through stereo viewing glasses, the stereo viewing glasses having left and right lenses, at least one of the lenses modifying the source image to produce at least two images having a different spacial appearance (modified stereo images) when viewed through the lenses.
- modifying the source image to produce modified stereo images having a different spacial appearance is performed in real time.
- the modified stereo images are created and stored prior to being displayed.
- Modifying the source image can involve an analog to analog, analog to digital, digital to digital, digital to analog, and/or in combination with optical to optical signal conversion.
- a method for converting a signal encoding a source image into recorded signals encoding two or more spatially different images which form a synthesized stereoscopic image of the source image when displayed in combination.
- the method includes taking a signal encoding a source image, forming two or more image signals encoding the source image, modifying at least one of the two or more image signals such that at least two of the image signals encode images which have a different spacial appearance than each other, and recording signals encoding the modified stereo images.
- the signals encoding the modified stereo images may be recorded on a variety of different media including, for example magnetic media, VHS video tape, Betacam SP or DI tape, nonlinear edit system, frame store, computer floppy disc, computer hard drive, RAM, CDROM, laserdisc, DVD, MPEG and other optical or digital device.
- Modifying at least one of the image signals can involve an analog to analog, analog to digital, digital to digital, digital to analog, and/or in combination with optical to optical signal conversion.
- a method is provided for synthesizing a stereoscopic image from a source image.
- the method includes taking a signal encoding a source image, forming two or more image signals encoding the source image, modifying at least one of the two or more image signals such that the two or more image signals encode at least a pair of images having a different spacial appearance, displaying on an image display the spatially different images encoded by the modified image signals, and viewing the image display through stereo viewing glasses which transmit the spatially different images to a left and right eye of the viewer to form a synthesized stereoscopic image of the source image.
- modification of the image signals is performed in real time. Modifying at least one of the image signals can involve analog to analog, analog to digital, digital to digital, digital to analog, and/or in combination with optical to optical signal conversion.
- the two or more image signals encoding the source image includes a signal which provides instructions to an image processor regarding how to modify at least one of the two or more image signals.
- These instructions may include, for example, image element by image element instructions or image frame by image frame instructions. These instructions can be simultaneously broadcasted the image signals and can be combined with the image signals. For example, the instructions can be embedded into the signal, for example, in the vertical sync.
- the present invention also relates to signals encoding two or more images which have a different spacial appearance which are derived from the same source image and which, when viewed in combination, form a synthesized stereoscopic image of the source image.
- These signals may be analog or digital signals
- the two or more images include two images which have a different spacial appearance than each other.
- the two or more images include three images which have a different spacial appearance than each other.
- These signals may be transmitted signals or signals which are recorded on a recording media. In one particular embodiment, these signals are derived from a computer generated image which has been modified according to the present invention.
- the present invention also relates to a recording of synthetic stereoscopic images according to the present invention.
- the recording includes a recording medium having recorded thereon two or more signals encoding at least two images having a different spacial appearance which, when viewed in combination, form a synthesized stereoscopic image.
- the recording may encode an analog or digital signal.
- the present invention also relates to an image processor for converting a signal encoding a source image into a signal encoding a synthesized stereoscopic image.
- the image processor includes a signal entry port for receiving a signal encoding a source image, logic for converting the source image into two or more image signals encoding the source image where at least one of the two or more image signals is modified such that the image signals encode two or more images having a different spacial appearance, and a signal exit port for transmitting the modified image signals.
- the logic may act to convert the source image to the synthesized stereoscopic image in real or non-real time.
- the logic may be programmable or controllable by the user to form different modified stereo images. For example, the user can program the logic to make certain elements of an image or certain images appear closer or farther away.
- the logic may be designed to dynamically control how the image processor transforms the source image based on the source image. This dynamic control may be on a program by program, image by image or element by element basis.
- the logic may include certain image element recognition protocols which cause certain elements to be selectively brought into the foreground or background when detected. In one embodiment, these protocols can be used to sense whether an image corresponds to a close-up or wide-angle perspective so that the image maybe modified accordingly.
- the logic may also include certain movement recognition protocols which detects movement of certain elements and causes the image to be modified in a particular manner in response to the detected movement. This adaptation is particularly useful in pan and scan applications where it is desirable to keep certain image elements, such as an actor, centered in the image.
- the image processor also receives one or more signals which accompany the signal encoding the source image and provides instructions to the image processor regarding how to modify the source image. These instructions may include, for example, image element by image element instructions or image frame by image frame instructions. For example, the instructions may direct image 1 to be given a first type of modification, image 2 to be given a second type of modification, image 3 to be given a third type of modification, etc. Each type of modification may be a global modification for the image or provide for multiple different modifications for different elements in the image.
- the present invention also relates to stereo viewing glasses for converting a source image into a synthesized stereoscopic image.
- the stereo viewing glasses include a first and second lens for receiving the source image, and an optic associated with at least one of the first and second lenses which modifies the spacial appearance of the source image such that the pair of lenses provide a pair of images having a different spacial appearance which, when viewed in combination, form a synthesized stereoscopic image.
- the stereo viewing glasses can include only optics, the images being formed by a separate image display.
- the stereo viewing glasses can include one or more image displays. These stereo viewing glasses can also incorporate existing optical techniques which have been used to create stereo effects.
- synthesized stereoscopic images In conjunction with forming synthesized stereoscopic images according to any of the various embodiments of the present invention, it is preferred to alter the type of modification of the source image used to create the synthesized stereoscopic images over time.
- the present invention relies on the mind taking synthesizing pairs of modified stereo images to create a synthesized stereoscopic image. It is possible for the initial stereoscopic effect to diminish over time if the same modifications to the source image is employed. Accordingly, it is preferred to employ different modifications to the pairs of modified stereo images over time. It is believed employing different modifications will make it more difficult for the viewer's mind to perceive the individual modified stereo images being presented as opposed to a synthesized stereoscopic image.
- the methods for forming stereoscopic images may also include the step of modifying the source image in two or more different ways over time as well as transitioning between the two or more different ways of modifying the source image.
- the devices and systems of the present invention may also include logic for modifying the source image in two or more different ways over time and optionally logic for sensing scene changes or other opportune sequences in the source image during which the way in which the source image is being modified may be altered.
- the image signals and recorded images of the present invention may also include modifications to the source image in two or more different ways over time in order to preserve and/or enhance the stereoscopic effect achieved as described above.
- the various embodiments of the present invention can be employed in a variety of applications where it is desirable to create a three dimensional depiction.
- the present invention can be used to transform images formed in existing computer programs into stereoscopic images.
- the present invention can be used to enhance the appearance of cascaded windows being stacked over each other. Different windows can also be made to appear closer or farther from the observer.
- the various embodiments of the present invention can also be employed in combination with known pan and scan techniques to enhance the effectiveness of three dimensional imaging.
- motion and depth can be introduced into a still picture by panning across the still picture.
- depth can be introduced by modifying the panned image according the present invention.
- the modified, panned stereoscopic image has a significantly greater three dimensional appearance.
- Another application of the present invention is in the area of medical tomography such as MRI and CT scans where a series of images are taken at different known depths.
- the series of two dimensional images can be converted into a series of three dimensional images which more accurately reflect the tomography of the imaged body location.
- a sense of depth can be exaggerated in each two dimensional image, thus making it easier to distinguish elements in each image.
- the present invention can also be used in medical endoscopy where two dimensional images taken from an endoscope are converted into three dimensional images by distorting the two dimensional image provided by the endoscope.
- the present invention can also be used in combination with quicktime virtual reality applications where three dimension imaging is introduced into or enhanced in such applications.
- Figures 1A-1I illustrate a series stereo pairs where at least the right image has been modified in a different manner than the left image.
- Figure 1 A illustrates a reduced right image.
- Figure IB illustrates a magnified right image.
- Figure ID illustrates a right image which is skewed along the X axis as a function of the Y axis.
- Figure IE illustrates a right image which is parabolically skewed along the X axis as a function of the Y axis.
- Figure IF illustrates a right image which has a spherical distortion in causing the center of the image to appear at a different position in the Z axis relative to the X-Y plane.
- Figure 1G illustrates a portion of the right image being magnified.
- Figure 1H illustrates a center portion of the right image being magnified.
- Figure 1 1 illustrates a portion of the left image being magnified and a center portion of the right image being reduced.
- Figure 2 illustrates a typical stereoscopic display system.
- Figure 3 A is a timing diagram showing the transmission of left and right images to the eyes of a user of a time sequential stereoscopic system.
- Figure 3B illustrates an over-under format for displaying modified stereo images.
- Figure 3C illustrates a time sequence of a series of over-under image frames as in Figure 3B where either a first or second image is displayed in each image frame.
- Figure 4 shows the effect of optical lenses used in one embodiment of the invention.
- Figure 5 illustrates a generalization of a system for producing synthesized stereoscopic images.
- Figures 6A-6G represent particular embodiments of the system illustrated with regard to Figure 5.
- Figure 7A illustrates a set of tomographic images.
- Figure 7B illustrates a set of stereo pairs of the tomographic images illustrated in Figure 7A.
- Figure 7C illustrates a set of stereo pairs of the tomographic images illustrated in Figure 7 A where particular images in the set of tomographic images are modified relative to each other to provide an improved stereoscopic depiction of those particular images in the set.
- Figure 8 A illustrates a set of bowling pins.
- Figures 8B-8E illustrate a series of two-dimensional tomographic images of the set of bowling pins where each image shows a different row of pins.
- Figure 8F illustrates a stereoscopic image of the bowling pins created from the tomographic images illustrated in Figures 8B-8E.
- Figure 8G illustrates a top down view of the image illustrated in Figure 8F.
- Figure 8H illustrates the set of bowling pins illustrated in Figure 8A and a second viewing axis.
- Figures 8I-8L illustrates a series of two-dimensional tomographic images of the set of bowling pins taken along the second viewing axis.
- Figure 8M illustrates a stereoscopic image of the bowling pins created from the tomographic images illustrated in Figures 8I-8L.
- Figure 8N illustrates a top down view of the image illustrated in Figure 8M.
- Figure 9 illustrates different images on a computer monitor where each image has a user selectable three dimensional appearance.
- Figure 10 illustrates examples of different pairs of wire meshes which can be generated by modifying the wire frame mesh illustrated in the figure as the initial wire mesh.
- the present invention relates to the discovery that the mind perceives two or more spatially different versions of the same image presented to the left and right eye as a synthesized stereoscopic image, i.e., as a three dimensional image.
- the present invention exploits this realization through methods, logic, data signals, recorded data, devices and stereoscopic imaging systems which convert a source image into a synthesized stereoscopic image by forming two or more images from the source image and modifying at least one of those images relative to the source image such that at least two of the images do not have the same spacial appearance as each other with or without regard to their distance.
- synthesized stereoscopic images have been created using image shifting in combination with time delay, i.e., by controlling the time at which a same image is provided to the left and right eye of a viewer.
- time delay stereoscopic system An example of a time delay stereoscopic system is described in U.S. Patent No. 5,510,832 which is incorporated herein by reference.
- the present invention creates synthesized stereoscopic images by forming two or more images which have a different spacial appearance.
- the use of images with a different spacial appearance according to the present invention can be done in combination with time delay or independent of time delay or motion.
- Stereoscopic images have also been formed previously with the use of computers which calculate how the left and right eyes of a viewer would perceive each element of an image given the relative position of the left and right eyes to each element.
- the amount of computation required by this approach significantly limits its practicality for real time conversion of source images to stereoscopic images.
- stereoscopic images are formed according to the present invention by modifying the source image without regard to simulating how each element of the source image would appear to the left and right eyes of the user if those elements were actually being seen by the user.
- elements of the source object or the entire source object are modified by an algorithm which causes the element or entire image to appear at a certain distance in the resultant stereoscopic image.
- the computational demands involved in the present invention are significantly reduced as compared to computer generated stereoscopic images.
- the reduced computational demands involved in the present invention enable stereoscopic images to be generated according to the present invention in real time.
- the present invention can modify these images to simulate the known depth without having to precisely calculate how each pixel of the image should appear in order to simulate depth.
- Two or more images with different spacial appearances can be formed by magnifying, reducing, stretching (X axis or Y axis) rotating (X-Y plane) and/or tilting (rotating in X-Z and/or Y-Z planes) one or more of the images.
- a different spacial appearance can also be formed by modifying the relative position of elements in all or a portion of one or more of the images.
- modified stereo images are formed by taking a source image, forming two or more images from the source image, and transforming at least one of the two or more images using a function (f(x,y)) which alters the position along the X and/or Y axis of all or some of the objects appearing in the source image.
- the function may also serve to alter the image on a line by line basis when different functions can be used in different lines. Examples of transforming functions include distorting algorithms such as elliptical and aspheric algorithms, enlargements or offsets.
- the function is a non-linear distortion along the X and/or Y axes.
- the function distorts at least two regions of the source image such that at least one region appears to have a concave appearance while at least one region appears to have a convex appearance.
- the function distorts at least two regions of the source image such that the at least two regions appear to have a concave appearance, the at least two regions being separated by a region which appears to have an at least partially planar or convex appearance.
- the function may distort at least two regions of the source image such that the at least two regions appear to have a convex appearance, the at least two regions being separated by a region which appears to have an at least partially planar or concave appearance.
- any combination of the above modifications may be made on all or any portion of the right and/or left eye image in all or only some of the fields. More than one modification can also be made to the same image.
- all or some of the multiple frames may have modified stereo pairs.
- the modified stereo images may be formed by taking a source image and mapping the source image onto a wire mesh and distorting the image according to a function, such as those described herein and illustrated with regard to Figures 1 A- II.
- Figures 1A-1I illustrate a series of modified stereo image pairs where at least the right eye image has been modified relative to the source image, illustrated as a uniform horizontally and vertically oriented grid. Although pairs of images are illustrated, it should be noted that sets of three or more images can also be formed.
- Figure 1 A illustrates a reduced right eye image.
- Figure IB illustrates a magnified right eye image.
- Figure IC illustrates a rotated right eye image.
- Figure ID illustrates a right eye image which is skewed along the X axis as a function of the Y axis.
- Figure IE illustrates a right eye image which is parabolically skewed along the X axis as a function of the Y axis.
- Figure IF illustrates a right eye image which has a spherical distortion in causing the center of the image to appear at a different position in the Z axis relative to the X-Y plane.
- Figure IG illustrates a right eye image where a portion of the image has been magnified.
- Figure 1H illustrates a right eye image where the center of the image has been magnified.
- Figure 1 Illustrates a left eye image with a magnified center portion and a right eye image with a reduced center portion.
- Figures 1 A-I are intended to be illustrative of the various modifications which can be made to the source image in order to form modified stereo image pairs and are not intended to be exhaustive. These and other modifications to a source image to produce modified stereo image pairs can be created by a variety of digital or optical devices in real or non-real time without departing from the scope of the present invention.
- a pair of modified images are tilted or rotated in the X and/or Y and/or Z plane 10 percent or less relative to the other image.
- a pair of modified images are magnified or reduced 10 percent or less relative to each other.
- one of a pair of modified images is delayed one field or less relative to the other eye image.
- a unique feature of the present invention is that it is possible in some cases to present either image to either eye without pseudoscopy. This is made possible by the fact that the differences in the two or more images are created by modifying one or more of the images relative to a source image as opposed to relying on the relative spatial positions of the images at the time of photography or videography in order to cause the sensation of depth.
- Figure 3 A shows all the transforms being performed on one side, channel or element, in fact both sides, channels or elements may perform the same or different transforms to different degrees and such transforms or degrees may change over time in order to obtain maximum effect with minimum discomfort for the viewer.
- a typical stereoscopic display system is illustrated in Figure 2.
- the system includes an image source 12, an image processor 14, an image display 16, and stereo viewing glasses 18.
- the stereoscopic display system produces modified stereo images by taking a source image and forming a two or more images where at least one image has been modified relative to the source image and relative to the other images of the modified stereo images.
- modified stereo images can be performed by several different components of a stereoscopic display system.
- the image source 12 can provide image data corresponding to modified stereo images.
- the image processor 14 can receive image data from the image source 12 and convert the image data into modified stereo images according to the present invention.
- the image data can also include instructions regarding how to modify the source image, for example, element by element or image by image.
- optics may be positioned between the image display and the user which convert images formed by the image display 16 into modified stereo images.
- the lenses of stereo viewing glasses 18 may each receive the same image from the image display 16 and convert the image into modified stereo images according to the present invention.
- the stereo viewing glasses may optionally include the image display 16.
- the present invention also encompasses image processors which convert data for source images into image signals for modified stereo images according to the present invention.
- image processors which convert data for source images into image signals for modified stereo images according to the present invention.
- optics which may be used to optically convert images produced by an image display into modified stereo images according to the present invention.
- the present invention is also intended to encompass stereoscopic imaging systems which include an image source, image processor and/or optics which produce modified stereo images according to the present invention.
- components of the system which are not being used to modify a source image to produce modified stereo image pairs may be standard equipment which is currently used in existing stereoscopic imaging systems.
- the present invention is also intended to encompass methods employed by the above components of a stereoscopic imaging system for producing modified stereo images according to the present invention.
- a significant advantage of the methods, devices and stereoscopic imaging systems of present invention is their ability to convert existing two dimensional and three dimensional motion picture films, photos, computer images, videos and single cameras with ordinary lenses into a variety of stereo formats in real time or nonreal time.
- the methods and devices of the present invention may be used to convert unrecorded image signals, for example from a live broadcasts, video signals received via antenna or cable, or images from a computer program into a stereoscopic rendering of the program in full color and in real time.
- a further advantage of the present invention is the ability to convert the source images into a series of different stereoscopic images based on the way in which the stereo images are modified relative to each other.
- the methods, devices and systems of the present invention are designed to be programmable to provide a series of different visual effects.
- this stereosynthesis can be very sophisticated, real time and automatic.
- An object may be made to appear to recede into the background as it moves or becomes smaller.
- artificial movement of an image i.e. movement that occurs in the modified images but not in the source image, can be created by selecting a series of modification which introduce the sense of motion. Some elements in an image can be made to appear closer while other elements appear farther away.
- a further advantage of the present invention is the limited computation that is required to convert the source images to stereoscopic images.
- Stereoscopic images have previously been formed by calculating how the left and right eyes of a viewer would perceive each element of an image given the relative position of the left and right eyes to each element.
- the amount of computation required by this approach significantly limits its practicality for real time conversion of the source images to stereoscopic images.
- stereoscopic images are formed according to the present invention without regard to simulating how each element of the source image would appear to the left and right eyes of the user if those elements were actually being seen by the user.
- the computational demands involved in the present invention are significantly reduced as compared to computer generated stereoscopic images and thus can be generated in real time.
- a further advantage of the present invention is the ability of the methods, devices and systems to convert the source images into three dimensional images in a single step.
- the methods and devices of the present invention may be used with standard equipment. For example, except where modified stereo images are converted by the image viewer, full color stereoscopic images are viewable with inexpensive dual filter glasses (amaglyph glasses), LCD shutter glasses or other devices known in the art for controlled image transmission to the left and right eyes of the observer.
- a further advantage of the present invention is the color quality of the stereoscopic images produced. Because the conversion of source images to three dimensional images according to the present invention does not involve modifying the color of the source images, high quality full color stereoscopic images can be produced.
- the image source 12 may be any recorded media, an image signal transmitter or data set which provides image data to the image processor 14.
- Examples of sources of recorded source images which can be readily converted into three dimensional images by the present invention include two dimensional and three dimensional motion picture films, photographs, computer images, and videos.
- the images are images generated by computer software.
- These image sources may be in a digital or analog format or in a real image format.
- Examples of recorded formats include, but are not limited to, videotape, magnetic media such as floppy discs, digital recorded media such as digital memory, digital tape, compact discs, DVD and the like.
- image transmitters include broadcast antenna, cable, fiber and satellite.
- the image source may be a standard image source which provides image data encoding standard two dimensional source images which are later processed by a component of a system according to the present invention to form modified stereo image pairs.
- the image source may include a signal which provides instructions to an image processor regarding how to convert the signal encoding the source image into signals encoding the modified stereo images.
- These instructions may include, for example, image element by image element instructions or image frame by image frame instructions.
- the image source may include image data creating modified stereo images.
- the image source provides image data to the image processor corresponding to modified stereo images.
- the image processor receives the image data for the modified stereo images.
- the image source provides image data encoding modified stereo images
- the image data is preferably provided in an over/under or left/right format where two of the modified stereo images are placed on the same frame. In this format, a first image 21 from a set of modified stereo images is taken and condensed into an upper portion (or left portion) of an image frame. Meanwhile, a second image 23 from the set of modified stereo images is condensed into a lower portion (or right portion) of the same image frame, as illustrated in Figure 3B.
- the first and second images 21, 23, although each being placed in only a portion of the image frame, are preferably non- interlaced images containing all the image data for the image.
- the use of interlaced field images causes half the amount of image information to be stored per image frame.
- Figure 3C illustrates a time sequence of a series of over-under image frames as in Figure 3B where either a first or second image 21, 23 is displayed in each image frame. Either the first or second image or the over-under image can be displayed from each image frame at a given time. Accordingly, you can have any combination of first (1) and second (2) images displayed in time sequence, (e.g., 12121212 or 11221122 or 111222 or 112112 or 111112, etc.).
- This approach provides the system of the present invention with significantly greater flexibility for producing visual effects based on displaying different combinations of first and second images. For example, this approach can be used to create certain visual effects, such as the movement of an image element laterally across the screen. By controlling whether an image element is shown to the left or right eye, the appearance of movement by the image element laterally across the image can be created or enhanced.
- an over/under or left/right format has further advantage of enabling different visual effects to be performed. For example, line averaging between two left or two right eye frames can be performed.
- computer software is used as described in "UNIVERSAL STEREOSCOPIC INTERFACE," Provisional Application Serial No.: 60/013,738; filed: March 20, 1996; and"UNIVERSAL STEREOSCOPIC INTERFACE,” Application Serial No.: 08/280,570; filed: March 19, 1997, each of which are incorporated herein by reference.
- This computer software enables one to distinguish between and selectively display different images of a pair of images presented in an over/under format. This software is useful for increasing the range of display formats which can be used with the modified stereo image pairs used in the present invention.
- the image data preferably includes modified stereo images where the modifications relative to the source image are performed by two or more different transformations over time.
- the present invention relies on the mind taking synthesizing pairs of modified stereo images to create a synthesized stereoscopic image. It is possible for the initial stereoscopic effect to diminish over time if the same modifications to the source image is employed. Accordingly, it is preferred to employ display pairs of modified stereo images where the pairs are modified by different transformations over time.
- the image processor 14 receives image data via a signal entry port 15.
- the image processor 14 includes logic for converting the image data into image signals encoding the source image and for modifying at least one of the image signals such that the image signals encode images having a different spacial appearance.
- the image processor also includes a signal exit port 17 for transmitting the modified first and second image signals to an image display 16 which displays real images corresponding to the modified image signals. Examples of image displays include, but are not limited to, screens associated with projection systems, televisions, CRT screens, VR or holography.
- Standard image displays such as television sets have a refresh rate of 15.25 KHz which is above the frequency at which the human eye can perceive individual images.
- the image displays appear to present a continuous image.
- Presenting alternating left and right eye images effectively causes an image display to have an effective refresh rate that is half the normal refresh rate of that display. If the refresh rate falls below the frequency at which the human eye can perceive images, the mind perceives the display as flickering which can cause eye strain and headaches.
- the image processor receives image data from an image source corresponding to modified stereo image.
- the image processor can receive image data from an image source corresponding to a source image and produce an image signal for the source image.
- the image processor converts image data from an image source encoding a standard source image into image signals encoding modified stereo images.
- the image processor takes image data from the image source and forms signals for images where at least one of the images has been modified relative to the source image such that the modified images do not have the same spacial appearance. Processing of the image data from the image source into modified stereo images may be done in real time or not in real time by the image processor and recorded.
- the signals inputted into the image processor and outputted by the image processor may be analog or digital signals.
- the image processor includes logic for performing multiple different transformations of a source image to produce different modified stereo images based on a source image.
- the different modified stereo images are produced by different transformations of the source image to create different visual effects. For example, certain transformations increase or decrease the depth of field or field of view provided.
- the image processor includes logic for switching between different transformations at selected scene changes or other selected periods of time.
- the present invention relies on the mind taking synthesizing pairs of modified stereo images to create a synthesized stereoscopic image. It is possible for the initial stereoscopic effect to diminish over time if the same modifications to the source image is employed. Accordingly, it is preferred to employ different modifications to the pairs of modified stereo images over time. Two or more different transformations can be used sequentially to form synthesized stereoscopic images, each different transformation being employed for a period of time.
- the logic in the processor can be used to switch between the different transformations after selected time periods, at scene breaks or other selected times. The logic can also be used to sense when to switch between the different transformations.
- the different transformations performed by the image processor may be selectable by the observer, thereby providing the observer with control over the visual effects produced.
- the user can select transformations which cause certain elements of an image to appear closer or farther away relative to their appearance in a source image.
- the user can also select among different transformations which introduce different degrees of spatially transformations to increase or decrease the disparity perceived between close and distant elements in the same image, ln this regard, the observer can design the appearance of the resulting stereoscopic images.
- the ability of an observer to customize the stereoscopic imagery produced is an important feature of the present invention since how stereoscopic images are perceived varies from person to person.
- the image processor can include logic which controls how the image processor transforms the source image based on the source image.
- This logic can correspond to software or a computer chip encoding the logic.
- the logic can be used to enable the image processor to dynamically adjust itself to optimize the stereoscopic appearance of different images or elements in images.
- the logic can be used to detect motion in the source image and modify the source image to make the images appear closer or farther away based on the motion detected.
- the logic can also be used for special effects to create the appearance of motion of elements of the image which are stationary in the source image.
- the logic is used to identify a depth map or area, using grey scale values, shadows, motion detection (things moving in front of another), hue, size changes, edge detection or other detection techniques. These areas are then assigned modifications according to their desired depths.
- the source image is then used as a texture map that is mapped onto or modified according to the depth map to modify the source image.
- the depth maps may be further refined with left and right eye renderings using the texture maps rendered from "2" eye point of view.
- a two dimensional source image may be converted to a three dimensional stereo pair by creating a depth map using a detected characteristic of the source image such as a grey scale value, hue, edge detection and or areas of motion.
- a wire frame mesh is then formed from the depth map.
- Relative depth is then assigned to the outline based on the stereoscopic effect to be achieved.
- a pair of wire meshes are then formed from the wire frame mesh where at least one mesh has been modified using the modification techniques of the present invention. Texture mapping of the original picture onto the wire meshes is then performed to produce the pair of modified images.
- the logic of the image processor can also be used as an editing tool to make certain objects appear closer or farther away.
- the ability to magnify or reduce the appearance of elements within an image is an important application of the present invention in conjunction with film editing, such as pan and scan.
- This type of editing tool is also useful in conjunction allowing an observer to customize the appearance of a stereoscopic image, for example by allowing an observer to select which images or elements of an image to magnify, reduce or distort.
- Another application of this editing tool is in the computer software arena where an observer selects which elements of a stationary computer generated image the observer wishes to bring into the foreground or the background.
- the image processor For optimal depth with minimal eyestrain, it is desirable to carefully adjust the effects on each shot of a program and to have them vary timewise. It is also desirable to design the image processor to permit the user to control which image transformation is performed. For instance, the entire right eye frame or parts of a frame may be delayed one field, magnified 5 percent and horizontally shifted 7 percent while the right half of the image is tilted back in the Z axis 6 percent, an object in the middle convexly projected on a spherical surface of diameter 3 times the screen height, while the left eye image may be untransformed or have an object in its center magnified 8 percent, the left half given vertical rugose transforms, the top half concavely projected on a dodecahedron with smallest diameter 4 times the screen width etc. These transforms could change in kind and vary in degree and from one eye's field to the other either slowly or as rapidly as every field with all these parameters preprogrammed or under user control.
- Conversion of image data encoding a source image to an image signal encoding modified stereo images can be performed in real time by analog and/or digital devices or by computer software. Conversion could also be done with a single video stream from digital or analog source or from dual streams. In cases where field delay or field advancement is desired, this could be introduced by running two copies of the same tape, disc or digital source one or more fields out of sync or by picking fields as desired from an analog or digital store in real or nonreal time. Common examples of such sources and stores are betacam SP or DI tapes, nonlinear edit systems, frame store, computer hard drives and
- RAM random access memory
- CDROM compact disc and DVD.
- parts of the source image can be designated for particular transforms and followed from frame to frame by the program. For instance, a human figure can be identified, given a five percent convex transform and one field delay in the left eye until it disappeared from the scene or the scene changed. Parts of the image can be transformed in this manner and composited and recomposited into the image. Delays or advancement of one or more field or parts of fields can be part of the synthesized difference between the images presented to the two eyes. Ideally, the delay will be one field or less and will depend on the direction and speed of motion.
- Non-real time shot to shot editing is essential for use of delay and is desirable for all the other image transforms as well. For many sequences, no delay will give the best results. Delay can be applied to only part of the field instead of or in addition to the whole field.
- Stereo Viewing Glasses The stereo viewing glasses 18 operate in combination with the image display
- Figure 3 A illustrates an image display 16 for use with flicker glasses to control what images are observed by the left and right eyes of the observer.
- the left lens 29 of the stereo viewing glasses 18 transmits the left image while the right lens 31 blocks or absorbs the left image 28. This allows the left eye of the observer to see the left image 28, while the right eye does not see the left image 28.
- the lens transmissions are reversed so that the right eye sees the right image and the left eye does not see the left image.
- the observer's brain receives both images and synthesizes the two images into a single image which the brain interprets as having depth.
- the stereo viewing glasses 18 may have optical, electrooptic or mechanical lenses. In one preferred embodiment, each lens is activated electronically to block the light reaching the eyes alternately.
- the glasses may be gelatin, glass, dichroic polarized, anaglyphic, or holographic filters or optical or electrooptic and need only have appropriate optical characteristics.
- any other display modality may be used such as displaying right and left stereo pairs in frame, field, line or pixel sequential form or as simultaneous pairs for stereoscopic or autostereoscopic viewing.
- the images are presented autostereoscopically by known means without the use of stereo viewing glasses.
- the stereo viewing glasses can have polarized lens. Then, by controlling the polarization of the light, different images can be directed to the left and right eyes.
- the stereo viewing glasses can have red/green and blue filters on the left and right eyes respectively. Then, by transmitting one image in red and green light and the other image in blue light, different images can be selectively transmitted to the left and right eyes of the observer.
- Other mechanisms for presenting different images to the left and right eyes of the observer that are known in the art or later developed and are intended to be used in the present invention.
- modified image pairs may be formed by optics positioned between the image display and the observer which selectively modify left or right eye images produced by the image display.
- the image display or the stereo viewing glasses may include optics which modify a source object produced by the image display such that modified pairs of left and right eye images are produced.
- Figure 4 shows the effect of one type of lens used in accordance with the present invention.
- the left eye lens 48 does not transform the image produced by the image display.
- the right eye lens 50 modifies the right eye image according to the present invention.
- the brain combines the left and right images to produce a single image which is perceived as having depth.
- Figure 5 illustrates a generalization of the overall system.
- FIGS 6A-6G represent particular embodiments of the system illustrated with regard to Figure 5.
- A represents a two dimensional data set of a source image.
- the data set can be an analog or digital data set.
- the data set is copied by device B which can be an optical image duplicator or signal distribution amplifier.
- Signals C and C represent at least two copies of the data set encoding the source image which are conveyed to separate sets of image processors, represented as processors D, E and F and D', E' and FV A greater or lesser number of image processors can be used than are illustrated.
- the data sets encoding the source image are spatially altered by the processors according to one or more functions for modifying the image, such as the functions that have been described herein.
- the signals from the processors can be resent through the different processors as many times as desired, as illustrated by long arrows 81.
- Device K controls the qualities of the spatial distortions introduced into the image signals M, M' by the processors, further modifications of the image signals M, M' by devices G and G', and/or the signals N, N' that are sent to image display devices H, H ⁇ Devices G and G' may perform a variety of functions including, for example, frequency doubling or other modifications which provide image enhancement.
- Device J represents the different techniques for combining two or more images to produce stereoscopic images including, for example, sequential and optical field multiplexing, such as shutter glasses or polarized lenses.
- Device J can also represent a device for recording the image signals as digital or analog data sets.
- Figure 6A illustrates an embodiment of the system illustrated in Figure 5 where A represents any video signal source including for example, a camera, VCR, broadcast signal or DVD.
- Device B represents a signal distribution amplifier.
- Image processing in this system is performed by a dual channel video effects device (DVE) D, D' which is controlled by controller K.
- image display devices H, H' are virtual reality (VR) glasses with dual video displays, one for each eye.
- Figure 6B illustrates an alternate embodiment of the system illustrated in Figure 5 where A represents any video signal source and device B represents a signal distribution amplifier.
- Image processing in this system is performed by a single DVE D which is controlled by controller K.
- An optional second DVE D' is also illustrated which can optionally alter image signal N' relative to the source image.
- image display devices H, H' are virtual reality (VR) glasses with dual video displays, one for each eye.
- Figure 6C illustrates another alternate embodiment of the system illustrated in Figure 5 where A represents any video signal source and device B represents a signal distribution amplifier. Image processing in this system is performed by
- DVE D D' which is controlled by controller K.
- a multiplexer G field sequentially multiplex signals M and M'.
- the multiplexed image signals M and M' are displayed on video display H (e.g., a TV or computer monitor).
- Shutter glasses J combine the multiplexed images displayed on video display H. J can alternatively be polarized lenses where the video display H displays multiplexed images formed of light with different polarizations.
- Figure 6D illustrates another alternate embodiment of the system illustrated in Figure 5 where A represents any video signal source, device B represents a signal distribution amplifier, D, D' represent an DVE image processor and K represents a controller.
- a multiplexer Gl field sequentially multiplexes signals M and M' while line doubler G2 increases the refresh rate at which images are displayed on video display H.
- Line doubler G2 can also include the functionality of a picture enhancer for providing edge improvement.
- Shutter glasses J combine the multiplexed images displayed on video display H.
- Figure 6E illustrates a variation of Figure 6D where the system further includes a motion detector P which detects motion of elements of an image, e.g., whether a camera is panning from right to left, left to right, up, down, etc.
- Device P can alternately be used to detect whether a particular image element is close or distant in order to determine what transformaton to use.
- the controller K determines which image (encoded by M and M') to display first.
- motion detector P can also be used to determine which image (encoded by M and M') the display H should display first.
- the motion detector P can also be used to detect motion in an image, the detection of which is used by the controller K to direct the processor D, D' to modify the image signals to show that an element of an image is moving in a particular direction.
- the detection of an element moving toward the screen can be used by the controller to direct the processor to modify the images to make the element appear closer in the stereoscopic image, thereby enhancing the stereoscopic effect.
- Figure 6F illustrates an alternate variation of Figure 6D where the system further includes a motion detector P which is only connected to the multiplexer.
- the motion detector P is used to detect motion of elements in an image, e.g., whether a camera is panning from right to left, left to right, up, down, etc, and uses this information to determine which image (encoded by M and M') the image display H should display first.
- motion detector P can also be used to determine which image (encoded by M and M') the display H should display first.
- Stereographic Imaging Of Sets Of Tomographic Images illustrates the modification of a set of tomographic images to provide improved stereographic imaging of the set of tomographic images.
- a set of tomographic images, illustrated in Figure 7 A is taken and converted into stereo pairs of the tomographic images, illustrated in Figure 7B, where the pair of images formed of each slice has a different spacial appearance relative to each other.
- Figure 7C only particular images of the set of tomographic images are modified relative to each other to provide an improved stereoscopic depiction of those particular images in the set. 7.
- Figure 8A illustrates a three dimensional object, in this case a set of bowling pins viewed along axis 71.
- Figures 8B-8E illustrate a series of two-dimensional tomographic images of the set of bowling pins taken along axis 71 where each image shows a different row of pins, intended to represent a different depth layer of the three dimensional image.
- a stereoscopic image of the bowling pins can be created, as illustrated in Figure 8F.
- a data matrix can then be formed of the different elements (bowling pins) as they appear in the overlaid stereoscopic image.
- Figure 8G illustrates a top down view of the data matrix illustrated in Figure 8F.
- Figure 8H illustrates the set of bowling pins illustrated in Figure 8 A and a viewing axis 73.
- Figures 8I-8L illustrates a series of two- dimensional tomographic images of the set of bowling pins taken along axis 73.
- Figure 8M illustrates a top down depiction of the stereoscopic image illustrated in Figure 8M.
- Figures 8 A-8N it is possible to convert a series of tomographic images of an object and use the methods and devices of the present invention to create different synthesized stereoscopic images of the object along different viewing axes.
- This application of the present invention is particularly useful in medical applications like MRI and CT imaging where it is desirable to be able to create different three dimensional renderings of an object in order to identify the relative positions of different elements of the object. For example, by viewing a three dimensional rendering of an object at different angles, it is possible to more accurately determine the relative positions of different elements within an object.
- an observer attaches a mechanism to his or her head which can detect head motion.
- the head movement mechanism is coupled to the system for generating the stereoscopic images and is used to direct different stereoscopic images to be generated based on the angle or position of the observer's head, as determined by the head movement mechanism. This enables an observer, for example, to turn his or her head sideways or up or down in order to view the stereoscopic image of the object at different angles.
- the system in a preferred embodiment, is able to track the head movement and modify the stereoscopic image accordingly in real time.
- FIG. 9 Illustrated in Figure 9 are regions DI through D10 which represent images displayed on a computer monitor according to the present invention which appear to have separate three dimensional appearances when viewed using shutter glasses, polarized plates or anaglyph techniques or other three dimensional imaging techniques.
- the software program receive data encoding a source image as input, forms modified stereo images of the source image, and then texture maps the image by applying a function to the source image, and then texture maps the image to modify the appearance of the entire source image based on that function, as opposed to forming a stereographic image by calculating how each pixel of each image would appear to the left and right eyes of the user.
- the software program can be used to independently texture map the different images represented by DI through D10 in order to create independent stereographic appearances for each image based on the particular modified stereo images formed, i.e., different modifications cause the images to have a different stereographic appearance.
- the software program can generate the stereographic appearance of these images automatically.
- the user can direct the software program to perform different modifications such that each image has a user selected stereographic appearance.
- the software enables the user to create his or her own stereographic display of a series of images which can be modified as the user desires.
- the modified stereo images may be formed by taking a source image and forming two more modified images according to the present invention. This may be done, for example, by mapping the source image onto two wire frame meshes and then distorting at least one of the images by the distorting one or both of the wire frame meshes onto which the source image is mapped.
- An equivalent effect to distorting the wire frame meshes can be done via mathematical manipulation of data encoding the source image.
- Figure 10 illustrates examples of different pairs of wire meshes which can be generated by modifying the wire mesh illustrated in the figure as the initial wire mesh. By mapping a source image onto two initial wire meshes for the left and right eyes and then distorting one or both of the wire meshes as illustrated in the figure, a stereo pair of images can be created. 10. Image Data Management Protocol For Image Displays
- image data for a given line of an image is received and modified according to the present invention into data for a stereo pair of the given line.
- each line of the stereo pair of lines is then displayed to the left and right eye respectively before new image data for same given line is received and displayed. This is accomplished by increasing the frequency at which a given line is presented to the left or right eye.
- the same number of lines are presented over time to each of the left and right eyes as would be presented to the left and right eyes if the line data were not modified according to the present invention.
- the above protocol may be used with interlaced or noninterlaced displays known as multiple frequency monitors.
Abstract
Methods, devices and systems are provided which produce a synthesized stereoscopic image of a source image by forming at least two images from a source image where at least one image has been modified relative to the source image such that the images have a different spacial appearance than each other. A wide variety of modifications may be made to the source object in order to form modified stereo images. These modifications may involve modifying one or more images relative to the source image. For example, at least one image may be magnified, reduced, or rotated in the X, Y and/or Z plane relative to the source image. Alternatively or in addition, the position of one or more elements of one of the images may also be changed relative to the source image. Alternatively or in addition, at least one of the images may be transformed relative to the source image using a function which alters the position of elements of the image along the X or Y axis relative to the source image. In one variation, the function is a distorting algorithm, for example an elliptical or aspheric distorting algorithm which is not dependent on the depth information of the source image.
Description
METHOD FOR PRODUCING A SYNTHESIZED STEREOSCOPIC IMAGE
FIELD OF THE INVENTION The present invention relates to methods, devices and display systems for converting source images to synthesized stereoscopic images and more specifically for forming two or more images from a source image where at least one image of the two or more images has been modified relative to the source image such that synthesized stereoscopic images are formed.
BACKGROUND OF THE INVENTION
Stereoscopic imagery for television and computer monitors is performed by presenting a different image to each eye of the observer. Images for each eye may be created and presented electronically, electrooptically and/or purely optically such that the human observer perceives a three dimensional image. Using these methods, it is possible to create complex three dimensional presentations which may be used in a wide variety of applications.
Most systems for producing stereoscopic imagery have depended on the use of true stereo pairs of images created by complex and costly optical or computer systems. Although there have been attempts to convert two dimensional images to three dimensional images using field delay with image shifting (e.g. U.S.
Patent No. 5,510,832), such conversions have not produced three dimensional imagery having sufficient quality. For example, it is not possible to create a
quality three dimensional image of a still image using prior art methods since field delay may be required. Other transformations of such pairs of images from one encoding method to another has been also been difficult and costly because they generally require depth information and computation. Furthermore, it has been necessary to generate stereo pairs of images using two separate cameras or a single camera with special lenses. Such arrangements are costly and difficult to use.
A need therefore exists for a method and device for providing stereoscopic images which can be done rapidly and inexpensively. The device and method should also be able to form the stereoscopic image without substantially degrading the image or color.
SUMMARY OF THE INVENTION
The present invention relates to methods, logic, data signals, recorded data, devices and systems for use in providing a synthesized stereoscopic image from a source image by forming two or more modified stereo images from the source image where at least one of the modified stereo images has been modified relative to the source image such that at least two of the modified stereo images have a different spacial appearance than each other. The stereoscopic image formed may be used in a holographic display. As used herein, "different spacial appearance" refers to a difference in the size and/or orientation of two images and/or a difference in the spacial relationship between elements of one image and the corresponding elements of the other image. The different spacial appearance can be perceived when the two images are displayed and compared. However, the images with a different spacial appearance can be expressed as data encoding the images which, when displayed, have a different spacial appearance. The images can also be expressed as data encoding a source image in combination with one or more
algorithms for modifying the source image and producing two images which have a different spacial appearance.
As also used herein, "modified stereo images" refers to the two or more images derived from the source image where at least one image of the two or more images has been modified relative to the source image such that at least two of the two or more images have a different spacial appearance. These modifications to the images do not required knowledge of the relative depts of particular elements of the image. Modified stereo images can be in the form of two or more images being displayed as well as data encoding the two or more images which when displayed would have a different spacial appearance.
Modified stereo images can be in the form of data encoding a source image in combination with one or more algorithms for modifying the source image and producing two or more images which have a different spacial appearance. A wide variety of modifications may be made to the source object in order to form the modified stereo images. These modifications may involve modifying one or more of the images forming the modified stereo images relative to the source image. For example, at least one image of the modified stereo images may be magnified, reduced, or rotated in the X, Y and/or Z plane relative to the source image. Alternatively or in addition, the position of one or more elements of one of the modified stereo images may also be changed relative to the source image. Alternatively or in addition, at least one image of the modified stereo images may be transformed relative to the source image using a function which alters the position of elements of the image along the X or Y axis relative to the source image. In one variation, the function is a distorting algorithm, for example an elliptical or aspheric distorting algorithm. In another variation, the function distorts at least two regions of the source image such that at least one region appears to have a concave appearance while at least one region appears to have a convex appearance. In yet another variation, the function distorts at least two regions of the source image such that the at least two regions appear to
have a concave appearance, the at least two regions being separated by a region which appears to have an at least partially planar or convex appearance. Alternatively, the function may distort at least two regions of the source image such that the at least two regions appear to have a convex appearance, the at least two regions being separated by a region which appears to have an at least partially planar or concave appearance.
As used herein, the source image may be any two or three dimensional image, holographic image or set of two or three dimensional images. Examples of source images include, but are not limited to motion picture film, photographs, computer images, video and tomographic data sets, such as those derived from
MRI or CT data. The source images may be recorded, for example on VHS video tape, betacam SP or DI tape, nonlinear edit system, time base corrector, computer floppy disc, computer hard drive, RAM, CDROM, laserdisc and DVD. The source image may also be provided via a live broadcast, video signals, or generated by a software program. The source image can also be an analog or digital set of data corresponding to one or more two or three dimensional images.
In one embodiment of the invention, a method is provided for producing a synthesized stereoscopic image by displaying modified stereo images having a different spacial appearance, and viewing the modified stereo images through stereo viewing glasses which transmit the spatially different images to left and right eyes of the viewer to form a synthesized stereoscopic image of the source image, such as a VR headset. In one variation of this embodiment, an image display is used to transmit the modified stereo images to the stereo viewing glasses. In a further variation, image signals encoding the modified stereo images are transmitted to the image display by a device which converts recorded data encoding the modified stereo images into image signals which are received by the image display. In yet another variation, image signals encoding the modified stereo images are transmitted to the image display by an image
processor which converts a signal encoding the source image into signals encoding the modified stereo images. According to this variation, conversion of the signal encoding the source image into the signals encoding the modified stereo images is performed in real time. Conversion may alternatively be delayed. Conversion of the image signals may be an analog to analog, analog to digital, digital to digital, digital to analog, and/or in combination with optical to optical signal conversion.
Further, according to this variation, the signal encoding the source image may include a signal which provides instructions to the image processor regarding how to convert the signal encoding the source image into signals encoding the modified stereo images. These instructions may be simultaneously broadcast and may include, for example, image element by image element instructions or image frame by image frame instructions. In another embodiment of the invention, a method is provided for producing a synthesized stereoscopic image using stereo viewing glasses which form modified stereo images. In this embodiment, the method includes displaying on an image display a source image, and viewing the source image through stereo viewing glasses, the stereo viewing glasses having left and right lenses, at least one of the lenses modifying the source image to produce at least two images having a different spacial appearance (modified stereo images) when viewed through the lenses. In one variation of this embodiment, modifying the source image to produce modified stereo images having a different spacial appearance is performed in real time. Alternatively, the modified stereo images are created and stored prior to being displayed. Modifying the source image can involve an analog to analog, analog to digital, digital to digital, digital to analog, and/or in combination with optical to optical signal conversion.
In another embodiment of the invention, a method is provided for converting a signal encoding a source image into recorded signals encoding two or more spatially different images which form a synthesized stereoscopic image of the
source image when displayed in combination. In this embodiment, the method includes taking a signal encoding a source image, forming two or more image signals encoding the source image, modifying at least one of the two or more image signals such that at least two of the image signals encode images which have a different spacial appearance than each other, and recording signals encoding the modified stereo images. The signals encoding the modified stereo images may be recorded on a variety of different media including, for example magnetic media, VHS video tape, Betacam SP or DI tape, nonlinear edit system, frame store, computer floppy disc, computer hard drive, RAM, CDROM, laserdisc, DVD, MPEG and other optical or digital device.
Modifying at least one of the image signals can involve an analog to analog, analog to digital, digital to digital, digital to analog, and/or in combination with optical to optical signal conversion. In another embodiment of the invention, a method is provided for synthesizing a stereoscopic image from a source image. According to this embodiment, the method includes taking a signal encoding a source image, forming two or more image signals encoding the source image, modifying at least one of the two or more image signals such that the two or more image signals encode at least a pair of images having a different spacial appearance, displaying on an image display the spatially different images encoded by the modified image signals, and viewing the image display through stereo viewing glasses which transmit the spatially different images to a left and right eye of the viewer to form a synthesized stereoscopic image of the source image. In one variation of this embodiment, modification of the image signals is performed in real time. Modifying at least one of the image signals can involve analog to analog, analog to digital, digital to digital, digital to analog, and/or in combination with optical to optical signal conversion.
In a variation of this embodiment, the two or more image signals encoding the source image includes a signal which provides instructions to an image
processor regarding how to modify at least one of the two or more image signals. These instructions may include, for example, image element by image element instructions or image frame by image frame instructions. These instructions can be simultaneously broadcasted the image signals and can be combined with the image signals. For example, the instructions can be embedded into the signal, for example, in the vertical sync.
The present invention also relates to signals encoding two or more images which have a different spacial appearance which are derived from the same source image and which, when viewed in combination, form a synthesized stereoscopic image of the source image. These signals may be analog or digital signals In one variation, the two or more images include two images which have a different spacial appearance than each other. In another variation, the two or more images include three images which have a different spacial appearance than each other. These signals may be transmitted signals or signals which are recorded on a recording media. In one particular embodiment, these signals are derived from a computer generated image which has been modified according to the present invention.
It is envisioned that larger bandwidth systems, such as HDTV, will be able to transmit and receive separate signals for each modified image used to form the synthesized stereoscopic image. Accordingly, the present invention is also intended to be used in combination with such higher bandwidth systems where two or more signals each encoding a modified stereo image is transmitted and/or received. The present invention also relates to a recording of synthetic stereoscopic images according to the present invention. The recording includes a recording medium having recorded thereon two or more signals encoding at least two images having a different spacial appearance which, when viewed in combination, form a synthesized stereoscopic image. The recording may encode an analog or digital signal. Any recording media capable of storing dual
image signals may be used, including, but not limited to VHS video tape, betacam SP or DI tape, nonlinear edit system, computer floppy disc, computer hard drive, RAM, CDROM, laserdisc and DVD. The present invention also relates to an image processor for converting a signal encoding a source image into a signal encoding a synthesized stereoscopic image. In one embodiment, the image processor includes a signal entry port for receiving a signal encoding a source image, logic for converting the source image into two or more image signals encoding the source image where at least one of the two or more image signals is modified such that the image signals encode two or more images having a different spacial appearance, and a signal exit port for transmitting the modified image signals.
The logic may act to convert the source image to the synthesized stereoscopic image in real or non-real time. The logic may be programmable or controllable by the user to form different modified stereo images. For example, the user can program the logic to make certain elements of an image or certain images appear closer or farther away. Alternatively or in addition, the logic may be designed to dynamically control how the image processor transforms the source image based on the source image. This dynamic control may be on a program by program, image by image or element by element basis. For example, the logic may include certain image element recognition protocols which cause certain elements to be selectively brought into the foreground or background when detected. In one embodiment, these protocols can be used to sense whether an image corresponds to a close-up or wide-angle perspective so that the image maybe modified accordingly. The logic may also include certain movement recognition protocols which detects movement of certain elements and causes the image to be modified in a particular manner in response to the detected movement. This adaptation is particularly useful in pan and scan applications where it is desirable to keep certain image elements, such as an actor, centered in the image.
In a variation of this embodiment, the image processor also receives one or more signals which accompany the signal encoding the source image and provides instructions to the image processor regarding how to modify the source image. These instructions may include, for example, image element by image element instructions or image frame by image frame instructions. For example, the instructions may direct image 1 to be given a first type of modification, image 2 to be given a second type of modification, image 3 to be given a third type of modification, etc. Each type of modification may be a global modification for the image or provide for multiple different modifications for different elements in the image.
The present invention also relates to stereo viewing glasses for converting a source image into a synthesized stereoscopic image. In one embodiment, the stereo viewing glasses include a first and second lens for receiving the source image, and an optic associated with at least one of the first and second lenses which modifies the spacial appearance of the source image such that the pair of lenses provide a pair of images having a different spacial appearance which, when viewed in combination, form a synthesized stereoscopic image. The stereo viewing glasses can include only optics, the images being formed by a separate image display. Alternatively, the stereo viewing glasses can include one or more image displays. These stereo viewing glasses can also incorporate existing optical techniques which have been used to create stereo effects.
In conjunction with forming synthesized stereoscopic images according to any of the various embodiments of the present invention, it is preferred to alter the type of modification of the source image used to create the synthesized stereoscopic images over time. The present invention relies on the mind taking synthesizing pairs of modified stereo images to create a synthesized stereoscopic image. It is possible for the initial stereoscopic effect to diminish over time if the same modifications to the source image is employed. Accordingly, it is preferred to employ different modifications to the pairs of
modified stereo images over time. It is believed employing different modifications will make it more difficult for the viewer's mind to perceive the individual modified stereo images being presented as opposed to a synthesized stereoscopic image. For example, two or more different modification functions can be used sequentially to form synthesized stereoscopic images, each different modification function being employed for a period of time. Transitions between each modification function can occur within scenes but are preferably made at scene changes or other times where the transition between modification functions is least likely to be noticed by the viewer. Accordingly, the methods for forming stereoscopic images may also include the step of modifying the source image in two or more different ways over time as well as transitioning between the two or more different ways of modifying the source image. The devices and systems of the present invention may also include logic for modifying the source image in two or more different ways over time and optionally logic for sensing scene changes or other opportune sequences in the source image during which the way in which the source image is being modified may be altered. The image signals and recorded images of the present invention may also include modifications to the source image in two or more different ways over time in order to preserve and/or enhance the stereoscopic effect achieved as described above.
The various embodiments of the present invention can be employed in a variety of applications where it is desirable to create a three dimensional depiction. For example, the present invention can be used to transform images formed in existing computer programs into stereoscopic images. Accordingly, the present invention can be used to enhance the appearance of cascaded windows being stacked over each other. Different windows can also be made to appear closer or farther from the observer.
The various embodiments of the present invention can also be employed in combination with known pan and scan techniques to enhance the effectiveness
of three dimensional imaging. For example, motion and depth can be introduced into a still picture by panning across the still picture. Meanwhile, depth can be introduced by modifying the panned image according the present invention. In combination, the modified, panned stereoscopic image has a significantly greater three dimensional appearance.
Another application of the present invention is in the area of medical tomography such as MRI and CT scans where a series of images are taken at different known depths. By distorting each image according to its known depth, the series of two dimensional images can be converted into a series of three dimensional images which more accurately reflect the tomography of the imaged body location. In addition to creating a three dimensional image based on an overlay of a set of two dimensional images, a sense of depth can be exaggerated in each two dimensional image, thus making it easier to distinguish elements in each image. The present invention can also be used in medical endoscopy where two dimensional images taken from an endoscope are converted into three dimensional images by distorting the two dimensional image provided by the endoscope. By providing a stereoscopic endoscopic image as opposed to a two dimensional endoscopic image, the doctor is better able to determine the relative position of different objects in the field of view of the endoscope.
The present invention can also be used in combination with quicktime virtual reality applications where three dimension imaging is introduced into or enhanced in such applications.
BRIEF DESCRIPTION OF THE DRAWINGS Figures 1A-1I illustrate a series stereo pairs where at least the right image has been modified in a different manner than the left image. Figure 1 A illustrates a reduced right image. Figure IB illustrates a magnified right image.
-I I-
Figure IC illustrates a rotated right image.
Figure ID illustrates a right image which is skewed along the X axis as a function of the Y axis.
Figure IE illustrates a right image which is parabolically skewed along the X axis as a function of the Y axis.
Figure IF illustrates a right image which has a spherical distortion in causing the center of the image to appear at a different position in the Z axis relative to the X-Y plane.
Figure 1G illustrates a portion of the right image being magnified. Figure 1H illustrates a center portion of the right image being magnified.
Figure 1 1 illustrates a portion of the left image being magnified and a center portion of the right image being reduced.
Figure 2 illustrates a typical stereoscopic display system.
Figure 3 A is a timing diagram showing the transmission of left and right images to the eyes of a user of a time sequential stereoscopic system.
Figure 3B illustrates an over-under format for displaying modified stereo images.
Figure 3C illustrates a time sequence of a series of over-under image frames as in Figure 3B where either a first or second image is displayed in each image frame.
Figure 4 shows the effect of optical lenses used in one embodiment of the invention.
Figure 5 illustrates a generalization of a system for producing synthesized stereoscopic images. Figures 6A-6G represent particular embodiments of the system illustrated with regard to Figure 5.
Figure 7A illustrates a set of tomographic images.
Figure 7B illustrates a set of stereo pairs of the tomographic images illustrated in Figure 7A.
Figure 7C illustrates a set of stereo pairs of the tomographic images illustrated in Figure 7 A where particular images in the set of tomographic images are modified relative to each other to provide an improved stereoscopic depiction of those particular images in the set. Figure 8 A illustrates a set of bowling pins.
Figures 8B-8E illustrate a series of two-dimensional tomographic images of the set of bowling pins where each image shows a different row of pins.
Figure 8F illustrates a stereoscopic image of the bowling pins created from the tomographic images illustrated in Figures 8B-8E. Figure 8G illustrates a top down view of the image illustrated in Figure 8F.
Figure 8H illustrates the set of bowling pins illustrated in Figure 8A and a second viewing axis.
Figures 8I-8L illustrates a series of two-dimensional tomographic images of the set of bowling pins taken along the second viewing axis. Figure 8M illustrates a stereoscopic image of the bowling pins created from the tomographic images illustrated in Figures 8I-8L.
Figure 8N illustrates a top down view of the image illustrated in Figure 8M.
Figure 9 illustrates different images on a computer monitor where each image has a user selectable three dimensional appearance. Figure 10 illustrates examples of different pairs of wire meshes which can be generated by modifying the wire frame mesh illustrated in the figure as the initial wire mesh.
DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS The present invention relates to the discovery that the mind perceives two or more spatially different versions of the same image presented to the left and right eye as a synthesized stereoscopic image, i.e., as a three dimensional image. The present invention exploits this realization through methods, logic, data signals, recorded data, devices and stereoscopic imaging systems which convert
a source image into a synthesized stereoscopic image by forming two or more images from the source image and modifying at least one of those images relative to the source image such that at least two of the images do not have the same spacial appearance as each other with or without regard to their distance. In previous stereoscopic systems, synthesized stereoscopic images have been created using image shifting in combination with time delay, i.e., by controlling the time at which a same image is provided to the left and right eye of a viewer. An example of a time delay stereoscopic system is described in U.S. Patent No. 5,510,832 which is incorporated herein by reference. By contrast to time delay based stereoscopic systems, the present invention creates synthesized stereoscopic images by forming two or more images which have a different spacial appearance. The use of images with a different spacial appearance according to the present invention can be done in combination with time delay or independent of time delay or motion. Stereoscopic images have also been formed previously with the use of computers which calculate how the left and right eyes of a viewer would perceive each element of an image given the relative position of the left and right eyes to each element. The amount of computation required by this approach significantly limits its practicality for real time conversion of source images to stereoscopic images. In addition, it is necessary to know in advance the true depth of each element in the image at a given time in order to do the computation. By contrast, stereoscopic images are formed according to the present invention by modifying the source image without regard to simulating how each element of the source image would appear to the left and right eyes of the user if those elements were actually being seen by the user. Instead, elements of the source object or the entire source object are modified by an algorithm which causes the element or entire image to appear at a certain distance in the resultant stereoscopic image. By avoiding the need to precisely calculate how each element of the source object would appear to the left and
right eyes of the user, the computational demands involved in the present invention are significantly reduced as compared to computer generated stereoscopic images. The reduced computational demands involved in the present invention enable stereoscopic images to be generated according to the present invention in real time. In instances where the relative depth of elements in an image are known, the present invention can modify these images to simulate the known depth without having to precisely calculate how each pixel of the image should appear in order to simulate depth.
Two or more images with different spacial appearances can be formed by magnifying, reducing, stretching (X axis or Y axis) rotating (X-Y plane) and/or tilting (rotating in X-Z and/or Y-Z planes) one or more of the images. A different spacial appearance can also be formed by modifying the relative position of elements in all or a portion of one or more of the images.
In one embodiment, modified stereo images are formed by taking a source image, forming two or more images from the source image, and transforming at least one of the two or more images using a function (f(x,y)) which alters the position along the X and/or Y axis of all or some of the objects appearing in the source image. The function may also serve to alter the image on a line by line basis when different functions can be used in different lines. Examples of transforming functions include distorting algorithms such as elliptical and aspheric algorithms, enlargements or offsets. In a preferred embodiment, the function is a non-linear distortion along the X and/or Y axes.
In another preferred embodiment, the function distorts at least two regions of the source image such that at least one region appears to have a concave appearance while at least one region appears to have a convex appearance. In a further variation of this embodiment, the function distorts at least two regions of the source image such that the at least two regions appear to have a concave appearance, the at least two regions being separated by a region which appears to have an at least partially planar or convex appearance. Alternatively, the
function may distort at least two regions of the source image such that the at least two regions appear to have a convex appearance, the at least two regions being separated by a region which appears to have an at least partially planar or concave appearance. Any combination of the above modifications may be made on all or any portion of the right and/or left eye image in all or only some of the fields. More than one modification can also be made to the same image. In the case of stereoscopic imaging involving multiple frames (e.g., video), all or some of the multiple frames may have modified stereo pairs. In embodiments where the present invention is used to generate three dimensional stereo graphics of a computer image, the modified stereo images may be formed by taking a source image and mapping the source image onto a wire mesh and distorting the image according to a function, such as those described herein and illustrated with regard to Figures 1 A- II. Figures 1A-1I illustrate a series of modified stereo image pairs where at least the right eye image has been modified relative to the source image, illustrated as a uniform horizontally and vertically oriented grid. Although pairs of images are illustrated, it should be noted that sets of three or more images can also be formed. Figure 1 A illustrates a reduced right eye image. Figure IB illustrates a magnified right eye image. Figure IC illustrates a rotated right eye image. Figure ID illustrates a right eye image which is skewed along the X axis as a function of the Y axis. Figure IE illustrates a right eye image which is parabolically skewed along the X axis as a function of the Y axis. Figure IF illustrates a right eye image which has a spherical distortion in causing the center of the image to appear at a different position in the Z axis relative to the X-Y plane. Figure IG illustrates a right eye image where a portion of the image has been magnified. Figure 1H illustrates a right eye image where the center of the image has been magnified. Figure 1 Illustrates a left eye image with a
magnified center portion and a right eye image with a reduced center portion. Figures 1 A-I are intended to be illustrative of the various modifications which can be made to the source image in order to form modified stereo image pairs and are not intended to be exhaustive. These and other modifications to a source image to produce modified stereo image pairs can be created by a variety of digital or optical devices in real or non-real time without departing from the scope of the present invention.
In one particular embodiment, a pair of modified images are tilted or rotated in the X and/or Y and/or Z plane 10 percent or less relative to the other image. In another particular embodiment, a pair of modified images are magnified or reduced 10 percent or less relative to each other. In yet another particular embodiment, one of a pair of modified images is delayed one field or less relative to the other eye image. In all of the above embodiments, a convincing depth synthesis of these modified stereo image pairs occurs. One example of a preferred transformation is the simultaneous warps of
Figures IB, IF and 1H with 1 field or frame delay, horizontal offset of 3% and multiple target tracking with motion detection determining the direction of delay (i.e., on field 1 or 2), a 3% magnification of one image with the image mapped on a sphere twice the diameter of the screen with its axis displaced 2/3 up the height of the screen and with these effects oscillating from right to left eye image at 10Hz.
Careful adjustment of all parameters of the video signal(or photographic, printed or film or computer image) as well as minimizing of parallax is necessary for good depth, minimal ghosting and comfortable viewing. With ordinary stereo imaging, it is necessary to present the right eye image only to the right eye and the left eye image only to the left eye. If this is not done, confusing pseudoscopic images which cause eyestrain result. A unique feature of the present invention is that it is possible in some cases to present either image to either eye without pseudoscopy. This is made possible by the
fact that the differences in the two or more images are created by modifying one or more of the images relative to a source image as opposed to relying on the relative spatial positions of the images at the time of photography or videography in order to cause the sensation of depth. Consequently, though Figure 3 A for example shows all the transforms being performed on one side, channel or element, in fact both sides, channels or elements may perform the same or different transforms to different degrees and such transforms or degrees may change over time in order to obtain maximum effect with minimum discomfort for the viewer. A typical stereoscopic display system is illustrated in Figure 2. The system includes an image source 12, an image processor 14, an image display 16, and stereo viewing glasses 18. According to the present invention, the stereoscopic display system produces modified stereo images by taking a source image and forming a two or more images where at least one image has been modified relative to the source image and relative to the other images of the modified stereo images.
The formation of modified stereo images can be performed by several different components of a stereoscopic display system. For example, the image source 12 can provide image data corresponding to modified stereo images. Alternatively, the image processor 14 can receive image data from the image source 12 and convert the image data into modified stereo images according to the present invention. In this variation, the image data can also include instructions regarding how to modify the source image, for example, element by element or image by image. Alternatively, optics may be positioned between the image display and the user which convert images formed by the image display 16 into modified stereo images. For example, the lenses of stereo viewing glasses 18 may each receive the same image from the image display 16 and convert the image into modified stereo images according to the present
invention. The stereo viewing glasses may optionally include the image display 16.
It is intended for the present invention to encompass image sources which provide image data encoding for modified stereo images according to the present invention as well as instrumentation for producing image data encoding for modified stereo images. The present invention also encompasses image processors which convert data for source images into image signals for modified stereo images according to the present invention. The present invention also encompasses optics which may be used to optically convert images produced by an image display into modified stereo images according to the present invention.
The present invention is also intended to encompass stereoscopic imaging systems which include an image source, image processor and/or optics which produce modified stereo images according to the present invention. In this regard, components of the system which are not being used to modify a source image to produce modified stereo image pairs may be standard equipment which is currently used in existing stereoscopic imaging systems.
The present invention is also intended to encompass methods employed by the above components of a stereoscopic imaging system for producing modified stereo images according to the present invention.
A significant advantage of the methods, devices and stereoscopic imaging systems of present invention is their ability to convert existing two dimensional and three dimensional motion picture films, photos, computer images, videos and single cameras with ordinary lenses into a variety of stereo formats in real time or nonreal time. In the case of real time conversions, the methods and devices of the present invention may be used to convert unrecorded image signals, for example from a live broadcasts, video signals received via antenna or cable, or images from a computer program into a stereoscopic rendering of the program in full color and in real time.
A further advantage of the present invention is the ability to convert the source images into a series of different stereoscopic images based on the way in which the stereo images are modified relative to each other. Hence, the methods, devices and systems of the present invention are designed to be programmable to provide a series of different visual effects. In addition to the general solidizing or roundness of objects, one can make objects protrude from the screen and when combined with target acquisition and tracking performed on the same machine simultaneously with the other image manipulations described here, this stereosynthesis can be very sophisticated, real time and automatic. An object may be made to appear to recede into the background as it moves or becomes smaller. Further, artificial movement of an image, i.e. movement that occurs in the modified images but not in the source image, can be created by selecting a series of modification which introduce the sense of motion. Some elements in an image can be made to appear closer while other elements appear farther away. One can also detect foreground objects based on size or speed of motion and then magnify, delay, spherically warp or otherwise manipulate them selectively in real time automatically or for best effects, nonrealtime with more sophisticated algorithms or with human intervention using known means such as digitizing tablets and the like. It is also possible by these automatic means or with user intervention to select people or other objects by combining image acquisition with speech detection to create various types of games or contests from ordinary or specially encoded broadcast or prerecorded video.
A further advantage of the present invention is the limited computation that is required to convert the source images to stereoscopic images. Stereoscopic images have previously been formed by calculating how the left and right eyes of a viewer would perceive each element of an image given the relative position of the left and right eyes to each element. The amount of computation required by this approach significantly limits its practicality for real time conversion of the source images to stereoscopic images. By contrast, stereoscopic images are
formed according to the present invention without regard to simulating how each element of the source image would appear to the left and right eyes of the user if those elements were actually being seen by the user. As a result, the computational demands involved in the present invention are significantly reduced as compared to computer generated stereoscopic images and thus can be generated in real time.
A further advantage of the present invention is the ability of the methods, devices and systems to convert the source images into three dimensional images in a single step. In this regard, aside from the component or components which are being used to form the pair of modified stereo images, the methods and devices of the present invention may be used with standard equipment. For example, except where modified stereo images are converted by the image viewer, full color stereoscopic images are viewable with inexpensive dual filter glasses (amaglyph glasses), LCD shutter glasses or other devices known in the art for controlled image transmission to the left and right eyes of the observer.
Also, ordinary analog or digital video and/or computer hardware and/or software can be used.
A further advantage of the present invention is the color quality of the stereoscopic images produced. Because the conversion of source images to three dimensional images according to the present invention does not involve modifying the color of the source images, high quality full color stereoscopic images can be produced.
1. Image Source
In general, the image source 12 may be any recorded media, an image signal transmitter or data set which provides image data to the image processor 14.
Examples of sources of recorded source images which can be readily converted into three dimensional images by the present invention include two dimensional and three dimensional motion picture films, photographs, computer images, and
videos. In one particular example, the images are images generated by computer software. These image sources may be in a digital or analog format or in a real image format. Examples of recorded formats include, but are not limited to, videotape, magnetic media such as floppy discs, digital recorded media such as digital memory, digital tape, compact discs, DVD and the like.
Examples of image transmitters include broadcast antenna, cable, fiber and satellite.
According to the present invention, the image source may be a standard image source which provides image data encoding standard two dimensional source images which are later processed by a component of a system according to the present invention to form modified stereo image pairs.
When the image source is an unmodified image as opposed to a modified stereo image, the image source may include a signal which provides instructions to an image processor regarding how to convert the signal encoding the source image into signals encoding the modified stereo images. These instructions may include, for example, image element by image element instructions or image frame by image frame instructions.
Alternatively, the image source may include image data creating modified stereo images. In this embodiment, the image source provides image data to the image processor corresponding to modified stereo images. The image processor receives the image data for the modified stereo images. This embodiment is a non-real time embodiment of the invention in the sense that the conversion of source images to modified stereo images is not being performed in real time by the system. This embodiment of the invention enables standard video recording media
(video tape, laser disc, DVD) to be used to store data encoding the stereoscopic images used in the present invention. By storing the data as opposed to converting source images to modified stereo images in real time, the processing speed requirements of the stereoscopic imaging system is greatly reduced.
In embodiments where the image source provides image data encoding modified stereo images, the image data is preferably provided in an over/under or left/right format where two of the modified stereo images are placed on the same frame. In this format, a first image 21 from a set of modified stereo images is taken and condensed into an upper portion (or left portion) of an image frame. Meanwhile, a second image 23 from the set of modified stereo images is condensed into a lower portion (or right portion) of the same image frame, as illustrated in Figure 3B. The first and second images 21, 23, although each being placed in only a portion of the image frame, are preferably non- interlaced images containing all the image data for the image. By contrast, the use of interlaced field images causes half the amount of image information to be stored per image frame.
Figure 3C illustrates a time sequence of a series of over-under image frames as in Figure 3B where either a first or second image 21, 23 is displayed in each image frame. Either the first or second image or the over-under image can be displayed from each image frame at a given time. Accordingly, you can have any combination of first (1) and second (2) images displayed in time sequence, (e.g., 12121212 or 11221122 or 111222 or 112112 or 111112, etc.). This approach provides the system of the present invention with significantly greater flexibility for producing visual effects based on displaying different combinations of first and second images. For example, this approach can be used to create certain visual effects, such as the movement of an image element laterally across the screen. By controlling whether an image element is shown to the left or right eye, the appearance of movement by the image element laterally across the image can be created or enhanced.
The use of an over/under or left/right format has further advantage of enabling different visual effects to be performed. For example, line averaging between two left or two right eye frames can be performed.
In one embodiment, computer software is used as described in "UNIVERSAL STEREOSCOPIC INTERFACE," Provisional Application Serial No.: 60/013,738; filed: March 20, 1996; and"UNIVERSAL STEREOSCOPIC INTERFACE," Application Serial No.: 08/280,570; filed: March 19, 1997, each of which are incorporated herein by reference. This computer software enables one to distinguish between and selectively display different images of a pair of images presented in an over/under format. This software is useful for increasing the range of display formats which can be used with the modified stereo image pairs used in the present invention. In embodiments where the image source provides image data encoding modified stereo images, the image data preferably includes modified stereo images where the modifications relative to the source image are performed by two or more different transformations over time. The present invention relies on the mind taking synthesizing pairs of modified stereo images to create a synthesized stereoscopic image. It is possible for the initial stereoscopic effect to diminish over time if the same modifications to the source image is employed. Accordingly, it is preferred to employ display pairs of modified stereo images where the pairs are modified by different transformations over time.
2. Image Processor and Image Display
In general, the image processor 14 receives image data via a signal entry port 15. The image processor 14 includes logic for converting the image data into image signals encoding the source image and for modifying at least one of the image signals such that the image signals encode images having a different spacial appearance. The image processor also includes a signal exit port 17 for transmitting the modified first and second image signals to an image display 16 which displays real images corresponding to the modified image signals. Examples of image displays include, but are not limited to, screens
associated with projection systems, televisions, CRT screens, VR or holography.
Standard image displays such as television sets have a refresh rate of 15.25 KHz which is above the frequency at which the human eye can perceive individual images. As a result, the image displays appear to present a continuous image. Presenting alternating left and right eye images effectively causes an image display to have an effective refresh rate that is half the normal refresh rate of that display. If the refresh rate falls below the frequency at which the human eye can perceive images, the mind perceives the display as flickering which can cause eye strain and headaches. In order to eliminate this effect, it is preferred to increase the rate at which lines or entire images are presented. This can be done by line doubling, frequency doubling or frame multiplication techniques which can be performed before or after the source image is modified. In the embodiment described in Section 1, the image processor receives image data from an image source corresponding to modified stereo image.
Alternatively, the image processor can receive image data from an image source corresponding to a source image and produce an image signal for the source image. In a particular embodiment of the invention, the image processor converts image data from an image source encoding a standard source image into image signals encoding modified stereo images. In this embodiment, the image processor takes image data from the image source and forms signals for images where at least one of the images has been modified relative to the source image such that the modified images do not have the same spacial appearance. Processing of the image data from the image source into modified stereo images may be done in real time or not in real time by the image processor and recorded. The signals inputted into the image processor and outputted by the image processor may be analog or digital signals.
In a preferred embodiment, the image processor includes logic for performing multiple different transformations of a source image to produce different modified stereo images based on a source image. The different modified stereo images are produced by different transformations of the source image to create different visual effects. For example, certain transformations increase or decrease the depth of field or field of view provided.
In a preferred embodiment, the image processor includes logic for switching between different transformations at selected scene changes or other selected periods of time. The present invention relies on the mind taking synthesizing pairs of modified stereo images to create a synthesized stereoscopic image. It is possible for the initial stereoscopic effect to diminish over time if the same modifications to the source image is employed. Accordingly, it is preferred to employ different modifications to the pairs of modified stereo images over time. Two or more different transformations can be used sequentially to form synthesized stereoscopic images, each different transformation being employed for a period of time. The logic in the processor can be used to switch between the different transformations after selected time periods, at scene breaks or other selected times. The logic can also be used to sense when to switch between the different transformations. The different transformations performed by the image processor may be selectable by the observer, thereby providing the observer with control over the visual effects produced. For example, the user can select transformations which cause certain elements of an image to appear closer or farther away relative to their appearance in a source image. The user can also select among different transformations which introduce different degrees of spatially transformations to increase or decrease the disparity perceived between close and distant elements in the same image, ln this regard, the observer can design the appearance of the resulting stereoscopic images. The ability of an observer to customize the
stereoscopic imagery produced is an important feature of the present invention since how stereoscopic images are perceived varies from person to person.
The image processor can include logic which controls how the image processor transforms the source image based on the source image. This logic can correspond to software or a computer chip encoding the logic. The logic can be used to enable the image processor to dynamically adjust itself to optimize the stereoscopic appearance of different images or elements in images. For example, the logic can be used to detect motion in the source image and modify the source image to make the images appear closer or farther away based on the motion detected. The logic can also be used for special effects to create the appearance of motion of elements of the image which are stationary in the source image.
In one particular embodiment, the logic is used to identify a depth map or area, using grey scale values, shadows, motion detection (things moving in front of another), hue, size changes, edge detection or other detection techniques. These areas are then assigned modifications according to their desired depths. The source image is then used as a texture map that is mapped onto or modified according to the depth map to modify the source image. The depth maps may be further refined with left and right eye renderings using the texture maps rendered from "2" eye point of view. According to this embodimnet, a two dimensional source image may be converted to a three dimensional stereo pair by creating a depth map using a detected characteristic of the source image such as a grey scale value, hue, edge detection and or areas of motion. A wire frame mesh is then formed from the depth map. Relative depth is then assigned to the outline based on the stereoscopic effect to be achieved. A pair of wire meshes are then formed from the wire frame mesh where at least one mesh has been modified using the modification techniques of the present invention. Texture mapping of the original picture onto the wire meshes is then performed to produce the pair of modified images.
The logic of the image processor can also be used as an editing tool to make certain objects appear closer or farther away. The ability to magnify or reduce the appearance of elements within an image is an important application of the present invention in conjunction with film editing, such as pan and scan. This type of editing tool is also useful in conjunction allowing an observer to customize the appearance of a stereoscopic image, for example by allowing an observer to select which images or elements of an image to magnify, reduce or distort. Another application of this editing tool is in the computer software arena where an observer selects which elements of a stationary computer generated image the observer wishes to bring into the foreground or the background.
For optimal depth with minimal eyestrain, it is desirable to carefully adjust the effects on each shot of a program and to have them vary timewise. It is also desirable to design the image processor to permit the user to control which image transformation is performed. For instance, the entire right eye frame or parts of a frame may be delayed one field, magnified 5 percent and horizontally shifted 7 percent while the right half of the image is tilted back in the Z axis 6 percent, an object in the middle convexly projected on a spherical surface of diameter 3 times the screen height, while the left eye image may be untransformed or have an object in its center magnified 8 percent, the left half given vertical rugose transforms, the top half concavely projected on a dodecahedron with smallest diameter 4 times the screen width etc. These transforms could change in kind and vary in degree and from one eye's field to the other either slowly or as rapidly as every field with all these parameters preprogrammed or under user control.
Conversion of image data encoding a source image to an image signal encoding modified stereo images can be performed in real time by analog and/or digital devices or by computer software. Conversion could also be done with a single video stream from digital or analog source or from dual streams. In cases
where field delay or field advancement is desired, this could be introduced by running two copies of the same tape, disc or digital source one or more fields out of sync or by picking fields as desired from an analog or digital store in real or nonreal time. Common examples of such sources and stores are betacam SP or DI tapes, nonlinear edit systems, frame store, computer hard drives and
RAM, CDROM, laserdisc and DVD.
With advanced hardware and/or software and especially with non-real time systems, with human operator input or with automatic image segmentation and pattern recognition techniques known in the art, parts of the source image can be designated for particular transforms and followed from frame to frame by the program. For instance, a human figure can be identified, given a five percent convex transform and one field delay in the left eye until it disappeared from the scene or the scene changed. Parts of the image can be transformed in this manner and composited and recomposited into the image. Delays or advancement of one or more field or parts of fields can be part of the synthesized difference between the images presented to the two eyes. Ideally, the delay will be one field or less and will depend on the direction and speed of motion. With most sequences of moving images, such delay introduces spurious vertical parallax and pseudoscopic imagery which looks odd and creates eyestrain. Non-real time shot to shot editing is essential for use of delay and is desirable for all the other image transforms as well. For many sequences, no delay will give the best results. Delay can be applied to only part of the field instead of or in addition to the whole field.
3. Stereo Viewing Glasses The stereo viewing glasses 18 operate in combination with the image display
16 to coordinate what images are provided to the left and right eyes 22, 24 of the observer 20. By controlling which images are provided to the left and right eyes and/or the timing of when those images are provided, an impression of a
three dimensional image can be created from a pair of two dimensional images. For example, Figure 3 A illustrates an image display 16 for use with flicker glasses to control what images are observed by the left and right eyes of the observer. As illustrated, when a left image 28 is presented, the left lens 29 of the stereo viewing glasses 18 transmits the left image while the right lens 31 blocks or absorbs the left image 28. This allows the left eye of the observer to see the left image 28, while the right eye does not see the left image 28. When the right image 30 is presented, the lens transmissions are reversed so that the right eye sees the right image and the left eye does not see the left image. By alternating between the left and right image, the observer's brain receives both images and synthesizes the two images into a single image which the brain interprets as having depth.
The stereo viewing glasses 18 may have optical, electrooptic or mechanical lenses. In one preferred embodiment, each lens is activated electronically to block the light reaching the eyes alternately. The glasses may be gelatin, glass, dichroic polarized, anaglyphic, or holographic filters or optical or electrooptic and need only have appropriate optical characteristics.
Although the invention is described primarily with regard to filter or shutter glasses, any other display modality may be used such as displaying right and left stereo pairs in frame, field, line or pixel sequential form or as simultaneous pairs for stereoscopic or autostereoscopic viewing. In one embodiment of the present invention, the images are presented autostereoscopically by known means without the use of stereo viewing glasses. For example, the stereo viewing glasses can have polarized lens. Then, by controlling the polarization of the light, different images can be directed to the left and right eyes.
Alternatively, the stereo viewing glasses can have red/green and blue filters on the left and right eyes respectively. Then, by transmitting one image in red and green light and the other image in blue light, different images can be selectively transmitted to the left and right eyes of the observer. Other mechanisms for
presenting different images to the left and right eyes of the observer that are known in the art or later developed and are intended to be used in the present invention. In another embodiment of the invention, modified image pairs may be formed by optics positioned between the image display and the observer which selectively modify left or right eye images produced by the image display. For example, the image display or the stereo viewing glasses may include optics which modify a source object produced by the image display such that modified pairs of left and right eye images are produced. Figure 4 shows the effect of one type of lens used in accordance with the present invention. In this embodiment, the left eye lens 48 does not transform the image produced by the image display. Meanwhile, the right eye lens 50 modifies the right eye image according to the present invention. The brain combines the left and right images to produce a single image which is perceived as having depth. In this simplest case, there is no electronic processing of the two dimensional image and the glasses are composed of optical elements which distort or transform the image differently for each eye such that the viewer obtains a sensation of depth.
4. Systems For Forming Synthesized Stereoscopic Images
The following example describes different systems for forming synthesized stereoscopic images. Figure 5 illustrates a generalization of the overall system.
Figures 6A-6G represent particular embodiments of the system illustrated with regard to Figure 5.
As illustrated in Figure 5, A represents a two dimensional data set of a source image. The data set can be an analog or digital data set. The data set is copied by device B which can be an optical image duplicator or signal distribution amplifier. Signals C and C represent at least two copies of the data set encoding the source image which are conveyed to separate sets of image processors, represented as processors D, E and F and D', E' and FV A greater
or lesser number of image processors can be used than are illustrated. The data sets encoding the source image are spatially altered by the processors according to one or more functions for modifying the image, such as the functions that have been described herein. The signals from the processors can be resent through the different processors as many times as desired, as illustrated by long arrows 81.
Device K controls the qualities of the spatial distortions introduced into the image signals M, M' by the processors, further modifications of the image signals M, M' by devices G and G', and/or the signals N, N' that are sent to image display devices H, H\ Devices G and G' may perform a variety of functions including, for example, frequency doubling or other modifications which provide image enhancement. Device J represents the different techniques for combining two or more images to produce stereoscopic images including, for example, sequential and optical field multiplexing, such as shutter glasses or polarized lenses. Device J can also represent a device for recording the image signals as digital or analog data sets.
Figure 6A illustrates an embodiment of the system illustrated in Figure 5 where A represents any video signal source including for example, a camera, VCR, broadcast signal or DVD. Device B represents a signal distribution amplifier. Image processing in this system is performed by a dual channel video effects device (DVE) D, D' which is controlled by controller K. In this embodiment, image display devices H, H' are virtual reality (VR) glasses with dual video displays, one for each eye. Figure 6B illustrates an alternate embodiment of the system illustrated in Figure 5 where A represents any video signal source and device B represents a signal distribution amplifier. Image processing in this system is performed by a single DVE D which is controlled by controller K. An optional second DVE D' is also illustrated which can optionally alter image signal N' relative to the
source image. In this embodiment, image display devices H, H' are virtual reality (VR) glasses with dual video displays, one for each eye.
Figure 6C illustrates another alternate embodiment of the system illustrated in Figure 5 where A represents any video signal source and device B represents a signal distribution amplifier. Image processing in this system is performed by
DVE D, D' which is controlled by controller K. In this embodiment, a multiplexer G field sequentially multiplex signals M and M'. The multiplexed image signals M and M' are displayed on video display H (e.g., a TV or computer monitor). Shutter glasses J combine the multiplexed images displayed on video display H. J can alternatively be polarized lenses where the video display H displays multiplexed images formed of light with different polarizations.
Figure 6D illustrates another alternate embodiment of the system illustrated in Figure 5 where A represents any video signal source, device B represents a signal distribution amplifier, D, D' represent an DVE image processor and K represents a controller. In this embodiment, a multiplexer Gl field sequentially multiplexes signals M and M' while line doubler G2 increases the refresh rate at which images are displayed on video display H. Line doubler G2 can also include the functionality of a picture enhancer for providing edge improvement. Shutter glasses J combine the multiplexed images displayed on video display H.
Figure 6E illustrates a variation of Figure 6D where the system further includes a motion detector P which detects motion of elements of an image, e.g., whether a camera is panning from right to left, left to right, up, down, etc. Device P can alternately be used to detect whether a particular image element is close or distant in order to determine what transformaton to use. Based on the type of motion detected by motion detector P, the controller K determines which image (encoded by M and M') to display first. When the multiplexer Gl is used to combine the images encoded by M and M1 in an over-under format,
motion detector P can also be used to determine which image (encoded by M and M') the display H should display first.
The motion detector P can also be used to detect motion in an image, the detection of which is used by the controller K to direct the processor D, D' to modify the image signals to show that an element of an image is moving in a particular direction. For example, the detection of an element moving toward the screen can be used by the controller to direct the processor to modify the images to make the element appear closer in the stereoscopic image, thereby enhancing the stereoscopic effect. Figure 6F illustrates an alternate variation of Figure 6D where the system further includes a motion detector P which is only connected to the multiplexer. In this embodiment, the motion detector P is used to detect motion of elements in an image, e.g., whether a camera is panning from right to left, left to right, up, down, etc, and uses this information to determine which image (encoded by M and M') the image display H should display first. When the multiplexer Gl is used to combine the images encoded by M and M' in an over-under format, motion detector P can also be used to determine which image (encoded by M and M') the display H should display first.
6. Stereographic Imaging Of Sets Of Tomographic Images The following example illustrates the modification of a set of tomographic images to provide improved stereographic imaging of the set of tomographic images. In this example, a set of tomographic images, illustrated in Figure 7 A is taken and converted into stereo pairs of the tomographic images, illustrated in Figure 7B, where the pair of images formed of each slice has a different spacial appearance relative to each other. As illustrated in Figure 7C, only particular images of the set of tomographic images are modified relative to each other to provide an improved stereoscopic depiction of those particular images in the set.
7. Tomographic Imaging Application
The following example describes a tomography application for the devices and methods of the present invention for generating a synthesized stereoscopic image from a two dimensional source image. Figure 8A illustrates a three dimensional object, in this case a set of bowling pins viewed along axis 71.
Figures 8B-8E illustrate a series of two-dimensional tomographic images of the set of bowling pins taken along axis 71 where each image shows a different row of pins, intended to represent a different depth layer of the three dimensional image. By taking the four tomographic images illustrated in Figures 8B-8E, distorting each image in relation to its relative position within the three dimensional image, and overlaying the four images, a stereoscopic image of the bowling pins can be created, as illustrated in Figure 8F. A data matrix can then be formed of the different elements (bowling pins) as they appear in the overlaid stereoscopic image. Figure 8G illustrates a top down view of the data matrix illustrated in Figure 8F.
Using the data matrix illustrated in Figures 8F and 8G, it is possible to generate different sets of two dimensional images taken along different viewing axes. For example, Figure 8H illustrates the set of bowling pins illustrated in Figure 8 A and a viewing axis 73. Figures 8I-8L illustrates a series of two- dimensional tomographic images of the set of bowling pins taken along axis 73.
Using the set of two dimensional images, it is then possible to distort each image as a function of its position along axis 73 and overlay the images to form a new stereoscopic image illustrating the appearance of the object along the viewing axis 73, as illustrated in Figure 8M. Figure 8N illustrates a top down depiction of the stereoscopic image illustrated in Figure 8M.
As illustrated by Figures 8 A-8N, it is possible to convert a series of tomographic images of an object and use the methods and devices of the present invention to create different synthesized stereoscopic images of the object along different viewing axes. This application of the present invention is particularly
useful in medical applications like MRI and CT imaging where it is desirable to be able to create different three dimensional renderings of an object in order to identify the relative positions of different elements of the object. For example, by viewing a three dimensional rendering of an object at different angles, it is possible to more accurately determine the relative positions of different elements within an object.
In one variation of this example, an observer attaches a mechanism to his or her head which can detect head motion. The head movement mechanism is coupled to the system for generating the stereoscopic images and is used to direct different stereoscopic images to be generated based on the angle or position of the observer's head, as determined by the head movement mechanism. This enables an observer, for example, to turn his or her head sideways or up or down in order to view the stereoscopic image of the object at different angles. The system, in a preferred embodiment, is able to track the head movement and modify the stereoscopic image accordingly in real time.
8. Stenographic Display Of Computer Images
The following example describes a method for forming stereographic display of a computer image. Illustrated in Figure 9 are regions DI through D10 which represent images displayed on a computer monitor according to the present invention which appear to have separate three dimensional appearances when viewed using shutter glasses, polarized plates or anaglyph techniques or other three dimensional imaging techniques.
According to this example, the software program receive data encoding a source image as input, forms modified stereo images of the source image, and then texture maps the image by applying a function to the source image, and then texture maps the image to modify the appearance of the entire source image based on that function, as opposed to forming a stereographic image by
calculating how each pixel of each image would appear to the left and right eyes of the user.
The software program can be used to independently texture map the different images represented by DI through D10 in order to create independent stereographic appearances for each image based on the particular modified stereo images formed, i.e., different modifications cause the images to have a different stereographic appearance. The software program can generate the stereographic appearance of these images automatically. Alternatively, the user can direct the software program to perform different modifications such that each image has a user selected stereographic appearance. Thus the software enables the user to create his or her own stereographic display of a series of images which can be modified as the user desires.
9. Stereographic Display Of Computer Images
In embodiments where the present invention is used to generate three dimensional stereo graphics of a computer image, the modified stereo images may be formed by taking a source image and forming two more modified images according to the present invention. This may be done, for example, by mapping the source image onto two wire frame meshes and then distorting at least one of the images by the distorting one or both of the wire frame meshes onto which the source image is mapped. An equivalent effect to distorting the wire frame meshes can be done via mathematical manipulation of data encoding the source image.
Figure 10 illustrates examples of different pairs of wire meshes which can be generated by modifying the wire mesh illustrated in the figure as the initial wire mesh. By mapping a source image onto two initial wire meshes for the left and right eyes and then distorting one or both of the wire meshes as illustrated in the figure, a stereo pair of images can be created.
10. Image Data Management Protocol For Image Displays
In one embodiment, image data for a given line of an image is received and modified according to the present invention into data for a stereo pair of the given line. According to this protocol, each line of the stereo pair of lines is then displayed to the left and right eye respectively before new image data for same given line is received and displayed. This is accomplished by increasing the frequency at which a given line is presented to the left or right eye. As a result, the same number of lines are presented over time to each of the left and right eyes as would be presented to the left and right eyes if the line data were not modified according to the present invention. The above protocol may be used with interlaced or noninterlaced displays known as multiple frequency monitors.
While the present invention is disclosed by reference to the preferred embodiments and examples detailed above, it is to be understood that these examples are intended in an illustrative rather than limiting sense, as it is contemplated that modifications will readily occur to those skilled in the art, which modifications will be within the spirit of the invention and the scope of the appended claims.
Claims
1. A method for producing a synthesized stereoscopic image comprising: displaying at least two images derived from a source image which have a different spacial appearance relative to each other which when viewed in combination form a synthesized stereoscopic image of the source image, the at least two spatially different images forming the synthesized stereoscopic image through a modification of at least one of the two spatially different images relative to the source image that does not simulate how a physical three dimensional model of the source image would independently appear to a left and right eye of an observer.
2. The method according to claim 1, wherein the step of displaying the spatially different images includes transmitting the image signals encoding the spatially different images to an image display.
3. The method according to claim 2 wherein at least one of the image signals is an analog signal.
4. The method according to claim 2 wherein at least one of the image signals is a digital signal.
5. The method according to claim 2, wherein the image signals are transmitted to the image display by a device which converts recorded data encoding the spatially different images into the image signals.
6. The method according to claim 2, wherein the image signals are transmitted to the image display by a device which converts a signal encoding the source image into the image signals.
7. The method according to claim 6, wherein converting the signal encoding the source image into the image signals is performed by the device in real time.
8. The method according to claim 1, wherein only one image of the spatially different images is modified relative to the source image.
9. The method according to claim 1, wherein at least two of the spatially different images are modified relative to the source image.
10. The method according to claim 1 , wherein one of the spatially different images is magnified, reduced, or rotated in the X, Y and/or Z plane relative to another of the spatially different images.
11. The method according to claim 10, wherein one of the spatially different images is tilted or rotated in the X and/or Y and/or Z plane 10 percent or less relative to another of the spatially different images.
12. The method according to claim 1, wherein images for a given eye are displayed at a frequency that is greater than the frequency at which that eye can perceive individual images.
13. The method according to claim 1, wherein a position of one or more elements in one of the spatially different images is different than another of the spatially different images.
14. The method according to claim 1, wherein at least one of the spatially different images is transformed relative to the source image using a function which alters the position of elements of the image along the at least one of the X and X axes.
15. The method according to claim 14, wherein the function is a distorting algorithm.
16. The method according to claim 14, wherein the function is an elliptical or aspheric algorithm relative to the source image.
17. The method according to claim 14, wherein the function is nonlinear along at least one of the X and Y axes.
18. A method for viewing a synthesized stereoscopic image comprising: displaying a source image on an image display; and viewing the source image through stereo viewing glasses, the stereo viewing glasses having left and right lenses, at least one of the lenses modifying the source image to produce at least two images derived from the source image which have a different spacial appearance relative to each other which when viewed in combination form a synthesized stereoscopic image of the source image, the at least two spatially different images forming the synthesized stereoscopic image through a modification of at least one of the two spatially different images relative to the source image that does not simulate how a physical three dimensional model of the source image would independently appear to a left and right eye of an observer.
19. The method according to claim 18, wherein modifying the source image to produce images having a different spacial appearance is performed in real time.
20. The method according to claim 18, wherein only one of the spatially different images is modified relative to the source image.
21. The method according to claim 18, wherein at least two of the spatially different images are modified relative to the source image.
22. The method according to claim 18, wherein one of the spatially different images is magnified, reduced, or rotated in the X, Y and/or Z plane relative to the another of the spatially different images.
23. The method according to claim 22, wherein one of the spatially different images is tilted or rotated in the X and/or Y and or Z plane 10 percent or less relative to another of the spatially different images.
24. The method according to claim 18, wherein images for a given eye are displayed at a frequency that is greater than the frequency at which that eye can perceive individual images.
25. The method according to claim 18, wherein a position of one or more elements in one of the spatially different images is different than another of the spatially different images.
26. The method according to claim 18, wherein at least one of the spatially different images is transformed relative to the source image using a function which alters the position of elements of the image along at least one of the X or Y axes.
27. The method according to claim 26, wherein the function is nonlinear along at least one of the X and Y axes.
28. A method for synthesizing a stereoscopic image from a source image comprising: taking a signal encoding a source image and forming at least two image signals encoding the source image; modifying at least one of the image signals such that the image signals encode at least two images derived from a source image which have a different spacial appearance relative to each other which when viewed in combination form a synthesized stereoscopic image of the source image, the at least two spatially different images forming the synthesized stereoscopic image through a modification of at least one of the two spatially different images relative to the source image that does not simulate how a physical three dimensional model of the source image would independently appear to a left and right eye of an observer; and displaying on an image display the spatially different images encoded by the modified image signals.
29. The method according to claim 28, wherein modifying at least one of the image signals is performed in real time.
30. The method according to claim 28 wherein the modification of the source image involves a signal conversion selected from the group consisting of digital to digital, digital to analog, analog to digital and analog to analog.
31. The method according to claim 28, wherein only one of the spatially different images is modified relative to the source image.
32. The method according to claim 28, wherein at least two of the spatially different images are modified relative to the source image.
33. The method according to claim 28, wherein one of the spatially different images is magnified, reduced, or rotated in the X, Y and/or Z plane relative to another of the spatially different images.
34. The method according to claim 33, wherein one of the spatially different images is tilted or rotated in the X and/or Y and/or Z plane 10 percent or less relative to another of the spatially different images.
35. The method according to claim 28, wherein a position of one or more elements in one of the spatially different images is different than in another of the spatially different images.
36. The method according to claim 28, wherein at least one of the spatially different images is transformed relative to the source image using a function which alters the position of elements of the image along at least one of the X and Y axes.
37. The method according to claim 36, wherein the function is nonlinear along at least one of the X and Y axes.
38. The method according to claim 36, wherein the function is a distorting algorithm.
39. The method according to claim 36 wherein the function is an elliptical or aspheric algorithm relative to the source image.
40. The method according to claim 28 wherein at least one of the spatially different images includes a horizontal or vertical shift relative to the source image.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AU41687/97A AU4168797A (en) | 1996-08-30 | 1997-08-28 | Method for producing a synthesized stereoscopic image |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US2490096P | 1996-08-30 | 1996-08-30 | |
US60/024,900 | 1996-08-30 | ||
US08/905,574 US6108005A (en) | 1996-08-30 | 1997-08-04 | Method for producing a synthesized stereoscopic image |
US08/905,574 | 1997-08-04 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO1998009445A1 true WO1998009445A1 (en) | 1998-03-05 |
Family
ID=26699006
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US1997/015216 WO1998009445A1 (en) | 1996-08-30 | 1997-08-28 | Method for producing a synthesized stereoscopic image |
Country Status (3)
Country | Link |
---|---|
US (2) | US6108005A (en) |
AU (1) | AU4168797A (en) |
WO (1) | WO1998009445A1 (en) |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
FR2901086A1 (en) * | 2006-05-11 | 2007-11-16 | Sagem Defense Securite | Vision restitution method, involves electronically moving one of images to adjust its position such that images are arranged identical to relative position to be occupied by images if axes of displays are in relative reference position |
WO2008004005A2 (en) * | 2006-07-05 | 2008-01-10 | James Amachi Ashbey | Improvements in stereoscopic motion pictures |
EP2102822A4 (en) * | 2007-01-10 | 2010-06-02 | Real Image Corp | Method and apparatus for generating stereoscopic image from two-dimensional image by using mesh map |
WO2011041219A1 (en) * | 2009-09-30 | 2011-04-07 | Rovi Technologies Corporation | Systems and methods for navigating a three-dimensional media guidance application |
CN103004217A (en) * | 2011-06-08 | 2013-03-27 | 松下电器产业株式会社 | Parallax image generation device, parallax image generation method, program and integrated circuit |
EP2582144A4 (en) * | 2010-06-08 | 2015-06-24 | Lg Electronics Inc | Image processing method and image display device according to the method |
EP3057066A1 (en) * | 2015-02-10 | 2016-08-17 | DreamWorks Animation LLC | Generation of three-dimensional imagery from a two-dimensional image using a depth map |
US9897806B2 (en) | 2015-02-10 | 2018-02-20 | Dreamworks Animation L.L.C. | Generation of three-dimensional imagery to supplement existing content |
Families Citing this family (170)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7907793B1 (en) | 2001-05-04 | 2011-03-15 | Legend Films Inc. | Image sequence depth enhancement system and method |
US8253729B1 (en) * | 1983-05-09 | 2012-08-28 | Geshwind David M | Trimming depth buffer during 2D to 3D conversion |
AUPN732395A0 (en) * | 1995-12-22 | 1996-01-25 | Xenotech Research Pty Ltd | Image conversion and encoding techniques |
US7190371B2 (en) | 1996-02-08 | 2007-03-13 | Palm Charles S | 3D stereo browser for the internet |
WO1997031305A2 (en) * | 1996-02-08 | 1997-08-28 | Synthonics Incorporated | 3d stereo browser for the internet |
US7091991B2 (en) * | 1996-02-08 | 2006-08-15 | Palm Charles S | 3D stereo browser for the internet |
US6346940B1 (en) * | 1997-02-27 | 2002-02-12 | Kabushiki Kaisha Toshiba | Virtualized endoscope system |
US6816158B1 (en) * | 1998-10-30 | 2004-11-09 | Lemelson Jerome H | Three-dimensional display system |
US7271803B2 (en) * | 1999-01-08 | 2007-09-18 | Ricoh Company, Ltd. | Method and system for simulating stereographic vision |
US6438254B1 (en) * | 1999-03-17 | 2002-08-20 | Matsushita Electric Industrial Co., Ltd. | Motion vector detection method, motion vector detection apparatus, and data storage media |
US6674440B1 (en) * | 1999-04-05 | 2004-01-06 | 3Dlabs, Inc., Inc. Ltd. | Graphics processor for stereoscopically displaying a graphical image |
KR100334722B1 (en) * | 1999-06-05 | 2002-05-04 | 강호석 | Method and the apparatus for generating stereoscopic image using MPEG data |
AU2447101A (en) * | 1999-12-22 | 2001-07-03 | Trustees Of The University Of Pennsylvania, The | Judging changes in images of the eye |
US7254265B2 (en) * | 2000-04-01 | 2007-08-07 | Newsight Corporation | Methods and systems for 2D/3D image conversion and optimization |
DE10016074B4 (en) * | 2000-04-01 | 2004-09-30 | Tdv Technologies Corp. | Method and device for generating 3D images |
CA2418089A1 (en) * | 2000-08-04 | 2002-02-14 | Dynamic Digital Depth Research Pty Ltd. | Image conversion and encoding technique |
US6760469B1 (en) * | 2000-10-05 | 2004-07-06 | Canon Kabushiki Kaisha | Flipping stereoscopic radiographs |
IL139995A (en) * | 2000-11-29 | 2007-07-24 | Rvc Llc | System and method for spherical stereoscopic photographing |
CA2328795A1 (en) | 2000-12-19 | 2002-06-19 | Advanced Numerical Methods Ltd. | Applications and performance enhancements for detail-in-context viewing technology |
DE10102477A1 (en) | 2001-01-19 | 2002-07-25 | Storz Endoskop Gmbh Schaffhaus | Anastomosis of two ends of a blood vessel using laser surgery, has light guiding instrument that directs light from a source along an optical fiber to the joint where the light is deflected outwards to weld the tissue together |
US10742965B2 (en) | 2001-01-23 | 2020-08-11 | Visual Effect Innovations, Llc | Faster state transitioning for continuous adjustable 3Deeps filter spectacles using multi-layered variable tint materials |
US9781408B1 (en) | 2001-01-23 | 2017-10-03 | Visual Effect Innovations, Llc | Faster state transitioning for continuous adjustable 3Deeps filter spectacles using multi-layered variable tint materials |
CA2345803A1 (en) | 2001-05-03 | 2002-11-03 | Idelix Software Inc. | User interface elements for pliable display technology implementations |
US8416266B2 (en) | 2001-05-03 | 2013-04-09 | Noregin Assetts N.V., L.L.C. | Interacting with detail-in-context presentations |
US9286941B2 (en) | 2001-05-04 | 2016-03-15 | Legend3D, Inc. | Image sequence enhancement and motion picture project management system |
US6965379B2 (en) | 2001-05-08 | 2005-11-15 | Koninklijke Philips Electronics N.V. | N-view synthesis from monocular video of certain broadcast and stored mass media content |
US7084886B2 (en) | 2002-07-16 | 2006-08-01 | Idelix Software Inc. | Using detail-in-context lenses for accurate digital image cropping and measurement |
US7213214B2 (en) | 2001-06-12 | 2007-05-01 | Idelix Software Inc. | Graphical user interface with zoom for detail-in-context presentations |
US9760235B2 (en) | 2001-06-12 | 2017-09-12 | Callahan Cellular L.L.C. | Lens-defined adjustment of displays |
DE10128530A1 (en) * | 2001-06-13 | 2002-12-19 | Basf Ag | Water-dilutable concentrate giving long-life low electrical conductivity cooling systems for fuel cell systems in e.g. vehicles is based on alkylene glycols and also contains azole derivatives |
JP4610799B2 (en) * | 2001-06-25 | 2011-01-12 | オリンパス株式会社 | Stereoscopic observation system and endoscope apparatus |
WO2003009218A1 (en) * | 2001-07-18 | 2003-01-30 | Intel Zao | Dynamic gesture recognition from stereo sequences |
US7274800B2 (en) * | 2001-07-18 | 2007-09-25 | Intel Corporation | Dynamic gesture recognition from stereo sequences |
CA2361341A1 (en) * | 2001-11-07 | 2003-05-07 | Idelix Software Inc. | Use of detail-in-context presentation on stereoscopically paired images |
JP5020458B2 (en) * | 2001-11-24 | 2012-09-05 | フェニックス スリーディー インコーポレイテッド | Generation of stereoscopic image sequence from 2D image sequence |
US20030128175A1 (en) * | 2002-01-09 | 2003-07-10 | International Business Machines Corporation | Stereoscopic display system and method |
US7319720B2 (en) * | 2002-01-28 | 2008-01-15 | Microsoft Corporation | Stereoscopic video |
CA2370752A1 (en) | 2002-02-05 | 2003-08-05 | Idelix Software Inc. | Fast rendering of pyramid lens distorted raster images |
CA2478671C (en) * | 2002-03-13 | 2011-09-13 | Imax Corporation | Systems and methods for digitally re-mastering or otherwise modifying motion pictures or other image sequences data |
US20030212552A1 (en) * | 2002-05-09 | 2003-11-13 | Liang Lu Hong | Face recognition procedure useful for audiovisual speech recognition |
US7209883B2 (en) * | 2002-05-09 | 2007-04-24 | Intel Corporation | Factorial hidden markov model for audiovisual speech recognition |
US7165029B2 (en) | 2002-05-09 | 2007-01-16 | Intel Corporation | Coupled hidden Markov model for audiovisual speech recognition |
US7804995B2 (en) * | 2002-07-02 | 2010-09-28 | Reald Inc. | Stereoscopic format converter |
US8120624B2 (en) | 2002-07-16 | 2012-02-21 | Noregin Assets N.V. L.L.C. | Detail-in-context lenses for digital image cropping, measurement and online maps |
CA2393887A1 (en) | 2002-07-17 | 2004-01-17 | Idelix Software Inc. | Enhancements to user interface for detail-in-context data presentation |
CA2406047A1 (en) | 2002-09-30 | 2004-03-30 | Ali Solehdin | A graphical user interface for digital media and network portals using detail-in-context lenses |
CA2449888A1 (en) | 2003-11-17 | 2005-05-17 | Idelix Software Inc. | Navigating large images using detail-in-context fisheye rendering techniques |
US7171043B2 (en) | 2002-10-11 | 2007-01-30 | Intel Corporation | Image recognition using hidden markov models and coupled hidden markov models |
US7321682B2 (en) * | 2002-11-12 | 2008-01-22 | Namco Bandai Games, Inc. | Image generation system, image generation method, program, and information storage medium |
US7680322B2 (en) * | 2002-11-12 | 2010-03-16 | Namco Bandai Games Inc. | Method of fabricating printed material for stereoscopic viewing, and printed material for stereoscopic viewing |
CA2411898A1 (en) | 2002-11-15 | 2004-05-15 | Idelix Software Inc. | A method and system for controlling access to detail-in-context presentations |
US7472063B2 (en) * | 2002-12-19 | 2008-12-30 | Intel Corporation | Audio-visual feature fusion and support vector machine useful for continuous speech recognition |
US7203368B2 (en) * | 2003-01-06 | 2007-04-10 | Intel Corporation | Embedded bayesian network for pattern recognition |
JP4128505B2 (en) * | 2003-09-05 | 2008-07-30 | オリンパス株式会社 | Capsule endoscope |
US7643025B2 (en) * | 2003-09-30 | 2010-01-05 | Eric Belk Lange | Method and apparatus for applying stereoscopic imagery to three-dimensionally defined substrates |
JP3746506B2 (en) * | 2004-03-08 | 2006-02-15 | 一成 江良 | Stereoscopic parameter embedding device and stereoscopic image reproducing device |
US7486302B2 (en) | 2004-04-14 | 2009-02-03 | Noregin Assets N.V., L.L.C. | Fisheye lens graphical user interfaces |
US8106927B2 (en) | 2004-05-28 | 2012-01-31 | Noregin Assets N.V., L.L.C. | Graphical user interfaces and occlusion prevention for fisheye lenses with line segment foci |
WO2005119376A2 (en) | 2004-06-01 | 2005-12-15 | Vesely Michael A | Horizontal perspective display |
US9317945B2 (en) | 2004-06-23 | 2016-04-19 | Callahan Cellular L.L.C. | Detail-in-context lenses for navigation |
US7714859B2 (en) | 2004-09-03 | 2010-05-11 | Shoemaker Garth B D | Occlusion reduction and magnification for multidimensional data presentations |
US7995078B2 (en) | 2004-09-29 | 2011-08-09 | Noregin Assets, N.V., L.L.C. | Compound lenses for multi-source data presentation |
US7643672B2 (en) * | 2004-10-21 | 2010-01-05 | Kazunari Era | Image processing apparatus, image pickup device and program therefor |
US20060088225A1 (en) | 2004-10-26 | 2006-04-27 | The Secretary Of State For The Home Department | Comparison |
US20060126927A1 (en) * | 2004-11-30 | 2006-06-15 | Vesely Michael A | Horizontal perspective representation |
JP4449723B2 (en) * | 2004-12-08 | 2010-04-14 | ソニー株式会社 | Image processing apparatus, image processing method, and program |
KR100782811B1 (en) * | 2005-02-04 | 2007-12-06 | 삼성전자주식회사 | Method and apparatus for making stereo video according to the frequency characteristic of input video, method for transmitting and receiving stereo video, reproducing method and apparatus thereof |
KR100716992B1 (en) * | 2005-02-04 | 2007-05-10 | 삼성전자주식회사 | Method for encoding and decoding of stereo video, and apparatus thereof |
US8004515B1 (en) * | 2005-03-15 | 2011-08-23 | Nvidia Corporation | Stereoscopic vertex shader override |
US7580036B2 (en) | 2005-04-13 | 2009-08-25 | Catherine Montagnese | Detail-in-context terrain displacement algorithm with optimizations |
US8717423B2 (en) * | 2005-05-09 | 2014-05-06 | Zspace, Inc. | Modifying perspective of stereoscopic images based on changes in user viewpoint |
US7907167B2 (en) | 2005-05-09 | 2011-03-15 | Infinite Z, Inc. | Three dimensional horizontal perspective workstation |
WO2006121956A1 (en) * | 2005-05-09 | 2006-11-16 | Infinite Z, Inc. | Biofeedback eyewear system |
US7872665B2 (en) * | 2005-05-13 | 2011-01-18 | Micoy Corporation | Image capture and processing |
US7656403B2 (en) * | 2005-05-13 | 2010-02-02 | Micoy Corporation | Image processing and display |
KR101423592B1 (en) * | 2005-05-26 | 2014-07-30 | 리얼디 인크. | Ghost compensation for improved stereoscopic projection |
US7875132B2 (en) * | 2005-05-31 | 2011-01-25 | United Technologies Corporation | High temperature aluminum alloys |
US8253062B2 (en) * | 2005-06-10 | 2012-08-28 | Chrysler Group Llc | System and methodology for zero-gap welding |
JP4665166B2 (en) * | 2005-06-29 | 2011-04-06 | ソニー株式会社 | Stereo image processing apparatus, stereo image processing method, and stereo image processing program |
JP4687279B2 (en) * | 2005-06-29 | 2011-05-25 | ソニー株式会社 | Image reproduction apparatus, image reproduction method, and image reproduction program |
KR100932977B1 (en) * | 2005-07-05 | 2009-12-21 | 삼성모바일디스플레이주식회사 | Stereoscopic video display |
KR100913173B1 (en) * | 2005-07-05 | 2009-08-19 | 삼성모바일디스플레이주식회사 | 3 dimension graphic processor and autostereoscopic display device using the same |
WO2007011748A2 (en) | 2005-07-14 | 2007-01-25 | Molsoft, Llc | Structured documents for displaying and interaction with three dimensional objects |
EP1750460A1 (en) * | 2005-08-05 | 2007-02-07 | Samsung SDI Co., Ltd. | 3D graphics processor and autostereoscopic display device using the same |
US20070040905A1 (en) * | 2005-08-18 | 2007-02-22 | Vesely Michael A | Stereoscopic display using polarized eyewear |
US20070043466A1 (en) * | 2005-08-18 | 2007-02-22 | Vesely Michael A | Stereoscopic display using polarized eyewear |
JP4761528B2 (en) * | 2005-10-11 | 2011-08-31 | 任天堂株式会社 | GAME PROGRAM AND GAME DEVICE |
US8031206B2 (en) | 2005-10-12 | 2011-10-04 | Noregin Assets N.V., L.L.C. | Method and system for generating pyramid fisheye lens detail-in-context presentations |
US8157381B2 (en) * | 2005-12-21 | 2012-04-17 | International Business Machines Corporation | Method to synchronize stereographic hardware to sequential color rendering apparatus |
US8182099B2 (en) | 2005-12-21 | 2012-05-22 | International Business Machines Corporation | Noise immune optical encoder for high ambient light projection imaging systems |
US8167431B2 (en) * | 2005-12-21 | 2012-05-01 | International Business Machines Corporation | Universal stereographic trigger peripheral for electronic equipment |
US8189038B2 (en) | 2005-12-21 | 2012-05-29 | International Business Machines Corporation | Stereographic projection apparatus with passive eyewear utilizing a continuously variable polarizing element |
US8152303B2 (en) | 2005-12-21 | 2012-04-10 | International Business Machines Corporation | Signal synthesizer for periodic acceleration and deceleration of rotating optical devices |
US8172399B2 (en) | 2005-12-21 | 2012-05-08 | International Business Machines Corporation | Lumen optimized stereo projector using a plurality of polarizing filters |
US20070147671A1 (en) * | 2005-12-22 | 2007-06-28 | Eastman Kodak Company | Analyzing radiological image using 3D stereo pairs |
CA2636858C (en) * | 2006-01-27 | 2015-11-24 | Imax Corporation | Methods and systems for digitally re-mastering of 2d and 3d motion pictures for exhibition with enhanced visual quality |
US8274448B1 (en) | 2006-03-29 | 2012-09-25 | Nvidia Corporation | Stereoscopic display system, method and computer program product |
US7983473B2 (en) | 2006-04-11 | 2011-07-19 | Noregin Assets, N.V., L.L.C. | Transparency adjustment of a presentation |
CA2653815C (en) | 2006-06-23 | 2016-10-04 | Imax Corporation | Methods and systems for converting 2d motion pictures for stereoscopic 3d exhibition |
KR100850931B1 (en) * | 2006-06-29 | 2008-08-07 | 성균관대학교산학협력단 | Rectification System ? Method of Stereo Image in Real Time |
US8152304B2 (en) | 2006-08-30 | 2012-04-10 | International Business Machines Corporation | Stereographic imaging system using open loop magnetomechanically resonant polarizing filter actuator |
US8264525B2 (en) * | 2006-08-30 | 2012-09-11 | International Business Machines Corporation | Closed loop feedback control to maximize stereo separation in 3D imaging systems |
US8162482B2 (en) * | 2006-08-30 | 2012-04-24 | International Business Machines Corporation | Dynamic projector refresh rate adjustment via PWM control |
US8810636B2 (en) | 2006-12-20 | 2014-08-19 | Ge Inspection Technologies, Lp | Inspection apparatus method and apparatus comprising selective frame output |
US8213676B2 (en) | 2006-12-20 | 2012-07-03 | Ge Inspection Technologies Lp | Inspection apparatus method and apparatus comprising motion responsive control |
US9633426B2 (en) | 2014-05-30 | 2017-04-25 | General Electric Company | Remote visual inspection image capture system and method |
US11275242B1 (en) * | 2006-12-28 | 2022-03-15 | Tipping Point Medical Images, Llc | Method and apparatus for performing stereoscopic rotation of a volume on a head display unit |
US8384771B1 (en) * | 2006-12-28 | 2013-02-26 | David Byron Douglas | Method and apparatus for three dimensional viewing of images |
US11315307B1 (en) | 2006-12-28 | 2022-04-26 | Tipping Point Medical Images, Llc | Method and apparatus for performing rotating viewpoints using a head display unit |
US10795457B2 (en) | 2006-12-28 | 2020-10-06 | D3D Technologies, Inc. | Interactive 3D cursor |
US11228753B1 (en) * | 2006-12-28 | 2022-01-18 | Robert Edwin Douglas | Method and apparatus for performing stereoscopic zooming on a head display unit |
US9349183B1 (en) * | 2006-12-28 | 2016-05-24 | David Byron Douglas | Method and apparatus for three dimensional viewing of images |
US9473766B2 (en) * | 2006-12-28 | 2016-10-18 | David Byron Douglas | Method and apparatus for three dimensional viewing of images |
US9026938B2 (en) | 2007-07-26 | 2015-05-05 | Noregin Assets N.V., L.L.C. | Dynamic detail-in-context user interface for application access and content access on electronic displays |
US8323182B2 (en) | 2007-12-18 | 2012-12-04 | Manohara Harish M | Endoscope and system and method of operation thereof |
US20130044104A1 (en) * | 2008-01-22 | 2013-02-21 | Jaison Bouie | Methods and Apparatus for Displaying an Image with Enhanced Depth Effect |
KR20110015452A (en) * | 2008-06-06 | 2011-02-15 | 리얼디 인크. | Blur enhancement of stereoscopic images |
KR101546828B1 (en) * | 2008-06-10 | 2015-08-24 | 엘지전자 주식회사 | Display Apparatus |
GB0819069D0 (en) | 2008-10-17 | 2008-11-26 | Forensic Science Service Ltd | Improvements in and relating to methods and apparatus for comparison |
CN102144396A (en) * | 2009-08-25 | 2011-08-03 | 松下电器产业株式会社 | Stereovision-image editing device and stereovision-image editing method |
US9294751B2 (en) | 2009-09-09 | 2016-03-22 | Mattel, Inc. | Method and system for disparity adjustment during stereoscopic zoom |
JP4763822B2 (en) * | 2009-09-30 | 2011-08-31 | パナソニック株式会社 | Video signal processing apparatus and video signal processing method |
US8711204B2 (en) * | 2009-11-11 | 2014-04-29 | Disney Enterprises, Inc. | Stereoscopic editing for video production, post-production and display adaptation |
US10095953B2 (en) | 2009-11-11 | 2018-10-09 | Disney Enterprises, Inc. | Depth modification for display applications |
US9445072B2 (en) | 2009-11-11 | 2016-09-13 | Disney Enterprises, Inc. | Synthesizing views based on image domain warping |
WO2011060296A2 (en) * | 2009-11-13 | 2011-05-19 | California Institute Of Technology | Stereo imaging miniature endoscope with single imaging chip and conjugated multi-bandpass filters |
US8717360B2 (en) | 2010-01-29 | 2014-05-06 | Zspace, Inc. | Presenting a view within a three dimensional scene |
US20110242355A1 (en) | 2010-04-05 | 2011-10-06 | Qualcomm Incorporated | Combining data from multiple image sensors |
US8896668B2 (en) | 2010-04-05 | 2014-11-25 | Qualcomm Incorporated | Combining data from multiple image sensors |
TW201031179A (en) * | 2010-04-15 | 2010-08-16 | Xiu-Ping Zhou | Image transform method, imaging methods and system for stereo vision |
WO2011162227A1 (en) * | 2010-06-24 | 2011-12-29 | 富士フイルム株式会社 | Stereoscopic panoramic image synthesis device, image capturing device, stereoscopic panoramic image synthesis method, recording medium, and computer program |
KR101723235B1 (en) * | 2010-10-04 | 2017-04-04 | 삼성전자주식회사 | Apparatus and method for attenuating three dimensional effect of stereoscopic image |
US20120098931A1 (en) * | 2010-10-26 | 2012-04-26 | Sony Corporation | 3d motion picture adaption system |
PL2458467T3 (en) * | 2010-11-26 | 2014-03-31 | Abb Research Ltd | Method and system for monitoring an industrial system |
US9380285B2 (en) | 2010-12-20 | 2016-06-28 | Samsung Display Co., Ltd. | Stereo image processing method, stereo image processing device and display device |
US9602799B2 (en) | 2011-01-14 | 2017-03-21 | Panasonic Intellectual Property Management Co., Ltd. | Device, method, and computer program for three-dimensional video processing |
US9407904B2 (en) | 2013-05-01 | 2016-08-02 | Legend3D, Inc. | Method for creating 3D virtual reality from 2D images |
US9288476B2 (en) | 2011-02-17 | 2016-03-15 | Legend3D, Inc. | System and method for real-time depth modification of stereo images of a virtual reality environment |
US9241147B2 (en) | 2013-05-01 | 2016-01-19 | Legend3D, Inc. | External depth map transformation method for conversion of two-dimensional images to stereoscopic images |
US9282321B2 (en) | 2011-02-17 | 2016-03-08 | Legend3D, Inc. | 3D model multi-reviewer system |
US9113130B2 (en) | 2012-02-06 | 2015-08-18 | Legend3D, Inc. | Multi-stage production pipeline system |
US20120229595A1 (en) * | 2011-03-11 | 2012-09-13 | Miller Michael L | Synthesized spatial panoramic multi-view imaging |
EP2710550A2 (en) * | 2011-05-17 | 2014-03-26 | Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V. | Methods and device for processing digital stereo image content |
US8786529B1 (en) | 2011-05-18 | 2014-07-22 | Zspace, Inc. | Liquid crystal variable drive voltage |
KR20120132240A (en) * | 2011-05-27 | 2012-12-05 | 삼성전자주식회사 | Display method and dual view driving method for providing plural images to plural users and display apparatus and dual view glass applying the method |
US9295375B2 (en) | 2012-09-27 | 2016-03-29 | Hrayr Karnig Shahinian | Programmable spectral source and design tool for 3D imaging using complementary bandpass filters |
US9456735B2 (en) | 2012-09-27 | 2016-10-04 | Shahinian Karnig Hrayr | Multi-angle rear-viewing endoscope and method of operation thereof |
US8687470B2 (en) | 2011-10-24 | 2014-04-01 | Lsi Corporation | Optical disk playback device with three-dimensional playback functionality |
US20130127861A1 (en) * | 2011-11-18 | 2013-05-23 | Jacques Gollier | Display apparatuses and methods for simulating an autostereoscopic display device |
US9286658B2 (en) * | 2012-03-22 | 2016-03-15 | Qualcomm Incorporated | Image enhancement |
JP2014045466A (en) * | 2012-08-29 | 2014-03-13 | Lenovo Singapore Pte Ltd | Stereoscopic video display system, setting method and observation position changing method of stereoscopic video data |
US9148651B2 (en) * | 2012-10-05 | 2015-09-29 | Blackberry Limited | Methods and devices for generating a stereoscopic image |
US9161018B2 (en) | 2012-10-26 | 2015-10-13 | Christopher L. UHL | Methods and systems for synthesizing stereoscopic images |
US9007365B2 (en) | 2012-11-27 | 2015-04-14 | Legend3D, Inc. | Line depth augmentation system and method for conversion of 2D images to 3D images |
US9547937B2 (en) | 2012-11-30 | 2017-01-17 | Legend3D, Inc. | Three-dimensional annotation system and method |
US9007404B2 (en) | 2013-03-15 | 2015-04-14 | Legend3D, Inc. | Tilt-based look around effect image enhancement method |
US9571812B2 (en) | 2013-04-12 | 2017-02-14 | Disney Enterprises, Inc. | Signaling warp maps using a high efficiency video coding (HEVC) extension for 3D video coding |
US9438878B2 (en) | 2013-05-01 | 2016-09-06 | Legend3D, Inc. | Method of converting 2D video to 3D video using 3D object models |
US11547446B2 (en) | 2014-01-13 | 2023-01-10 | Trice Medical, Inc. | Fully integrated, disposable tissue visualization device |
US9861261B2 (en) | 2014-03-14 | 2018-01-09 | Hrayr Karnig Shahinian | Endoscope system and method of operation thereof |
KR20160026298A (en) * | 2014-08-29 | 2016-03-09 | 삼성전자주식회사 | Magnetic resonance imaging apparatus, method for controlling the same, and head coil for the magnetic resonance imaging apparatus |
US10368104B1 (en) * | 2015-04-01 | 2019-07-30 | Rockwell Collins, Inc. | Systems and methods for transmission of synchronized physical and visible images for three dimensional display |
US9609307B1 (en) | 2015-09-17 | 2017-03-28 | Legend3D, Inc. | Method of converting 2D video to 3D video using machine learning |
WO2017087128A1 (en) | 2015-11-16 | 2017-05-26 | Virtual Dreamware, LLC | Symmetrically mirrored video segmet |
CN105681777B (en) * | 2016-01-20 | 2017-09-05 | 深圳创维-Rgb电子有限公司 | A kind of bore hole 3D display methods and system |
US11252391B2 (en) | 2017-03-06 | 2022-02-15 | Nevermind Capital Llc | Methods and apparatus for packing images into a frame and/or including additional content or graphics |
US10567733B2 (en) * | 2017-03-06 | 2020-02-18 | Nextvr Inc. | Methods and apparatus for communicating and/or using frames including a captured image and/or including additional image content |
US10735707B2 (en) | 2017-08-15 | 2020-08-04 | International Business Machines Corporation | Generating three-dimensional imagery |
JP7014660B2 (en) * | 2018-03-28 | 2022-02-01 | オリンパス株式会社 | Endoscope device, optical system switching determination method, program, and recording medium in the endoscope device |
US20200288112A1 (en) | 2019-03-07 | 2020-09-10 | Alibaba Group Holding Limited | Method, apparatus, medium, and device for processing multi-angle free-perspective video data |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO1983002706A1 (en) * | 1982-01-27 | 1983-08-04 | Stereographics Corp | Stereoscopic television system |
EP0716330A2 (en) * | 1994-12-07 | 1996-06-12 | Canon Kabushiki Kaisha | Apparatus for displaying image recognized by observer as stereoscopic image, and image pick-up apparatus |
Family Cites Families (43)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US3621127A (en) * | 1969-02-13 | 1971-11-16 | Karl Hope | Synchronized stereoscopic system |
GB2062281B (en) * | 1979-10-16 | 1983-06-22 | Secr Defence | Method and apparatus for producing three-dimensional displays |
JPS56168484A (en) * | 1980-05-29 | 1981-12-24 | Matsushita Electric Ind Co Ltd | Stereoscopic television system |
US4562463A (en) * | 1981-05-15 | 1985-12-31 | Stereographics Corp. | Stereoscopic television system with field storage for sequential display of right and left images |
US4925294A (en) * | 1986-12-17 | 1990-05-15 | Geshwind David M | Method to convert two dimensional motion pictures for three-dimensional systems |
US4582117A (en) * | 1983-09-21 | 1986-04-15 | Electric Power Research Institute | Heat transfer during casting between metallic alloys and a relatively moving substrate |
US4625290A (en) * | 1983-11-02 | 1986-11-25 | University Of Florida | Apparatus and method for producing a three-dimensional display on a video display device |
US4740836A (en) * | 1983-12-05 | 1988-04-26 | Craig Dwin R | Compatible 3D video display using commercial television broadcast standards and equipment |
US4683217A (en) * | 1984-04-13 | 1987-07-28 | Union Carbide Corporation | Iron-aluminum-phosphorus-silicon-oxide molecular sieves |
US4739418A (en) * | 1985-03-20 | 1988-04-19 | Victor Company Of Japan, Ltd. | Information signal recording disc recorded with stereoscopic television signal |
FR2588681B1 (en) * | 1985-10-16 | 1988-06-10 | Thomson Csf | METHOD FOR DETECTING CONTOURS IN IMAGING, AND LOCAL ESTIMATION OF THEIR PARAMETERS, IN A SPACE WITH AT LEAST THREE DIMENSIONS, AND DEVICE FOR CARRYING OUT SAID METHOD |
US4862292A (en) * | 1987-08-26 | 1989-08-29 | Canon Kabushiki Kaisha | Digital information signal recording apparatus |
JP2667835B2 (en) * | 1987-10-09 | 1997-10-27 | 株式会社日立製作所 | Computer Graphics Display |
US5155750A (en) * | 1987-12-24 | 1992-10-13 | Lockheed Missiles & Space Company, Inc. | Stereoscopic radiographic inspection system |
US4877307A (en) * | 1988-07-05 | 1989-10-31 | Kaiser Aerospace & Electronics Corporation | Stereoscopic display |
US4979033A (en) * | 1988-09-27 | 1990-12-18 | Stephens Berton H | Stereoscopic video system and method with field synchronization and intensity control |
JPH02177784A (en) * | 1988-12-28 | 1990-07-10 | Canon Inc | Image signal recording method |
US5175806A (en) * | 1989-03-28 | 1992-12-29 | Computer Design, Inc. | Method and apparatus for fast surface detail application to an image |
US5212562A (en) * | 1989-07-25 | 1993-05-18 | Canon Kabushiki Kaisha | Image signal reproducing apparatus having memory function |
EP0449478A3 (en) * | 1990-03-29 | 1992-11-25 | Microtime Inc. | 3d video special effects system |
US5165013A (en) * | 1990-09-26 | 1992-11-17 | Faris Sadeg M | 3-D stereo pen plotter |
JP3189010B2 (en) * | 1990-06-19 | 2001-07-16 | キヤノン株式会社 | Image processing device |
GB9027881D0 (en) * | 1990-12-21 | 1991-02-13 | Delta System Design Ltd | Improvements in 3d imaging systems |
US5416510A (en) * | 1991-08-28 | 1995-05-16 | Stereographics Corporation | Camera controller for stereoscopic video system |
US5193000A (en) * | 1991-08-28 | 1993-03-09 | Stereographics Corporation | Multiplexing technique for stereoscopic video system |
US5239372A (en) * | 1991-12-31 | 1993-08-24 | Stereographics Corporation | Stereoscopic video projection system |
WO1993022699A1 (en) * | 1992-04-24 | 1993-11-11 | Depth Enhancement Inc | Three-dimensional visualization by altered multiple two-dimensional perspective imagery |
US5659625A (en) * | 1992-06-04 | 1997-08-19 | Marquardt; Stephen R. | Method and apparatus for analyzing facial configurations and components |
US5293227A (en) * | 1992-07-24 | 1994-03-08 | Tektronix, Inc. | Self-synchronizing optical state controller for infrared linked stereoscopic glasses |
GB2270243B (en) * | 1992-08-26 | 1996-02-28 | Namco Ltd | Image synthesizing system |
US5434613A (en) * | 1992-12-10 | 1995-07-18 | Dasso; John M. | Method and apparatus for generating a three-dimensional effect for two-dimensional images |
US5564810A (en) * | 1992-12-31 | 1996-10-15 | Honeywell Inc. | Full color stereoscopic display with color multiplexing |
US5467446A (en) * | 1993-01-29 | 1995-11-14 | Colorage, Inc. | System and method for rasterizing images subject to orthogonal rotation |
JP3151770B2 (en) * | 1993-03-26 | 2001-04-03 | キヤノン株式会社 | Compound eye image display |
US5490240A (en) * | 1993-07-09 | 1996-02-06 | Silicon Graphics, Inc. | System and method of generating interactive computer graphic images incorporating three dimensional textures |
DE69424129T2 (en) * | 1993-11-29 | 2000-11-16 | Canon Kk | Device for processing stereoscopic images |
US5510832A (en) * | 1993-12-01 | 1996-04-23 | Medi-Vision Technologies, Inc. | Synthesized stereoscopic imaging system and method |
KR960702716A (en) * | 1994-03-28 | 1996-04-27 | 제임스 앨린 이반스 | TWO-DIMENSIONAL AND THREE-DIMENSIONAL IMAGING DEVICE |
US5701154A (en) * | 1994-11-01 | 1997-12-23 | Dasso; John M. | Electronic three-dimensional viewing system |
AUPN003894A0 (en) * | 1994-12-13 | 1995-01-12 | Xenotech Research Pty Ltd | Head tracking system for stereoscopic display apparatus |
US5606348A (en) * | 1995-01-13 | 1997-02-25 | The United States Of America As Represented By The Secretary Of The Army | Programmable display interface device and method |
US5850352A (en) * | 1995-03-31 | 1998-12-15 | The Regents Of The University Of California | Immersive video, including video hypermosaicing to generate from multiple video views of a scene a three-dimensional video mosaic from which diverse virtual video scene images are synthesized, including panoramic, scene interactive and stereoscopic images |
AUPN732395A0 (en) * | 1995-12-22 | 1996-01-25 | Xenotech Research Pty Ltd | Image conversion and encoding techniques |
-
1997
- 1997-08-04 US US08/905,574 patent/US6108005A/en not_active Ceased
- 1997-08-28 AU AU41687/97A patent/AU4168797A/en not_active Abandoned
- 1997-08-28 WO PCT/US1997/015216 patent/WO1998009445A1/en active Application Filing
-
2002
- 2002-08-22 US US10/226,807 patent/USRE39342E1/en not_active Expired - Lifetime
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO1983002706A1 (en) * | 1982-01-27 | 1983-08-04 | Stereographics Corp | Stereoscopic television system |
EP0716330A2 (en) * | 1994-12-07 | 1996-06-12 | Canon Kabushiki Kaisha | Apparatus for displaying image recognized by observer as stereoscopic image, and image pick-up apparatus |
Cited By (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
FR2901086A1 (en) * | 2006-05-11 | 2007-11-16 | Sagem Defense Securite | Vision restitution method, involves electronically moving one of images to adjust its position such that images are arranged identical to relative position to be occupied by images if axes of displays are in relative reference position |
WO2008004005A2 (en) * | 2006-07-05 | 2008-01-10 | James Amachi Ashbey | Improvements in stereoscopic motion pictures |
WO2008004005A3 (en) * | 2006-07-05 | 2008-06-05 | James Amachi Ashbey | Improvements in stereoscopic motion pictures |
EP2102822A4 (en) * | 2007-01-10 | 2010-06-02 | Real Image Corp | Method and apparatus for generating stereoscopic image from two-dimensional image by using mesh map |
US8970669B2 (en) | 2009-09-30 | 2015-03-03 | Rovi Guides, Inc. | Systems and methods for generating a three-dimensional media guidance application |
US8291322B2 (en) | 2009-09-30 | 2012-10-16 | United Video Properties, Inc. | Systems and methods for navigating a three-dimensional media guidance application |
CN102763061A (en) * | 2009-09-30 | 2012-10-31 | 联合视频制品公司 | Systems and methods for navigating a three-dimensional media guidance application |
EP3441851A1 (en) * | 2009-09-30 | 2019-02-13 | Rovi Guides, Inc. | Systems and methods for navigating a three-dimensional media guidance application |
WO2011041219A1 (en) * | 2009-09-30 | 2011-04-07 | Rovi Technologies Corporation | Systems and methods for navigating a three-dimensional media guidance application |
CN102763061B (en) * | 2009-09-30 | 2017-03-22 | 乐威指南公司 | Systems and methods for navigating a three-dimensional media guidance application |
EP2582144A4 (en) * | 2010-06-08 | 2015-06-24 | Lg Electronics Inc | Image processing method and image display device according to the method |
CN103004217B (en) * | 2011-06-08 | 2016-08-03 | 松下知识产权经营株式会社 | Parallax image generating apparatus, anaglyph generate method, program and integrated circuit |
US9147278B2 (en) | 2011-06-08 | 2015-09-29 | Panasonic Intellectual Property Management Co., Ltd. | Parallax image generation device, parallax image generation method, program, and integrated circuit |
CN103004217A (en) * | 2011-06-08 | 2013-03-27 | 松下电器产业株式会社 | Parallax image generation device, parallax image generation method, program and integrated circuit |
EP3057066A1 (en) * | 2015-02-10 | 2016-08-17 | DreamWorks Animation LLC | Generation of three-dimensional imagery from a two-dimensional image using a depth map |
US9721385B2 (en) | 2015-02-10 | 2017-08-01 | Dreamworks Animation Llc | Generation of three-dimensional imagery from a two-dimensional image using a depth map |
US9897806B2 (en) | 2015-02-10 | 2018-02-20 | Dreamworks Animation L.L.C. | Generation of three-dimensional imagery to supplement existing content |
US10096157B2 (en) | 2015-02-10 | 2018-10-09 | Dreamworks Animation L.L.C. | Generation of three-dimensional imagery from a two-dimensional image using a depth map |
Also Published As
Publication number | Publication date |
---|---|
US6108005A (en) | 2000-08-22 |
USRE39342E1 (en) | 2006-10-17 |
AU4168797A (en) | 1998-03-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6108005A (en) | Method for producing a synthesized stereoscopic image | |
US6496598B1 (en) | Image processing method and apparatus | |
EP0540137B1 (en) | Three-dimensional image display using electrically generated parallax barrier stripes | |
EP0746949B1 (en) | Synthesized stereoscopic imaging system and method | |
JP4427032B2 (en) | Image conversion and encoding technology | |
US8384766B2 (en) | Apparatus for processing a stereoscopic image stream | |
US9094674B2 (en) | Generation of three-dimensional movies with improved depth control | |
CA2352272C (en) | Improvement in and relating to method and apparatus for producing anaglyphic 3d images | |
Schmidt et al. | Multiviewpoint autostereoscopic dispays from 4D-Vision GmbH | |
Ezra et al. | New autostereoscopic display system | |
EP0791274A1 (en) | Methods and apparatus for zooming during capture and reproduction of 3-dimensional images | |
JP2009500963A (en) | System and method for capturing visual and non-visual data for multi-dimensional video display | |
KR19990053446A (en) | Three-dimensional stereoscopic image generation device using multiple liquid crystal slits | |
KR100496513B1 (en) | Image conversion method and image conversion system, encoding method and encoding system | |
de Beeck et al. | Three dimensional video for the home | |
Pastoor | Human factors of 3DTV: an overview of current research at Heinrich-Hertz-Institut Berlin | |
KR102042914B1 (en) | Floating-type hologram displaying apparatus using multi-layered displaying planes and multi-video processing method for said apparatus | |
WO1995013564A1 (en) | Method and apparatus for visualizing two-dimensional motion picture images in three dimensions | |
JP2002101430A (en) | Three-dimensional color display unit | |
Sheat et al. | Three-dimensional imaging for video telephony | |
AU8964598A (en) | Image processing method and apparatus | |
MXPA00002201A (en) | Image processing method and apparatus |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): AU CA CN JP KR US |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): AT BE CH DE DK ES FI FR GB GR IE IT LU MC NL PT SE |
|
DFPE | Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101) | ||
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
NENP | Non-entry into the national phase |
Ref country code: JP Ref document number: 1998511944 Format of ref document f/p: F |
|
122 | Ep: pct application non-entry in european phase | ||
NENP | Non-entry into the national phase |
Ref country code: CA |