US 20060268104 A1
A method and system for reducing ghost images in plano-stereoscopic image transmissions is provided. The method comprises establishing a plurality of expected ghosting profiles associated with a plurality of predetermined regions on a screen, and compensating for leakage in each predetermined region of a projected left and right eye images by removing an amount of ghost images leaking from the projected left eye image into the projected right eye image and vice versa. The system comprises a processor configured to receive the quantity of ghost artifacts and compute ghost compensation quantities for left eye images and right eye images. The processor is further configured to remove an amount of actual image ghost artifacts leaking from a projected left eye image into a projected right eye image and vice versa. The processor is also configured to compute ghost compensation quantities for each of a plurality of zones, each zone corresponding to a region on a screen having an expected ghosting profile associated therewith.
1. A method for reducing visual ghost artifacts in plano-stereoscopic image transmissions, comprising:
dividing a hypothetical screen representation into a plurality of regions corresponding to regions on a projection screen;
computing at least one ghost artifact coefficient depending upon expected ghosting within an associated region established by said dividing, each ghost artifact coefficient representing ghost artifacts leaking from a left eye image into a right eye image and from the right eye image into the left eye image;
applying at least one ghost artifact coefficient for a left eye projected image to a right eye projected image to form a compensated right eye projected image; and
removing the compensated right eye projected image from the right eye projected image.
2. The method of
performing a calibration function comprising assessing ghost artifacts using a test pattern comprising at least one right eye test image and at least one left eye test image projected to each of a right eye element and a left eye element of a selection device and establishing an expected ghosting profile based on the assessed ghost artifacts;
wherein the computing comprises computing the ghost artifact coefficients based on the expected ghosting profile.
3. The method of
performing a calibration function comprising computing an expected ghosting profile based on a computer model of a theatre;
wherein the computing comprises computing the ghost artifact coefficients based on the expected ghosting profile.
4. The method of
applying at least one ghost artifact coefficient for a right eye projected image to a left eye projected image to form a compensated left eye projected image; and
removing the compensated left eye projected image from the left eye projected image.
5. The method of
6. The method of
7. The method of
8. The method of
9. A method for reducing ghost images in plano-stereoscopic image transmissions, comprising:
establishing a plurality of expected ghosting profiles associated with a plurality of predetermined regions on a screen; and
compensating for leakage in each predetermined region of a projected left eye image and a projected right eye image by removing an amount of ghost images leaking from the projected left eye image into the projected right eye image and from the projected right eye image into the projected left eye image.
10. The method of
assessing ghost artifacts using a test pattern comprising at least one right eye test image and at least one left eye test image projected to each of a right eye element and a left eye element of a selection device and establishing an expected ghosting profile based on the assessed ghost artifacts;
wherein the compensating comprises compensating for leakage based on the expected ghosting profile.
11. The method of
computing an expected ghosting profile based on a computer model of a theatre;
wherein the computing comprises computing the ghost artifact coefficients based on the expected ghosting profile.
12. The method of
13. The method of
14. The method of
15. The method of
16. A system for reducing ghost images in plano-stereoscopic image transmissions, comprising:
a processor configured to receive the quantity of ghost artifacts and compute ghost compensation quantities for left eye images and right eye images and further configured to remove an amount of actual image ghost artifacts leaking from a projected left eye image into a projected right eye image and from the projected right eye image into the projected left eye image;
wherein the processor is configured to compute ghost compensation quantities for each of a plurality of zones, each zone corresponding to a region on a screen having an expected ghosting profile associated therewith.
17. The system of
18. The system of
a calibration arrangement configured to transmit a plano-stereoscopic test pattern to a screen; and
a selector device and at least one sensor configured to receive transmissions from said screen and assess a quantity of ghost artifacts received in a left eye of the selector device from a right image from the test pattern and vice versa.
19. The system of
20. The system of
This application claims the benefit of U.S. Provisional Patent Application Ser. No. 60/685,368, filed May 26, 2005, entitled “Ghost Compensation for Improved Stereoscopic Projection,” inventors Matt Cowan et al., the entirety of which is incorporated herein by reference.
1. Field of the Invention
The present design relates to the projection of stereoscopic images, and in particular to reducing the effects of image leakage between left eye and right eye views, also referred to as crosstalk or “ghosting.”
2. Description of the Related Art
Stereoscopic images are created by supplying the viewer's left and right eyes with separate left and right eye images showing the same scene from respective left and right eye perspectives. This is known as plano-stereoscopic image display. The viewer fuses the left and right eye images and perceives a three dimensional view having a spatial dimension that extends into and out from the plane of the projection screen. Good quality stereoscopic images demand that the left and right eyes are presented independent images uncorrupted by any bleed-through of the other eye's image. In other words, stereoscopic selection or channel isolation must be complete. Stereoscopic selection can be accomplished to perfection using isolated individual optical paths for each eye, as in the case of a Brewster stereoscope. But when using temporal switching (shuttering) or polarization for image selection, the left channel will leak to some extent into the right eye and vice versa. The effect of this leaking is referred to as ghosting or crosstalk.
Various designers have attempted to reduce crosstalk or the ghosting artifact in stereoscopic displays. Most notably, Levy, in U.S. Pat. Nos. 4,266,240, 4,287,528, and 4,517,592, lays out the basic technology for subtracting a portion of one image from the other to reduce the ghosting effect. Levy's implementations were directed to stereoscopic television systems. Ensuing solutions draw heavily on Levy's work and add relatively small improvements.
In the motion picture realm, many degrading artifacts have been cited in the literature as detracting from the enjoyment of the projected plano-stereoscopic motion picture experience, including the breakdown of convergence and accommodation, unequal field illumination, and lack of geometric congruence. None of these artifacts are more important than leakage between left eye and right eye images. Stereoscopic movies show deep, vivid images that create a significant, realistic perception of a spatial dimension that extends into and out from the plane of the projection screen, and this effect is most degraded by crosstalk.
Certain solutions have been proposed to address ghosting, but many of the proposed solutions tend to be uniform across an image or screen surface, i.e. remove the same ghosting artifacts in the same way regardless of screen position, environment, or any other pertinent factor.
The present design seeks to address the issue of ghosting or crosstalk in a projected plano-stereoscopic motion picture environment. It would be advantageous to offer a design that enhances or improves the display of projected plano-stereoscopic motion pictures or images by reducing the crosstalk associated with such motion picture or image displays over designs previously made available.
According to a first aspect of the present design, there is provided a method for reducing ghost images in plano-stereoscopic image transmissions. The method comprises establishing a plurality of expected ghosting profiles associated with a plurality of predetermined regions on a screen, and compensating for leakage in each predetermined region of a projected left eye image and a projected right eye image by removing an amount of ghost images leaking from the projected left eye image into the projected right eye image and from the projected right eye image into the projected left eye image.
According to a second aspect of the present design, there is provided a system for reducing ghost images in plano-stereoscopic image transmissions. The system comprises a processor configured to receive the quantity of ghost artifacts and compute ghost compensation quantities for left eye images and right eye images. The processor is further configured to remove an amount of actual image ghost artifacts leaking from a projected left eye image into a projected right eye image and from the projected right eye image into the projected left eye image. The processor is configured to compute ghost compensation quantities for each of a plurality of zones, each zone corresponding to a region on a screen having an expected ghosting profile associated therewith.
These and other advantages of the present invention will become apparent to those skilled in the art from the following detailed description of the invention and the accompanying drawings.
Preferred embodiments of the present design focus on large screen projection for entertainment, scientific, and visual modeling applications. Such projection alternates the left and right image on the same screen area using temporal switching or polarization to select the appropriate images for each eye. In the case of temporal switching, which may be combined with polarization modulation, the display alternately transmits left and right eye images, and an electro-optical or similar polarization modulator is employed as part of the selection system to direct the appropriate image to the appropriate eye. The modulator is best located at the projector and used in conjunction with analyzer glasses worn by audience members. An alternate method is to use shuttering eyewear and dispense with the polarization switching approach. Selection devices are synchronized with the frame or field output of the projector to ensure that the frame or field can be perceived by the appropriate eye.
In such projection systems, crosstalk results from a variety of sources, including the imperfect polarization modulation of the displayed image, a timing mismatch between polarization switching and the frame or field output of the display, the imperfect phase of the switch, allowing the wrong eye to leak through at the beginning or end of the frame, imperfect or leaking analyzers for viewing the polarized light, polarization state contamination caused by projection screen depolarization; polarization state contamination caused by airborne dust or dust on the port glass or modulator surface, and, in a linear polarizer selection system, relatively slight rotation of the analyzer glasses.
The present design addresses these sources of crosstalk in projection applications through an empirical calibration process characterizing the crosstalk specific to the projection equipment, image polarization or shuttering equipment, projection screen, viewer image selection equipment, and environment of a given installation. This process yields “ghosting coefficients” that characterize the measured crosstalk and are used to compensate image data at the projection site to provide installation-specific crosstalk cancellation.
Crosstalk is a linear phenomenon that affects each part of the image to the same proportion. Crosstalk may be color dependent in so far as the primary colors that make up the image may have different crosstalk characteristics. In such cases each color may be compensated individually.
The present design may be applied to the class of displays in which the entire image area is addressed or displayed simultaneously. In this case, the predicted ghosting is uniform across the entire screen, and characterization of crosstalk is preferably done by making a single measurement of crosstalk for each primary color to obtain a complete characterization with a single coefficient for each primary color. Alternative embodiments may utilize displays in which the display is written to the screen in lines, segments, or blocks. Where the image is displayed in segments, the ghosting depends on the timing of the display of the segment, related to the switching speed of the modulator or shutter and their temporal characteristics. For segmented displays, characterization may be done for each segment, or for a sample of segments and then interpolated for the other segments.
The present design may also be applied in systems where the level of ghosting is different in different areas across the screen. In this case the system creates a segmented ghosting map where different ghost coefficients are applied in different areas of the screen. This is particularly applicable with polarized projection on silver screens, where the level of ghosting tends to be highly dependent on the projection angle and the viewing angle of the images.
The present design benefits both linear and circular polarization implementations. Linear polarization has higher extinction but greater angular dependency of the polarizer with respect to the analyzer and shows degradation when the viewer tips his head to one side, whereas circular polarization selection has lower extinction but is far more forgiving with regard to head tipping. Using circular polarization for image selection can exhibit low crosstalk comparable to crosstalk obtained when employing linear selection. Using linear polarization for image selection in accordance with the present design can provide an improved head tilting range comparable to that obtained when circular polarization selection is employed.
In this system, a primary cause of ghosting is imperfect polarization of the analyzer glasses 110. Sometimes the depolarization artifact exhibits a color dependency, resulting in more ghosting in one color than another. In addition, imperfect synchronization or phasing of the modulator with respect to the field rate may result in ghosting. In liquid crystal technologies used for modulation, a switching time on the order of hundreds of microseconds may be required for a change in state. If a field or frame is projected during this transitional period, ghosting will be introduced.
The systems identified in
The basic process for characterizing the ghosting or crosstalk in a given system is to use test patterns that provide a full luminance (white or a primary color) image for one eye and a zero luminance (black) image for the other eye. These images are displayed or projected by the system in L-R-L-R sequence. While a test pattern is displayed, the amount of light passing through the left and right eye portions of a pair of analyzer glasses located in a normal use position can be measured. The amount of light arriving at each eye location in response to the test patterns empirically characterizes the effects of all sources of crosstalk in the optical path between the projector and the viewer's eyes.
Using the test pattern of
As mentioned above, ghosting may be color dependent. In such cases, the full luminance images are primary color images, and measurements as described above are made for each separate primary color, a feature available in various photosensing devices.
While these illustrations assume that the analyzer glasses used for the measurements are oriented in a horizontally non-tilted alignment with respect to the projection screen, in alternative embodiments it may be desirable to characterize the ghosting effects with the glasses positioned at a slight horizontal tilt. Such testing can yield a ghosting characterization that is slightly increased compared to that of the non-tilted position, however the slight overcompensation that may result may produce a demonstrably better acceptable head tilt range as discussed below with respect to
The foregoing assumes that a calibration procedure occurs within a specific environment. Alternately, the system may calibrate using a model of a specific theater or other computer simulation, or may simply make assumptions about the proposed environment and create GCs based on expected viewing conditions.
Once all measurements are made, a ghosting coefficient (GC) for each eye channel may be calculated by dividing the leakage luminance by the full luminance. The ghosting coefficients GC provide a characterization of the crosstalk from one eye to the other that is created by the particular equipment used in the particular installation where the measurements were made. Where ghosting is color dependent, a separate ghosting coefficient is calculated for each primary color.
As an example, leakage luminance may be computed in each of the red, blue, and green color realms as 10, 15, and 5, respectively, with total or full luminance values of 100, 100, 100. The GC for red (GCR) would be 0.10, or 10 per cent, representing 10 leakage luminance divided by 100 full luminance values. Blue and green ghosting coefficients in this example would be a GCB of 0.15 and a GCG of 0.05.
The ghosting coefficients are used to compensate images in a manner that reduces the inherent crosstalk of the display system through cancellation, such that the final images perceived by the eyes exhibit reduced or imperceptible ghosting. More specifically, the ghosting coefficients are used to calculate ghosting components of the type illustrated in
The design produces each compensated image using an original image and a ghosting component derived from the corresponding opposite eye image of the image pair as follows:
Rf is the final compensated image for the right eye;
Ri is the original image for the right eye;
Lf is the final compensated image for the left eye;
Li is the original image for the left eye; and
GC is the ghosting coefficient.
Through substitution, these equations may be used to characterize the ghost-compensated images in terms of the original images as follows:
In the case where the ghosting coefficient is small, the GC2 term becomes small, and the equations may be approximated as:
In systems that exhibit color-dependent ghosting, the system calculates compensated sub-images for each primary color using the ghosting coefficient corresponding to each color.
As demonstrated below with respect to
Ghosting compensation is preferably implemented in digital display systems in which images are represented as digital data that can be mathematically operated upon to perform image processing in accordance with the ghosting correction equations provided above.
After linear transformation, the system computes the ghost contribution from each eye image at point 504 using the formulas and coefficients discussed above. The ghost contribution calculated for each images is then subtracted at point 506 from the original opposite eye image to yield compensated linear image data. The compensated linear images may be converted back into a non-linear form by applying the inverse of the linear transformation applied above at point 508. Application of the inverse to convert back to non-linear form involves setting the range of representation and applying the non-linear transformation and offset. The output of this processing is compensated right and left eye images 510 and 511.
In implementations where the ghost compensation is integrated into a display device such as a projector, the display device may not be required to put the image representation back into a non-linear representation since the linear image data may be fed directly to the image display elements of the display device. In other words, blocks 508 may not be needed and the output of blocks 506 may be applied directly to the image display elements of the display device and may be displayed.
In general, ghost compensation may be performed in both real-time and non-real-time implementations. Examples of each are provided in
A second approach for mastering is to use a non real-time process to render the ghost compensation into the images. This system provides an off-line processor 606 that saves a ghost compensated master 608 which may be supplied later for viewing. The ghost compensated master may be used for internal review or may be used as a master for producing distribution copies of the content. In the latter case, the ghosting coefficients used in the compensation processing are typically selected to be an average of the estimated ghosting coefficients present in various viewing installations, as opposed to a value optimized for a specific installation. The real-time and off-line compensation may be implemented either in software, firmware or hardware.
Various real-time embodiments for use in viewing installations such as cinemas are now discussed with respect to
Although the image processing of the compensation module may be performed by a microprocessor acting under the control of software or firmware, such as the native processing elements of the server itself, image processing may alternately be performed by a field programmable gate array (FPGA) 804 configured to receive image data and ghosting coefficients as inputs and to process the image data in the manner discussed with respect to
The primary functionality provided by the hardware is the subtraction of ghosting properties from the left eye and right eye images according to Equations (1) through (6). Compensation for ghosting thus requires calculation of the appropriate coefficients, applying the coefficients to the existing data, and subtracting the ghosted inverse from the image to produce the de-ghosted image. To perform this, particularly when three components such as red, green, and blue are employed and ghost removal occurs for each component of every pixel. Thus the design shifts a great deal of data in and out in a very short amount of time, and primary processing is loading data, performing a subtraction, and transferring the compensated data from the processor or processing device.
The FPGA 804 may be set in a bypass mode in which no compensation processing is performed. However, in more robust implementations, the programming interface may comprise a serial port or a network interface and related circuitry for receiving ghosting coefficients as well as receiving and executing compensation module control commands. The programming interface of the compensation module 802 may communicate through the communications subsystem of the projector, enabling the compensation module to be addressed through a communications port of the projector such as an Ethernet port to receive ghosting coefficients and commands.
The compensation module of the projector embodiment obtains left eye and right eye data from deserializers 820 and 821 of the projector. The projector architecture typically has the capability of accepting serial (HDSDI) or DVI inputs. The linearized compensated images generated by the compensation module may be supplied to the image rendering elements of the projector.
In accordance with another alternative embodiment, the substantial computational capability of a computer graphics output card may perform compensation in real-time on image data sent from a computer to a display device. This embodiment uses the capability of the graphics card to perform the numerical computations of the compensation algorithm, in real-time, operating from content processed or played from or through a processing device such as a personal computer.
The computing device executes a calibration application that automates the test pattern display and analysis and the setting of ghosting coefficients described herein.
After all readings are obtained, the calibration application computes the ghosting coefficients of the left eye and right eye channels in the manner described above. The calibration application then sends the ghosting coefficients to the compensation module in the projector along with any commands necessary to store the ghosting coefficients and enable compensation processing using those coefficients. Ghosting coefficients may take any of a variety of forms appropriate for the specific implementation, such as in an array or arrays or via a set of data values in a stream or listing. For example, if a region, including a pixel, has a red GC of 0.3, the value of 0.3 and the coordinate of that pixel may be transmitted to the compensation module, and similar red coefficients for all regions or pixels in the image are transmitted, typically indexed by region or pixel numbers or locations. Similar GC values may be transmitted for green and blue in the manner discussed.
Similarly, ghost compensation in accordance with embodiments disclosed provides enhanced performance for circular polarization applications, enabling dynamic ranges comparable to those of linear polarization systems to be achieved.
In a display system, factors that create ghosting are generally different in different parts of the display. Such differences are generally the result of differences in the angle at which light passes through the optical elements and the differences in angle of reflection off the screen. Screen composition may contribute to the artifacts or ghosts perceived. In such a construction, different ghost factors are required to optimize the ghost image depending on ghost position on the screen. Typically more ghosting exists at the edges and corners of the image than in the center of the screen.
The optimum correction for the theatre is created by characterizing the ghosting factor across the area of the screen, generally by sampling or modeling the amount of ghosting in each part of the screen and creating a segmented correction map. For example, if Red/Green/Blue components are treated separately, blue ghosting may be significant at an edge or all edges of the screen. The blue GC at an outer region or zone, toward the edge of the image, may be 0.4, while at the center of the screen blue ghosting may not be as significant and may therefore have a smaller GC, such as 0.15. Each zone may have different GCs or may employ different ghosting properties depending on the particular environment.
The foregoing outlines a general case where the ghost factor is potentially different for every point on the screen. From a more practical point of view, the correction may be applied in the horizontal direction only such as is illustrated in
We have described a means for improving the projection of stereoscopic motion picture images, for a variety of uses but primarily for the theatrical motion picture industry. The application of ghost compensation technology allows for clearer, sharper, deeper stereoscopic movies with better off-screen effects. Preferred embodiments use real-time pre-compensation based on ghosting characteristics measured at the installation site so that the compensation is tailored to the characteristics of the individual screening room or theatre. An advantage of local ghosting characterization and processing is that only one type of print needs to be distributed for all theatres. Thus this print may be used in any theater for either planar exhibition or stereoscopic exhibition. Thus, by the real-time addition of the ghost pre-compensation at the projector or server, the distributors and exhibitors enjoy the economic and logistical advantages of using a single inventory of prints for all applications.
The circuits, devices, processes and features described herein are not exclusive of other circuits, devices, processes and features, and variations and additions may be implemented in accordance with the particular objectives to be achieved. For example, devices and processes as described herein may be integrated or interoperable with other devices and processes not described herein to provide further combinations of features, to operate concurrently within the same devices, or to serve other purposes. Thus it should be understood that the embodiments illustrated in the figures and described above are offered by way of example only. The invention is not limited to a particular embodiment, but extends to various modifications, combinations, and permutations that fall within the scope of the claims and their equivalents.
The design presented herein and the specific aspects illustrated are meant not to be limiting, but may include alternate components while still incorporating the teachings and benefits of the invention. While the invention has thus been described in connection with specific embodiments thereof, it will be understood that the invention is capable of further modifications. This application is intended to cover any variations, uses or adaptations of the invention following, in general, the principles of the invention, and including such departures from the present disclosure as come within known and customary practice within the art to which the invention pertains.
The foregoing description of specific embodiments reveals the general nature of the disclosure sufficiently that others can, by applying current knowledge, readily modify and/or adapt the system and method for various applications without departing from the general concept. Therefore, such adaptations and modifications are within the meaning and range of equivalents of the disclosed embodiments. The phraseology or terminology employed herein is for the purpose of description and not of limitation.