|Publication number||US4558359 A|
|Application number||US 06/547,610|
|Publication date||Dec 10, 1985|
|Filing date||Nov 1, 1983|
|Priority date||Nov 1, 1983|
|Publication number||06547610, 547610, US 4558359 A, US 4558359A, US-A-4558359, US4558359 A, US4558359A|
|Inventors||Gilbert G. Kuperman, Donald L. Wallquist|
|Original Assignee||The United States Of America As Represented By The Secretary Of The Air Force|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (9), Non-Patent Citations (10), Referenced by (60), Classifications (14), Legal Events (6)|
|External Links: USPTO, USPTO Assignment, Espacenet|
The invention described herein may be manufactured and used by or for the Government of the United States for all governmental purposes without the payment of any royalty.
This invention relates to the fields of electronic image processing and stereoscopic image analysis.
Stereoscopic imagery in which physically displaced views of the same image are presented simultaneously for conveying the illusion of depth is well known in the art. The patent of A. Rosenfield, U.S. Pat. No. 2,910,910 is an example of stereo imaging wherein two slightly displaced photographs of the same terrain are viewed simultaneously using red and green filtering for the purpose of automatically plotting terrain contour maps. The Rosenfield patent also shows the combination of stereo imaging apparatus and digital computing apparatus.
Another example of stereo imaging apparatus is found in the patent of J. W. Knauf, U.S. Pat. No. 3,464,766 wherein a stereo image is viewed by a person with the aid of a synchronized shutter which alternates the views received by the left and right eye.
The U.S. patent of J.E. Ihms, U.S. Pat. No. 4,133,602 is a recent example of stereo image optical apparatus and discloses the use of a single photographic medium for recording the two images needed for a stereoscopic view. The Ihms patent includes an interesting discussion of the history and state of the art in one portion of the stereoscopic image technology. Another recent example of stereo imaging apparatus is found in the U.S. patent of H. J. Wehrli, U.S. Pat. No. 4,290,694 wherein an apparatus for detecting changes between two photographic views of the same image is disclosed. The Wehrli patent also includes a listing and discussion of prior art in stereo imaging. Another recent example of stereo imaging apparatus is the U.S. patent of A. P. Colvocoresses, U.S. Pat. No. 4,313,678 wherein an apparatus for determining height or elevation of a photographed object is disclosed. The Colvocoresses patent contemplates the use of satellite photography and the measurements of elevation on a terrestrial body.
The U.S. patent of G. G. Langdon et al, U.S. Pat. No. 4,225,861 is of interest with respect to the present invention since it discloses the use of digital electronic processing to change a photographic image into an image having new color components for analysis; the changes being made in response to dimensional elements from the original photograph. The Langdon et al patent does not contemplate the use of stereoscopic viewing or the changing of image element locations however.
The International Imaging Systems (I2 S) Model 70E processor used in the preferred embodiment of the invention is described in several application publications which are listed below for both the purpose of incorporation by reference and for consideration as prior art in the field of image processing.
1. "Applications of Digital Image Processing Techniques and Hardware to Regional Growth Modeling" by Ted Driscoll, Proceedings of the American Society of Photogrammetry Vol. 1 45th Annual Meeting, Mar. 18-24, 1979, Washington, D.C. The bibliography and Technical appendix of this article covering the Model 70 processor are of additional interest.
2. "A Low Cost Transportable Image Processing System" by John R. Adams and Edward C. Driscoll, First ASSP Workshop on Two-Dimensional Digital Signal Processing, Oct. 3-4, 1979 Lawrence Hall of Science, Berkley, Calif.
3. "High Speed, Hardware Classification of Multispectral Imagery duplicated in a General Purpose Image Display" by Edward C. Driscoll, Jr. of I2 S, Journal of Applied Photographic Engineering June 1982 Vol 8 #3.
4. Evolution of Image Processing Algorithms from Software to Hardware by Ted Driscoll and Chris Walker of I2 S. Proc. of Society of Photooptical Instrumentation Engineers, Vol 271 February 1981 Paper #17.
5. New concepts in Display Technology by John Adams and Robert Wallis, Computer, August 1977 p61-69.
The inventors of the present invention also have a copending U.S. patent application for another invention which uses the Model 70E I2 S image processor. This application is U.S. Ser. No. 06/556,861 filed Dec. 1, 1983, and contains descriptive material relating to the Model 70E processor; the contents of this application are hereby incorporated by reference.
An object of the present invention is to electronically modify or transform image density information which may be monochrome or black and white in nature so that a reconstructed color image using the modified information is more easily comprehended by an image analyst. The main characteristic of the invention is the addition of picture elements (pixels or pels) in segregatable color combinations to the original dimensions of an image feature.
A further object of the invention is to provide an image analysis apparatus and method that is usable in analyzing images developed in a large number of different technologies. An exemplary but not exhaustive list of potential applications includes images obtained from radar (especially synthetic aperture radar), sonar, Xray (both medical and technical), sound equipment (sonograms), photographic equipment, temperature or heat sensors (thermograms), seismology or mineral exploration, oceanography, population displays, and mapping.
FIG. 1 is a block diagram of the preferred embodiment of the invention.
FIG. 2 is a flow diagram for the processing of image information in accordance with the invention.
FIG. 3 including FIGS. 3a, 3b and 3c indicates the manner of collecting, processing and displaying electrical signals in the invention.
FIG. 4, including FIGS. 4a, 4b and 4c is a flow diagram.
FIG. 1 is a block diagram of a system for generating anaglyphic stereoscopic images according to the present invention. The term anaglyph is defined in the Random House College Dictionary as being an ornament sculptured or embossed in low relief, as a cameo.
In the present invention, the term anaglyph is understood to refer to an image having relief properties in a third or Z dimension in addition to features defined in the usual X and Y coordinate directions.
The system of FIG. 1 includes a digital computer 100 which may be a Digital Equipment Corporation PDP 11/34 computer or one of many equivalent computers, an image array processor 110; such as the Model 70E Image Array processor manufactured by International Imaging Systems, Inc. of 1500 Buckeye Drive, Milpitas, Calif. 95035 or an equivalent processor; a three color cathode ray tube display 142, optical filter elements 150 and 152; and an observer or analyst 154. The image array processor 110 includes several channels of refresh memory, three of which are utilized in the preferred embodiment of the invention and indicated at 102, 104 and 106 in FIG. 1; these memories are embodied in the form of digital hardware. The I2 S processer also includes a plurality of hardware embodied lookup tables indicated at 112-128 in FIG. 1, several hardware output functions memories indicated at 130-134 in FIG. 1, several digital to analog converters indicated at 136-140 in FIG. 1 and a display controller indicated at 108 in FIG. 1. Certain of the elements of the image array processor 110 are shown unused in the embodiment of the present invention; these elements are shown in the interest of clarity and to allow reference to published information for the I2 S model 70 processor. The green and blue hardware lookup tables 114 and 116 and the red and blue lockup tables 118 and 122 in FIG. 1 are in this unused group. These tables are software deselected and have no influence on the displayed image. In addition, both the active hardware lookup tables 112, 120 and 124-128 and the output function memories 130-134 are not required for the processing of the present invention and are therefore loaded with 1 to 1 correspondence information so the output of the refresh memories passes unchanged to the display 142.
The three color display 142 in FIG. 1 is similar in appearance and function to the display of a conventional color television receiver except that it is arranged to display images having 512 pixels of information along both the horizontal and vertical picture axes in lieu of the 525 lines used in the NTS television system. Images to be presented by the display 142 are stored in the refresh memories 102,104 and 106 in FIG. 1. Each of these memories has the capability of storing 512×512 words of 8 bits each, the image on the display 142 being defined therefore by 262,144 (512×512) discrete elements or pixels each having a resolution of 8 binary bits or 28 or 256 intensity levels. The three channels of refresh memory allow storing of this quantity of information for three discrete colors which are indicated as red, green and blue in the FIG. 1 apparatus; a gray image usable in an alternate embodiment of the invention can be achieved by exciting all of the display 142 electron guns 144, 146 and 148 as indicated by the connection of outputs from the lookup table memories 124, 126, and 128.
The channel 1 and channel 2 refresh contents are coupled exclusively to the red and green electron guns of the display 142; the channel 1 information is comprised of picture elements to be displayed at or on the left of an original image picture element while channel 2 information is to be displayed at or to the right of an original image picture element. It is to be understood, of course, that the directions of right and left and the assignments of channels and colors are all arbitrary and can be altered with suitable adjustments to the optical filters 150 and 152 and the arrangement of data from the digital computer 100 without changing the essence of the invention.
The display controller 108 in FIG. 1 merely illustrates the existence of some arrangement in the computer 100 and the image array processor 110 for conveying information from the computer and its output bus into a designated refresh memory for display operation. The refresh memories after receiving information from the computer retain the information for repeated access by the display 142 in order to maintain a stable and permanent display image. The refresh memories can also be used to assemble information which is processed and presented by the computer 100 in segments of less than a complete image.
The digital to analog converters 136, 138 and 140 in FIG. 1 provide analog voltage signals suitable for driving the three electron gun inputs of the color display 142. The input to these converters is digital encoded information and the operation of such converters is well known in the art. The color display 142 in FIG. 1 is intended for viewing by a human observer indicated at 154, a different view is presented to the observer's right and left eyes 156 and 158 by way of the green and red color separation filters 150 and 152. In both the green and red filtered view of the reconstructed image the original image which is reproduced in blue will be significantly attenuated or totally suppressed.
The image processing to be accomplished in the present invention is shown diagrammatically in the sequence chart of FIG. 2; this sequence can best be understood by reference to the examples shown in the three views of FIG. 3 wherein FIG. 3a showns a simulated image such as might be encountered in an aerial photograph or a radar map. The objects in FIG. 3a include a building 310 having a light colored frontal surface 302, a somewhat darker colored side surface 304, and one or more windows 306, together with a segment of darkly colored road surface 309. It is to be understood that the type of image and the objects of FIG. 3a are selected merely for the purpose of illustration and that the density or relative light and dark colorations of the surfaces depicted in this figure are of interest in the electronic processing to be accomplished; other images such as those obtained with X-ray or thermogram apparatus (with smooth and continuous changes of object density) would provide more graphic but more difficult to draw and explain examples of processing done in accordance with the invention. The analysis of images obtained with medical X-ray equipment and human subjects for example is found to be particularly aided by the use of the present invention.
Scanning of the objects shown in FIG. 3a by way of a flying spot scanner or an image orthicon or a solid state photosensor array or other techniques which are known in the art produces a video signal generally as shown in FIG. 3b--when the scanning is done along the line 312 shown in FIG. 3a. In the FIG. 3b video signal the background 308 is assumed to be totally black as indicated by the low amplitude at 316 and 324 while the very light color of the frontal surface 302 is indicated by the high amplitude signal 318. The successively darker appearance of the window 306, the side surface 304, and the road 309 are shown respectively at 322, 320 and 326 in FIG. 3b.
It is the intention of the invention that supplemental and physically offset image elements or pixels be added to the image elements of the objects shown in FIG. 3a at locations which are displaced in proportion to the density of the original object picture element as observed by the scanning apparatus. The processing therefore involves generating for each pixel of image feature information in a scan, additional pixels of information which are to be located immediately preceeding and immediately succeeding the original pixel in a reconstructed image. The preceeding pixels of information are displayed in a second color different from that of the original pixel and the succeeding pixels are displayed in a third color different from that of the original pixels and the preceeding pixel. The location of the added or offset pixels with respect to the original pixel is directly related to the density or intensity or amplitude of the orignal pixel. The preferred embodiment of the invention contemplates that the reconstructed image include therefore pixels of red, green and blue color for example even though the original image was monochromatic in nature. The preferred embodiment further contemplates that the three color reconstructed composite or summation image will be viewed stereoscopically or binocularly using color separation filters to achieve a view which appears to have depth or three dimensions.
When the scanning of FIGS. 3a and 3b is performed by the FIG. 1 apparatus there will be generated 512 pixels of image data, each pixel being of modulus 28 or 256 intensity levels with the overall video signal having the appearance of the waveform shown in FIG. 3b when the scan is performed as indicated at 312 and 314.
FIG. 3c is a partial representation of a reconstructed image which has been processed in accordance with the invention from the scans shown at 312 and 314. In FIG. 3c the original FIG. 3a image component is reproduced pixel for pixel excepting for slight drawing modifications allowing illustration of the supplemental or offset pixels that will result from certain features in the FIG. 3a scene. A complete drawing of a reconstructed image from FIG. 3a would require the use of colors and be visually complex. The shading which would indicate the blue color of the FIG. 3c original image features, according to the preferred embodiment of the invention, is not shown in the interest of preserving drawing clarity in FIG. 3c. The original image features in FIG. 3c can also be reproduced in grey as is described below.
FIG. 3c illustrates reconstructed image features generated by the processing of the present invention in response to scanning the first pixels inside the FIG. 3a boundary lines 303 and 307, assuming the video signal from each FIG. 3a scan is substantially as shown in FIG. 3b. The first pixels inside the boundary 303 are of high intensity as illustrated at 318 in FIG. 3b and will result in a large separation between the offset and the causing pixels as indicated by the wide spacing 331 for the left offset pixels 330. A similar wide spacing occurs for the right offset pixels 332 the intent of the invention being that the offset spacing be in proportion to the original image pixel intensity of brightness or density or amplitude of height--whichever is an appropriate description of the video signal amplitude in the image being processed. An X-ray image is usually described in terms of image density for example while a photograph of the sun could be described in terms of brightness and a radar image could be described in terms of return signal amplitude and an aerial photograph in terms of height or elevation. These terms are used interchangeably in describing the invention herein.
It is intended that the offset pixels be presented in a first color such as red as indicated by the shading 333 for the left offset pixels and that the right offset pixels be of a second color such as green as indicated by the shading at 334. The red, green and blue colors are of course examples and could be replaced with other mutually exclusive colors or with other mutually exclusive image presentations such as vertical and horizontal light polarizations, as is known in the art.
The lower video signal amplitude 326 representing the road 309 in FIG. 3a results in offset image pixels being displaced by a smaller distance as is shown at 335 in FIG. 3c. The offset pixels 339 and 337 are generated in response to the original image pixels adjacent line 307. The offset pixels from these first pixels of the road 309 are also displayed in red and green colors as indicated at 338 and 336.
The different displacements of the offset pixels resulting from the differing video levels at 318 and 326 as shown by the dimensions 331 and 335 in FIG. 3c illustrate the intention of the invention that the displacement of offset pixels be predictably related to the density of the original image; this relationship may be linear or nonlinear in nature without departing from the spirit of the invention. A linear relationship found useful in the preferred embodiment of the invention is shown in Table 1.
TABLE 1______________________________________Input Image Pixel Left and Right(Density, Height, Intensity) Offset Pixel Displacement______________________________________ 0-31 032-63 164-95 2 96-127 3128-159 4160-191 5192-223 6224-255 7______________________________________
As illustrated by these values it is found useful in embodying the invention to divide the 256 possible pixel intensities into eight equal ranges of 32 intensity levels each and assign one increment of offset space or one pixel location of space to each of these eight ranges. As described below a maximum displacement of 8 pixel locations was found to be most effective in embodying the invention; however, other maximum displacements could be and have been employed. A nonlinear relationship between input image density and the number of offset pixels could, for example, change the numbers in the right hand column of Table 1 in accordance with a logarithmic or exponential or other mathematical function; the increment between column 2 numbers would in such instance be non constant and vary according to the selected mathematical relationship.
From the foregoing description of the invention it will be apparent that there can develop at certain pixels of the reproduced image a competition for the utilization of a particular pixel as part of the original image component, part of one or more left offset image components and part of one or more right offset image components. Generally, this competetion is resolved in the image processing by supplying to the reconstructed image display, signals which will first reconstruct the original image component in blue color. With the original image displayed, competetion between two left displaced or red pixel signals is resolved by displaying the stronger of the two competing pixels. Similarly competition between two right displaced or green pixel signals is resolved by displaying the stronger of the green signals. When the competition is between a left and right offset pixel signal the competition is resolved by illuminating both a red and green color for the pixel, the two electron guns and shadow mask inherent in a cathode ray tube embodiment of the display permitting simultaneous excitation of two different colors.
Near the edges of the reconstructed image, it is apparent that some of the offset pixels could extend beyond the boundary of the original image by the number of pixels selected for displacing early and late original image pixels. Extension beyond the original boundaries can be resolved by either omitting the over extending pixels or providing additional space on the reconstructed image display, omission being preferable in most embodiments.
FIG. 3c illustrates the supplementing of original image pixels without discussing the status of reconstructed image intermediate pixels--pixels indicated at 327 which lie in the locations between the original and displaced pixels. These intermediate pixels may of course be illuminated as a part of the original image or as part of the left displaced or right displaced pixels from subsequent original image pixels. A final possibility for the intermediate pixels is that they be dark or of zero intensity in which case the filtering and smoothing functions described below may cause artificial illumination for cosmetic or appearance purposes. If none of these mechanisms operate to illuminate an intermediate pixel it will remain dark.
It will also be apparent from the discussion of FIG. 3a and 3b that conditions will arise wherein differing maximum (and minimum) video signal amplitudes require that different increments of video signal be used in order that the video signal be resolved into the 256 equal increments desired in the preferred embodiment apparatus. This scaling or gain control function is indicated in the flow diagram of FIG. 2 at 200 and can be accomplished by a variety of techniques which are known in the art including both digital and analog signal processing. A digital scaling arrangement is used in the preferred embodiment of the invention as is described in connection with the computer program subsequently.
FIG. 2 shows a flow diagram by which an input image may be scanned, the resulting signals subjected to signal processing in accordance with the present invention and the processed signals used to form a reconstructed image. In FIG. 2 the block 200 indicates preprocessing of the image intensity values such that the processed image employs the 256 discrete levels of intensity or image density value. As indicated above, this preprocessing is accomplished by digital techniques in the preferred embodiment of the invention and can include a mathematical operation of subtracting the lowest density value in the image to be processed from the highest density value and dividing the difference by the number 256 to determine the intensity or density value to be assigned to each increment of preprocessed image or density. This preprocessing could be performed on partial segments of the image to be processed such as 1 scan line but is preferably performed on the entire image.
At block 202 in FIG. 2 the preprocessed image data is transmitted, preferably in increments of one scanning line, or 512 pixels of intensity modulus 28, into the processing apparatus, the digital computer 100. The processing steps accomplished in the digital computer are indicated by the box 214 in FIG. 2 and include assignment of left and right offset pixed displacement values by way of a software table look-up as shown at block 204 and the smoothing or filtering of pixel intensities for cosmetic purposes as shown at block 206. In the preferred embodiment of the invention the filtering/smoothing of pixel intensities was selected to replace offset pixels of zero intensity with some intermediate pixel intensity where the zero intensity pixel is located near a non zero offset pixel. Other smoothing/filtering algorithms could of course be employed without departing from the spirit of the invention.
Following the assignment of offset values and smoothing/filtering of these values, information for forming the reconstructed image is transmitted to the image display apparatus shown in the blocks 110 and 142 in FIG. 1 as is generally indicated by blocks 208, 210 and 212 in FIG. 2.
The computer program used in embodying blocks 204 and 206 in FIG. 2 is shown below in this specification. A related flow diagram is shown in FIG. 4 of the drawings in three portions. FIG. 4a, FIG. 4b and FIG. 4c. This program employs the Fortran IV programming language and includes programming comments to explain the function being performed. This annotated program in a common programming language will generally be understood by persons skilled in programming art without lengthy or detailed additional comments; a few comments to to allow faster comprehension of the program are therefore the extent of the present description. In the FIG. 4 diagram the numbers located within flow diagram boxes correspond to line numbers in the program listing. The boxes in the FIG. 4 flow diagram have the shape-to-performed function relationships known in the computer art.
As will be readily appreciated by readers skilled in the programming art, the letter C appearing before certain lines in the program listing, such as the lines between lines 1 and 2, denote comments which are ignored by the computer; additionally, certain lines in the program such as lines 14 and 30 call for the execution of subroutines which may be used repeatedly or which may be used in other programs. The function performed by a called subroutine is annotated in the notes immediately preceeding the call statement. The entire program listing shown herein is itself actually a subroutine which is named PSS and which is called for execution by a general operating system level program. The operating system program serves to host or provide a supporting framework for the program shown herein and could be one of many operating systems which are known in the art.
Generally the program shown in this specification down to the comment regarding setting up of the offset table following line 21 is comprised of housekeeping functions involving the assignment of variable types, definition of variables so as to be accessible between the listed program and its operating system program, the declaration of starting values, clearing of the image array processer memories, and the testing for operator elected input parameters.
The program step at line 21 involves correlating or fitting the 256 intensity levels in the preprocessed image data into the elected maximum pixel displacement. A maximum displacement of eight pixel locations is used in the preferred embodiment as is described in connection with Table 1 above.
The computer program steps establishing the look-up table for offset pixel displacement appear following line 23 in the program listing. In the preferred embodiment of the invention 32 intensity levels are assigned to each increment of pixel displacement with intensity levels 1 through 32 receiving no offset as was also indicated in Table 1.
Following development of an appropriate lookup table for offset value assignment, the program deals with the assignment of image array processor channels to the display color input and treats the four quadrant input capability of the image array processer. The program then enters a series of nested loops to perform pixel information processing. The nested loops, two of which are indicated by the statements DO 300 and DO 400, at lines 31 and 33 respectively require execution of subsequent statements, down through the labels 300 and 400 respectively, for a number of times determined by the values assumed by the variables ROWPTR and COLPTR between limits of 1 and NUMROW and 1 and NUMCOL respectively. These nested loops compute pixel element offset values and transmit information between the computer disc memory and the image array processor and are shown in FIG. 4b of the FIG. 4 flow diagram.
The actual determination of right and left offset displacement is accomplished by the IOFF=statements within the nested loops, for each scanning line of the reconstructed image. Overflow of offset images beyond the extent of the display screen is handled by the two IF IOFF statements. Resolution of possible competetion for intensities is resolved by the IF (RIGHT) statement. Smoothing of the reconstructed image data over the selected range of plus and minus 2 pixel locations is achieved by the statements following the DO 510 instruction at line 43 of the program and in FIG. 4c of the flow diagram. The transmitting of computed information for the reconstructed image to the image array processor is accomplished by the subroutine IMAGE which is called by the three statements at lines 49-51 following the smoothing operations.
In general the present invention contemplates the electronic generating of supplemental or offset image elements which are located on either side of the causing original image pixel and are displayed in separately viewable fashion to obtain a stereoscopic appearing view. This stereoscopic view is found to be more readily comprehended by human facilities as to image detail than was the original planar view of the image. The ability to use a viewer's depth perception ability which is notably sensitive and well developed in most persons has been found especially useful in analyzing complex and finely detailed features in an image such as is obtained with radar or X-ray equipment.
The disclosed embodiment of the invention employs a digital computer and associated software for embodying certain portions of the invention; these elements of the embodiment could of course be replaced with electronic hardware including ROM and RAM memory integrated circuit chips if desired. A hardware embodiment of these portions of the invention would probably increase the operating speed of the image processing operation at the expense of an increase in apparatus fabrication cost.
As disclosed in the preferred embodiment of the invention, equally displaced left and right offset pixels are contemplated except as the illustration of the offset pixels may be affected by smoothing or pixel competetion; the use of non-symetric offsets or the absence of either the left or right offset pixels could of course be readily arranged in the preferred embodiment processing and could be desirable for some uses of the invention. In particular the reconstructed image has the stereoscopic appearance of leaning in one direction where the left and right offset displacements are intentionally made non-symetric.
Another embodiment of the invention is also possible wherein the left offset pixels and the right offset pixels are displayed in monochrome photographs or monochrome displays of any of the type known in the art. An embodiment of this type would allow use of the invention by operators who are unable to distinguish colors. In this embodiment the left and right monochrome images are viewed stereoscopically in a manner known in the art; such viewing might include the apparatus of the earlier referred to patents of J. W. Knauf, U.S. Pat. No. 3,464,766 or H. J. Wehrli, U.S. Pat. No. 4,290,694, which are hereby incorporated by reference. An embodiment of this type would not require the image array processor apparatus 110 which was used in the multiple color display embodiment described above but could apply displaced pixel signal information directly to an imaging apparatus. An embodiment of this type could for example use facsimile equipment which converts directly from electronic signals to an output image without the intermediate step of displaying the image on a screen.
Another photographic related embodiment of the invention could employ color sensitive photographic film which is exposed in separate colors from a multicolor electronic display and subsequently viewed stereoptically.
Yet another embodiment of the invention is implied in the monochromatic photographic embodiment described above and involves omission of the original image features from the reconstructed image of the preferred embodiment. It has been found that presence of the original image is visually pleasing but not essential in the reconstructed image. Omission of the original image in the preferred embodiment would include omitting the blue displayed features or component and the channel 3 apparatus in the image array processor 110. The blue displayed original component is found to be useful primarily when the reconstructed image is viewed with the naked eye--without stereoscopic separation filters; the blue original component is suppressed by the red and green stereoscopic filters 152 and 150 and therefore absent in the normal stereoscopic view of the reconstructed image.
The outputs from the hardware lookup tables 124 and 126 which are shown as dotted lines in FIG. 1 can be used in another embodiment of the invention wherein the original image components are displayed in shades of gray in lieu of the preferred blue. Use of gray original image components has been found to be less desirable and lacking the degree of clarity found with the blue colored original component embodiment of the invention.
While the apparatus and method herein described constitute a preferred embodiment of the invention, it is to be understood that the invention is not limited to this precise form of apparatus or method and that changes may be made therein without departing from the scope of the invention which is defined in the appended claims. ##SPC1##
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US2865988 *||May 26, 1953||Dec 23, 1958||Itt||Quasi-stereoscopic systems|
|US2910910 *||Dec 20, 1957||Nov 3, 1959||Sperry Rand Corp Ford Instr Co||Automatic stereoplotter|
|US3464766 *||Oct 28, 1966||Sep 2, 1969||Us Interior||Stereo-image alternator system|
|US3833759 *||Jul 24, 1972||Sep 3, 1974||Agency Ind Science Techn||Electronic system for displaying artifically produced environment patterns|
|US3943561 *||Aug 6, 1973||Mar 9, 1976||Westinghouse Electric Corporation||System for optically detecting moving targets|
|US4133602 *||Nov 19, 1976||Jan 9, 1979||Ihms James E||Method for producing color structure change through interaction of light sources and application to improved apparatus for stereoscopic photography|
|US4225861 *||Dec 18, 1978||Sep 30, 1980||International Business Machines Corporation||Method and means for texture display in raster scanned color graphic|
|US4290694 *||Mar 7, 1980||Sep 22, 1981||Kern Instruments, Inc.||Photogrammetric plotting apparatus|
|US4313678 *||Sep 24, 1979||Feb 2, 1982||The United States Of America As Represented By The Secretary Of The Interior||Automated satellite mapping system (MAPSAT)|
|1||*||A Low Cost Transportable Image Proc System by J. Adams and E. Driscoll, 1st ASSP Workshop on Two Dimensional Digital Sig Proc, Oct. 3 4, 1979, Lawrence Hall of Science, Berkley, CA.|
|2||A Low Cost Transportable Image Proc System by J. Adams and E. Driscoll, 1st ASSP Workshop on Two-Dimensional Digital Sig Proc, Oct. 3-4, 1979, Lawrence Hall of Science, Berkley, CA.|
|3||*||Appl of Digital Image Proc Tech and Hdware to Regional Growth Modeling, T. Driscoll, Proc of the Am Soc of Photogrammetry vol. 1, 45 Annual Meeting, Mar. 18 24, 1979, Wash DC.|
|4||Appl of Digital Image Proc Tech and Hdware to Regional Growth Modeling, T. Driscoll, Proc of the Am Soc of Photogrammetry vol. 1, 45 Annual Meeting, Mar. 18-24, 1979, Wash DC.|
|5||*||Evolution of Image Proc Algorithms fr Sfware to Hdware by T. Driscoll & Co Walker of I 2 S. Proc of Soc of Photooptical Instrumentation Engs. vol. 271, Feb. 1981, Paper 17.|
|6||Evolution of Image Proc Algorithms fr Sfware to Hdware by T. Driscoll & Co Walker of I2 S. Proc of Soc of Photooptical Instrumentation Engs. vol. 271, Feb. 1981, Paper #17.|
|7||*||High Speed, Hdware Class of Multispectral Imagery dup in a Gen Purpose Image Display, by E. Driscoll of I 2 S, Journal of Applied Photographic Eng. Jan. 82, vol. 3.|
|8||High Speed, Hdware Class of Multispectral Imagery dup in a Gen Purpose Image Display, by E. Driscoll of I2 S, Journal of Applied Photographic Eng. Jan. 82, vol. #3.|
|9||*||New Concepts in Display Tech by J. Adams and R. Wallis, Computer Magazine, Aug. 1977, pp. 61 69.|
|10||New Concepts in Display Tech by J. Adams and R. Wallis, Computer Magazine, Aug. 1977, pp. 61-69.|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US4647965 *||May 24, 1984||Mar 3, 1987||Imsand Donald J||Picture processing system for three dimensional movies and video systems|
|US4658246 *||Aug 14, 1984||Apr 14, 1987||The United States Of America As Represented By The Secretary Of The Air Force||Matrix display simulation|
|US4723159 *||Jan 27, 1986||Feb 2, 1988||Imsand Donald J||Three dimensional television and video systems|
|US4807202 *||Apr 17, 1986||Feb 21, 1989||Allan Cherri||Visual environment simulator for mobile viewer|
|US4809065 *||Dec 1, 1986||Feb 28, 1989||Kabushiki Kaisha Toshiba||Interactive system and related method for displaying data to produce a three-dimensional image of an object|
|US4819064 *||Nov 25, 1987||Apr 4, 1989||The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration||Television monitor field shifter and an opto-electronic method for obtaining a stereo image of optimal depth resolution and reduced depth distortion on a single screen|
|US4829373 *||Aug 3, 1987||May 9, 1989||Vexcel Corporation||Stereo mensuration apparatus|
|US4841292 *||Aug 11, 1986||Jun 20, 1989||Allied-Signal Inc.||Third dimension pop up generation from a two-dimensional transformed image display|
|US4851901 *||Aug 28, 1987||Jul 25, 1989||Kabushiki Kaisha Toshiba||Stereoscopic television apparatus|
|US4994898 *||Jun 22, 1988||Feb 19, 1991||Aspex Limited||Color television system for processing signals from a television camera to produce a stereoscopic effect|
|US5034987 *||Apr 25, 1989||Jul 23, 1991||Nippon Identograph Co., Ltd.||Continuous photographing and observing of a three-dimensional image|
|US5262838 *||Feb 4, 1991||Nov 16, 1993||Vx Optronics Corp.||Apparatus and means for binocular, switchable to binocular rangefinder without light loss|
|US5491646 *||Nov 20, 1992||Feb 13, 1996||Erskine; Stephen P. E.||Production of anaglyphs|
|US5517419 *||Jul 22, 1993||May 14, 1996||Synectics Corporation||Advanced terrain mapping system|
|US5524188 *||Mar 1, 1993||Jun 4, 1996||Halpern Software, Inc.||Viewing three dimensional objects by rotational wobble about multiple axes|
|US5564000 *||Jun 7, 1995||Oct 8, 1996||Halpern Software, Inc.||Method and apparatus for viewing three dimensional objects|
|US5617332 *||Aug 10, 1988||Apr 1, 1997||Fressola; Alfred A.||Method and system for producing stereographic images of celestial objects|
|US5973700 *||Sep 16, 1992||Oct 26, 1999||Eastman Kodak Company||Method and apparatus for optimizing the resolution of images which have an apparent depth|
|US6175371 *||May 23, 1996||Jan 16, 2001||Philippe Schoulz||Process for transforming images into stereoscopic images, images and image series obtained by this process|
|US6393095||Apr 21, 2000||May 21, 2002||The Nottingham Trent University||Automatic defect detection|
|US6438260 *||Feb 4, 1994||Aug 20, 2002||The Nottingham Trent University||Visual presentation of information derived from a 3D image system|
|US7116323||Dec 10, 2002||Oct 3, 2006||In-Three, Inc.||Method of hidden surface reconstruction for creating accurate three-dimensional images converted from two-dimensional images|
|US7784107 *||Jun 1, 2007||Aug 24, 2010||Victor B. Kley||High speed measurement, analysis and imaging systems and methods for length scales from meter to sub-nanometer|
|US7907793||Aug 17, 2009||Mar 15, 2011||Legend Films Inc.||Image sequence depth enhancement system and method|
|US8073247||Nov 1, 2010||Dec 6, 2011||Legend3D, Inc.||Minimal artifact image sequence depth enhancement system and method|
|US8078006||Nov 1, 2010||Dec 13, 2011||Legend3D, Inc.||Minimal artifact image sequence depth enhancement system and method|
|US8160390||Nov 1, 2010||Apr 17, 2012||Legend3D, Inc.||Minimal artifact image sequence depth enhancement system and method|
|US8385684||Feb 17, 2011||Feb 26, 2013||Legend3D, Inc.||System and method for minimal iteration workflow for image sequence depth enhancement|
|US8396328||Oct 27, 2010||Mar 12, 2013||Legend3D, Inc.||Minimal artifact image sequence depth enhancement system and method|
|US8401336||Dec 22, 2010||Mar 19, 2013||Legend3D, Inc.||System and method for rapid image sequence depth enhancement with augmented computer-generated elements|
|US8704879||Sep 10, 2010||Apr 22, 2014||Nintendo Co., Ltd.||Eye tracking enabling 3D viewing on conventional 2D display|
|US8730232||Feb 1, 2011||May 20, 2014||Legend3D, Inc.||Director-style based 2D to 3D movie conversion system and method|
|US8776261||Aug 4, 2009||Jul 8, 2014||Victor B. Kley||Tool tips with scanning probe microscopy and/or atomic force microscopy applications|
|US9007365||Nov 27, 2012||Apr 14, 2015||Legend3D, Inc.||Line depth augmentation system and method for conversion of 2D images to 3D images|
|US9007404||Mar 15, 2013||Apr 14, 2015||Legend3D, Inc.||Tilt-based look around effect image enhancement method|
|US9098112||Mar 5, 2014||Aug 4, 2015||Nintendo Co., Ltd.||Eye tracking enabling 3D viewing on conventional 2D display|
|US9241147||May 1, 2013||Jan 19, 2016||Legend3D, Inc.||External depth map transformation method for conversion of two-dimensional images to stereoscopic images|
|US9270965||Aug 17, 2015||Feb 23, 2016||Legend 3D, Inc.||Multi-stage production pipeline system|
|US9282321||Aug 17, 2015||Mar 8, 2016||Legend3D, Inc.||3D model multi-reviewer system|
|US9286941||May 11, 2015||Mar 15, 2016||Legend3D, Inc.||Image sequence enhancement and motion picture project management system|
|US9288476||Aug 17, 2015||Mar 15, 2016||Legend3D, Inc.||System and method for real-time depth modification of stereo images of a virtual reality environment|
|US9342861||Feb 25, 2014||May 17, 2016||Deluxe 3D Llc||Alternate viewpoint rendering|
|US9407904||May 11, 2015||Aug 2, 2016||Legend3D, Inc.||Method for creating 3D virtual reality from 2D images|
|US9423693||Mar 12, 2014||Aug 23, 2016||Victor B. Kley||In-plane scanning probe microscopy tips and tools for wafers and substrates with diverse designs on one wafer or substrate|
|US9438878||Sep 17, 2015||Sep 6, 2016||Legend3D, Inc.||Method of converting 2D video to 3D video using 3D object models|
|US9443555||Feb 22, 2016||Sep 13, 2016||Legend3D, Inc.||Multi-stage production pipeline system|
|US9547937||Nov 30, 2012||Jan 17, 2017||Legend3D, Inc.||Three-dimensional annotation system and method|
|US9595296||Sep 12, 2016||Mar 14, 2017||Legend3D, Inc.||Multi-stage production pipeline system|
|US9609307||Dec 14, 2015||Mar 28, 2017||Legend3D, Inc.||Method of converting 2D video to 3D video using machine learning|
|US9615082||Mar 14, 2016||Apr 4, 2017||Legend3D, Inc.||Image sequence enhancement and motion picture project management system and method|
|US20050104878 *||Dec 10, 2002||May 19, 2005||Kaye Michael C.||Method of hidden surface reconstruction for creating accurate three-dimensional images converted from two-dimensional images|
|US20050195368 *||Mar 1, 2005||Sep 8, 2005||Bissinger Stanley T.||Three dimensional shadow projection system and method for home use|
|US20050219463 *||Mar 1, 2005||Oct 6, 2005||Bissinger Stanley T||Portable three dimensional shadow projection system and method|
|US20050219466 *||Mar 1, 2005||Oct 6, 2005||Bissinger Stanley T||Three dimensional shadow projection system and method|
|US20050231505 *||Jun 30, 2004||Oct 20, 2005||Kaye Michael C||Method for creating artifact free three-dimensional images converted from two-dimensional images|
|US20080135750 *||Jun 1, 2007||Jun 12, 2008||Kley Victor B||High Speed Measurement, Analysis and Imaging Systems and Methods For Length Scales From Meter to Sub-Nanometer|
|US20100031405 *||Aug 4, 2009||Feb 4, 2010||Kley Victor B||Tool Tips with Scanning Probe Microscopy and/or Atomic Force Microscopy Applications|
|EP2611172A1 *||Dec 28, 2011||Jul 3, 2013||Thomson Licensing||Method for generating an anaglyph image|
|WO1988004804A1 *||Dec 17, 1987||Jun 30, 1988||David Geshwind||A method to convert two-dimensional motion pictures for three-dimensional systems|
|WO1996038753A1 *||May 23, 1996||Dec 5, 1996||Philippe Schoulz||Process for transforming images into stereoscopic images, images and image series obtained by this process|
|U.S. Classification||348/44, 348/E13.037, 348/46, 345/419, 359/478|
|International Classification||H04N13/00, G01S7/06, G01S7/52|
|Cooperative Classification||H04N13/0431, G01S7/062, G01S7/52071|
|European Classification||H04N13/04G1, G01S7/06D, G01S7/52S8B4|
|Jan 4, 1984||AS||Assignment|
Owner name: UNITED STATES OF AMERICA, AS REPRESENTED BY THE SE
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST.;ASSIGNORS:KUPERMAN, GILBERT G.;WALLQUIST, DONALD;REEL/FRAME:004205/0303;SIGNING DATES FROM 19831014 TO 19831020
|Dec 16, 1986||CC||Certificate of correction|
|Jan 13, 1989||FPAY||Fee payment|
Year of fee payment: 4
|Jul 13, 1993||REMI||Maintenance fee reminder mailed|
|Dec 12, 1993||LAPS||Lapse for failure to pay maintenance fees|
|Feb 22, 1994||FP||Expired due to failure to pay maintenance fee|
Effective date: 19931212