US 3778541 A
A given scene such as that retained by a colored photograph is scanned both by a high resolution scanner that detects image density gradients and a color analyzer with lower resolution. Controlled by the gradient detecting scanner is a printer that produces, on an appropriate work surface, a line drawing delineating prominent object boundaries in the image. The color information extracted from discrete areas of the image by the color analyzer is compared by a computer with colors available in a preselected finite set of distinctly colored substances and an appropriate selection is made from the set for each area analyzed. In response to the computer selections, a color indicia printer produces on the work surface a color outline of surface areas geometrically corresponding in position to the discrete areas of the image. The color outline is superimposed on the line drawing but is distinguishable therefrom. Also produced by the color printer within each surface area is indicia representing the particular colored substance selected for application thereto.
Description (OCR text may contain errors)
United States Patent [1 1 Bowker SYSTEM FOR ANALYZING MULTICOLORED SCENES  Inventor: John Kent Bowker, Marblehead,
 Assignee: ltek Corporation, Lexington, Mass.
 Filed: Sept. 3, 1971  Appl. No.: 181,141
 US Cl. 178/51 R, 178/5.2 A  Int. Cl. H04n 1/22  Field of Search l78/5.2 A, 5.2 D, 178/52 R, 6.6 B, DIG. 34, 6.7 A, 6.7 R, DIG. 28; 355/38.4l; 356/175-178, 2
 References Cited UNITED STATES PATENTS 2,972,012 2/l961 Farber l78/5.2 A 3,100,815 8/1963 Drake et al. 3,181,987 5/1965 Polevitzky l78/5.2 R 2,799,722 7/1957 Deugebauer l78/5.2 A
Primary ExaminerRobert L. Griffin Assistant ExaminerJohn C. Martin Attorneyl'lomer 0. Blair et a1.
 ABSTRACT A given scene such as that retained by a colored photograph is scanned both by a high resolution scanner that detects image density gradients and a color analya! as: K as 1 Dec. 11, 1973 zer with lower resolution. Controlled by the gradient detecting scanner is a printer that produces, on an appropriate work surface, a line drawing delineating prominent object boundaries in the image. The color information extracted from discrete areas of the image by the color analyzer is compared by a computer with colors available in a preselected finite set of distinctly colored substances and an appropriate selection is made from the set for each area analyzed. In response to the computer selections, a color indicia printer produces on the work surface a color outline of surface areas geometrically corresponding in position to the discrete areas of the image. The color outline is superimposed on the line drawing but is distinguishable therefrom. Also produced by the color printer within each surface area is indicia representing the particular colored substance selected for application thereto.
In a preferred embodiment, the density gradient and color analyzer scans are made simultaneously in parallel adjacent paths across the image. The line printer responds instantaneously to the output of the high resolution gradient detector. The color analyzer, however, supplies information at intervals along the scan and the color indicia printer prints on each scan line information from the preceding scan line and in a position corresponding to the area from which the information was obtained. The delay in color indicia printout provides time for color evaluation by the computer.
27 Claims, 23 Drawing Figures SPECIMEN SCANNER STANDARDlZER GRADlANT DETECTOR a4 e26 PRINTOUT m ZONE DETECTOR PATENTED 1 3. 778.541
sum 02 or 16 15282931303: John Kent Boll/9993 a fltioraeg;
PAIENTEDUEE 1 1 ms 3378541 mm 11 or 16 GREEN G I weniow:
Jam .KQZIZBOWM 21,9 E. "'wv PMENIEUBH: 1 1 ms SHEET 12 HF 16 v 122191522303 John KMBoww,
d1 z' zzeg l SYSTEM FOR ANALYZING MULTICOLORED SCENES BACKGROUND OF THE INVENTION This invention relates generally to an automatic system for independently analyzing and recording both image detail boundaries and color distribution in particular multi-colored scenes. The system of the present invention is particularly well suited for producing diagrammed work surfaces on which relatively unskilled persons can create renderings of an originalscene such as one first recorded on photographic film.
There are available commercially various types of products designed to assist a user in the creation of an artistic rendering. Such products include, for example, fabrics imprinted with designs used during application of decorative stitching and other needlework, imprinted diagrams used during assembly of ceramic mosaics, various types of paint receiving surfaces bearing individually designated color outlines to be followed during the application of oil paints or water colors, etc. One of the best known of the'foregoing product types involves the so-called paint-by-number techniques for creating oil paintings. According to this technique, a popular oil painting masterpiece is used as a model by a commercial artist who generates what might be described as a color contour diagram of the original. Such a contour diagram outlines a plurality of individual areas each bearing a designation for a particular colored paint to be applied thereto. The various distinctly colored paints required are suppliedas a palette with the color contour diagram. The paint colors-provided in a given palette are determined by the commercial artist who attempts to select for each of the designated areas a paint color corresponding as nearly as possible to the color present in the corresponding area of the original painting. Generally, to minimize cost andreduce the intricacy of the color contour diagram, a limited number of individual colors is provided, typically between and 30.
Because of the substantial human effort requiredto generate a color contour diagram and'to select an appropriate paint palette for use therewith, the varietyof original paintings available in paint-by-number form ismacy in end results obviously would enhance market potential. Both of these objectives would accrue if a customer could select for rendering-any scene with which he is intimately connected and which had been previously recorded on photographic film. For example, a much larger segment of the public would be interested in creating an original oil painting based on a colored photograph of a relative, a close friend, a familiar landscape, an admired architectural object such as ones own home, etc.
Thus, one problem presented above was to provide a set of premixed pigment colors that could be used to create a tonally correct and harmonious oil painting of any preselected photograph. A solution to the problem, however, was not available with the conventionalcolor reproduction techniques employed, for example, in the fields of photography, color television. and printing. Color reproduction systems in these fields rely on Newtonian theory that sets forth the generalization that all colors can be defined in tern 1 of fixed primary colors R, G and B. A specific color 0 is then definedgs a veg; tor in three-dimensional space equal to rR gG plus bB where the values r, g and b are the tristimulus values of the color with respect to the particular set of primary colors R, G and B. A color to be reproduced is first spectrally analyzed by a suitable device such as a color television camera, to determine the component values r, g and b. These values are then used to selectively control the proportionalities of primary color light sources used to reproduce the color. The reproduction can entail an additive process in which appropriate values of the three primary colors, such as the commonly used red, green and blue, are added or a subtractive system in which a tri-color set such as cyan (minus red magenta (minus green) and yellow (minus blue) absorbs desired amounts of incoming primary colors. Color television, for example, is strictly an additive process in which red, green and blue phosphors are selectively activated to produce a desired color while color photography is a subtractive process in which appropriate thicknesses of color layers subtract light from incident white light to produce the desired color in either transmitted or reflected light. In all such analytical color reproduction systems, however, a substantially infinite variation of the reproduction color stimuli is available to reproduce the measured tristimulus values of the original color. Thus, the reproduced color comprises appropriate values of each of the primary colors that synthesize the color desired.
ltwill be apparent that these conventional color reproduction techniques are not applicable to the present problem in which color selections must be made from a palette consisting of premixed paints. The color space represented by such a palette is similar to the digitized hyperspace of n dimensions common to object recognition, and is quite different from conventional analytic color space.
In addition, classifying all areas of the photograph as being one of the available colors does not convey sufficient information to the artist to permit him to complete the rendering. Sharp detail boundaries in the input photograph should be distinguished from gradual color transitions if these characteristics are to be recreated in the rendering. For example, in creating the rendition of a spherical object such as an apple, the various colors applied to the object should be blended rather than applied so as to define sharp boundaries. Conversely, a sharp discontinuity such as would appear between the object and a different colored background should appear also as a sharp boundary in the rendition. Therefore, the work surface provided should distinguish between sharp detail boundaries and gradual transitions so as to apprise the artist of where and when not to blend the applied paints.
The object of this invention, therefore, is to provide an automatic system for producing sets of diagrammed paint boards and associated palettes that can be used to generate tonally correct and harmonious oil paint renderings of original scenes as retained, for example, by color photographs.
SUMMARY OF THE INVENTION The present invention is characterized by an automatic system for producing a diagrammed work surface that both illustrates sharp detail boundaries and identifies gradual color transitions in a given multi-colored scene. In a specific application, distinctly colored substances are then selectively applied to the work surface to create an artistic rendering of the original scene. Preferably, a colored photograph is used as a basis for producing the work surface retaining diagrams that define general areas on which distinctly colored paints can be applied to create a rendering of the picture imaged on the photograph. According to the invention there are selected and identified, in a given colorimetric system, the boundaries of color domains corresponding generally to color tonalities of the photograph. A distinctly colored oil paint is then provided to represent each of the color domains and each paint color is given an identifying designation. Scanning through discrete portions of the photograph with a color analyzer establishes in the given colorimetric system the coordinate positions of the colors present in each of the portions scanned. Next, a computer search is made to determine which particular one of the selected color domains encompasses the color coordinate position of each of the analyzed colors in the photograph. Finally, discrete zones on the paint receiving work surface that correspond geometrically in position to the discrete portions scanned in the photograph are located and there is applied by a printout mechanism to each work surface zone the designation for that paint color representing the color domain that encompasses the color coordinate position of the analyzed color in the geometrically corresponding portion of the photograph. A gradient detector scans in synchronism with the color analyzer and detects outlines of distinguishable objects present in the photograph. The detected object boundary outlines are superimposed by the printout mechanism on the color designated zones of the work surface. Together, the zone and boundary outlines guide the artist during the application of paints to the work surface.
According to a preferred embodiment of the invention, the above described step of determining which color domain encompasses the color coordinate position of each analyzed color entails the prior step of selecting in the given colorimetric system a plurality of particular color coordinate points such that planes established by other points equally spaced from the particular selected color coordinate points define the boundaries of the color domains. A comparison is then made by computer to determine which of the particular selected color coordinate points is nearest the coordinate position of each analyzed color from the photograph. Because of the above noted coordinate point selection method, the nearest particular point lies in and thereby establishes the color domain encompassing the color coordinate position of each analyzed color. This method of coordinate point comparison permits the location of an appropriate domain for each analyzed color with conventional computer memory techniques.
One feature of the invention is the use of a threedimensional tristimulus colorimetric system in the methods described above. Utilization of the tristimulus colorimetric system facilitates a determination of the color coordinate positions of colors in the photograph in that conventional primary color analyzer systems can be employed to analyze the photographs. According to a preferred embodiment of the invention, however, the original selection of color domains is first made in a three-dimensional polar coordinate colorimetric system utilizing hue, saturation and lightness as color components. Such a colorimetric system is more perceptible psychologically and therefore simplifies the selection of appropriate color domains. Once selected, the coordinates defining boundaries of the selected color domains are mathematically transformed into equivalent coordinates of the tristimulus colorimetric system desired for analysis of the photograph.
According to the featured embodiment of the system described above, the step of selecting a plurality of color domains and then providing a palette consisting of a distinctly colored paint for each domain entails the selection of a plurality of sets of color domains and a corresponding palette for each. The tonal variations in each selected color domain and corresponding palette set are unique. For example, one set might approximate the tonalities present in a photographic subject of light skin and blond hair while another set might correspond to the tonalities present in a photograph of a subject with dark skin and black hair. The particular set of color domains and corresponding palette used in the above described system is then selected from this plurality of sets after a comparison thereof with the particular photograph to be rendered. In this way individual palettes, each composed of a relatively small number of distinctly colored paints, can be employed to produce harmonious and relatively tonally correct renderings of photographs with widely different tonal representation. Another feature of the invention is a synchronization system that facilitates simultaneous and synchronized scans of the photograph by the color analyzer and the gradient detector and of the work surface by the printout mechanism. Preferably, a rotatably reciprocating two-sided mirror is used as a synchronizer. The photograph is illuminated and the analyzing scanning beam is generated as the reciprocating mirror reflects light from different areas of the photograph onto a focusing lens. The work surface is generated on a sheet of photosensitive material by a printout beam reflected from the other side of the mirror. This insures both spacial and temporal synchronization of the input and output, provided the delay between scanning and printout is small. Another feature of the invention is the use of distinct and separately modulated printout beams for recording the information derived by the gradient detector and the color analyzer. The separate printout beams are complemented by a synchronized delay system that produces intermittent color sampling by the color analyzer and subsequent printout in an interlace pattern. Since the gradient detector is a substantially instantaneously responsive device, the image detail boundary information detected thereby can be printed out continuously on the work surface by the synchronized writing beam output of the two-sided mirror, as described above. However, analyzation and identification of the various color zones by the color analyzer requires several milliseconds, and scan rates slow enough to accommodate immediate printout of color identification information would seriously limit the output capacity of the system. Therefore, after any zone is sampled and identified, the color information concerning that zone is placed in a memory of an interlace control system. Subsequently, during a following scan, when the position of the color output scanning beam corresponds with the position of the aforementioned zone, the interlace system retrieves the information in the memory and applies it to the printout mechanism. The path of the color printout beam is displaced with respect to the boundary printout beam to compensate for the spacing between successive scans.
Another feature that improves the efficiency of the system is a resolution control that establishes different resolutions for the gradient boundary ancl color zone detectors. In a preferred embodiment the gradient detecor is provided with substantially greater resolution than the zone detector. Detail boundaries are therefore detected from small and closely spaced sampling areas as compared to those used for color analysis. Preferably, the zone detector samples anarea of the magnitude of 100 times the size of that sampled by the gradient detector. This relationship provides the high resolution desired for detail boundary detection without seriously limiting scanning speed by burdening the zone detector with an excessive number of computations. In this connection, it should be realized that reduced resolution in the zone detector does not degrade the ultimate performance of the system. According to conventional techniques, oil paints of different colors are substantially blended near boundaries to create desired results rather than being applied to distinct areas. Such blending techniques would tend to negate the effect of a high resolution zone detector.
Still another feature of this invention is the provision of a standardizer that operates to produce input scenes of uniform size and tone. Standardization. of the input scene enhances the speed capability of the system. Preferably, all input scenes are reproduced in a given format such as on' a '70 millimeter transparency strip. This technique accommodates a simultaneous recording of auxiliary information useful in the overall process. For example, fiducial, or code marks are recorded to provide control signals for the computers employed in the system. In addition to the obvious benefit of properly cropping each input image while photographing, the standardizer compensates for variations in quality and color balance in the input prints with corrective filters. Also, the system is simplified in that scanning can be controlled by the fiducial marks rather than computations such as scan counting. Printout on a continuous roll is also facilitated permitting the size and shape of the prints to be altered without any change in the system.
DESCRIPTION OF THE DRAWINGS These and other objects and features of the present invention will become more apparent upon a perusal of the following description taken in conjunction with the accompanying drawings wherein:
FIG. 1 shows a basic block diagram of a system for analyzing multi-colored scenes;
FIG. 2 shows a sample input specimen;
FIG. 3 shows an output printobtained by the opera tion of the system shown in FIG. 1 on the specimen shown in FIG. 2;
FIG. 4 shows a preferred physical layout for the optical components of the system shown in FIG. 1;
FIG. 5 shows a block diagram of the system shown in FIG. 1;
FIG. 6 shows the format of the transparency strip used to record the input specimen; 7
FIG. 7 shows a preferred embodiment. of a camera used to produce the transparencystrip shown in FIG.
FIG. 8 isa schematic diagram of'the viewing system used in the camera shown in FIG. 7;
FIG. 9 shows a preferred operator control panel;
FIG. 10 shows the scan position control used in the embodiment shown in FIG. 1;
FIG. 11 shows the resolution control aperture and associated photodetectors used in the gradient detector in the embodiment shown in FIG. 1;
FIG. 12 is a block diagram of the gradient detector;
FIG. 13' is a munsel color diagram;
FIG. 14 shows a constant value plane of the munsel diagram that has been divided into distinct color domains;
FIG. 15 is a three-dimensional color diagram comprised ofa plurality of constant value planes such as the one shown in FIG. l4;
FIG. 16 is a schematic diagram of the zone detector used in the embodiment shown in FIG. 1;
FIG. 17 shows the second radiant energy source used in the embodiment shown in FIG. 1;
FIG. 18 shows the support used in the radiant energy source used in FIG. 17;
FIG. 19 shows the aperture mask used in the radiant energy source shown in FIG. 17;
FIG. 20 shows a color block and associated circuitry used in the embodiment shown in FIG. 1;
FIG. 21 shows waveforms present within the circuitry shown in FIG. 20;
FIG. 22 shows an interlace pattern used in the embodiment shown in FIG. 1; and
FIG. 23 shows a typical palette that is supplied with the printout guide.
DESCRIPTION OF THE PREFERRED EMBODIMENTS Referring first to FIG. 1 there is shown a basic block diagram of a preferred system 21 for automatically analyzing multicolor scenes. A multicolor scene (not shown) is photographed by a standardizer 22. A plurality of scenes or input specimens are used and the standardizer 22 produces a strip of mm transparencies from the plurality of input specimens. This film strip (not shown) is placed in a scanner 23 that detects from the images depicted thereon certain boundary detail and color information as described below. An output 24 of the scanner 23 (a beam of light) is fed into a gradient detector 25 and a zone detector 26'. The zone detector comprises a color analyzer 26 and a color comparator 30. The aforementioned boundary detail information is analyzed in the gradient detector 25 and the color information is analyzed in the zone detector 26'. A printout scanner 29 receives the boundary detail information on a gradient detector output line 27 and the decoded color information on a zone detector output line 31.
The input specimens are multicolor photographic prints and transparencies. The output print is a sheet of photosensitive material marked with suitable indicia to serve as a guide to assist an artist in creating an original rendition of the image portrayed on the input specimen.
In order that the following detailed description of the system 21 be best understood, it is important that the objectives and functions of the system be known. For this purpose asample input specimen and a sample output print will be examined. Attention is directed to FIG. 2, which shows a reproduction of a sample input specimen 33, that specimen being a photograph ofa still life comprised ofa pear 34 on a plate 35, and to