US 3925776 A
A system of display terminals each receiving from an image generator edges which commonly represent a selected two-dimensional projection of three-dimensional objects. Each edge is defined by its intended position on a two-dimensional screen and by attributes for a portion of the screen adjacent the edge. These attributes may include color, brightness, shading and the like. Each display terminal has an edge memory which stores a set of edges ordered by the y-coordinate of their tops and are partially x-ordered. The stored edges are decoded into scanline segments, each segment being the scanline interval between two adjacent edges intersecting that scanline. Each segment is defined by its intended position on the display screen and by its other attributes such as color brightness. The segments are subsequently decoded into video raster points, each point being defined by its position on the display screen and by attributes such as brightness and color. Since only edges are transmitted from the image generator to the terminals, as opposed to a video raster, a relatively low capacity communication link can be used. Since edges are buffered at each display terminal, rather than video raster data, the edge memories inside the terminals need not be excessively large. The conversion of edges to segments and of segments to video raster is fast and efficient, and allows flickerless display of complex color images.
Description (OCR text may contain errors)
' United States Patent Swallow 5] Dec. 9, 1975 DISPLAY TERMINAL SYSTEM image generator edges which commonly represent a 75 I t I R al selected two-dimensional pro ection of threenven or on d J Swallow Upper dimensional objects. Each edge 18 defined by its in- Marlboro, Md.
tended position on a two-dimensional screen and by  A lgn R sear rporation, New York, attributes for a portion of the screen adjacent the N.Y. Filed: Apr. 18, 1974 Appl. No.: 462,171
Primary ExaminerMarshall M. Curtis Attorney, Agent, or FirmCooper, Dunham, Clark, Griffin & Moran  ABSTRACT A system of display terminals each receiving from an edge. These attributes may include color, brightness, shading and the like. Each display terminal has an edge memory which stores a set of edges ordered by the y-coordinate of their tops and are partially xordered. The stored edges are decoded into scanline segments, each segment being the scanline interval between two adjacent edges intersecting that scanline. Each segment is defined by its intended position on the display screen and by its other attributes such as color brightness. The segments are subsequently decoded into video raster points, each point being defined by its position on the display screen and by attributes such as brightness and color. Since only edges are transmitted from the image generator to the terminals, as opposed to a video raster, a relatively low capacity communication link can be used. Since edges are buffered at each display tenninal, rather than video raster data, the edge memories inside the terminals need not be excessively large. The conversion of edges to segments and of segments to video raster is fast and efficient, and allows flickerless display of complex color images.
36 Claims, 21 Drawing Figures comm M4 7/0 Sheet 8 of 15 3,925,776
BLOCK L BLOCK L U.S. Patent Dec. 9, 1975 BLOC/V514,-
BLOCK L-/ I in E army PORT/0N 0F scmv LJA/E DECODING BVCLE U.S. Patent Dec. 9, 1975 Sheetllof 15 3,925,776
CRY Ab (MAG-l-S/GU) c v (ram/a4) P/PE CLOCK A X SHIFT 0,1022 0)? SELECT KA R0 M (256 wmos) I GATE IVE 7W URK A X meager/0N PEG/$75 e0 54 GATES GATE 6? v v *1 1 L v fi, PC AX LTF r00 BIT Ab F c v vuL (1 BIT) mrzas/e V T0 LEVEL. 6
Tim 1 BLOCK D/AGRAM OF LEVEL 5 U.S. Patent Dec.9, 1975 Sheet 14 of 15 3,925,776
OUT/ 07 come/15f Vounes SUPPLV Tiqla. AA T/ L06. AMPL,
DISPLAY TERMINAL SYSTEM BACKGROUND OF THE INVENTION The invention is in the field of computer graphics and relates specifically to a system of display terminals for displaying graphical information received from a central image generator.
In computer graphics, a representation of a generally three-dimensional world is stored in a memory, and selected two-dimensional projections of selected portions ing as a memory and storing a selected set of two-dimensional views of a three-dimensional world, combined with a television set. A more complex example is a specially programmed digital computer system which stores a three-dimensional object such as a cube by identifying the coordinates of its edges, generates selected two-dimensional projections of the three-dimensional object and the coordinates of the lines making up the projection, and either transmits the coordinates of the lines representing the projection to a stroke type display device or converts the line coordinates of the projection to a set of raster points and transmits that set to a raster display device such as a television receiver.
A survey of computer graphic techniques may be found in Sutherland, I.E., A Characterization of Ten Hidden Surface Algorithms, ACM Computing Surveys, Volume 6, No. 1, March 1974, pages l55; and in the references listed at page 45 of the article, and particularly in Newman, W.M. et al., Principles of Interactive Computer Graphics, Mc-Graw-Hill, 1973.
A major factor which has prevented widespread use of computer graphics has been the cost of storing great amounts of information and of transmitting information to the display devices at a high rate. For example, if the purpose of a system is to show on a display surface any selected view of a three-dimensional object such as a cube, it is theoretically possible to store a nearly infinite number of views on a video tape and to find and display a selected one on a television screen, but this would be prohibitively expensive. A great reduction in the amount of stored information results when a three-dimensional object is stored in computer memory not as it looks in a specific two-dimensional view but as it actually is in three-dimensions, e.g., by storing the three-dimensional coordinates of the apices or the edges of a cube. This three-dimensional information can be computer-processed to generate almost any perspective view of the three-dimensional object, to thereby reduce storage costs as compared to a video tape storage. The remaining question is then how to transmit a representation of the two dimensional view of the object to a display device and how to display it. The answer to this question must take into account and reconcile a number of conflicting factors, such as: the desirability of a low transmission rate so as to avoid expensive communication links between the central computer and the display device, the desirability of fast operation of the display device so as to be able to show a complex image without flicker and the desirability of having minimal storage at the display device so as to minimize cost.
In various approaches to reconciling these conflicting factors, some prior art systems use stroke-type display 2 devices (IBM 2250 and Tectronics) which form lines by random positioning and stroking of the CRT beam, others use video gating overaTV raster (Anagraph) and still others use random point plotting (Plasma). .All
these types of display devices refresh the display either by repeated image generation from encoded form such as lines (IBM 2250), by reading a video storage device such as a video disc (Anagraph) or shift register memory (TICCIT), by reading a storage tube (Tectronics) or by use of a special memory display panel (Plasma).
While the stroke type display devices afford relatively low transmission rate betweenthe images generator and the display device, sinceline identification data is transmitted and not video raster, these devices cannot display complex images because of limited stroking rates. While the other types of display devices discussed immediately above can theoretically display complex images, they are severely limited in resolution because of the high cost of locally storing each point of an image and becauseof the required high transmission rate.
There are only two systems known to applicant which use point display devices and are'capable of displaying area graphics where arbitrary shapes or surfaces can be simulated: the system developed by the Evans and Sutherland Computer Corporation and the system de veloped by GB. for pilot simulator-like application.
Both are systems in which the terminal displaydevice includes a special purpose image generator computer. Both systems generate colored perspective views from three-dimensional descriptions of objects, but both are limited to one display device since the output of the image generator is a color video raster to a color monitor. Modification of these two systems by the use of a blackboard memory in the display monitor, in order to share the image generator for several users, is uneconomical because of the high cost of buffering colored video raster data. Additionally, both of these two'systerns are limited in speed, only one picture per second being possible.
A need remained therefore, prior to this invention,
for a graphics display system using a relatively low transmission rate between the image generator and the display terminals, capable of supporting a number of display terminals by the same image generator, needing minimal storage of data at the display terminal and capable of producing flickerless display of complex images, all this at a relatively low price per terminal;
SUMMARY OF THE INVENTION The invention is in the field of computer graphics and relates specifically to a system of display terminals associated with an image generator providing information such as definitions of selected two-dimensional projections of a three-dimensional world.
The invented system each identified the design goals of relatively low transmission rate between the image generator and thedisplay terminals, of relatively small storage capacity at the display terminals and of flickerless display of complex images by (l) transmitting to the display terminals pictorial information coded as visible two-dimensional edges each indentified by its intended position on a two-dimensional display screen and by desired attributes of a portion of the display screen associated with the edge and extending adjacent the edge, such as color, brightness, shading and the like, thus affording relatively low transmission ratebetween the image generator and the display terminals,
l/30th t2) buffering the edges at the display terminal. rather than buffering video raster information, so as to need relatively low storage capacity at the terminals, and (3) providing fast but relatively inexpensive translation of edges to video raster, so that complex images can be displayed with no flicker and at low cost.
Each display terminal includes an edge memory for storing a selected set of edges which are y-ordered and x-ordered in a specific manner to facilitate their conversion to video raster. The edges may be divided into blocks, each block including all edges that have portions within a selected span of the screen y-coordinate. Then, the memory may be divided into a small section, of sufficient size to store a complete block of edges, and a large section storing the remaining blocks of edges. In edge decoding, only the small section of the memory has to be accessed for any given scanline for as long as that scanline belongs to the span of the block currently residing in the small section in memory.
Each edge defines a continuous line on the screen and selected attributes for an associated portion of the screen. The line is continuous, as opposed to a line defined by series of raster point, and it is a straight line in the disclosed embodiment of the invention, although the invented principles would apply to a curved line as well. The screen portion associated with an edge is the screen area adjacent the edge and extending to the right of it up to any other edges, but the invented principles apply to other relationships between an edge line and an associated screen area, such as a screen portion to the left of an edge line. The screen portion associated with an edge is a continuous area, rather than an area defined by a collection of raster points.
Additionally, each edge defines other attributes of the edge line and of the associated screen portion, such as brightness, color, brightness gradient and brightness variation. These attributes provide the invented system with the capability of displaying realistic color and realistic shading so that three-dimensional objects can be simulated on the two-dimensional display screen. For example, the edge defines the desired brightness at the top of the edge line, the color to the right of the edge line, the gradient of brightness downward along the edge line, a brightness variation flag which if set means that the color of a screen portion between two edges will vary linearly, and a flag which if set means that this is the first edge of a set.
The conversion of edges to video raster takes place in two steps: first the edges are converted to scanline segments, each segment being the portion of a scanline disposed between the intersections of that scanline by edge lines or between an edge line and the end of the screen, each segment being identified by its position on the screen as by the attributes of the screen portion with which it coincides; and second, the scanline segments are converted to a video raster. The'conversion of edges to segments takes place in a pipeline decoder whose input is the word defining an edge and whose output is a word defining the position of the segment and the attributes of that segment. Several of the segment defining words output from the edge-to-segment decoder are buffered in a ping-pong buffer, and the buffer output is converted into a video raster which is in color, at a resolution several times that of a conventional TV receiver, and may include smooth shading for more realistic depiction of three-dimensional information on a two-dimensional screen. The output of the segment-to-video decoder is applied to a commercial color TV receiver modified in accordance with the invention to operate at a higher resolution and to accept the specifically formatted output of the segment-tovideo decoder.
A major aspect of the invented display terminals is that they receive and store edges, rather than video raster, but display a video raster rather than strokes. By this feature, the invented display terminal combines the desirable goals of relatively low need for storage capacity at the display terminal and the capability of displaying flickerless complex images. An additional major aspect of the invented display terminals is that the edges stored at the display terminal are y-ordered and xordered, so that no sorting needs to be done as the edges are converted to video raster. Still another major aspect of the invention is the blocking of edges, which reduces the time necessary for cycling the edge memory. Still another major aspect of the invention is that it allows for flexibity in defining edges (for example, it is possible under the principles of the invention to define curved edge lines and to define associated screen areas which bear different relationship to the defined edge line). Still another major aspect of the invention is the provision for smooth shading of the displayed information, to provide a more realistic appearance of simulated three dimensional objects. Other aspects of the invention are discussed in the detailed description of the invention and are brought out in the appended claims.
BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram of the invented system of display terminals, as connected to a central image generator through a communication controller.
FIG. 2 illustrates an edge definition word.
FIG. 3 illustrates the manner of defining an edge by its position on a display screen and by the color and other attributes to the right of the edge until a next edge is encountered, as used in this invention.
FIG. 4 is a block diagram of an edge memory forming a part of each display terminal.
FIG. 5 is a more detailed block diagram of one bank of the memory shown in FIG. 4.
FIG. 6 illustrates the sorting of a set of edges into blocks of edges in accordance with the invention.
FIGS. 7, 8, 9, and 10 illustrate the contents of the memory shown in FIG. 4 during different stages of the operation thereof.
FIG. 11 is a generalized block diagram of an edge-tosegment decoder.
FIGS. 12, 13, 14 and 15 are more detailed block diagrams of portions of the decoder shown in FIG. 11.
FIG. 16 is a block diagram of a segment buffer receiving the output of the edge-to-segment decoder of FIG. 11. 7
FIG. 17 is a partly block and partly circuit diagram of a segment-to-video decoder.
FIG. 18 is a block diagram of a TV receiver modified in accordance with the invention.
FIG. 19 is a circuit diagram of a portion of an antilog amplifier as used in the embodiment of FIG. 18.
FIGS. 20 and 21 are timing diagrams illustrating the operation of the invention.
DETAILED DESCRIPTION Overview FIGS. 1, 2, and 3 Referring to FIG. 1, an image generator generates edges each defining the intended location on a two-dimensional display screen of a line that typically represents a portion of the two-dimensional projection of a three-dimensional object, as well as desired attributes of a portion of the display screen associated with the edge and extending adjacent the edge.
Referring to FIGS. 2 and 3, each edge may be defined by a 76-bit word X,Y,H,S,B,C,G,F,E where:
a. X x, an ll bit x-coordinate of the edges top;
b. Y y, an 11 bit y-coordinate of the edges top;
0. H y" y, an l1 bit height, where y is an 1 1 bit I y-coordinate of the edges bottom;
d. S (x x) 2 /(y" y), a 13 bit signed integer (12 bit integer plus 1 bit sign) representing the slope where x" is an 11 bit x-coordinate of the edges bottom and where M is the number of consecutive leading zeros from the left in the l 1 bit H up to 10 zeros;
e. B b, where b is the 7 bit brightness (log) at the top of the edge;
f. C 0, color to the right made up of a 4 bit red, a 4 bit blue, and a 4 bit green component (each a log function);
g. G (b b) 2 /(yy'), a 9 bit signed integer representing the gradient of brightness defined downward along the edge where b" is the 7 bit brightness (log) at the bottom of the edge;
h. F, a I bit flag which, if zero, implies that the brightness to the right is to vary linearly (in the log domain) toward the value of the next edge;
i. E, a l bit flag marking the beginning of a set of edges in memory.
Edges may be generated by a system similar to that utilized in the computer graphics system of the Evans and Sutherland Computer Corporation, but since the image generator does not form a part of this invention, it may be assumed for simplicity that the image generator 10 is a storage device, such as a drum memory or a disc memory, storing a plurality of edges each defined by a 76-bit word of the type shown in FIG. 2, the collection of stored edges representing a desired two-dimensional image, such as a selected two-dimensional projection of a three-dimensional world. The edges stored in the image generator 10 are only those which should be visible on the display screen, and they are x-ordered and y-ordered according to rules discussed in detail below. Additionally the generator 10 stores control information of the type discussed in detail below.
A plurality of edges of the type shown in FIG. 2 together with eight bits of control information per edge, making up a total of 84 bits per edge, are transmitted via a communication link 11, which may be a coaxial cable of sufficient capacity, to a communication controller 12 comprising a receiver 14, a timing generator 16 and a terminal load control 18. The communication controller 12 serves a number of display terminals 1, 2 R. The receiver 14 is a serial-to parallel converter for the edges and control information transmitted from the image generator 10, the timing generator 16 provides the necessary timing and control signals for transmission to each of the display terminals via a timing bus 20, and the terminal load control 18 decodes the edge data and control data from the coax receiver 14 and ap- 5 plurality of memory banks and a control for the memory banks, and temporarily stores a selected plurality of edges that are x-ordered and y-ordered in a selected manner; a decoder 26, which comprises an edge-to-segment decoder 28, a segment buffer 30 and a segmentto-video decoder 32; and a display monitor 34, which may be in color. The purpose of the decoder 26 is to convert the edges from the memory 24 into color video raster for driving the display monitor 34, and it carries out the conversion by decoding the edges to segments of scanlines, buffering the segments and converting the buffered segments to a color video raster.
Because the edges are a highly compressed representation of pictorial information, the communication link 11 can be of a relatively low capacity, and the same image generator 10 can serve up to hundreds of display terminals, unlike the prior art known to applicant where each terminal must have its own image genera tor. Because the memory 24 of each terminal stores edges rather than video raster, and because of the memory 24 is organized in a novel and efficient manner, it can provide sufficient information to the display monitor 34 by storing only a few thousand edges. Because the memory 24 outputs for each scanline of the display monitor 34 only the edges that intersect it, and because of the novel and efficient organization of the decoder 26, the decoder can operate with sufficient speed to ensure flickerless display at the monitor 34 of complex images, and it can operate with a segment buffer 28 comprising only a few shift registers.
It is noted that a polygon of arbitrary shape and color can be represented by edges in accordance with this invention. Because the brightness may very linearly along the edges bounding the polygon and linearly in x between edges, a polygon can represent a portion of a curved surface. By means of such polygons, both flat and curved surfaces can be represented in any illumination environment, limited only by the capacity of the edge memory 24 and by the number of bits used to define an edge. For a planar surface and a point light source at infinity, the edges bounding the left side of a polygon require F to be 1 and G to be 0. The result is a polygon of arbitrary but constant brightness and color. The F, B and G of the edges bounding the right side of this type of a polygon do not influence the brightness within the polygon. Planar surfaces. which are illuminated by neighboring light sources require a varying shading and thus fall into the category of curved shaded surfaces. The brightness within the polygon is a piecewise linear function of that at the comers of the polygon because of the way brightness is defined along an edge at scanline y by:
b(y) (y Y) G 2"' b (Equation which reduces to My) (y (,v" y) B (Equation 2) shaded surface is defined by 7 where .x and x are the x intercepts of the two edges and where the form of the equation is that of a linear interpolation.
The edges on the left of curved shaded polygons have their F set to zero. The edges on the right of these polygons must, of course, have the appropriate Gs and Bs. However, their Fs depend upon the nature of the surface to their right. If a discontinuity in brightness is required to the right of a curved shaded surface, the edges on the right of the shaded polygon must be followed by another set of parallel edges which redefine the brightness and color further to the right.
Aside from the obvious cost advantage afforded by the data compression due to edge encoding, there is a second advantage related to image generation. The generation of 2-D perspectives out of 3-D objects requires planar approximations to curved surfaces so as to allow low cost, high speed visible surface calculations. The perspectives of these planar surfaces are polygons which are conveniently represented by edges. Also, the brightness of the 3-D objects is generated only for the comers of the facets representing curved 3-D surfaces where linear interpolations in the perspective domain are assumed for all other portions of the surfaces. This interpolation is identical to that used at the display terminal according to this invention, permitting a perfect match between image generation and display terminal decoding.
Overview of Operation (FIGS. 1-3) Prior to operating the disclosed system of display devices, the image generator must be able to provide one or more sets of edges and control data. Although a specially programmed computer system can be used to provide the necessary data, since the image generator does not form a part of this invention, it can be assumed for the purposes of this disclosure that the image generator 10 is a memory (a drum or disc memory) that is loaded through a keyboard (through an intermediate tape storage, if necessary) with the required data. The required data includes one or more sets of edges each defined by a word of the type illustrated in FIG. 2, and each accompanied by several bits of control information, e.g. 8 bits. Each set of edges in the image generator 10 would normally represent a two-dimensional projection of a three-dimensional object, but this need not be the case, and the set of edges may represent any suitable pictorial information. The edges of a set are yordered and x-ordered as described in detail later in this specification. Further, the edges are blocked, again as described in detail later in this specification, and include marker edges defining the blocks of edges and include null edges, as necessary. The several bits of control information accompanying in each edge definition word indicate to the terminal load control 18 which of the several display terminals is to receive a specific set of edges, which of the edge defining words already stored in a specific memory 24 of a specific display terminal should be replaced, updated or deleted, what timing the timing generator 16 should provide, and the like. Since these control bits are used to carry out functions which are well known in the prior art, such as loading and updating selected ones of a plurality of memories, they are not described in detail in this specification.
In actual operation of the invented system of display terminals, the image generator 10 delivers over the communication link 11 a set of edges, e.g. an integer multiple of 1282 edges. As the edges are being received by the communications controller 12, the receiver 14 converts the serially received data to parallel data, the timing generator 16, which includes internal clocks, generates the necessary timing information for loading the selected memory 24, and the terminal load control 18 determines which memory 24 receives the edges and directs the edges to that memory 24 in the format suitable for loading it.
The set of edges is loaded in the selected memory 24 in the manner described in detail later in this specification. Then, the edge-to-segment decoder 28 looks at a subset of the edges stored in the memory 24 (looks at a block of edges) to see which, if any, cross the scanline of the monitor 34 which is currently being decoded. Based on detecting the edge crossings of'that scanline, the edge-to-segment decoder 28 divides the scanline into segments, and defines each segment by its position on the scanline and by the attributes (color, brightness and the like) corresponding to the edge crossings bounding the segment. The segment buffer 30 buffers a few of the segments provided by the decoder 28, and the segment-to-video decoder 32 takes the buffered segments and decodes them into a video raster, each point of which is defined by its position on the screen of the monitor 34 and by its attributes such as brightness and color. The video raster from the decoder 32 is applied to the monitor 34 for display at a resolution several times that of the conventional color TV receiver (e.g. 1600 points in x and 1200 points in y).
It is noted that while a specific hardware embodiment of the invention is described in detail in this specification, many variations are possible within the scope of the invention. For example, edges may be defined by words differing in format from the words shown in FIG. 2, the edge may define the attributes of a screen area to its left, as well as to its right, or some other associated area of the screen, and different components may be used for specific functions. Further, parts, or all of the invented display terminal system may be simulated on specially programmed general purpose digital computers.
The invention is described in detail below under the general headings of: memory 24; decoder 26, including edge-to-segment decoder 28, segment buffer 30 and segment-to-video decoder 32; and display monitor 34.
Memory 24 FIGS. 4 and 5 The memory 24 (FIG. 1) is interposed between the memory input bus 22 and the decoder 26, and its purpose is to temporarily store (buffer) a set of edges and to provide to the decoder 26 those edges which cross the scanline that is currently being decoded. In order not to have to read out all of the edges in the memory 24 during each scanline period (e.g. each 64 microseconds) the memory is organized into two sections, a small section which is read out once for each scanline period and a large section which is read out only once for each vertical trace period (e.g. every l/th of a second) of the display 'monitor 34.
Referring to FIG. 4, the memory 24 of each of the display terminals comprises N banks 1, 2 N. (where N is four, for example) each bank comprising a large section 36 capable of storing 1,024 edges, and a small section 38 capable of storing 2 58 edges. The set of the small sections 38 of the N banks is referred to below as the small section of the memory 24 and the set of the N large sections 36 as the large section of the me m-