WO2006031797A2

WO2006031797A2 - Multi-application graphic display environment

Info

Publication number: WO2006031797A2
Application number: PCT/US2005/032530
Authority: WO
Inventors: Christopher John Hall
Original assignee: Seaweed Systems
Priority date: 2004-09-10
Filing date: 2005-09-09
Publication date: 2006-03-23
Also published as: EP1787282A2; CA2579222A1; US20060055702A1; EP1787282A4; EP1787282B1; WO2006031797A3; US8006196B2

Abstract

Graphics information resulting from operation of applications is directed to a set of visible windows. A set of graphics contexts is provided having fixed associations with the applications. A set of variable associations is provided between the graphics contexts and the windows, so that varying the windows to which graphics information resulting from operation of the applications is directed is accomplished by varying the associations between the graphics contexts and the windows.

Description

MULTI-APPLICATION GRAPHIC DISPLAY ENVIRONMENT

Priority Claim

[0001] Priority is hereby claimed to the United States utility patent application entitled MULTI-APPLICATION GRAPHIC DISPLAY ENVIRONMENT, having application number 10/938,281 , filed on September 10, 2004.

Technical Field

[0002] The present disclosure relates to window management in a graphical computing environment.

Background Art

[0003] It is common in graphical computing environments to partition the graphical display into windows. Windows are commonly rectangular areas comprising graphical data associated with a particular computing application.

[0004] General purpose computing environments often provide for large numbers of windows in arbitrary overlapping configurations. It may be complex and time-consuming to properly account for changes (e.g., update) such arbitrary configurations, and to properly safeguard the accuracy of the content of the display. In safety-critical time- constrained applications, it may be especially important that updates to the graphical display are carried out within a time-bounded fashion, and in a manner that protects the visual content of safety-critical windows, and the display overall.

Disclosure of Invention

[0005] The following summary is intended to highlight and introduce some aspects of the disclosed embodiments, but not to limit the scope of the invention. Thereafter, a detailed description of illustrated embodiments is presented, which will permit one skilled in the relevant art to make and use aspects of the invention. One skilled in the relevant art can obtain a full appreciation of aspects of the invention from the subsequent detailed description, read together with the figures, and from the claims (which follow the detailed description).

[0006] Graphics information resulting from operation of applications is directed to a set of visible windows. A set of graphics contexts is provided having fixed associations with the applications. A set of variable associations is provided between the graphics contexts and the windows, so that varying the windows to which graphics information resulting from operation of the applications is directed is accomplished by varying the associations between the graphics contexts and the windows.

Brief Description of the Drawings

[0007] The headings provided herein are for convenience only and do not necessarily affect the scope or meaning of the claimed invention.

[0008] In the drawings, the same reference numbers and acronyms identify elements or acts with the same or similar functionality for ease of understanding and convenience. To easily identify the discussion of any particular element or act, the most significant digit or digits in a reference number refer to the figure number in which that element is first introduced.

[0009] Figure 1 is a block diagram of an embodiment of a data processing device.

[0010] Figure 2 is a block diagram of an embodiment of a graphical computing environment.

[0011] Figure 3 is a block diagram of an embodiment of windows in a graphical computing environment.

[0012] Figure 4 is a block diagram of an embodiment of a window pixel value.

[0013] Figure 5 is a block diagram of an embodiment of a process of determining a display pixel value. [0014] Figure 6 is a flow chart of an embodiment of a process of rendering window pixels as display pixels. [0015] Figure 7 is a flow chart of an embodiment of a process of rendering window pixels as display pixels. [0016] Figure 8 is a block diagram of an embodiment of a graphical computing environment. [0017] Figure 9 is an action diagram of an embodiment of a process of associating the contents of windows with applications. [0018] Figure 10 is a block diagram of an embodiment of a graphical computing environment.

Industrial Applicability and Modes for Carrying Out the Invention [0019] The invention will now be described with respect to various embodiments. The following description provides specific details for a thorough understanding of, and enabling description for, these embodiments of the invention. However, one skilled in the art will understand that the invention may be practiced without these details. In other instances, well known structures and functions have not been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments of the invention. References to "one embodiment" or "an embodiment" do not necessarily refer to the same embodiment, although they may. [0020] Herein, "logic" refers to any configuration of circuits and/or signals that may be applied to affect operations within a device. Logic may comprise signals stored in a device memory and applied to a processing device, such as a microprocessor, digital signal processor, microcontroller, and so on. Software is one example of such logic. Examples of device memories that may comprise logic include RAM (random access memory), flash memories, ROMS (read-only memories), EPROMS (erasable programmable read-only memories), mass storage, cache memories, and EEPROMS. Logic may also be comprised by digital and/or analog electrical hardware circuits. Logic may be formed from combinations of software and hardware. [0021] Figure 1 is a block diagram of an embodiment of a data processing device. A data processing device 102 (such as a vehicular display system) comprises a processor 104 and various types of memory. The types of memory may include a processor cache 106, volatile random access memory (RAM) 108, and non-volatile RAM 110 (read-only memory, magnetic and optical discs or other media, flash memory, and so on).

[0022] The data processing device 102 may also comprise other logic and circuits 1 12 to perform processing that is not central to the present discussion. The data processing device 102 comprises a graphics subsystem 1 14 that includes memory 116, display logic and circuits 118, and a graphics processor 119, among other things.

[0023] The volatile RAM 108 may comprise logic 120 that, when applied to the processor, results in collection, configuration, and display of graphics information. At any particular time, portions/versions 122 of the logic 120 may be comprised by non¬ volatile RAM 110. Likewise, the cache 106 may at times comprise portions/versions of the logic 120.

[0024] Figure 2 is a block diagram of an embodiment of a graphical computing environment. A graphics subsystem 114 comprises a memory 116, display logic and circuits 118, and a scissor register 218. The display logic and circuits 1 18 are coupled to a display 202. The scissor register 218 is coupled to display logic and circuits 118. The display 202 comprises windows 204-206 (the number of windows may be more or less than illustrated).

[0025] The environment includes an operating system 212 to manage computing resources. The operating system 212 comprises a window manager 214 to manage the display and control of windows. A memory 220 of the data processing device, which may comprise, among other elements, memories 106, 108, and 1 10, comprises various logic applications 208-210. Logic applications 208-210 are applied to one or more processors 104 to carry out various graphical computing operations, such as updating the contents of windows 204- 205. [0026] The memory 220 further comprises a GID buffer 216. The purpose of the GID buffer 216 is explained in more detail in conjunction with later figures.

[0027] Figure 3 is a block diagram of an embodiment of windows in a graphical computing environment. The display 202 comprises windows 304-309. The windows 304-309 are organized into layers. A layer represents a window's display priority. For example, windows in higher layers may have a lower display priority than windows in lower layers, so that when windows in the different layers having overlapping display locations, the window with the lower layer (higher display priority) will display in the overlapping locations. Windows having the same layer are not permitted to overlap.

[0028] Each window 304-309 has a scissor rectangle, e.g. a bounding rectangle. One or more data structure(s) associated with a window defines a "graphics context", and may comprise content corresponding to display locations extending beyond the scissor rectangle (scrolling windows are one example). Content within the scissor rectangle is displayed, and content outside of the scissor rectangle is clipped (not displayed).

[0029] Figure 4 is a block diagram of an embodiment of a window pixel value. By way of example, the window 306 comprises a pixel value 414,. among others. A pixel value is logic representing display characteristics for a display pixel, e.g. a basic display unit. One format of pixel values is the RGBA format, where bits of the pixel value represent the red, green, blue, and alpha components of the display pixel. The pixel value may also comprise bits (GID) representing the layer of the window comprising the pixel value. The pixel value may comprise other bits as well.

[0030] Figure 5 is a block diagram of an embodiment of a process of determining a display pixel value. The GID buffer 216 comprises a pixel GID value 511 (e.g. the pixel layer id) determined by successively writing corresponding pixel GID values of windows in different layers. [0031] Window 110 502 (a window in layer 10) comprises pixel GID value 508. Window I5 503 (a window in layer 5) comprises a pixel GID value 509. Window I2 504 (a window in layer 2) comprises pixel GID value 510. The pixel GID values 508-510 are corresponding, e.g. they apply to display pixels having identical locations on the display 202. The GID buffer 216 has pixel GID value locations for each display pixel. The pixel GID value 511 corresponds to the pixel GID values 508-510 of the windows. The pixel GID values 508-510 are written successively according to the GID display priority. In an environment where higher GID values (higher layers) have lower display priority, pixel GID value 508 is written first, then overwritten by pixel GID value 509, which is overwritten by pixel GID value 510. Pixel GID value 511 is thus assigned the value of the corresponding window pixel GID value having the highest display priority.

[0032] Figure 6 is a flow chart of an embodiment of a process of rendering window pixels as display pixels. The content of a window is rendered on the display 202. If at 602 there are more pixels of the region of the contents lying within the scissor rectangle to render, the next pixel to render is identified. If there are not more pixels within the scissor rectangle to process, the process concludes at 610.

[0033] A check is made at 606 to determine if the pixel GID value is equal to an active GID value. The active GID value is the GID value (e.g. layer id) of the layer currently processed for display. If at 606 the pixel value is not equal to the active GID value, the process continues at 602. If at 606 the pixel value is equal to the active GID value, the pixel is rendered on the display 608 at 608 and processing returns to 602. Otherwise the process concludes at 610.

[0034] Figure 7 is a flow chart of an embodiment of a process of rendering window pixels as display pixels. At 702 the active layer is set to the layer furthest back (e.g. the layer with lowest display priority) having windows. At 704 the active window (e.g. the window to process) is set to a first window of the active layer. At 706 each pixel GID value of the active window is written to corresponding screen positions of the GID buffer 216. If at 708 there are not more windows in the active layer, processing continues at 712. If there are more windows in the active layer, the active window is set to a next window of the active layer at 710.

[0035] If at 712 there are not more layers with windows, processing concludes at 716. Otherwise, at 714 the active layer is set to the next furthest back layer having windows (e.g. to the layer with windows having the next-lowest display priority). At 716 the process concludes.

[0036] Thus, for each of a plurality of overlapping display windows, the

GID buffer 216 is z-buffered with identifications of display layers of the windows.

[0037] Figure 8 is a block diagram of an embodiment of a graphical computing environment. The display 202 comprises windows 810 and 811. A first application 802 comprises a first graphics context 806 (e.g. the first application 802 is associated with and controls the content of the first graphics context 806). A second application 803 comprises a second graphics context 807. The first graphics context 806 is associated with window 810, and the second graphics context is associated with window 811. The first application 802 directs display content to the first window 810 by directing the content to the first graphics context 806. The second application 802 directs display content to the second window 811 by directing the content to the second graphics context 807.

[0038] In a safety-critical graphical computing environment, it may be advantageous to employ a trusted process to create all windows for the environment. This same or another trusted process may direct the association of processes (e.g. processes of applications) with windows. The windows may be created at the time the operating system 212 is initialized, and then left unchanged in size and position. Thus, once created, windows cannot be resized, moved, or change layers. However, many of these features may be accomplished by employing a trusted process to change the association of windows and processes. In such environments, it may prove simpler to protect the accuracy of the display information, and updating the display may be accomplished faster than in environments where the windows may be moved, sized, and arranged arbitrarily.

[0039] A mechanism may be provided in such environments to enable applications to direct their display content to different windows. An application may direct its display content to a different window of the same layer, or to a window of a different layer, by providing a set of fixed associations between the graphics contexts and applications. A set of variable associations is provided between the graphics contexts and the windows. Varying the windows to which graphics information resulting from operation of the applications is directed is accomplished by varying the associations between the graphics contexts and the windows.

[0040] Figure 9 is an action diagram of an embodiment of a process of associating the contents of windows with applications. Figure 9 demonstrates how two applications may switch the use of windows, without moving the windows or changing the layers of the windows. At 900 and 901 the window manager logic 214 creates the first window (window 1 ) and the second window (window 2). The first application (app1) creates the first graphics context (gd ) at 902. The second application (app2) creates the second graphics context (gc2) at 904.

[0041] At 906 app1 signals the window manager 214 to request to attach gd to window 1 , and at 908 the window manager 214 attaches gd to window 1. At 910 app2 signals the window manager 214 to request to attach gc2 to window 2, and at 912 the window manager 214 attaches gc2 to window 2.

[0042] Display content (graphics information) is sent via gd from app1 to window 1 at 914. Display content is sent via gc2 from app2 to window 2 at 916. Figure 8 represents the state of a graphics environment after act 916.

[0043] At 918 app2 requests to unattach gc2 from window 2, and the window manager 214 unattaches gc2 from window 2 at 920. At 922 app1 requests to unattach gd from window 1 , and the window manager 214 unattaches gd from window 1 at 922. [0044] App1 requests to attach gd to window 2 at 928. The window manager 214 attaches gd to window 2 at 930. App2 and requests to attach gc2 to window 1 at 934. The window manager 214 attaches gc2 to window 2 at 936.

[0045] App1 directs display information to GC1 , which directs the information to window 2 at 938. App2 directs display information to GC2, which directs the information to window 1 at 940.

[0046] Figure 10 is a block diagram of an embodiment of a graphical computing environment. Figure 10 represents the state of the graphics environment of Figure 8 after act 940. As in Figure 8, the display 202 comprises windows 810 and 811. The first application 802 now comprises the third graphics context 1002, and the second application 803 now comprises the fourth graphics context 1003. The first application 802 directs display content to the second window 811 by directing the content to the third graphics context 1002. The second application 802 directs display content to the first window 810 by directing the content to the fourth graphics context 1003. The applications have switched windows, without moving the windows or changing the layers of the windows.

[0047] Unless the context clearly requires otherwise, throughout the description and the claims, the words "comprise," "comprising," and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is to say, in the sense of "including, but not limited to." Words using the singular or plural number also include the plural or singular number respectively. Additionally, the words "herein," "above," "below" and words of similar import, when used in this application, shall refer to this application as a whole and not to any particular portions of this application. When the claims use the word "or" in reference to a list of two or more items, that word covers all of the following interpretations of the word: any of the items in the list, all of the items in the list and any combination of the items in the list.

Claims

ClaimsWhat is claimed is:

1. A method comprising: associating a first graphic content with a first application; associating a second graphic context with a second application; attaching the first graphics context to a first window, to direct to the first window graphics resulting from operation of the first application; attaching the second graphics context to a second window, to direct to the second window graphics resulting from operation of the second application; detaching the first graphics context from the first window; detaching the second graphics context from the second window; attaching the second graphics context to the first window; and attaching the first graphics context to the second window.

2. The method of claim 1 further comprising: assigning a first identifier to a first display layer, the first window associated with the first layer; assigning a second identifier to a second display layer, the second window associated with the second layer; filling memory locations of a buffer with the first identifier, the memory locations corresponding to a region of the first window; and filling memory locations of the buffer with the second identifier, the memory locations filled with the second identifier corresponding to a region of the second window overlapping the region of the first window.

3. A method comprising: providing a set of visible windows to which may be directed graphics information resulting from operation of applications; providing a set of graphics contexts; providing a set of fixed associations between the graphics contexts and the applications; and providing a set of variable associations between the graphics contexts and the windows, whereby varying the windows to which graphics information resulting from operation of the applications is directed is accomplished by varying the associations between the graphics contexts and the windows.

4. The method of claim 3 further comprising: for each of the set of visible windows, some of the visible windows overlapping others, z-buffering a memory region having locations corresponding to pixels with identifications of display layers of the windows.

5. A method comprising: assigning a first identifier to a first display layer; assigning a second identifier to a second display layer; filling memory locations of a buffer with the first identifier, the memory locations corresponding to a region of a first display window associated with the first display layer; and filling memory locations of the buffer with the second identifier, the memory locations filled with the second identifier corresponding to a region of a second display window overlapping the region of the first display window, the second display window associated with the second display layer.

6. A method comprising: for each of a plurality of display windows, some of the display windows overlapping others, z-buffering a memory region having locations corresponding to pixels with identifications of display layers of the display windows.