WO2007030283A1 - High level graphics stream - Google Patents

High level graphics stream Download PDF

Info

Publication number
WO2007030283A1
WO2007030283A1 PCT/US2006/031718 US2006031718W WO2007030283A1 WO 2007030283 A1 WO2007030283 A1 WO 2007030283A1 US 2006031718 W US2006031718 W US 2006031718W WO 2007030283 A1 WO2007030283 A1 WO 2007030283A1
Authority
WO
WIPO (PCT)
Prior art keywords
graphics
tree
channel
data packets
composition
Prior art date
Application number
PCT/US2006/031718
Other languages
French (fr)
Inventor
Sriram Subramanian
Leonardo E. Blanco
Andrei Baioura
Oreste Dorin Ungureanu
Sriya Sampath
Pawel Olszta
Gilles Khouzam
Joy Chik
Paul David
Ivan Leichtling
Original Assignee
Microsoft Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Microsoft Corporation filed Critical Microsoft Corporation
Priority to AU2006287781A priority Critical patent/AU2006287781A1/en
Priority to JP2008530060A priority patent/JP4827923B2/en
Priority to CA002621076A priority patent/CA2621076A1/en
Priority to CN2006800327888A priority patent/CN101258478B/en
Priority to BRPI0615723-8A priority patent/BRPI0615723A2/en
Priority to EP06801462.0A priority patent/EP1922634B1/en
Publication of WO2007030283A1 publication Critical patent/WO2007030283A1/en
Priority to NO20080928A priority patent/NO20080928L/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/003D [Three Dimensional] image rendering
    • G06T15/005General purpose rendering architectures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators

Definitions

  • the application programs or processes may send graphics commands to operating system components or other processes that use the graphics commands to render the images or graphics.
  • the graphics commands which are also referred to as higher-level graphics commands or primitives, might specify or define colors, lines, shapes, and other graphics constructs.
  • the operating system components or processes receiving the graphics commands may interpret or convert the graphics commands into lower-level graphics information such as individual pixel values or bitmaps which are used to render graphics on a display device.
  • an interface i.e., GDI interface
  • GDI interface may be part of or supports a component or process that receives the graphics commands, hi particular, the GDI interface receives parameters from an application, where the parameters are for an image or graphic to be represented.
  • the GDI interface produces the image by sending commands to a process or component which in turn renders the image to a display or output device such as a monitor, printer, etc.
  • the application programs or processes sending such graphics commands reside on devices or computers that are separate from devices or computers hosting the operating system components or processes that receive the graphics commands.
  • trans- machine systems examples of which include terminal service systems where applications reside on a central server computer and remote client computers receive the graphics commands that are rendered locally (i.e., at the client computers).
  • the application programs and operating system components (processes) receiving the graphics commands reside on the same device or computer and may be referred to as trans-process systems.
  • Other implementations may send graphics commands using the same process; however, such implementations may use different threads in which the graphics commands are passed. These implementations may be referred to as trans-thread systems.
  • the graphics commands sent by the application programs are typically transitory, hi other words, once the graphics commands are sent by the application for processing and received, they are immediately consumed or processed. This is particularly true where GDI is used, where the interface receives parameters, creates graphics commands, and the graphics commands are immediately consumed. Therefore, if a particular graphics or image is lost, the application program or process must resend the graphics commands (or parameters) to the receiving component or process. Furthermore, since graphics commands may be consumed as they are received, if a user desires to replicate the graphics or image particular to the graphics commands, the specific graphics commands must be resent. [0006] m certain situations, graphics commands are optimized or compressed.
  • This optimization or compression typically is performed in trans-machine implementations, such as terminal-service systems, where communication between a central server computer and remote client computers is through a communications medium such as a network. Bandwidth over the communication medium or network is often limited. Therefore, optimization or compression may be needed. Optimization or compression may result in compromised image quality. For example, in certain situations, it is desirable to magnify the graphic or image at the receiving computer or device; however, since the graphics commands have been compressed, magnification may result in image degradation. [0007] Furthermore, in certain cases, different applications may support or provide different types or dissimilar graphics commands. For example, one application may support a legacy graphics command format such as GDI, while another application may support a new or different graphics command format. However, it may be desirable to support different graphics command formats and allow graphics to be processed from applications using different graphics command formats.
  • Graphics commands are provided in data packets in a graphics stream.
  • the data packets represent objects and resources of an editable model such a visual tree.
  • the graphics stream or data packets are sent through a dedicated channel, where an application may be introduced in the channel that modifies the graphics stream or data packets.
  • the data packets are received by a composition engine that creates a composition tree that is analogous to the visual tree.
  • Fig. 1 is a block diagram of a system that implements a channel to provide graphics in the form of a visual tree.
  • Fig. 2 is a diagram of a visual tree representing graphics or an image, and an analogous composition tree.
  • FIG. 3 is a block diagram of a computing device that implements a visual tree and composition trees to provide graphics or images.
  • Fig. 4 is a block diagram of a system that implements an application and application program interfaces that modify a composition tree of graphics or an image.
  • Fig. 5 is a diagram of a revised composition tree.
  • FIG. 6 is a flowchart illustrating a process.
  • FIG. 7 is a flowchart illustrating a process.
  • Fig. 1 shows a system 100 that provides and renders graphics or images.
  • the system 100 implements the use of visual trees and composition trees in providing and rendering graphics or images.
  • System 100 may be implemented as a trans-machine system where elements or processes of system 100 reside in more than one machine or device, and communication between the machines or devices is through a communication medium such as a network that includes intranets and the Internet.
  • Examples of trans-machine systems include terminal- service systems, where a central server computer supports one or more client computers.
  • application programs are resident at the central server computer and accessed or used by the client computers.
  • the system 100 may also be implemented as a trans-process system, where elements or processes reside on the same machine or device.
  • Another exemplary implementation of system 100 is a trans- thread system, where graphics are sent through different threads within the same process.
  • system 100 shows a single application program 102; however, system 100 may include multiple application programs.
  • the system 100 further includes a framework 104.
  • An example of framework 104 includes Windows® Presentation Foundation provided by the Microsoft Corporation.
  • Framework 104 accesses one or more application programs, such as application program 102.
  • graphics commands are provided by application program 102 to framework 104.
  • Framework 104 may implement or include a particular application program interface (API) in communicating with application program 102.
  • System 100 includes a core 106 which receives graphics commands from framework 104 and creates a visual tree 108.
  • Visual tree 108 is a representation of graphics or an image provided by the framework 104, where the graphics or image is representative of the graphics commands from application program 102.
  • the visual tree 106 is made up of visual objects and resources as described in further detail below.
  • the visual objects and resources may be translated into particular data packets that make up a data or graphics stream.
  • the graphics stream is communicated over a dedicated channel 110 to a composition engine 112.
  • Channel 110 is a dedicated channel in the sense that when other application programs are included in system 100, each application program has its own dedicated channel. In a trans-thread system, where the graphics stream is communicated over different threads, each dedicated channel is representative of a particular thread.
  • the channel 110 may also be partitioned into different functional parts, including a top half channel and a bottom half channel.
  • the data packets of the graphics stream may be provided in a particular format.
  • An exemplary data packet format is a 32-bit data packet size value and a 32- bit unsigned packet identification value followed by packet data of one or more commands. Furthermore, when multiple channels are implemented, each cannel may be associated with a specific connection.
  • a data packet may provide a message that specifies the channel connection. Such a message may be implemented in the following format that includes exemplary fields "Connection ID", followed by "Channel ID”, followed by "Command E)", followed by "Command Data”.
  • a back channel 114 may be provided that allows the composition engine 112 to send back messages to the core 106, framework 104, and application program 102. Back channels such as back channel 114 may be provided with each dedicated channel (e.g., channel 110). The messages that are sent back through back channel 114 may include hardware status and/or changes to hardware or hardware configuration that are supported by the composition engine 112. Other examples of messages returned by the composition engine 112 include out of memory information, frame rate consumption information, hardware changes, etc.
  • Data packets may be arranged in batches, where batches of data packets are sent over the channel 108 to composition engine 112. For example, visual objects and resources of a visual tree 108 are sent over as a batch to composition engine 112. Composition engine 112 waits to receive the entire batch of visual objects and resources, before creating an analogous composition tree 116. hi addition to the objects and resources that are sent in the graphics stream, commands or instructions are included the graphics stream. The commands or instructions are particularly used to create the composition tree 116. When system 100 includes multiple application programs, because each application program has its own dedicated channel in which graphics streams, and particularly batches of data packets, are communicated, composition engine 112 does not have to wait for other batches or graphics streams to be communicated over the channel.
  • the dedicated channels prevent interference of communication of graphics streams (data packet batches) from different application programs. Furthermore, dedicated channels allow inherent synchronization of graphics streams such that data packets are received in their relative chronological order.
  • the composition tree 116 includes the same information (i.e., analogous objects and resources) as the visual tree 108; however, the composition tree 116 is formatted to be used by a process/function 118.
  • the process/function 118 may be an operating system component or application program component that renders graphics or images using the composition tree 116.
  • Process/function 118 may interpret or convert the composition tree 116 into lower-level graphics information such as individual pixel values or bitmaps which are used to render graphics on a display device (not shown).
  • composition tree 116 objects and resources of composition tree 116, along with other commands or instructions as to reconstruct the composition tree 116 may be stored as a file in a local memory for future use or use in cases such as "refresh" when the graphics or image represented by the composition is lost.
  • optimization and compression may be performed on the graphics streams as they are sent over the channel 110 or other channels.
  • the optimization or compression may implement one of various optimization and/or compression methods.
  • the graphic stream messages or graphics stream may be transported using one of various transport protocols (e.g., RPC, RDP, TCP/IP, etc.) and/or methods.
  • FIG. 2 shows an exemplary visual tree 108 and an exemplary composition tree 116.
  • Visual tree 108 is made up of hierarchical graphic or visual objects Vl 200(1) to VN 200(N).
  • the visual tree 108 represents graphics or an image.
  • Each of the visual objects 200 is further described by commands, instructions, or resources (collectively referred to as "resources").
  • resources are used to describe how to draw the visual objects. Resources may have higher order or lower order resources.
  • the graphics object V3 200(3) is defined by a "draw line” resource 202.
  • the "draw line” resource 202 is further described by a "pen” resource 204.
  • the "pen” resource 204 is further described by a "brush” resource 206.
  • the visual objects V 200 along with the resources
  • composition tree 116 (e.g., resources 202, 204, 206) are translated into data packets and sent in a graphics stream, along with commands or instructions.
  • the commands or instructions are used to construct the composition tree 116 using analogous composition objects Cl 208(1) to CN 208(N), and resources defining each of the composition objects C 208 (e.g., "draw line” resource 210, "pen” resource 212, and "brush” resource 214).
  • a composition engine which receives the graphics stream of visual objects, resources, commands, and instructions may wait for all of the data packets to be received before constructing the composition. Therefore, the visual objects, resources, commands, and instructions may be sent as a batch. [00029] Fig.
  • Computing device 300 which implements visual trees and composition trees to provide and render graphics or images.
  • Computing device 300 is an implementation of a trans-process system; however, it will become evident, that elements in computing device 300 may also be implemented as part of, or have analogous elements in trans-machine and trans-thread systems.
  • Computing device 300 may be a conventional desktop personal computer (PC), which includes a local operating system, processing unit or processor(s) 302, and a storage system or memory 304.
  • the computing device 300 is a general- purpose PC implemented with a Windows® brand operating system from the Microsoft Corporation.
  • application program 306 and legacy application program 308 may be part of multiple application programs resident in memory 304 or which reside separate from memory 304.
  • Programs 306 and 308 are particularly accessed and controlled by processor(s) 302.
  • the legacy application program 308 may include application programs that implement graphics command formats such as GDI.
  • Computing device 300 includes a framework and core 310 that supports application program 306.
  • Framework and core 310 includes framework 104 and core 106 described above, hi this example, legacy application program 308 is supported by a legacy framework and core 312.
  • a single framework and core may be used to support multiple application programs.
  • Framework and core 310 and legacy framework and core 312 are particularly configured to create and provide visual trees, such as visual tree 108, based on commands received from respective application program 306 and legacy application program 308.
  • Framework and core 310 is connected to a redirector 314 which may be an interface to a dedicated channel "A" that supports application program 306.
  • Redirector 314 may be considered as a "top half of the channel "A”.
  • Legacy framework and core 312 is likewise connected to a legacy redirector 316 which is an interface to a separate channel "B" that supports the legacy application program 308.
  • redirector 316 may be considered as a "top half of the channel "B”.
  • Redirectors 314 and 316 maybe connected to or use a transport layer or transport 318.
  • a redirection layer 320 is a bottom half of channel "A”
  • a redirection layer 322 is a bottom half of channel "B”.
  • composition engine “A” 324 receives a graphics stream through channel “A” and creates a composition tree for use by process/function "A” 326.
  • composition engine “B” 328 receives a graphics stream through channel “B” and creates a composition tree for use by process/function "B” 330.
  • process/function "A” 326 and process/function "B” 330 uses the created composition trees to render or generate graphics or images to a local display or output device.
  • Fig. 4 shows a system 400 that implements an application and application program interfaces used to modify a composition tree of graphics or an image.
  • system 400 is a modification of system 100 described above, hi certain cases, it is desirable to modify graphics or images that are provided by application programs, such as application program 306 and legacy application program 308. Considering that optimization or compression may take place when graphic stream messages or graphics streams are sent, and particularly in trans- machine systems, it is desirable to minimize degradation of the graphics or image when modification takes place.
  • an application 402 is located between a top half of channel 404 and a bottom half of channel 406.
  • the application 402 may perform a graphics or image modification, or in certain cases application 402 may be configured as an accessibility application that is able to access or look at a graphics stream that is being passed through the channel (i.e., between the channel halves 404 and 406). hi general, application 402 is considered as an intermediary that may be placed within a channel.
  • An example of graphics or image modification is enlarging or magnifying the graphics or image.
  • the application 402 particularly modifies the graphics stream or particular data packets passing through the channel, such that a revised composition tree 408 is created.
  • the modification may be made on a particular node or nodes of the composition tree, regardless of the hierarchy of the node or nodes in the composition. If a relatively higher-level node is modified, all the nodes below it on the composition tree are affected (i.e., modified).
  • a graphics stream application program interface (API) 410 and playback API 412 may also be provided and exposes the application 402 to the core 104 and composition engine 110.
  • graphics stream API 410 is used to access a graphics stream from the core 104 and playback API 412 is used by composition engine 110 to open the graphics stream from application 402.
  • Graphics stream API 410 may include a "graphics stream open” API which opens the graphics stream for reading.
  • a pointer may be implemented and provided to application 402 by the "graphics stream open” API to the particular function or process which composition engine 110 supports.
  • Graphics stream API 410 may further include a "graphics stream close” API which closes the graphics stream for reading.
  • the "graphics stream close” API causes a close message to be sent to composition engine 110 or to the particular function or process which composition engine 110 supports.
  • Graphics stream API 410 may also include a "graphics stream set transform hint" API that provides a hint or message to the core 104, or in the case of trans-machine system (e.g., terminal-service system) a graphics server (i.e., core 104 resident in the graphics server), that the application 402 intends to perform an action or modification (e.g., magnification) to the graphics stream.
  • the "graphics stream transform hint" API provides a message back to application program 102 to account for enlargement or magnification.
  • a pointer may be provided as to the particular revised composition tree 404. The message may be sent through a back channel, such as back channel 114.
  • Playback API 412 provides the ability for composition engine 112 to draw or create the revised composition tree 408. In general the playback API 412 exposes the bottom half of channel 406 and the composition engine 112 to the application 402.
  • Fig. 5 is a diagram of a modified or revised composition tree.
  • the revised composition tree 408 as modified by application 402 is illustrated.
  • the modification is to enlarge or magnify a prior composition tree (e.g., composition tree 116).
  • the modification is performed on the composition tree, and particularly to a node or nodes of the composition tree. Depending on which node or nodes or modified, other nodes may be affected. In other words, if a particular node is modified, all lower-level nodes of branches from that particular node are affected or modified.
  • the node Cl 208(1) is modified by a "magnify" node 500. Therefore, the prior composition tree 116 is modified (i.e., magnified) and is provided as revised composition tree 408.
  • Fig. 6 shows a process 600 that provides for graphics streams representative of visual trees to a process or function.
  • Process 600 may further be implemented as an API that supports modification of the graphics stream.
  • the process 600 is illustrated as a collection of blocks in a logical flow graph, which represent a sequence of operations that can be implemented in hardware, software, firmware, or a combination thereof.
  • the blocks represent computer instructions that, when executed by one or more processors, perform the recited operations.
  • certain blocks may take place concurrently or in a different order.
  • the process 600 may be implemented, for example, by the system 100 of Fig. 1 and/or computing device 300 as discussed in Fig. 3, although the process 600 may be implemented by other architectures.
  • graphics commands are sent to and received by an application that is located in a channel.
  • the application may be an accessibility application or an application that performs a modification.
  • An example of such an application is application 402.
  • the graphics stream may include data packets which are representative of visual objects and resources that make up a visual tree.
  • the application opens or accesses the graphics stream.
  • the opening may be related to merely reading the graphics streams and data packets of the graphics stream, or a modification may be performed on the graphics stream. Furthermore, a pointer may be provided to a function or processes supported by a composition engine receiving the graphics stream. [00046]
  • the actual modification of the graphics stream and/or data packets is performed. Examples of modification include magnification.
  • a message may be provided to a core that provides the graphics streams, where the message indicates that modification such as magnification is to take place.
  • the modified stream is passed on to the composition engine which may create an composition tree based on the original visual tree, and changes or modifications from the modification.
  • the graphics stream may be closed.
  • Fig. 7 shows a process 700 that allows for communication between a framework and composition engine.
  • Process 700 may further be implemented as a protocol to provide such communication.
  • the process 700 is illustrated as a collection of blocks in a logical flow graph, which represent a sequence of operations that can be implemented in hardware, software, firmware, or a combination thereof.
  • the blocks represent computer instructions that, when executed by one or more processors, perform the recited operations.
  • the process 700 may be implemented, for example, by the system 100 of Fig. 1 and/or computing device 300 as discussed in Fig. 3, although the process 600 may be implemented by other architectures.
  • a dedicated channel is created which allows communication between the framework or a core, and the composition engine.
  • a dedicated back channel may also be created to allow the composition engine to send back messages to the framework or core.
  • data packets are sent over or provided over the dedicated channel.
  • the data packets may be part of a graphics stream and sent as a batch.
  • the batch of data packets includes all objects and resources that are necessary to create an editable model.
  • the model is representative of graphics or an image, and for example, may be a hierarchical tree such as visual tree 108 described above.
  • an application may be introduced in the channel to modify the graphics stream or data packets. Examples of the application include a graphics stream reader or a magnifier. A message may be returned to the framework or core, indicating that a modification is to be performed.
  • the editable model is built using the unmodified or modified data packets.
  • the editable model may be analogous hierarchical tree to the visual tree created at the core.
  • the editable model is considered or referred to as a composition tree, such as composition trees 116 and 408.

Abstract

A system that provides graphics commands that represent graphics or an image, to a consuming process. The graphics commands are represented by a visual tree which includes objects and resources describing the objects. The objects and resources are communicated to a composition engine that creates an analogous composition tree that is analogous to the visual tree. The composition tree is used to render the graphics or image.

Description

HIGH LEVEL GRAPHICS STREAM
[0001] The present application claims priority under 35 U.S.C. §119(e) to U.S.
Provisional Application No. 60/714,880, filed September 7, 2005, the disclosure of which is incorporated herein.
BACKGROUND [0002] When application programs or processes provide images or graphics to be displayed, the application programs or processes may send graphics commands to operating system components or other processes that use the graphics commands to render the images or graphics. The graphics commands, which are also referred to as higher-level graphics commands or primitives, might specify or define colors, lines, shapes, and other graphics constructs. The operating system components or processes receiving the graphics commands may interpret or convert the graphics commands into lower-level graphics information such as individual pixel values or bitmaps which are used to render graphics on a display device. [0003] An example of graphics commands is GDI which represents "Graphical Device Interface", "Graphics Device Interface", or "Graphics Display Interface", hi a GDI implementation, an interface (i.e., GDI interface) may be part of or supports a component or process that receives the graphics commands, hi particular, the GDI interface receives parameters from an application, where the parameters are for an image or graphic to be represented. The GDI interface produces the image by sending commands to a process or component which in turn renders the image to a display or output device such as a monitor, printer, etc. [0004] In certain implementations, the application programs or processes sending such graphics commands reside on devices or computers that are separate from devices or computers hosting the operating system components or processes that receive the graphics commands. These implementations may be referred to as trans- machine systems, examples of which include terminal service systems where applications reside on a central server computer and remote client computers receive the graphics commands that are rendered locally (i.e., at the client computers). In other implementations, the application programs and operating system components (processes) receiving the graphics commands reside on the same device or computer and may be referred to as trans-process systems. Other implementations may send graphics commands using the same process; however, such implementations may use different threads in which the graphics commands are passed. These implementations may be referred to as trans-thread systems.
[0005] Regardless of the particular implementation or system, whether it be trans-machine, trans-process, or trans-thread, the graphics commands sent by the application programs are typically transitory, hi other words, once the graphics commands are sent by the application for processing and received, they are immediately consumed or processed. This is particularly true where GDI is used, where the interface receives parameters, creates graphics commands, and the graphics commands are immediately consumed. Therefore, if a particular graphics or image is lost, the application program or process must resend the graphics commands (or parameters) to the receiving component or process. Furthermore, since graphics commands may be consumed as they are received, if a user desires to replicate the graphics or image particular to the graphics commands, the specific graphics commands must be resent. [0006] m certain situations, graphics commands are optimized or compressed.
This optimization or compression typically is performed in trans-machine implementations, such as terminal-service systems, where communication between a central server computer and remote client computers is through a communications medium such as a network. Bandwidth over the communication medium or network is often limited. Therefore, optimization or compression may be needed. Optimization or compression may result in compromised image quality. For example, in certain situations, it is desirable to magnify the graphic or image at the receiving computer or device; however, since the graphics commands have been compressed, magnification may result in image degradation. [0007] Furthermore, in certain cases, different applications may support or provide different types or dissimilar graphics commands. For example, one application may support a legacy graphics command format such as GDI, while another application may support a new or different graphics command format. However, it may be desirable to support different graphics command formats and allow graphics to be processed from applications using different graphics command formats. SUMMARY
[0008] Graphics commands are provided in data packets in a graphics stream.
The data packets represent objects and resources of an editable model such a visual tree. The graphics stream or data packets are sent through a dedicated channel, where an application may be introduced in the channel that modifies the graphics stream or data packets. The data packets are received by a composition engine that creates a composition tree that is analogous to the visual tree.
[0009] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
BRIEF DESCRIPTION OF THE CONTENTS
[00010] The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The use of the same reference numbers in different figures indicates similar or identical items. [00011] Fig. 1 is a block diagram of a system that implements a channel to provide graphics in the form of a visual tree. [00012] Fig. 2 is a diagram of a visual tree representing graphics or an image, and an analogous composition tree.
[00013] Fig. 3 is a block diagram of a computing device that implements a visual tree and composition trees to provide graphics or images. [00014] Fig. 4 is a block diagram of a system that implements an application and application program interfaces that modify a composition tree of graphics or an image.
[00015] Fig. 5 is a diagram of a revised composition tree.
[00016] Fig. 6 is a flowchart illustrating a process.
[00017] Fig. 7 is a flowchart illustrating a process.
DETAILED DESCRIPTION [00018] Fig. 1 shows a system 100 that provides and renders graphics or images. In particular, the system 100 implements the use of visual trees and composition trees in providing and rendering graphics or images. System 100 may be implemented as a trans-machine system where elements or processes of system 100 reside in more than one machine or device, and communication between the machines or devices is through a communication medium such as a network that includes intranets and the Internet. Examples of trans-machine systems include terminal- service systems, where a central server computer supports one or more client computers. In particular, application programs are resident at the central server computer and accessed or used by the client computers. The system 100 may also be implemented as a trans-process system, where elements or processes reside on the same machine or device. Another exemplary implementation of system 100 is a trans- thread system, where graphics are sent through different threads within the same process.
[00019] - For simplicity, system 100 shows a single application program 102; however, system 100 may include multiple application programs. The system 100 further includes a framework 104. An example of framework 104 includes Windows® Presentation Foundation provided by the Microsoft Corporation. Framework 104 accesses one or more application programs, such as application program 102. In particular, graphics commands are provided by application program 102 to framework 104. Framework 104 may implement or include a particular application program interface (API) in communicating with application program 102. [00020] System 100 includes a core 106 which receives graphics commands from framework 104 and creates a visual tree 108. Visual tree 108 is a representation of graphics or an image provided by the framework 104, where the graphics or image is representative of the graphics commands from application program 102. [00021] The visual tree 106 is made up of visual objects and resources as described in further detail below. The visual objects and resources may be translated into particular data packets that make up a data or graphics stream. The graphics stream is communicated over a dedicated channel 110 to a composition engine 112. Channel 110 is a dedicated channel in the sense that when other application programs are included in system 100, each application program has its own dedicated channel. In a trans-thread system, where the graphics stream is communicated over different threads, each dedicated channel is representative of a particular thread. As further discussed below, the channel 110 may also be partitioned into different functional parts, including a top half channel and a bottom half channel. [00022] The data packets of the graphics stream may be provided in a particular format. An exemplary data packet format is a 32-bit data packet size value and a 32- bit unsigned packet identification value followed by packet data of one or more commands. Furthermore, when multiple channels are implemented, each cannel may be associated with a specific connection. A data packet may provide a message that specifies the channel connection. Such a message may be implemented in the following format that includes exemplary fields "Connection ID", followed by "Channel ID", followed by "Command E)", followed by "Command Data". [00023] A back channel 114 may be provided that allows the composition engine 112 to send back messages to the core 106, framework 104, and application program 102. Back channels such as back channel 114 may be provided with each dedicated channel (e.g., channel 110). The messages that are sent back through back channel 114 may include hardware status and/or changes to hardware or hardware configuration that are supported by the composition engine 112. Other examples of messages returned by the composition engine 112 include out of memory information, frame rate consumption information, hardware changes, etc.
[00024] Data packets may be arranged in batches, where batches of data packets are sent over the channel 108 to composition engine 112. For example, visual objects and resources of a visual tree 108 are sent over as a batch to composition engine 112. Composition engine 112 waits to receive the entire batch of visual objects and resources, before creating an analogous composition tree 116. hi addition to the objects and resources that are sent in the graphics stream, commands or instructions are included the graphics stream. The commands or instructions are particularly used to create the composition tree 116. When system 100 includes multiple application programs, because each application program has its own dedicated channel in which graphics streams, and particularly batches of data packets, are communicated, composition engine 112 does not have to wait for other batches or graphics streams to be communicated over the channel. In other words, the dedicated channels prevent interference of communication of graphics streams (data packet batches) from different application programs. Furthermore, dedicated channels allow inherent synchronization of graphics streams such that data packets are received in their relative chronological order. [00025] The composition tree 116 includes the same information (i.e., analogous objects and resources) as the visual tree 108; however, the composition tree 116 is formatted to be used by a process/function 118. The process/function 118 may be an operating system component or application program component that renders graphics or images using the composition tree 116. Process/function 118 may interpret or convert the composition tree 116 into lower-level graphics information such as individual pixel values or bitmaps which are used to render graphics on a display device (not shown). As further discussed below, objects and resources of composition tree 116, along with other commands or instructions as to reconstruct the composition tree 116 may be stored as a file in a local memory for future use or use in cases such as "refresh" when the graphics or image represented by the composition is lost.
[00026] Optimization and compression may be performed on the graphics streams as they are sent over the channel 110 or other channels. The optimization or compression may implement one of various optimization and/or compression methods. Furthermore, the graphic stream messages or graphics stream may be transported using one of various transport protocols (e.g., RPC, RDP, TCP/IP, etc.) and/or methods.
[00027] Fig. 2 shows an exemplary visual tree 108 and an exemplary composition tree 116. Visual tree 108 is made up of hierarchical graphic or visual objects Vl 200(1) to VN 200(N). The visual tree 108 represents graphics or an image. Each of the visual objects 200 is further described by commands, instructions, or resources (collectively referred to as "resources"). In particular, resources are used to describe how to draw the visual objects. Resources may have higher order or lower order resources. For example, the graphics object V3 200(3) is defined by a "draw line" resource 202. The "draw line" resource 202 is further described by a "pen" resource 204. The "pen" resource 204 is further described by a "brush" resource 206. [00028] As described above, the visual objects V 200 along with the resources
(e.g., resources 202, 204, 206) are translated into data packets and sent in a graphics stream, along with commands or instructions. The commands or instructions are used to construct the composition tree 116 using analogous composition objects Cl 208(1) to CN 208(N), and resources defining each of the composition objects C 208 (e.g., "draw line" resource 210, "pen" resource 212, and "brush" resource 214). As discussed above, a composition engine which receives the graphics stream of visual objects, resources, commands, and instructions may wait for all of the data packets to be received before constructing the composition. Therefore, the visual objects, resources, commands, and instructions may be sent as a batch. [00029] Fig. 3 shows an exemplary computing device 300 which implements visual trees and composition trees to provide and render graphics or images. Computing device 300 is an implementation of a trans-process system; however, it will become evident, that elements in computing device 300 may also be implemented as part of, or have analogous elements in trans-machine and trans-thread systems. Computing device 300 may be a conventional desktop personal computer (PC), which includes a local operating system, processing unit or processor(s) 302, and a storage system or memory 304. As one example, the computing device 300 is a general- purpose PC implemented with a Windows® brand operating system from the Microsoft Corporation. [00030] In this example, application program 306 and legacy application program 308 may be part of multiple application programs resident in memory 304 or which reside separate from memory 304. Programs 306 and 308 are particularly accessed and controlled by processor(s) 302. The legacy application program 308 may include application programs that implement graphics command formats such as GDI. [00031] Computing device 300 includes a framework and core 310 that supports application program 306. Framework and core 310 includes framework 104 and core 106 described above, hi this example, legacy application program 308 is supported by a legacy framework and core 312. In other implementations, a single framework and core may be used to support multiple application programs. Framework and core 310 and legacy framework and core 312 are particularly configured to create and provide visual trees, such as visual tree 108, based on commands received from respective application program 306 and legacy application program 308. [00032] Framework and core 310 is connected to a redirector 314 which may be an interface to a dedicated channel "A" that supports application program 306. Redirector 314 may be considered as a "top half of the channel "A". Legacy framework and core 312 is likewise connected to a legacy redirector 316 which is an interface to a separate channel "B" that supports the legacy application program 308. Likewise, redirector 316 may be considered as a "top half of the channel "B". Redirectors 314 and 316 maybe connected to or use a transport layer or transport 318. A redirection layer 320 is a bottom half of channel "A", and a redirection layer 322 is a bottom half of channel "B". [00033] A composition engine "A" 324 receives a graphics stream through channel "A" and creates a composition tree for use by process/function "A" 326. Likewise, composition engine "B" 328 receives a graphics stream through channel "B" and creates a composition tree for use by process/function "B" 330. Each of process/function "A" 326 and process/function "B" 330 uses the created composition trees to render or generate graphics or images to a local display or output device.
[00034] Fig. 4 shows a system 400 that implements an application and application program interfaces used to modify a composition tree of graphics or an image. In this example, system 400 is a modification of system 100 described above, hi certain cases, it is desirable to modify graphics or images that are provided by application programs, such as application program 306 and legacy application program 308. Considering that optimization or compression may take place when graphic stream messages or graphics streams are sent, and particularly in trans- machine systems, it is desirable to minimize degradation of the graphics or image when modification takes place. [00035] In this example, an application 402 is located between a top half of channel 404 and a bottom half of channel 406. The application 402 may perform a graphics or image modification, or in certain cases application 402 may be configured as an accessibility application that is able to access or look at a graphics stream that is being passed through the channel (i.e., between the channel halves 404 and 406). hi general, application 402 is considered as an intermediary that may be placed within a channel.
[00036] An example of graphics or image modification is enlarging or magnifying the graphics or image. The application 402 particularly modifies the graphics stream or particular data packets passing through the channel, such that a revised composition tree 408 is created. As further discussed below, the modification may be made on a particular node or nodes of the composition tree, regardless of the hierarchy of the node or nodes in the composition. If a relatively higher-level node is modified, all the nodes below it on the composition tree are affected (i.e., modified). [00037] A graphics stream application program interface (API) 410 and playback API 412 may also be provided and exposes the application 402 to the core 104 and composition engine 110. In particular, graphics stream API 410 is used to access a graphics stream from the core 104 and playback API 412 is used by composition engine 110 to open the graphics stream from application 402. [00038] Graphics stream API 410 may include a "graphics stream open" API which opens the graphics stream for reading. Furthermore, a pointer may be implemented and provided to application 402 by the "graphics stream open" API to the particular function or process which composition engine 110 supports. Graphics stream API 410 may further include a "graphics stream close" API which closes the graphics stream for reading. The "graphics stream close" API causes a close message to be sent to composition engine 110 or to the particular function or process which composition engine 110 supports. [00039] Graphics stream API 410 may also include a "graphics stream set transform hint" API that provides a hint or message to the core 104, or in the case of trans-machine system (e.g., terminal-service system) a graphics server (i.e., core 104 resident in the graphics server), that the application 402 intends to perform an action or modification (e.g., magnification) to the graphics stream. For example, in the case of magnification, the "graphics stream transform hint" API provides a message back to application program 102 to account for enlargement or magnification. Furthermore, a pointer may be provided as to the particular revised composition tree 404. The message may be sent through a back channel, such as back channel 114. [00040] Playback API 412 provides the ability for composition engine 112 to draw or create the revised composition tree 408. In general the playback API 412 exposes the bottom half of channel 406 and the composition engine 112 to the application 402.
[00041] Fig. 5 is a diagram of a modified or revised composition tree. In particular, the revised composition tree 408 as modified by application 402 is illustrated. In this example, the modification is to enlarge or magnify a prior composition tree (e.g., composition tree 116). [00042] When a modification takes place as to graphics or images, the modification is performed on the composition tree, and particularly to a node or nodes of the composition tree. Depending on which node or nodes or modified, other nodes may be affected. In other words, if a particular node is modified, all lower-level nodes of branches from that particular node are affected or modified. In this example, the node Cl 208(1) is modified by a "magnify" node 500. Therefore, the prior composition tree 116 is modified (i.e., magnified) and is provided as revised composition tree 408.
[00043] Fig. 6 shows a process 600 that provides for graphics streams representative of visual trees to a process or function. Process 600 may further be implemented as an API that supports modification of the graphics stream. The process 600 is illustrated as a collection of blocks in a logical flow graph, which represent a sequence of operations that can be implemented in hardware, software, firmware, or a combination thereof. In the context of software, the blocks represent computer instructions that, when executed by one or more processors, perform the recited operations. Although described as a flowchart, it is contemplated that certain blocks may take place concurrently or in a different order. The process 600 may be implemented, for example, by the system 100 of Fig. 1 and/or computing device 300 as discussed in Fig. 3, although the process 600 may be implemented by other architectures. [00044] At block 602, graphics commands are sent to and received by an application that is located in a channel. The application may be an accessibility application or an application that performs a modification. An example of such an application is application 402. The graphics stream may include data packets which are representative of visual objects and resources that make up a visual tree. [00045] At block 604, the application opens or accesses the graphics stream.
The opening may be related to merely reading the graphics streams and data packets of the graphics stream, or a modification may be performed on the graphics stream. Furthermore, a pointer may be provided to a function or processes supported by a composition engine receiving the graphics stream. [00046] At block 606, the actual modification of the graphics stream and/or data packets is performed. Examples of modification include magnification. In certain cases, a message may be provided to a core that provides the graphics streams, where the message indicates that modification such as magnification is to take place. [00047] At block 608, the modified stream is passed on to the composition engine which may create an composition tree based on the original visual tree, and changes or modifications from the modification. [00048] At block 610, the graphics stream may be closed. In other words, the application is precluded from reading the graphics stream. A message may be sent to the composition engine indicating that the closing is occurring or has occurred. [00049] Fig. 7 shows a process 700 that allows for communication between a framework and composition engine. Process 700 may further be implemented as a protocol to provide such communication. The process 700 is illustrated as a collection of blocks in a logical flow graph, which represent a sequence of operations that can be implemented in hardware, software, firmware, or a combination thereof. In the context of software, the blocks represent computer instructions that, when executed by one or more processors, perform the recited operations. Although described as a flowchart, it is contemplated that certain blocks may take place concurrently or in a different order. The process 700 may be implemented, for example, by the system 100 of Fig. 1 and/or computing device 300 as discussed in Fig. 3, although the process 600 may be implemented by other architectures. [00050] At block 702, a dedicated channel is created which allows communication between the framework or a core, and the composition engine. A dedicated back channel may also be created to allow the composition engine to send back messages to the framework or core.
[00051] At block 704, data packets are sent over or provided over the dedicated channel. The data packets may be part of a graphics stream and sent as a batch. The batch of data packets includes all objects and resources that are necessary to create an editable model. The model is representative of graphics or an image, and for example, may be a hierarchical tree such as visual tree 108 described above. [00052] At block 706, an application may be introduced in the channel to modify the graphics stream or data packets. Examples of the application include a graphics stream reader or a magnifier. A message may be returned to the framework or core, indicating that a modification is to be performed. [00053] At block 708, the editable model is built using the unmodified or modified data packets. As discussed, the editable model may be analogous hierarchical tree to the visual tree created at the core. In particular, the editable model is considered or referred to as a composition tree, such as composition trees 116 and 408.
CONCLUSION
[00054] The above described system supports communicating a editable model such as visual tree to a composition engine that creates a composition tree used to render graphics or an image. Although the invention has been described in language specific to structural features and/or methodological acts, it is to be understood that the invention defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claimed invention.

Claims

CLAIMSWhat is claimed is:
1. A method of modifying graphics commands representative of a visual tree comprising: receiving from the graphics commands in a graphics stream; opening the graphics stream; modifying the graphics stream into a modified graphics stream; and passing the modified graphics stream to a composition engine that creates a modified composition tree analogous to the visual tree.
2. The method of claim 1 wherein the receiving the graphics command in the graphics stream includes data packets representative of visual objects and resources that make up the visual tree.
3. The method of claim 1 wherein the opening is performed by an accessibility application.
4. The method of claim 1 wherein the opening includes providing a pointer to an application supported by the composition engine.
5. The method of claim 1 wherein the modifying comprises a magnification operation on the graphics the stream.
6. The method of claim 1 further comprising closing the graphics stream from reading.
7. The method of claim 6 wherein the closing further comprises sending a message to the composition engine that the closing is occurring.
8. The method of claim 1 further comprising providing a message to the core as to the modifying the graphics stream.
9. A method of communication between a framework and composition engine comprising: creating a dedicated channel between the framework and composition engine; providing data packets over the channel, wherein the data packets are representative of elements of an editable model of graphics; and building the editable model of the graphics at the composition engine.
10. The method of claim 9 wherein the creating further comprises creating a back channel for the composition engine to send status information to the framework.
11. The method of claim 9 wherein the providing further comprises batching the data packets, wherein the composition engine builds the editable model only when the complete batch of data packets is received.
12. The method of claim 9 wherein the providing comprises sending a graphics stream that includes the data packets over the channel.
13. The method of claim 9 further comprising accessing and modifying the data packets prior to the building.
14. The method of claim 9 wherein the editable model is a tree that includes visual elements and resources that are represented in the data packets, and commands are provided in the data packets in the building of the editable model.
15. A system comprising: a framework that accesses graphics commands representing graphics and images from one or more application programs; a core which receives the graphics commands from the framework, and creates a visual tree; a channel that communicates data packets from the core that represent objects and resources of the visual tree; and a composition engine connected t the channel, that creates a composition tree that is analogous to the visual tree.
16. The system of claim 15 wherein the core creates the visual tree by creating hierarchical visual objects and resources, wherein the resources describe the visual objects.
17. The system of claim 15 wherein the channel is a dedicated channel to the core and composition engine.
18. The system of claim 15 wherein the channel is comprised of a top half and a bottom half, wherein an application may be implemented between the top half and the bottom half.
19. The system of claim 15 further comprising a back channel, wherein the composition engine sends messages to the core through the back channel.
20. The system of claim 15 wherein the system is one of the following: a trans-machine system, a trans-process system, or a trans-thread system.
PCT/US2006/031718 2005-09-07 2006-08-15 High level graphics stream WO2007030283A1 (en)

Priority Applications (7)

Application Number Priority Date Filing Date Title
AU2006287781A AU2006287781A1 (en) 2005-09-07 2006-08-15 High level graphics stream
JP2008530060A JP4827923B2 (en) 2005-09-07 2006-08-15 High level graphics system
CA002621076A CA2621076A1 (en) 2005-09-07 2006-08-15 High level graphics stream
CN2006800327888A CN101258478B (en) 2005-09-07 2006-08-15 High level graphics stream
BRPI0615723-8A BRPI0615723A2 (en) 2005-09-07 2006-08-15 high level graphics sequence
EP06801462.0A EP1922634B1 (en) 2005-09-07 2006-08-15 High level graphics stream
NO20080928A NO20080928L (en) 2005-09-07 2008-02-25 Hoyniva graphics stream

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US71488005P 2005-09-07 2005-09-07
US60/714,880 2005-09-07
US11/279,194 2006-04-10
US11/279,194 US7609280B2 (en) 2005-09-07 2006-04-10 High level graphics stream

Publications (1)

Publication Number Publication Date
WO2007030283A1 true WO2007030283A1 (en) 2007-03-15

Family

ID=37829623

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2006/031718 WO2007030283A1 (en) 2005-09-07 2006-08-15 High level graphics stream

Country Status (11)

Country Link
US (1) US7609280B2 (en)
EP (1) EP1922634B1 (en)
JP (1) JP4827923B2 (en)
KR (1) KR20080052574A (en)
CN (1) CN101258478B (en)
AU (1) AU2006287781A1 (en)
BR (1) BRPI0615723A2 (en)
CA (1) CA2621076A1 (en)
NO (1) NO20080928L (en)
RU (1) RU2431185C2 (en)
WO (1) WO2007030283A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010525428A (en) * 2007-03-30 2010-07-22 マイクロソフト コーポレーション Remoting applications based on Windows Presentation Framework in an unconfigured desktop

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8527563B2 (en) * 2005-09-12 2013-09-03 Microsoft Corporation Remoting redirection layer for graphics device interface
CA2616324C (en) * 2008-02-04 2015-06-16 Omnivex Corporation Digital signage display
CA2822771C (en) * 2008-02-04 2015-09-29 Omnivex Corporation Subscription based content delivery for a digital signage network
CA2620337C (en) * 2008-02-04 2012-11-27 Omnivex Corporation Digital signage network
US8230180B2 (en) * 2008-06-11 2012-07-24 Samsung Electronics Co., Ltd. Shared memory burst communications
KR101496340B1 (en) * 2008-10-31 2015-03-04 삼성전자주식회사 Processor and method for controling memory
GB2479103B (en) 2009-01-29 2014-01-01 Hewlett Packard Development Co Selectively communicating data of a peripheral device to plural sending computers
US8307103B2 (en) * 2009-03-09 2012-11-06 Microsoft Corporation Tear-free remote desktop protocol (RDP) display
US20130127849A1 (en) * 2011-05-26 2013-05-23 Sebastian Marketsmueller Common Rendering Framework and Common Event Model for Video, 2D, and 3D Content
US9361715B2 (en) * 2011-06-02 2016-06-07 Microsoft Technology Licensing, Llc Global composition system
US10031636B2 (en) * 2011-09-08 2018-07-24 Microsoft Technology Licensing, Llc Remoting desktop displays using move regions
TWI482118B (en) * 2012-10-18 2015-04-21 Wistron Corp System and method for processing load balance of graphic streams
US9542906B2 (en) 2013-05-10 2017-01-10 Microsoft Technology Licensing, Llc Shared compositional resources
US10796397B2 (en) 2015-06-12 2020-10-06 Intel Corporation Facilitating dynamic runtime transformation of graphics processing commands for improved graphics performance at computing devices

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5315698A (en) * 1991-08-21 1994-05-24 Digital Equipment Corporation Method and apparatus for varying command length in a computer graphics system
US6388679B1 (en) * 1998-12-29 2002-05-14 Intel Corporation Multi-resolution computer display system
US20030052886A1 (en) 2000-06-28 2003-03-20 Naegle Nathaniel D. Transferring a digital video stream through a series of hardware modules

Family Cites Families (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5662383A (en) * 1995-08-11 1997-09-02 Bemis Manufacturing Company Apparatus for attaching fabric to a chair frame
US5657390A (en) * 1995-08-25 1997-08-12 Netscape Communications Corporation Secure socket layer application program apparatus and method
US5842040A (en) * 1996-06-18 1998-11-24 Storage Technology Corporation Policy caching method and apparatus for use in a communication device based on contents of one data unit in a subset of related data units
US6434578B1 (en) * 1996-06-26 2002-08-13 Microsoft Corporation Generating instruction sequences using independently executable pane renderers
JPH1074137A (en) * 1996-08-30 1998-03-17 Hitachi Ltd Graphic user interface change system
KR0175570B1 (en) * 1996-10-16 1999-04-01 양승택 MPEG-2 TS Multiplexing Device for ATM Network Matching
US5862383A (en) 1996-11-27 1999-01-19 Nokia Research Center Generation of code by graphical representation
US7200809B1 (en) * 1999-08-04 2007-04-03 Oracle International Corporation Multi-device support for mobile applications using XML
US6452600B1 (en) * 1999-10-28 2002-09-17 Nintendo Co., Ltd. Graphics system interface
US6411301B1 (en) * 1999-10-28 2002-06-25 Nintendo Co., Ltd. Graphics system interface
US6538656B1 (en) * 1999-11-09 2003-03-25 Broadcom Corporation Video and graphics system with a data transport processor
JP2001273520A (en) * 2000-03-23 2001-10-05 Famotik Ltd System for integrally displaying multimedia document
US6798418B1 (en) * 2000-05-24 2004-09-28 Advanced Micro Devices, Inc. Graphics subsystem including a RAMDAC IC with digital video storage interface for connection to a graphics bus
US7630721B2 (en) * 2000-06-27 2009-12-08 Ortiz & Associates Consulting, Llc Systems, methods and apparatuses for brokering data between wireless devices and data rendering devices
JP2002024298A (en) * 2000-07-05 2002-01-25 Ricoh Co Ltd Three-dimensional shape processor, three-dimensional shape displaying method, and storage medium stored with program for implementing the same method
US7274368B1 (en) * 2000-07-31 2007-09-25 Silicon Graphics, Inc. System method and computer program product for remote graphics processing
US7002591B1 (en) * 2000-08-23 2006-02-21 Nintendo Co., Ltd. Method and apparatus for interleaved processing of direct and indirect texture coordinates in a graphics system
US6867781B1 (en) * 2000-08-23 2005-03-15 Nintendo Co., Ltd. Graphics pipeline token synchronization
US7196710B1 (en) * 2000-08-23 2007-03-27 Nintendo Co., Ltd. Method and apparatus for buffering graphics data in a graphics system
US6697074B2 (en) * 2000-11-28 2004-02-24 Nintendo Co., Ltd. Graphics system interface
US7379475B2 (en) * 2002-01-25 2008-05-27 Nvidia Corporation Communications processor
US20020103935A1 (en) * 2001-01-26 2002-08-01 Neil Fishman Pushing rich content information to mobile devices
JP4688316B2 (en) * 2001-02-28 2011-05-25 独立行政法人産業技術総合研究所 Simulation apparatus and simulation method, and video generation apparatus and video generation method
WO2002084879A2 (en) * 2001-04-13 2002-10-24 The Code Coproration System and method for encoding and decoding data and references to data in machine-readable graphical codes
US20030017846A1 (en) * 2001-06-12 2003-01-23 Estevez Leonardo W. Wireless display
US7216288B2 (en) * 2001-06-27 2007-05-08 International Business Machines Corporation Dynamic scene description emulation for playback of audio/visual streams on a scene description based playback system
US7191233B2 (en) * 2001-09-17 2007-03-13 Telecommunication Systems, Inc. System for automated, mid-session, user-directed, device-to-device session transfer system
US7064766B2 (en) * 2001-10-18 2006-06-20 Microsoft Corporation Intelligent caching data structure for immediate mode graphics
US7161599B2 (en) * 2001-10-18 2007-01-09 Microsoft Corporation Multiple-level graphics processing system and method
US7439982B2 (en) * 2002-05-31 2008-10-21 Envivio, Inc. Optimized scene graph change-based mixed media rendering
US7075538B2 (en) * 2002-08-30 2006-07-11 Sun Microsystems, Inc. Methods and apparatus for faster line drawing on remote displays
US20040177373A1 (en) * 2003-02-28 2004-09-09 Tsuyoshi Kawabe Graphic data distribution system and a data converter, and a graphic data distribution method
US7434166B2 (en) * 2003-06-03 2008-10-07 Harman International Industries Incorporated Wireless presentation system
WO2005039185A1 (en) * 2003-10-06 2005-04-28 Mindego, Inc. System and method for creating and executing rich applications on multimedia terminals
US7643731B2 (en) * 2004-01-23 2010-01-05 Osamu Kobayashi Low power DVD playback in a portable computing system
US7562131B2 (en) * 2004-06-25 2009-07-14 Intel Corporation UPnP user interface system and method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5315698A (en) * 1991-08-21 1994-05-24 Digital Equipment Corporation Method and apparatus for varying command length in a computer graphics system
US6388679B1 (en) * 1998-12-29 2002-05-14 Intel Corporation Multi-resolution computer display system
US20030052886A1 (en) 2000-06-28 2003-03-20 Naegle Nathaniel D. Transferring a digital video stream through a series of hardware modules

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010525428A (en) * 2007-03-30 2010-07-22 マイクロソフト コーポレーション Remoting applications based on Windows Presentation Framework in an unconfigured desktop

Also Published As

Publication number Publication date
EP1922634B1 (en) 2018-01-17
RU2008108840A (en) 2009-09-20
RU2431185C2 (en) 2011-10-10
US7609280B2 (en) 2009-10-27
EP1922634A4 (en) 2012-09-26
BRPI0615723A2 (en) 2012-04-10
CN101258478B (en) 2012-06-27
JP2009507306A (en) 2009-02-19
US20070052723A1 (en) 2007-03-08
EP1922634A1 (en) 2008-05-21
NO20080928L (en) 2008-04-03
CN101258478A (en) 2008-09-03
AU2006287781A1 (en) 2007-03-15
KR20080052574A (en) 2008-06-11
JP4827923B2 (en) 2011-11-30
CA2621076A1 (en) 2007-03-15

Similar Documents

Publication Publication Date Title
US7609280B2 (en) High level graphics stream
JP5373135B2 (en) Efficient encoding of alternative graphic sets
US9654586B2 (en) Resource server providing a rapidly changing resource
JP2011129153A (en) System and method for unified composition engine in graphics processing system
US20080250424A1 (en) Seamless Window Implementation for Windows Presentation Foundation based Applications
AU2004279201B2 (en) Protocol for remote visual composition
CN107402771B (en) Remote user interface collaboration application
CN112688992A (en) vue method and device for displaying flow chart auditing state

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 200680032788.8

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 2006801462

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2621076

Country of ref document: CA

WWE Wipo information: entry into national phase

Ref document number: 2006287781

Country of ref document: AU

WWE Wipo information: entry into national phase

Ref document number: KR

WWE Wipo information: entry into national phase

Ref document number: 2008108840

Country of ref document: RU

WWE Wipo information: entry into national phase

Ref document number: MX/a/2008/003311

Country of ref document: MX

Ref document number: 2008530060

Country of ref document: JP

Ref document number: 1982/DELNP/2008

Country of ref document: IN

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2006287781

Country of ref document: AU

Date of ref document: 20060815

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: PI0615723

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20080306