|Publication number||US7366816 B2|
|Application number||US 11/146,506|
|Publication date||Apr 29, 2008|
|Filing date||Jun 7, 2005|
|Priority date||Jun 22, 2004|
|Also published as||CA2509001A1, CN1712191A, EP1609546A1, US7263753, US20050278918, US20050283634|
|Publication number||11146506, 146506, US 7366816 B2, US 7366816B2, US-B2-7366816, US7366816 B2, US7366816B2|
|Inventors||Barinder Singh Rai|
|Original Assignee||Seiko Epson Corporation|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (17), Referenced by (16), Classifications (27), Legal Events (4)|
|External Links: USPTO, USPTO Assignment, Espacenet|
The invention relates to a method and apparatus for adaptively adjusting the bandwidth of a data transmission channel having multiple buffered paths. The invention is particularly well suited for adjusting the bandwidth of such a data transmission channel within a graphics controller that internally transmits image data corresponding to multiple windows over multiple display pipes for provision to a graphics display device.
Graphics display systems, such as cellular telephones, typically employ a graphics controller as an interface between one or more providers of image data and a graphics display device such as an LCD panel or panels. In a cellular telephone, the providers of image data are typically a host processor (CPU) and a camera (any and all such providers hereinafter being generally considered a “host”).
The image data are transmitted from a host to the graphics controller where the data are stored in a memory. The image data typically correspond to different “windows” for display by the display device. For example, one host, such as the CPU, may provide a “main window” for displaying associated main window image data and another host, such as the camera, may provide a “sub-window” for displaying associated sub-window image data that overlay the main window data. An artifact known as a “sprite” may also be provided. A typical example of a “sprite” is the cursor provided by the CPU. The “sprite” typically overlays both the main and sub-window image data.
The image data corresponding to the windows are typically fetched from the memory and transmitted or clocked through respective FIFO (“first-in-first out”) buffers or “display pipes” to a selecting circuit. The selecting circuit selects the image data, on a pixel-by-pixel basis, from one of the pipes for further transmission to the display device. For example, for a pixel located at a particular row and column on the display falling within a sub-window that overlays a main window, but which is not overlaid by a sprite, the selecting circuit selects the image data at the end of the display pipe for the sub-window and does not select the image data at the end of the display pipes corresponding to either the main window or the sprite.
The display pipes provide a buffering function that is advantageous because the rate at which image data may be fetched from the memory is typically higher than the rate at which the image data are clocked out to the display device. From this it can be seen that the memory may fill the display pipe relatively quickly, and while the display pipe is being emptied relatively slowly, the memory is freed for alternative uses. A “near-empty” signal is generated when the display pipe is nearly empty, which is used to trigger the memory accesses needed to refill the display pipe. It may be noted that the rate at which data are transmitted through the display pipe is often referred to as the “bandwidth” of the display pipe. The concept applies to other channels of data transmission as well, and can even apply to analog transmission channels.
The rate at which image data are introduced into or fill the display pipe is determined by the (clock) rate at which memory may be accessed (referred to as “MCLK”), and the rate at which the image data are clocked out of the display pipe is determined by the rate at which the graphics display device can accept the image data (referred to as “PCLK”).
The clock rates MCLK and PCLK are distinguished from the frequency of the near-empty signal used for triggering memory accesses or fetches. Where the image data corresponding to a particular display pipe are required relatively infrequently, the near-empty signal is likewise generated relatively infrequently. Where there are multiple display pipes or paths, the display pipes are typically not selected with the same frequency, and are therefore not emptied with the same frequency, even though they are emptied at the same rate. The bandwidth of a given one of the display pipes is therefore a function of both the frequency of occurrence of the near-empty signal and the rate at which data are output from the pipe. Particularly, the bandwidth is proportional to the product of this frequency and this rate.
The clock rate for memory fetches must at least be high enough to keep up with the requirements of all the display pipes; otherwise, one or more display pipes will be starved of data. This starvation will result in the appearance of an undesirable artifact on the display. To ensure that starvation does not occur, MCLK is set high enough to service all of the pipes under the worst case condition that all of the pipes require refilling at the same time.
In many graphics display systems, it is also desirable to decrease power consumption. This is particularly so in portable, battery powered systems such as cellular telephones. And in general, power consumption is greater, the greater the clock rate.
Accordingly, there is a need for a method and apparatus for adaptively adjusting the bandwidth of a data transmission channel having multiple paths that provides for optimizing the bandwidth to reduce power consumption.
According to the invention, a method and apparatus for adaptively adjusting the bandwidth of a data transmission channel having multiple buffered paths is disclosed. Each output path includes a buffer for holding respective portions of the data. A value representative of at least the number of said buffers that are nearly empty of data as compared to a threshold is determined, and the transmission rate of the input path is adjusted according to said value. Preferably, the value is determined dynamically as the number of buffers that are empty of data change over time. In addition, the buffers are preferably display pipes provided in a graphics controller chip for interfacing between one or more hosts and a graphics display device.
It is to be understood that this summary is provided as a means of generally determining what follows in the drawings and detailed description and is not intended to limit the scope of the invention. Objects, features and advantages of the invention will be readily understood upon consideration of the following detailed description taken in conjunction with the accompanying drawings.
As mentioned above, the invention relates to a method and apparatus for adaptively adjusting the bandwidth of a data transmission channel having multiple buffered paths. Reference will now be made in detail to the present preferred embodiments of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers are used in the drawings and the description to refer to the same or like parts.
The system 8 includes a primary host 12 and a graphics display device 14, and the graphics controller interfaces between the host and the display device. The graphics controller is typically and preferably a single IC separate from the host and separate from the display device 14.
The host 12 is preferably a microprocessor, but may be a computer or any other provider of image data. The system 8 typically has an associated system memory 13.
The system also typically, though not necessarily, includes a secondary host or camera 15 that also provides image data to the graphics controller 10. An asynchronous camera interface 15 a is employed to provide image data from the camera 15 to the graphics controller 10.
The graphics controller 10 receives data and instructions output from the host 12 onto a bus 16 through a host I/F 12 a. The bus 16 may be serial or parallel, and may be organized to transmit the data and instructions over the same line(s) or over separate line(s) of the bus.
The graphics controller 10 includes an internal memory 24 and a memory controller 28. Image data are provided by the host 12 and the camera 15 to the memory controller, which stores the data in the internal memory.
The pixels originate from the host 12, or the camera 15, and are transmitted to the display device through the graphics controller 10.
Image data may be displayed on the display device 14 within windows, each window defining, typically, a rectangular area within the display area 18. Image data corresponding to a window populate the window. There may be, and often are, more image data corresponding to a window than can be seen in the window.
Often the system provides for displaying image data corresponding to a larger “main” window 20 within which may be displayed image data corresponding to a smaller “sub-window” 22. The sub-window typically overlays the main window. The system also typically provides a “sprite” 23. Typically, the sprite overlays any and all main windows and sub-windows. Any number of main windows, sub-windows, or sprites (collectively “windows”) may be provided.
Referring back to
The graphics controller 10 includes a data transmission channel having at least one input path and at least two output paths. For illustrative purposes, the system 8 is described with a single input path from the memory controller 28, it being understood that with suitable modification additional input paths may be provided without departing from the principles of the invention.
Preferably, three parallel output paths, more particularly three display pipes 26 corresponding to the three windows, are provided for transferring the image data from the memory 24 to the graphics display device 14. Particularly, data from the main storage region 24 a are fetched and input to a main window pipe 26 a, data from the sub-window storage region 24 b are fetched and input to a sub-window pipe 26 b, and data from the sprite storage region 24 c are fetched and input to a sprite pipe 26 c. Typically, the host 12 provides the main image data and the sprite data, and either the camera 15 or the host 12 provides the sub-window image data, though this is not essential. Additional windows and additional pipes for transmitting or propagating image data associated with the additional windows may also be provided. The display pipes are preferably dual-port FIFO memories, but this is not essential to the invention and the display pipes may be, for example, a random access memory provided alone or in conjunction with a suitable controller.
The image data are fetched from the internal memory 24 and written to the display pipes 26 by the memory controller 28 at a clock rate MCLK. In turn, image data are fetched from the pipes 26 and transmitted over the display device interface 14 a to the graphics display device 14 by a graphics display device control module 30 at a clock rate PCLK. The clock rate PCLK is typically substantially lower than the clock rate MCLK.
The graphics display device control module 30 selects image data from one of the pipes for transmission to the display device. Such a selection is necessary when there are image data available from more than one pipe that correspond to the same pixel on the display device. In that case, image data available from a pipe corresponding to a window having priority “overlay” image data available from a pipe corresponding to all other windows having lesser priority. For example, it is typically desired to overlay sub-window image data over main window image data and to overlay sprite image data over either sub-window image data or main window image data. By overlaying image data, only the image data corresponding to the window having priority are displayed. Window priority is typically either assumed or specified by the host.
Fetching all of the image data corresponding to all of the windows and then selecting only the image data having priority is less than optimally efficient; however, it is standard practice and therefore exemplary. It should be understood that the invention may be used in accordance with any scheme or methodology for increasing the efficiency of data transfer through the system.
Each display pipe has associated therewith a level indicator L; more particularly a level indicator Lmain is associated with the display pipe 26 a; a level indicator Lsub is associated with the display pipe 26 b; and a level indicator Lsprite is associated with the display pipe 26 c. The level indicators indicate the number of pixels of data remaining in the associated display pipes. For example, the display pipe 26 a includes storage space for pixels P1, P2, . . . PN.
“Upon start-up of the graphics controller 10, the memory controller 28 will fill the display pipes 26, including the display pipe 26 a, with image data. For example, if N equals 64, the display pipe 26 a will contain 64 pixels of image data. Thereafter, data will be output from the display pipe 26 a according to demand, which cannot in general be predicted. The image data level in the pipe will first fall, and thereafter rise and fall according to the rate and frequency at which image data are written to the pipe and the rate and frequency at which image data are withdrawn from the pipe.”
The level indicators may keep track of the data remaining in the associated pipes simply by counting, i.e., summing the data input and subtracting the data output. The level indicators may therefore be implemented as increment/decrement modules. However, the level indicators may be implemented in alternative ways, including ways that do not provide the precision of counting each pixel of data. For example, the level indicators may indicate whether the pipe level exceeds (or does not exceed) a limited number of levels, such as 25%, 50% and 75%. In simplest form, the level indicators may simply indicate that a single level has been reached. For example, a level indicator according to the invention may provide an indication only when the associated pipe is nearly empty, e.g., the level declines to 20%. Such an indication is referred to herein as a “near-empty flag” or “NE flag.” For the three display pipes described above, three corresponding NE flags may be provided, namely, NEmain, NEsub, and NEsprite.
As discussed above, the clock generator 40 generates two types of clock signals. The clock generator circuit generates a clock signal MCLK for clocking the memory controller 28 and thereby controlling the rate at which data are fetched and input to the display pipes, and also generates a clock signal PCLK that determines the rate at which data are output from the display pipe to the graphics display device. The clock signal PCLK is determined so as to be compatible with the graphics display device.
However, according to the present invention, an adaptive bandwidth adjustment circuit 42 is provided including a clock generator 40 that, in contrast to the prior art, produces a number of alternative clock signals for use as the signal MCLK, each having a different frequency. The circuit 42 also includes a select circuit 44 that receives the values of the level indicators L as inputs “A,” “B,” and “C.” The frequency select circuit 44 outputs a select signal SEL that may be used to control a multiplexer 46 for selecting the desired clock signal. Other frequency selection or control methods may be used; for example, the signal SEL may be used to control a phase locked loop.
Where the level indicators L merely indicate whether the display pipe is nearly empty or not by use of NE flags, the inputs A, B and C are either “high” or “low.” The frequency select circuit 44 may produce a unique output for any value of ABC. As one illustrative example, the frequency select circuit 44 sums the flags for all of the pipes and produces an output or value according to the truth table below:
In this example, a simple sum of simple binary level indications is used to select from among the alternative clock frequencies 30, 45, 60, and 73.5 MHz, providing the outstanding advantage of dynamically adapting the MCLK frequency to provide the amount of bandwidth needed under different circumstances. For example, when the graphics controller 10 is first powered up, all of the display pipes 26 will be empty. At that time, all of the NE flags will be set at 1 and the circuit 42 will select the highest clock frequency for the clock signal MCLK (e.g., 73.5 MHz) to fill the pipes. Once the pipes are all filled, all of the NE flags will be set to 0 and the circuit 42 will select the lowest clock frequency for MCLK (e.g., 30 MHz). This lower clock frequency reduces power consumption to a desired minimum. Thereafter, as the display pipes are emptied at different frequencies, the NE flags for different combinations of the pipes will be set back to 1 at particular times at which an optimum intermediate frequency for MCLK can be used.
The truth table shown in
In general, the level indicators L may indicate the data level in a display pipe to any precision up to and including a precision equal to a single pixel. It will be readily appreciated that the output of the frequency select circuit 44 could be more highly tuned where precision is increased.
As will be readily appreciated, as an alternative to adjusting the clock rate MCLK for clocking the memory 24, the bandwidth of the data transmission channel through the graphics controller may be adjusted by adjusting the width of the input path, which in the example is the bus 16. For instance, a 24-bit parallel bus may be provided and bandwidth may be adjusted by alternately employing 8, 16, or 24 bits of the bus.
It is to be recognized that, while a particular method and apparatus for adaptively adjusting the bandwidth of a data transmission channel having multiple buffered paths has been shown and described as preferred, other configurations and methods could be utilized, in addition to those already mentioned, without departing from the principles of the invention. For example, while described in the context of a preferred, graphics system 8, the invention may be used in other types of systems and may employ other types of buffered paths in addition to electronic buffers, such as optical or acoustic buffers. The invention may also use input paths that bypass a memory such as the memory 24. For example, data could be input to the graphics controller from a host which writes directly to the display pipes. The invention is believed to have wide applicability.
It should be understood that, while preferably implemented in hardware, the invention may be implemented in a combination of hardware and software, or only in software, provided the graphics controller is suitably adapted. For example, a program of instructions stored in a machine readable medium may be provided for execution by an embedded processor included in the graphics controller.
The terms and expressions which have been employed in the foregoing specification are used therein as terms of description and not of limitation, and there is no intention in the use of such terms and expressions to exclude equivalents of the features shown and described or portions thereof, it being recognized that the scope of the invention is defined and limited only by the claims which follow.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US4998246 *||Jul 31, 1989||Mar 5, 1991||Mitsubishi Denki Kabushiki Kaisha||Method for transmission of cyclic data|
|US5191436 *||Apr 30, 1991||Mar 2, 1993||Sony Corporation||Method for recording coded motion picture data|
|US5809538||Feb 7, 1996||Sep 15, 1998||General Instrument Corporation||DRAM arbiter for video decoder|
|US6012109||Sep 9, 1997||Jan 4, 2000||National Instruments Corporation||Video capture device with adjustable frame rate based on available bus bandwidth|
|US6105086 *||Jun 4, 1998||Aug 15, 2000||Lsi Logic Corporation||Data communication circuit and method for buffering data between a shared resource and multiple interfaces|
|US6330647||Aug 31, 1999||Dec 11, 2001||Micron Technology, Inc.||Memory bandwidth allocation based on access count priority scheme|
|US6366325||Dec 7, 1998||Apr 2, 2002||Ati International Srl||Single port video capture circuit and method|
|US6563506||Dec 14, 1998||May 13, 2003||Ati International Srl||Method and apparatus for memory bandwith allocation and control in a video graphics system|
|US6662278||Sep 22, 2000||Dec 9, 2003||Intel Corporation||Adaptive throttling of memory acceses, such as throttling RDRAM accesses in a real-time system|
|US6720968||Dec 11, 1998||Apr 13, 2004||National Instruments Corporation||Video acquisition system including a virtual dual ported memory with adaptive bandwidth allocation|
|US6912612 *||Feb 5, 2003||Jun 28, 2005||Intel Corporation||Shared bypass bus structure|
|US7016303 *||Sep 13, 2000||Mar 21, 2006||Sony Corporation||Transmitting method transmitting system and transmitter|
|US7047374 *||Feb 5, 2003||May 16, 2006||Intel Corporation||Memory read/write reordering|
|US20030147634||Jul 15, 1999||Aug 7, 2003||Masayuki Takezawa||Signal processing apparatus, control method for signal processing apparatus, imaging apparatus recording/reproducing apparatus|
|US20030163619 *||Dec 19, 2002||Aug 28, 2003||Kabushiki Kaisha Toshiba||Buffer controller and buffer control method|
|US20040128437||Dec 30, 2002||Jul 1, 2004||Alon Regev||CAM with policy based bandwidth allocation|
|US20040263427 *||Jun 25, 2003||Dec 30, 2004||Horigan John W.||Lossless clock domain translation for a pixel stream|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7536490 *||Jul 20, 2006||May 19, 2009||Via Technologies, Inc.||Method for link bandwidth management|
|US7991938 *||Jul 26, 2007||Aug 2, 2011||Samsung Electronics Co., Ltd.||Bus width configuration circuit, display device, and method configuring bus width|
|US8209690 *||Jan 19, 2007||Jun 26, 2012||University Of Maryland||System and method for thread handling in multithreaded parallel computing of nested threads|
|US8218091 *||Apr 17, 2007||Jul 10, 2012||Marvell World Trade Ltd.||Shared memory multi video channel display apparatus and methods|
|US8264610||Apr 17, 2007||Sep 11, 2012||Marvell World Trade Ltd.||Shared memory multi video channel display apparatus and methods|
|US8284322||Apr 17, 2007||Oct 9, 2012||Marvell World Trade Ltd.||Shared memory multi video channel display apparatus and methods|
|US8402298 *||Feb 19, 2008||Mar 19, 2013||Renesas Electronics Corporation||Array-type processor having delay adjusting circuit for adjusting a clock cycle in accordance with a critical path delay of the data path|
|US8736757||Jun 15, 2012||May 27, 2014||Marvell World Trade Ltd.||Shared memory multi video channel display apparatus and methods|
|US8754991||Sep 14, 2012||Jun 17, 2014||Marvell World Trade Ltd.||Shared memory multi video channel display apparatus and methods|
|US8804040||Aug 9, 2012||Aug 12, 2014||Marvell World Trade Ltd.||Shared memory multi video channel display apparatus and methods|
|US20080022024 *||Jul 20, 2006||Jan 24, 2008||Jin-Liang Mao||Method for link bandwidth management|
|US20080055462 *||Apr 17, 2007||Mar 6, 2008||Sanjay Garg||Shared memory multi video channel display apparatus and methods|
|US20080055466 *||Apr 17, 2007||Mar 6, 2008||Sanjay Garg||Shared memory multi video channel display apparatus and methods|
|US20080055470 *||Apr 17, 2007||Mar 6, 2008||Sanjay Garg||Shared memory multi video channel display apparatus and methods|
|US20080201526 *||Feb 19, 2008||Aug 21, 2008||Nec Electronics Corporation||Array-type processor having delay adjusting circuit|
|US20090125907 *||Jan 19, 2007||May 14, 2009||Xingzhi Wen||System and method for thread handling in multithreaded parallel computing of nested threads|
|International Classification||B23P11/00, B21J15/04, B21J15/06, G06F1/12, G06F13/36, G06F13/40, G06F1/04, G06F5/06, G09G5/36, G09G5/395, G09G5/397|
|Cooperative Classification||Y10T29/5373, B21J15/105, G09G2340/12, B21J15/043, G09G5/397, G09G5/363, B21J15/045, G09G5/395, Y10T29/53739|
|European Classification||B21J15/10B, B21J15/04B, B21J15/04B2, G09G5/36C, G09G5/397, G09G5/395|
|Jun 7, 2005||AS||Assignment|
Owner name: EPSON RESEARCH AND DEVELOPMENT, INC., CALIFORNIA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:RAI, BARINDER SINGH;REEL/FRAME:016687/0701
Effective date: 20050531
|Jul 5, 2005||AS||Assignment|
Owner name: SEIKO EPSON CORPORATION, JAPAN
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:EPSON RESEARCH AND DEVELOPMENT, INC.;REEL/FRAME:016474/0055
Effective date: 20050627
|Sep 14, 2011||FPAY||Fee payment|
Year of fee payment: 4
|Oct 14, 2015||FPAY||Fee payment|
Year of fee payment: 8