|Publication number||US6324616 B2|
|Application number||US 09/835,279|
|Publication date||Nov 27, 2001|
|Filing date||Apr 13, 2001|
|Priority date||Nov 2, 1998|
|Also published as||US6233645, US20010014928|
|Publication number||09835279, 835279, US 6324616 B2, US 6324616B2, US-B2-6324616, US6324616 B2, US6324616B2|
|Inventors||George Z. Chrysos, P. Snyder II Wilson|
|Original Assignee||Compaq Computer Corporation|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (16), Non-Patent Citations (5), Referenced by (28), Classifications (19), Legal Events (7)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This application is a continuation of application Ser. No. 09/184,623, filed on Nov. 2, 1998. now U.S. Pat. No. 6,233,645. The entire teachings of the above application is incorporated herein by reference.
Modern microprocessors comprise many resources such as registers and memory for which various requesters contend. For example, a typical microprocessor has various layers of memory including local instruction and data caches, lower level backup caches (B-cache), main memory, storage devices such as disk drives, and storage available over a network. Generally, these layers form a hierarchical set of memories where at one extreme, the local caches are very fast but tend to be small, whereas at the other extreme, main memory, disk storage, and network storage tend to have very large capacities, but access times are also magnitudes larger than for local memory.
To reduce the impact of the long latency between processor and main memory or other long latency storage, hardware prefetching techniques are utilized. The CPU predicts which memory blocks it will utilize next and requests those blocks from the memory hierarchy before they are actually needed.
Because it cannot be predicted with certainty which branch a branch instruction in the instruction stream will take, these prefetched instructions are said to be speculative. Unfortunately, these prefetches can consume bandwidth to the lower level caches, main memory, etc., reducing the amount of available bandwidth for memory transactions that are needed as soon as possible, for example, for demand misses and cache-victim processing.
The present invention addresses this problem by providing a mechanism which dynamically detects when a high percentage of memory-access bandwidth is needed for high-priority operations such as demand misses, and which, based on this detection, shuts down low priority requesters such as hardware prefetchers until the demand for the bandwidth is reduced.
A preferred embodiment of the present invention realizes this mechanism by counting the number of high-priority utilizations from the last N opportunities in which the high-priority utilizations could have been requested. When the count exceeds some threshold, indicating high usage by the high-priority requester, a portion of low-priority requests are throttled or disabled. Priority may be determined, for example, by measuring the degree of speculative use of the resource. Thus, speculative prefetches have a lower priority than demand accesses.
In other words, the present invention works on the principle that if bandwidth utilization due to high-priority accesses has been high in the recent past, then it will continue to be high due to high-priority accesses in the near future. To that end, the present invention prohibits hardware prefetchers, which have a lower priority, from making requests during high bandwidth utilization periods to increase overall performance.
Accordingly, a method for arbitrating among a plurality of requesters of resources in a computer system comprises selecting a first requester based on a history of program execution. Other requesters which compete with the first requester are inhibited.
Preferably, the history is based on past utilization of the resources. Preferably, memory, including but not limited to main memory and cache memory, and associated data paths, are resources from which a demand fetcher and a speculative prefetcher request instructions or data. The demand fetcher is the first requester, while the speculative prefetcher is a competing requester.
In a preferred embodiment of the present invention, low-priority utilization of a resource in a digital processor, such as speculative prefetching, is limited in favor of high-priority utilization of the resource such as demand fetching, by determining a value which is predictive of high-priority utilization of the resource. If the predictive value is greater than some threshold, low-priority utilization of the resource is inhibited, or throttled. If the predictive value is less than or equal to the threshold, low-priority utilization of the resource is allowed. Preferably, the threshold is a predefined constant.
Furthermore, the predictive value is preferably determined by counting the number of opportunities in which the resource could have been utilized for a high-priority need, regardless of whether it actually was so utilized. The number of actual high-priority utilizations of the resource is also counted, and divided by the opportunity count, yielding the predictive value.
In a preferred embodiment, some actual high-priority utilizations are given more weight than others. Preferably, recent high-priority utilizations are weighted more heavily than less recent high-priority utilizations.
In another preferred embodiment, the number m of actual high-priority uses from the last N use opportunities is first determined, where N is predefined. The value m is compared with some threshold. Speculative accesses are inhibited if m is greater than a threshold, but are allowed otherwise.
In a preferred embodiment, m is determined by providing an N-bit shift register. A 1 is shifted into the N-bit shift register for each high-priority use opportunity in which there has been an actual high-priority use. A 0 is shifted into the shift register for all other use opportunities, such that a count of 1 s in the shift register provides a value for m. Optionally, hysteresis can be used about the threshold.
In a further preferred embodiment, a second shift register is provided. When a 0 is shifted in to the N-bit shift register and 1 is shifted out, the second register is shifted left, with a 0 being shifted in from the right. When a 1 is shifted in to the N-bit register and 0 is shifted out, the second register is shifted right, with a 1 being shifted in from the left. When a certain predesignated bit in the second shift register, associated with the threshold has a value of 1, m is deemed to exceed the threshold.
Similarly, where a plurality of options is available, the techniques of the present invention may be employed by associating a priority level with each option. A value is determined based on past behavior of at least one of the options. At least one high-priority option is selected if the determined value exceeds a threshold such that lower priority options are effectively inhibited. On the other hand, any option may be selected if the determined value does not exceed the threshold.
Finally, where multiple threads are executing, the techniques of the present invention may be employed to select a thread based on utilization of bandwidth, or based on a prioritization among the threads.
The foregoing and other objects, features and advantages of the invention will be apparent from the following more particular description of preferred embodiments of the invention, as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention.
FIG. 1 is a block diagram of a computer system having a main memory and backup cache.
FIG. 2 is a schematic diagram of a preferred embodiment of the present invention using a shift register and counting logic.
FIG. 3 is a schematic diagram of the embodiment of FIG. 2 with weights added.
FIG. 4 is a schematic diagram depicting the use of an upper and lower threshold to characterize a variation of the embodiments of the FIGS. 2 and 3 with hysteresis.
FIG. 5A is a schematic diagram of a preferred embodiment with two shift registers.
FIG. 5B is a schematic diagram of a preferred embodiment of the present invention using a counter.
FIG. 1 shows a general computer system 100. A microprocessor (CPU) 101 generally fetches its instructions and data from a main memory 113. However, this main memory and the system bus 111 over which instructions and data travel, tend to slow down the execution of the microprocessor 101 which must wait to receive the instructions or data.
To speed up execution, a modern microprocessor 101 typically has an on-board instruction cache, or 1-cache 103, and a data cache, or D-cache 105. Based on the fact that recently-fetched instructions or data tend to be reused, these caches 103, 105 provide the microprocessor 101 with near-instantaneous access to the instructions or data. While the discussion hereafter focuses only on instruction fetching, it will be recognized by those skilled in the art that the discussion applies equally to data fetching.
Memory hierarchies may comprise several caching levels. The first level instruction and data caches described above need to be accessed very quickly. This restriction limits the size of these first-level caches, creating the need for other caches which are larger but are slower to access. Therefore, many computer systems have backup caches, or B-caches 109, which may reside on-chip or off-chip, and which serve as intermediary caches between the first-level caches and main memory 113 or other storage. When needed, instructions or data are retrieved from the first-level caches 103, 105, or from lower levels of the memory hierarchy, as available.
Fetching hardware 108 fetches instructions or data typically as the need arises. Such fetches are denoted hereafter as “demand requests”. In addition, modem CPUs utilize hardware prefetching techniques to reduce the impact of the long latency between the CPU and main memory. The CPU prefetcher 107 predicts which memory blocks the CPU will utilize in the near future and requests those blocks from the memory hierarchy before they are actually needed. The CPU cannot be certain ahead of time which instructions and data it will need in the future. For instruction prefetch, this uncertainty is typically due to branch instructions in the code. This is referred to as “speculative prefetching”. Because the results are not needed immediately, timing is less critical with prefetching requests than with demand requests.
Unfortunately, prefetching consumes significant bandwidth to the memory hierarchy, thereby reducing the amount of available bandwidth for memory transactions that are needed as soon as possible, such as for demand misses, cache-victim processing, etc. This can be a problem even when the priority of the prefetch requests has been lowered to below that of other demand misses, because, for example, the prefetch cycle could have already begun. Thus there is a need for a mechanism which dynamically detects when a high percentage of the bandwidth is needed for high-priority accesses such as demand misses, and which shuts down the hardware prefetcher until the demand for bandwidth decreases.
FIG. 2 is a schematic diagram of a preferred embodiment of the present invention. Instruction fetching hardware 108 is responsible for fetching instructions from memory, e.g., the B-cache 109. Within the fetching hardware 108 is the hardware prefetcher 107 which is responsible for speculative prefetching. Of course, there could be multiple hardware prefetchers, for example, for instructions and for data.
The present invention works on the principle that if bandwidth utilization for high-priority requests has been high in the recent past, then it will continue to be high in the near future. When recent bandwidth utilization has been high for high-priority requests, speculative requests from the hardware prefetcher 107 are prohibited from making requests.
Information about the recent history of memory bandwidth utilization of a resource, such as a B-cache 109, is captured using a shift register 115 of arbitrary length N. Each bit 116 in the shift register 115 represents one usage opportunity for the memory resource. The fetcher 108 outputs a signal 119 which is indicative of resource usage. For example, if a high-priority demand request was made in the last opportunity, the fetcher 108 shifts a 1 into the register 115 via signal 119. Otherwise, the fetcher 108 shifts a 0 into the register 115. If the register 115 contains N bits, then it holds information about the last N usage opportunities. When the register 115 is full, the oldest information is shifted out of the register 115 as new information is shifted in. To avoid unnecessarily inhibiting the prefetcher 107, low-priority utilizations such as prefetches are preferably not recorded in the shift register 115 as consuming a usage opportunity.
In the present invention, prefetch requests are inhibited depending on some function based upon the contents of the shift register 115. Preferably, this function is a count of the bits 116 in the shift register 115 that are set to 1. This is represented schematically in FIG. 2 by counting logic 117 whose output 125 represents the number of 1 s in the shift register 115. This count 125 is compared by comparator 121 to some threshold 127, and if the count 125 exceeds the threshold 127, the comparator 121 generates a choke signal 123 which inhibits the prefetcher 107. Otherwise prefetching is not inhibited.
FIG. 3 is a schematic diagram of the embodiment similar to that of FIG. 2 in which the bits 116 of the shift register 115 are individually assigned weights 129. Thus, the counting logic 117 in FIG. 3 may give more weight to some bits 116 than others, for example, counting some bits 116 more than once. Preferably, more weight is given to more recent bits 120 in the register 115 than to the older bits 122. The ultimate output of counter logic 117 and operation of the remaining parts is the same as in FIG. 2 as previously discussed.
FIG. 4 is a schematic diagram depicting the use of two thresholds 135—an upper threshold ThH and a lower threshold ThL —to characterize a preferred embodiment of the present invention with hysteresis around a nominal threshold. The illustrated thresholds ThL and ThH are used instead of threshold 127 Th in FIGS. 2 and 3. Selector 131 selects the upper threshold ThH when the comparator output 123 indicates that the usage has been low and that prefetching is allowed, and selects the lower threshold ThL when the comparator output 123 indicates that the usage has been high and prefetching has been turned off.
FIG. 5A is a schematic diagram of a preferred embodiment using two shift registers, SRA and SRB. Assume that logic in the fetching hardware 108 produces either a 0 or 1 to be shifted into shift register SRA, according to whether the resource was actually used at the last usage opportunity. As before, SRA can be of arbitrary length N.
The shift register SRA's input bit 201 and output bit 209 are brought to logic circuit 203. The logic circuit 203 has the following truth table:
SRA input 201
SRA output 209
No change - no action
a 1 has been dropped
a 1 has been added
No change - no action
The SHL and SHR output signals from the logic circuit 203 command the second shift register SRB to shift left or right respectively. When shift register SRB is shifted to the left via the SHL signal, a 0 is shifted in from the right 213. On the other hand, when register SRB is shifted to the right via the SHR signal, a 1 is shifted in from the left 215. Thus, both registers hold the same number of 1s and 0s, but while register SRA's bits are ordered historically, register SRB groups all of the 1s contiguously to the left side 215 and groups all of the 0s contiguously to the right side 213. Of course, it is obvious to those skilled in the art that the terms left and right are used to differentiate one from the other, and may just as easily be reversed.
Referring to the above logic truth table, when the input and output bits 201, 209 to shift register SRA are the same, i.e., both 0s or both 1s, there is no change to the total number of 1s in the register, and so no action is taken with respect to shift register SRB. However, when the input 201 is a 0 and the output bit 209 is a 1, shift register SRA is losing a 1. Therefore register SRB is shifted left. Note again that when SRB is shifted left, a 0 is shifted in. Finally, when the input bit 201 is a 1 and the output bit 209 is a 0, shift register SRA is gaining a 1, and register SRB is shifted right, with a 1 shifting in from the left 215.
Now, to see if there are more than n actual uses out of the last N opportunities where n is the value of the threshold, i.e., to see if there are more than n 1s in shift register SRA or equivalently, register SRB, the hardware only needs to look at predesignated bit n of register SRB. Predesignated bit n is the n+1st bit from the left 215 end of register SRB as illustrated in FIG. 5 where the leftmost bit is 0. No counting is required. If the value 205 of register SRB bit n is 1, the threshold has been exceeded. In turn, prefetcher 107 is inhibited. Otherwise, when the value 205 of bit n is 0, the threshold is not met and prefetcher 107 remains active.
FIG. 5B shows an alternate embodiment of the present invention in which a counter 225 and comparator 227 perform essentially the same function as the second shift register SRB of FIG. 5A. The same logic circuit 203 is used, however, its output SHL and SHR signals become decrement and increment signals respectively to the counter 225, which keeps a running tally of the number of 1s in register SRA. As with the embodiment of FIG. 2, the counter's value 226 is compared with a threshold at comparator 227, which generates a choke signal 123 to inhibit the prefetcher 107.
It will be obvious to those skilled in the art that the above use of two shift registers with the logic described above can be used to measure bandwidth for other goals as well as throttling prefetching, e.g. performance evaluation with respect to any type of resource usage. The roles of 0s and 1s could similarly be reversed.
In another embodiment of the present invention, a thread is selected based on bandwidth, or prioritization among the threads.
While this invention has been particularly shown and described with references to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Those skilled in the art will recognize or be able to ascertain using no more than routine experimentation, many equivalents to the specific embodiments of the invention described specifically herein. Such equivalents are intended to be encompassed in the scope of the claims.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US4620278 *||Aug 29, 1983||Oct 28, 1986||Sperry Corporation||Distributed bus arbitration according each bus user the ability to inhibit all new requests to arbitrate the bus, or to cancel its own pending request, and according the highest priority user the ability to stop the bus|
|US5421020||Jan 8, 1993||May 30, 1995||International Business Machines Corporation||Counter register implementation for speculative execution of branch on count instructions|
|US5421022||Jun 17, 1993||May 30, 1995||Digital Equipment Corporation||Apparatus and method for speculatively executing instructions in a computer system|
|US5454117||Aug 25, 1993||Sep 26, 1995||Nexgen, Inc.||Configurable branch prediction for a processor performing speculative execution|
|US5506989||Jan 31, 1990||Apr 9, 1996||Ibm Corporation||Arbitration system limiting high priority successive grants|
|US5561776||Jun 6, 1995||Oct 1, 1996||Hyundai Electronics America||Processor architecture supporting multiple speculative branching|
|US5613083||Sep 30, 1994||Mar 18, 1997||Intel Corporation||Translation lookaside buffer that is non-blocking in response to a miss for use within a microprocessor capable of processing speculative instructions|
|US5625837||Jun 6, 1995||Apr 29, 1997||Hyundai Electronics America||Processor architecture having out-of-order execution, speculative branching, and giving priority to instructions which affect a condition code|
|US5627983||Jun 6, 1995||May 6, 1997||Hyundai Electronics America||Processor architecture providing out-of-order execution|
|US5754800||May 16, 1995||May 19, 1998||Seiko Epson Corporation||Multi processor system having dynamic priority based on row match of previously serviced address, number of times denied service and number of times serviced without interruption|
|US6035392||Feb 20, 1998||Mar 7, 2000||International Business Machines Corporation||Computer with optimizing hardware for conditional hedge fetching into cache storage|
|US6073132||Mar 27, 1998||Jun 6, 2000||Lsi Logic Corporation||Priority arbiter with shifting sequential priority scheme|
|US6073159||Dec 31, 1996||Jun 6, 2000||Compaq Computer Corporation||Thread properties attribute vector based thread selection in multithreading processor|
|EP0651324A1||Sep 20, 1994||May 3, 1995||Advanced Micro Devices Inc.||Speculative instruction queue for variable byte length instructions|
|EP0738962A2||Mar 15, 1996||Oct 23, 1996||International Business Machines Corporation||Computer processing unit employing aggressive speculative prefetching of instruction and data|
|EP0763793A2||Aug 20, 1996||Mar 19, 1997||International Business Machines Corporation||Cache controlled instruction pre-fetching|
|1||Appelbe, et al., "Hoisting Branch Conditions-Improving Super-Scalar Processor Performance," Proc. of Languages and Compilers for Parallel Computing, 8th International Workshop, LCPC, pp. 304-317, (1996).|
|2||Appelbe, et al., "Hoisting Branch Conditions—Improving Super-Scalar Processor Performance," Proc. of Languages and Compilers for Parallel Computing, 8th International Workshop, LCPC, pp. 304-317, (1996).|
|3||Chang, P-Y, et al., "Improving Branch Prediction Accuracy by Reducing Pattern History Table Interference," Proc. 1996 Conference on Parallel Architectures and Compilation Techniques (PACT' 96) Boston, MA, pp. 48-57, (Oct. 20-23, 1996).|
|4||Gonzales, J., and Gonzalez, A., "Speculative Execution via Address Prediction and Data Prefetching," Proc. of International Conference on Supercomputing 1997, ACM, NY, pp. 196-203 (1997).|
|5||Lee, D. and Baer, J-L., "Instruction Cache Fetch Policies for Speculative Execution," Proc. of 22nd Annual International Symposium on Computer Architecture, pp. 357-367 (1995).|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US6625744 *||Nov 19, 1999||Sep 23, 2003||Intel Corporation||Controlling population size of confidence assignments|
|US6918010 *||Oct 16, 2002||Jul 12, 2005||Silicon Graphics, Inc.||Method and system for prefetching data|
|US7054968 *||Sep 16, 2003||May 30, 2006||Denali Software, Inc.||Method and apparatus for multi-port memory controller|
|US7102997||Mar 4, 2002||Sep 5, 2006||Fujitsu Limited||Aggregate rate transparent LAN service for closed user groups over optical rings|
|US7712101||Sep 24, 2004||May 4, 2010||Imagination Technologies Limited||Method and apparatus for dynamic allocation of resources to executing threads in a multi-threaded processor|
|US7747806 *||Jun 1, 2007||Jun 29, 2010||Panasonic Corporation||Resource use management device, resource use management system, and control method for a resource use management device|
|US7752400 *||Dec 14, 2000||Jul 6, 2010||F5 Networks, Inc.||Arbitration and crossbar device and method|
|US8314807 *||Sep 16, 2010||Nov 20, 2012||Apple Inc.||Memory controller with QoS-aware scheduling|
|US8490102||Jul 29, 2004||Jul 16, 2013||International Business Machines Corporation||Resource allocation management using IOC token requestor logic|
|US8510521||Sep 16, 2010||Aug 13, 2013||Apple Inc.||Reordering in the memory controller|
|US8553042||Oct 16, 2012||Oct 8, 2013||Apple Inc.||QoS-aware scheduling|
|US8631213||Sep 16, 2010||Jan 14, 2014||Apple Inc.||Dynamic QoS upgrading|
|US8762653||Oct 24, 2013||Jun 24, 2014||Apple Inc.||Dynamic QoS upgrading|
|US9053058||Dec 20, 2012||Jun 9, 2015||Apple Inc.||QoS inband upgrade|
|US9135072||Sep 4, 2013||Sep 15, 2015||Apple Inc.||QoS-aware scheduling|
|US9229896||Dec 21, 2012||Jan 5, 2016||Apple Inc.||Systems and methods for maintaining an order of read and write transactions in a computing system|
|US9507596 *||Aug 28, 2014||Nov 29, 2016||Intel Corporation||Instruction and logic for prefetcher throttling based on counts of memory accesses to data sources|
|US20030165146 *||Mar 4, 2002||Sep 4, 2003||Robert Sultan||Aggregate rate transparent LAN service for closed user groups over optical rings|
|US20050060456 *||Sep 16, 2003||Mar 17, 2005||Denali Software, Inc.||Method and apparatus for multi-port memory controller|
|US20050235126 *||Sep 24, 2004||Oct 20, 2005||Ko Yin N||Dynamic load balancing|
|US20060026598 *||Jul 29, 2004||Feb 2, 2006||International Business Machines Corporation||Resource allocation management|
|US20060048158 *||Jul 29, 2004||Mar 2, 2006||International Business Machines Corporation||Methods and apparatus for aging a command|
|US20060123003 *||Dec 8, 2004||Jun 8, 2006||International Business Machines Corporation||Method, system and program for enabling non-self actuated database transactions to lock onto a database component|
|US20070204267 *||Feb 28, 2006||Aug 30, 2007||Cole Michael F||Throttling prefetching in a processor|
|US20070283065 *||Jun 1, 2007||Dec 6, 2007||Ryuji Fuchikami||Resource use management device, resource use management system, and control method for a resource use management device|
|US20080091866 *||Oct 12, 2006||Apr 17, 2008||International Business Machines Corporation||Maintaining forward progress in a shared L2 by detecting and breaking up requestor starvation|
|US20120069034 *||Sep 16, 2010||Mar 22, 2012||Sukalpa Biswas||QoS-aware scheduling|
|US20160062768 *||Aug 28, 2014||Mar 3, 2016||Intel Corporation||Instruction and logic for prefetcher throttling based on data source|
|U.S. Classification||710/244, 710/116, 711/151, 710/40, 712/E09.055, 712/E09.047|
|International Classification||G06F11/34, G06F9/50, G06F9/38|
|Cooperative Classification||G06F9/5016, G06F9/3802, G06F11/3409, G06F2201/88, G06F9/383, G06F2209/507, G06F2209/5021|
|European Classification||G06F9/38B, G06F9/38D2, G06F9/50A2M|
|Apr 13, 2001||AS||Assignment|
Owner name: COMPAQ COMPUTER CORPORATION, TEXAS
Free format text: MERGER;ASSIGNOR:DIGITAL EQUIPMENT CORPORATION;REEL/FRAME:011704/0119
Effective date: 19991209
|May 27, 2005||FPAY||Fee payment|
Year of fee payment: 4
|May 27, 2009||FPAY||Fee payment|
Year of fee payment: 8
|Sep 15, 2011||AS||Assignment|
Owner name: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P., TEXAS
Free format text: CHANGE OF NAME;ASSIGNOR:COMPAQ INFORMATION TECHNOLOGIES GROUP, L.P.;REEL/FRAME:026909/0243
Effective date: 20021001
Owner name: COMPAQ INFORMATION TECHNOLOGIES GROUP, L.P., TEXAS
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:COMPAQ COMPUTER CORPORATION;REEL/FRAME:026909/0193
Effective date: 20010531
|Jul 5, 2013||REMI||Maintenance fee reminder mailed|
|Nov 27, 2013||LAPS||Lapse for failure to pay maintenance fees|
|Jan 14, 2014||FP||Expired due to failure to pay maintenance fee|
Effective date: 20131127