|Publication number||US7103735 B2|
|Application number||US 10/722,884|
|Publication date||Sep 5, 2006|
|Filing date||Nov 26, 2003|
|Priority date||Nov 26, 2003|
|Also published as||CN1645341A, CN100377113C, US20050114605|
|Publication number||10722884, 722884, US 7103735 B2, US 7103735B2, US-B2-7103735, US7103735 B2, US7103735B2|
|Inventors||Ravishankar R. Iyer|
|Original Assignee||Intel Corporation|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (8), Non-Patent Citations (1), Referenced by (18), Classifications (7), Legal Events (4)|
|External Links: USPTO, USPTO Assignment, Espacenet|
The present disclosure relates generally to memory within processor systems, and more particularly, to methods and apparatus to process cache allocation requests based on priority.
Typically, cache is memory that a processor may access more quickly than random access memory (RAM) on a main memory chip. Cache may be identified based on how close and accessible a memory is to the processor. For example, a first-level unified (L1) cache may reside on the same chip as the processor. When the processor executes an instruction, for example, the processor first looks at its on-chip cache to find the data associated with that instruction to avoid performing a more time-consuming search for the data elsewhere (e.g., off-chip or on a RAM on a main memory chip).
Caches implemented in current processor systems are typically unaware of how cache lines are allocated to multiple incoming application streams. When a processor issues a load/store request for a data block in a cache, for example, the processor only checks for the data block in the cache. That is, if the data block is not in the cache, the cache controller issues a request to the main memory. Upon receiving a response from the main memory, the cache controller allocates the data block into the cache.
In processor systems employing multi-threaded cores, multi-core processors, multi-tasked cores, and/or virtualized cores, multiple incoming application streams may interfere with each other and as a result, may cause a shared cache to operate inefficiently. With multiple incoming application streams sharing cache space with equal priority often results in sub-optimal allocation of cache resources to the more important memory intensive application(s).
Although the following discloses example systems including, among other components, software or firmware executed on hardware, it should be noted that such systems are merely illustrative and should not be considered as limiting. For example, it is contemplated that any or all of the disclosed hardware, software, and/or firmware components could be embodied exclusively in hardware, exclusively in software, exclusively in firmware or in some combination of hardware, software, and/or firmware.
In the examples of
As noted above, the priority assignment unit 110 assigns a priority level to each of the cache allocation requests 210 of incoming application streams received by the cache controller 120. The priority assignment unit 110 may be implemented using an operating system (OS). For example, the OS may assign a priority level to an application thread running in a processor (e.g., the processor 1020 of
Regardless of the specific manner in which the priority assignment unit 110 is implemented, the priority assignment unit 110 assigns the priority level based on the nature of the cache allocation requests 210. For example, the priority assignment unit 110 may assign a higher priority to instructions than data, a higher priority to data loads than data stores, and/or a higher priority to demand requests than prefetches and speculative memory references. The priority assignment unit 110 may also assign the priority level based on the cache miss ratio of a particular thread. In other words, the priority assignment unit 110 may maintain a counter to track the number of cache allocation requests 210 and the miss rate for each cache. Based on the counter values, the priority assignment unit 110 may assign a higher priority to threads with higher cache allocation request rates, higher cache miss ratios, and/or higher hit ratios. Further, the priority assignment unit 110 may assign the priority level based on the origin of the incoming application streams (i.e., type of the source of the incoming application streams). If the cache 130 is a central processing unit (CPU) cache, for example, the priority assignment unit 110 may assign a higher priority to a CPU-initiated cache allocation request than other devices such as a memory controller and/or a network device that is attempting to send (i.e., push) data into the cache 130.
In the example priority table 300 of
The cache controller 120 assigns an allocation probability (AP) to each priority level. In the priority table 300, for example, the priority level P1 (i.e., the highest priority level) may be assigned an AP of 100%, the priority level P2 may be assigned an AP of 75%, the priority level P3 may be assigned an AP of 50%, and the priority level P4 (i.e., the lowest priority level) may be assigned an AP of 25%. For each thread requesting allocation, the cache controller 120 generates a random number between zero (0) and one hundred (100), and compares the random number to the AP assigned to each of the cache allocation requests 210. If the AP is greater than the random number, then the cache controller 120 allows allocation of the cache lines in the cache 130 to the requesting thread. On the other hand, if the AP is less than or equal to the random number, the cache controller 120 denies the allocation of cache lines in the cache 130 to the requesting thread.
For applications with different characteristics in different execution phases, the priority assignment unit 110 may dynamically change the priority levels assigned to the cache allocation requests 210. After a set of priority levels is established (e.g., as shown in
In particular, the cache controller 120 may assign APs to each of the cache allocation requests 210 based on time intervals. In the example priority table 400 of FIG. 4, the cache controller 120 assigns APs based on time intervals. For example, the cache controller 120 may assign a series of time intervals (T1, T2, T3, and T4) to each of the cache allocation requests 210. Each of the time intervals T1, T2, T3, and T4 may be, for example, one millisecond (1 ms). Initially, the cache controller 120 places the incoming application stream in interval T1 and moves into interval T2 after the elapsed time T1, and cycles through each of the intervals so that using the priority table 400, the incoming application stream returns to interval T1 after the interval T4 is completed. Thus, the AP of the incoming application stream is based on the priority level and the current time interval. Specifically, the AP may be a function of the priority level (P) and the current time interval (T) so that AP=(1/P)×(1/T)×100.
As noted above, the cache 130 may be shared by multiple incoming application streams. Persons of ordinary skill in the art will readily recognize that the cache 130 includes the cache lines 230 (i.e., data) and tags (i.e., state information and/or replacement information). The memory space of the cache 130 may be partitioned so that each partition is assigned a particular priority level. Then, when a cache allocation request is made to a particular memory address, the priority assignment unit 110 assigns a priority level to that cache allocation request by comparing the address location to a memory range lookup table to identify the priority level corresponding to that particular memory address.
While the stream types, the priority levels, the time intervals, and the APs of the priority tables 300 and 400 shown in
Referring back to
A flow diagram 500 representing one manner in which the cache allocation system 100 of
The flow diagram 500 begins with the priority assignment unit 110 assigning a priority level to each of the cache allocation requests 210 from multiple incoming application streams (block 510). For example, the priority assignment unit 110 may assign a priority level to the cache allocation request based on a stream type of the incoming application stream, a source type of the incoming application stream, and/or any other suitable methods as described in detail above. Based on the priority table 300, for example, the cache controller 120 identifies an AP corresponding to the priority level of each of the cache allocation requests 210 (block 520). The cache controller 120 may also identify an AP corresponding to the priority level and the current time interval of each of the cache allocation requests 210 using the priority table 300 and/or the priority table 400. To determine whether the cache allocation request 210 should be allowed or denied, the cache controller 120 identifies each of the cache allocation requests 210 using either an allocate condition or a bypass condition based on the AP. For example, the cache controller 120 may generate a random number (block 530), and compare the AP with the random number (block 540). Alternatively, the cache controller 120 may compare the AP to a pre-determined number. If the AP is greater than the random number (or the pre-determined number), the cache controller 120 identifies one of the cache allocation requests 210 with an allocate condition, and allocates a portion of the cache 130 (e.g., one or more cache lines 230) to the cache allocation request 210 (block 550). In contrast, if the AP is less than or equal to the random number (or the pre-determined number), the cache controller 120 identifies one of the cache allocation requests 210 with a bypass condition, and denies the cache allocation request (block 560). As a result, cache efficiency and performance are increased when multiple applications are sharing the cache 130.
The processor system 1000 illustrated in
As is conventional, the memory controller 1012 performs functions that enable the processor 1020 to access and communicate with a main memory 1030 including a volatile memory 1032 and a non-volatile memory 1034 via a bus 1040. The volatile memory 1032 may be implemented by Synchronous Dynamic Random Access Memory (SDRAM), Dynamic Random Access Memory (DRAM), RAMBUS Dynamic Random Access Memory (RDRAM), and/or any other type of random access memory device. The non-volatile memory 1034 may be implemented using flash memory, Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), and/or any other desired type of memory device.
The processor system 1000 also includes an interface circuit 1050 that is coupled to the bus 1040. The interface circuit 1050 may be implemented using any type of well known interface standard such as an Ethernet interface, a universal serial bus (USB), a third generation input/output interface (3GIO) interface, and/or any other suitable type of interface.
One or more input devices 1060 are connected to the interface circuit 1050. The input device(s) 1060 permit a user to enter data and commands into the processor 1020. For example, the input device(s) 1060 may be implemented by a keyboard, a mouse, a touch-sensitive display, a track pad, a track ball, an isopoint, and/or a voice recognition system.
One or more output devices 1070 are also connected to the interface circuit 1050. For example, the output device(s) 1070 may be implemented by display devices (e.g., a light emitting display (LED), a liquid crystal display (LCD), a cathode ray tube (CRT) display, a printer and/or speakers). The interface circuit 1050, thus, typically includes, among other things, a graphics driver card.
The processor system 1000 also includes one or more mass storage devices 1080 to store software and data. Examples of such mass storage device(s) 1080 include floppy disks and drives, hard disk drives, compact disks and drives, and digital versatile disks (DVD) and drives.
The interface circuit 1050 also includes a communication device such as a modem or a network interface card to facilitate exchange of data with external computers via a network. The communication link between the processor system 1000 and the network may be any type of network connection such as an Ethernet connection, a digital subscriber line (DSL), a telephone line, a cellular telephone system, a coaxial cable, etc.
Access to the input device(s) 1060, the output device(s) 1070, the mass storage device(s) 1080 and/or the network is typically controlled by the I/O controller 1014 in a conventional manner. In particular, the I/O controller 1014 performs functions that enable the processor 1020 to communicate with the input device(s) 1060, the output device(s) 1070, the mass storage device(s) 1080 and/or the network via the bus 1040 and the interface circuit 1050.
While the components shown in
The methods and apparatus disclosed herein are particularly well suited for use in a processor cache. However, persons of ordinary skill in the art will appreciate that the teachings of the disclosure may be applied to process cache allocation requests in other suitable environments.
Although certain example methods, apparatus, and articles of manufacture have been described herein, the scope of coverage of this patent is not limited thereto. On the contrary, this patent covers all methods, apparatus, and articles of manufacture fairly falling within the scope of the appended claims either literally or under the doctrine of equivalents.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US5539893 *||Nov 16, 1993||Jul 23, 1996||Unisys Corporation||Multi-level memory and methods for allocating data most likely to be used to the fastest memory level|
|US5875464 *||Mar 18, 1996||Feb 23, 1999||International Business Machines Corporation||Computer system with private and shared partitions in cache|
|US5913215 *||Feb 19, 1997||Jun 15, 1999||Seymour I. Rubinstein||Browse by prompted keyword phrases with an improved method for obtaining an initial document set|
|US5983313 *||Apr 10, 1996||Nov 9, 1999||Ramtron International Corporation||EDRAM having a dynamically-sized cache memory and associated method|
|US6601151 *||Feb 8, 1999||Jul 29, 2003||Sun Microsystems, Inc.||Apparatus and method for handling memory access requests in a data processing system|
|US6857046 *||Mar 28, 2002||Feb 15, 2005||Cisco Technology, Inc.||Caching for context switching applications|
|US20030154112 *||Oct 29, 2002||Aug 14, 2003||Steven Neiman||System and method for allocating computing resources|
|US20040193827 *||Aug 5, 2003||Sep 30, 2004||Kazuhiko Mogi||Computer system for managing performances of storage apparatus and performance management method of the computer system|
|1||Chiou et al., Application-Specific Memory Management for Embedded Systems Using Software-Controlled Caches, Proceedings of the 37<SUP>th </SUP>Conference on Design Automation, pp. 416-419 (2000).|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7624257||Nov 30, 2005||Nov 24, 2009||International Business Machines Corporation||Digital data processing apparatus having hardware multithreading support including a register set reserved for special class threads|
|US7769955 *||Apr 27, 2007||Aug 3, 2010||Arm Limited||Multiple thread instruction fetch from different cache levels|
|US8024731 *||Apr 25, 2007||Sep 20, 2011||Apple Inc.||Assigning priorities to threads of execution|
|US8250302 *||Oct 1, 2008||Aug 21, 2012||Hewlett-Packard Development Company, L.P.||Cache management using sampled values assigned to a request|
|US8250347 *||Oct 24, 2007||Aug 21, 2012||International Business Machines Corporation||Digital data processing apparatus having hardware multithreading support including cache line limiting mechanism for special class threads|
|US8255915||Oct 26, 2007||Aug 28, 2012||Hewlett-Packard Development Company, L.P.||Workload management for computer system with container hierarchy and workload-group policies|
|US8407705 *||Aug 16, 2011||Mar 26, 2013||Apple Inc.||Assigning priorities to threads of execution|
|US8566491||Jul 19, 2011||Oct 22, 2013||Qualcomm Incorporated||System and method for improving throughput of data transfers using a shared non-deterministic bus|
|US8595451||Nov 4, 2010||Nov 26, 2013||Lsi Corporation||Managing a storage cache utilizing externally assigned cache priority tags|
|US8848731||Jul 19, 2011||Sep 30, 2014||Qualcomm Incorporated||System and method for facilitating data transfer using a shared non-deterministic bus|
|US8886918||Nov 28, 2007||Nov 11, 2014||International Business Machines Corporation||Dynamic instruction execution based on transaction priority tagging|
|US20070124568 *||Nov 30, 2005||May 31, 2007||International Business Machines Corporation||Digital data processing apparatus having asymmetric hardware multithreading support for different threads|
|US20080040583 *||Oct 24, 2007||Feb 14, 2008||International Business Machines Corporation||Digital Data Processing Apparatus Having Asymmetric Hardware Multithreading Support for Different Threads|
|US20080270758 *||Apr 27, 2007||Oct 30, 2008||Arm Limited||Multiple thread instruction fetch from different cache levels|
|US20090138682 *||Nov 28, 2007||May 28, 2009||Capps Jr Louis B||Dynamic instruction execution based on transaction priority tagging|
|US20090138683 *||Nov 28, 2007||May 28, 2009||Capps Jr Louis B||Dynamic instruction execution using distributed transaction priority registers|
|US20100082907 *||Oct 1, 2008||Apr 1, 2010||Vinay Deolalikar||System For And Method Of Data Cache Managment|
|US20110302588 *||Aug 16, 2011||Dec 8, 2011||Apple Inc.||Assigning Priorities to Threads of Execution|
|U.S. Classification||711/158, 711/138, 711/E12.021|
|International Classification||G06F12/08, G06F12/00|
|Dec 19, 2003||AS||Assignment|
Owner name: INTEL CORPORATION, CALIFORNIA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:IYER, RAVISHANKAR R.;REEL/FRAME:014808/0470
Effective date: 20031125
|Feb 27, 2007||CC||Certificate of correction|
|Feb 19, 2010||FPAY||Fee payment|
Year of fee payment: 4
|Feb 19, 2014||FPAY||Fee payment|
Year of fee payment: 8