|Publication number||US7343595 B2|
|Application number||US 10/782,674|
|Publication date||Mar 11, 2008|
|Filing date||Feb 19, 2004|
|Priority date||Aug 30, 2003|
|Also published as||US7926044, US20050050517, US20080104372|
|Publication number||10782674, 782674, US 7343595 B2, US 7343595B2, US-B2-7343595, US7343595 B2, US7343595B2|
|Inventors||Adam D Iley, John J R Scott|
|Original Assignee||International Business Machines Corporation|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (21), Non-Patent Citations (12), Referenced by (8), Classifications (10), Legal Events (6)|
|External Links: USPTO, USPTO Assignment, Espacenet|
The invention relates to speculative pre-execution of portions of a computer program.
Computers have proliferated into all aspects of society and in today's increasingly competitive market-place, the performance of not only the machines themselves but also the software that runs on these machines, is of the utmost importance. Software developers are therefore continually looking for methods to improve the execution efficiency of the code (programs) they produce in order to meet the high expectations of software users.
One such method is by inserting pre-execution instructions into source code such that execution of such instructions cause a portion of the program defined by the source code to be pre-executed. This is described in U.S. Patent Application Publication U.S. 2002/0055964.
Further, U.S. Patent Application Publication U.S. 2002/0144083 describes a processor using spare hardware contexts to spawn speculative threads such that data is pre-fetched in advance of a main thread.
Another known method is “branch prediction” (also mentioned in U.S. 2002/0055964). Within a program there are typically a number of branch points. These are points which can return one of a finite number of results. Prediction techniques are used to determine the likely return result such that a branch point's subsequent instructions can be pre-executed on this assumption. “if . . . else” statements and “case” statements are two well known examples of branch points.
There are a number of branch prediction techniques known in the industry. Such techniques are common in RISC and processor architectures (e.g. The pSeries architecture).
Software schemes also exist. A paper “Static Correlated Branch Prediction” by Cliff Young and Michael D Smith (ACM Transactions on Programming Languages and Systems, Vol. 21, Issue 5 (September 1999). pages 1028-1075 describes how the repetitive behaviour in the trace of all conditional branches executed by a program can be exploited by a compiler. Another paper “A Comparative Analysis of Schemes for Correlated Branch Prediction” by Cliff Young, Michael D Smith and Nicholas Gloy (published in the Proceedings of the 22nd Annual International Symposium on Computer Architecture, June 1995) presents a framework that categorizes branch prediction schemes by the way in which they partition dynamic branches and by the kind of predictor they use.
The paper “Understanding Backward Slices of Performance Degrading Instructions” by C Zilles and G Sohi (published in the proceedings of the 27th Annual International Symposium on Computer Architecture (ISCA-2000), Jun. 12-14 2000) discusses the small fraction of static instructions whose behaviour cannot be anticipated using current branch predictors and caches. The paper analyses the dynamic instruction stream leading up to these performance degrading instructions to identify the operations necessary to execute them early.
Another paper “The Predictability of Computations that Produce Unpredictable Outcomes” by T Aamodt, A Moshovos and P Chow (an update of the paper that appeared in the Proceedings of the 5th Workshop on Multithreaded Execution, Architecture, and Compilation—pages 23-34, Austin, Tex., December 2001) studies the dynamic stream of slice traces that foil existing branch predictors and measures whether these slices exhibit repetition.
“Speculative Data-Driven Multithreading” by Amir Roth and Gurindar Sohi (appearing in the Proceedings of the 7th International Conference on High Performance Computer Architecture (HPCA-7), Jan. 22-24, 2001) describes the use of speculative data-driven multithreading (DDMT) for coping with mispredicted branches and loads that miss in the cache.
It is also known for the programmer to be able to provide branch prediction pragma.
Whilst branch prediction techniques are known, there is however a need in the industry for more efficient processing of software functions as opposed to branch points.
Accordingly the invention provides a method for executing a program comprising a function call and one or more subsequent instructions, the method comprising the steps of: processing, on a first thread, a function defined by the function call, the function having one or more programmer predefined typical return values; for each predefined return value, pre-processing, on an additional thread, the one or more subsequent instructions assuming that the function returned that pre-defined return value, thereby enabling said processor, on completion of processing said function, to make use of the pre-processing completed by the additional thread which used the actual return value.
Thus the present invention enables a programmer to define typical return values for a function such that the function can be pre-processed ahead of a main thread. Assuming that the function does actually return one of the predefined return values, performance can be much improved.
Note, preferably the additional threads operate in parallel.
Preferably the program comprises a plurality of subsequent instructions defining one or more additional functions and the plurality of subsequent instructions are pre-processed on each additional thread until a function is reached which is of external effect. Once such a function is reached by an additional thread that thread preferably blocks (waits) on said function until the actual return value is determined by the first thread.
Preferably each additional thread also blocks on reaching a function which is affected by an external event.
According to one aspect the invention provides an apparatus for executing a program comprising a function call and one or more subsequent instructions, the apparatus comprising: means for processing, on a first thread, a function defined by the function call, the function having one or more programmer predefined typical return values; means for pre-processing for each predefined return value, on an additional thread, the one or more subsequent instructions assuming that the function returned that pre-defined return value, thereby enabling said processor, on completion of processing said function, to make use of the pre-processing completed by the additional thread which used the actual return value.
The invention may be implemented in computer software.
According to another aspect, the invention provides a compiler for generating a computer program comprising a function call defining a function, having one or more programmer predefined typical return values, and one or more subsequent instructions, the compiler comprising means for generating executable code, said executable code for instructing a computer to process on a first thread the function and to pre-process, for each defined typical return value, on an additional thread the one or more subsequent instructions assuming that the function returned that pre-defined return value, thereby enabling said processor, on completion of processing said function, to make use of the pre-processing completed by the additional thread which used the actual return value.
It will be appreciated that the term compiler is intended to cover the whole compilation process optionally including linking.
A preferred embodiment of the present invention will now be described, by way of example only, and with reference to the following drawings:
It has been observed that within a program certain tasks (functions) require substantial amounts of processing time but frequently return the same result. In order to exploit this observation a new construct is preferably incorporated into existing programming languages. This construct enables programmers to mark certain functions as “restricted”. In this context, the keyword “restricted” preferably means that the marked function does not effect the global environment (e.g. by outputting to a file) and the syntax associated with the new keyword permits the values most commonly returned by the function to be specified by the programmer as part of the function's signature. Further preferably, a “restricted” function is not itself affected by the global environment. In other words, it always operates in the same way regardless of the results produced by other “restricted” functions.
From the code extract, it can be seen that the overdue function checks the status of each user's book to determine whether that book is: not yet due back at the library; is late back; or is very late back. If a user's book is not overdue, then the function does no processing in relation to that user. On the other hand, if a user's book is either late or very late, then the remind_late or remind_very_late function is called as appropriate.
Whilst the overdue function itself is thus relatively fast, both remind functions have long and complicated processing to do on behalf of the user in relation to which that function is called. This processing involves looking up the user's address; the name of the overdue book; the number of days the book is overdue by; and the list of those currently waiting for the book. If the book is very late, then the user's borrower history must also checked. Further, in both cases the outstanding fine has to be calculated and the appropriate letter text retrieved. All this information is then used to build an appropriate letter in memory for eventual dispatch to the user.
Whilst the processing of both remind functions is long and complicated, this processing also does not affect the global environment. Values are retrieved and held in volatile memory, but no data is inserted, updated, deleted or output to non-volatile memory, an external device etc. Thus these functions can also be marked as “restricted”, although in this instance it is not appropriate to associate either function with typical return values.
Once letters have been built in non-volatile memory for all user's with overdue books, then these letters are sent to the printer via the “send_to_printer” function. This function is not marked as “restricted” since it does effect the global environment.
The execution of code including the new “restricted” keyword will now be described with reference to
Note, as alluded to with reference to
Further, rather than spawning pre-execution threads, a thread pool may be used.
Thus by enabling the programmer to define functions with non-global effect/as not affected by the global environment and also typical return values for such functions, it is possible to speculatively pre-execute code. Assuming that the speculation proves correct, program execution performance can be dramatically improved—a pre-execution thread will have preferably performed the long and complicated processing in the background whilst the main thread is performing other tasks.
Note, in one embodiment the main thread is not finally terminated until it is verified that an appropriate pre-execution does exist. Indeed it may be the main thread that is responsible for terminating those pre-execution threads that are not associated with the correct return value.
Another example of a system in which the invention should prove useful is a menu system in which a program will display a number of menu options and then wait for the user to choose one. In accordance with the “restricted” construct defined by a preferred embodiment of the present invention, the programmer can define the options most likely to be selected and then the program can pre-execute each of those options as far as it can (i.e. until a global function is encountered).
As discussed above, the functionality of the present invention is preferably achieved by modification of existing programming languages. Executable programs are typically produced from compiled source code. The compilation process is thus modified such that the meaning of “restricted” keyword is understood and such that appropriate executable code is generated as a result of the compilation process.
Thus for completeness the operation of a compiler in accordance with a preferred embodiment of the present invention is described with reference to
A compiler 310 is provided with a program's source code 300 as input. The compiler processes this source code to produce object code 320 and this is then passed to a linker 330 which uses this code 320 to produce an executable 340.
Typically, there are three stages to the compilation process: lexical analysis; syntax analysis; and code generation. During the lexical analysis, symbols (e.g. alphabetic characters) are grouped together to form tokens. For example the characters P R I N T are grouped to form the command (token) PRINT. In some systems, certain keywords are replaced by shorter, more efficient tokens. This part of the compilation process also verifies that the tokens are valid.
In accordance with a preferred embodiment of the present invention, the lexical analyser is therefore modified to recognise “restricted” as a keyword and also to recognise expected return values when the programmer provides them.
Next, the syntax analyser checks whether each string of tokens forms a valid sentence. Again the syntax analyser is preferably modified to recognise that “restricted” keyword and the predefined typical return values are valid.
Finally, the code generation stage produces the appropriate object code. The code generator is thus also preferably modified to recognise the new “restricted” construct such that the appropriate object code is generated for any program employing the new construct (i.e. to achieve the result discussed with reference to
It is assumed that a person skilled in the art of compiler development will be familiar with the above process and thus this will not be discussed in any further detail.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US5669000 *||Jan 31, 1995||Sep 16, 1997||Apple Computer, Inc.||Interpreter for performing remote testing of computer systems|
|US5751981||Feb 9, 1996||May 12, 1998||Advanced Micro Devices, Inc.||High performance superscalar microprocessor including a speculative instruction queue for byte-aligning CISC instructions stored in a variable byte-length format|
|US5913925 *||Dec 16, 1996||Jun 22, 1999||International Business Machines Corporation||Method and system for constructing a program including out-of-order threads and processor and method for executing threads out-of-order|
|US6173442 *||Feb 5, 1999||Jan 9, 2001||Sun Microsystems, Inc.||Busy-wait-free synchronization|
|US6367070 *||Jan 13, 1998||Apr 2, 2002||Intel Corporation||Means and method for establishing loop-level parallelism|
|US6425120 *||Jan 14, 2000||Jul 23, 2002||Softwire Technology Llc||Repeating program object for use with a graphical program-development system|
|US6425121 *||Jan 14, 2000||Jul 23, 2002||Softwire Technology, Llp||Method and apparatus for resolving divergent paths in graphical programming environments|
|US6539541 *||Aug 20, 1999||Mar 25, 2003||Intel Corporation||Method of constructing and unrolling speculatively counted loops|
|US6634023 *||Jun 16, 1999||Oct 14, 2003||International Business Machines Corporation||Compile method, exception handling method and computer|
|US6760903 *||Aug 22, 2000||Jul 6, 2004||Compuware Corporation||Coordinated application monitoring in a distributed computing environment|
|US6931631 *||Jun 27, 2001||Aug 16, 2005||International Business Machines Corporation||Low impact breakpoint for multi-user debugging|
|US6961925 *||Apr 3, 2001||Nov 1, 2005||Cray Inc.||Parallelism performance analysis based on execution trace information|
|US6961930 *||Mar 31, 2000||Nov 1, 2005||Hewlett-Packard Development Company, L.P.||Efficient, transparent and flexible latency sampling|
|US6964043 *||Oct 30, 2001||Nov 8, 2005||Intel Corporation||Method, apparatus, and system to optimize frequently executed code and to use compiler transformation and hardware support to handle infrequently executed code|
|US7010787 *||Mar 29, 2001||Mar 7, 2006||Nec Corporation||Branch instruction conversion to multi-threaded parallel instructions|
|US7082601 *||Jul 17, 2002||Jul 25, 2006||Nec Corporation||Multi-thread execution method and parallel processor system|
|US7117481 *||Nov 6, 2002||Oct 3, 2006||Vmware, Inc.||Composite lock for computer systems with multiple domains|
|US7143401 *||Feb 20, 2001||Nov 28, 2006||Elbrus International||Single-chip multiprocessor with cycle-precise program scheduling of parallel execution|
|US7152170 *||Jul 31, 2003||Dec 19, 2006||Samsung Electronics Co., Ltd.||Simultaneous multi-threading processor circuits and computer program products configured to operate at different performance levels based on a number of operating threads and methods of operating|
|US20020055964||Dec 18, 2001||May 9, 2002||Chi-Keung Luk||Software controlled pre-execution in a multithreaded processor|
|US20020144083||Mar 30, 2001||Oct 3, 2002||Hong Wang||Software-based speculative pre-computation and multithreading|
|1||Aamodt et al, "The Predictability of Computations that Product Unpredictable Outcomes", Dec. 2001, pp. 23-34.|
|2||*||Cui et al, "Parallel replacement mechanism for multithread", IEEE, pp. 338-344, 1997.|
|3||*||Li et al, "A framework of reachability testing for Java multithread programs", IEEE, pp. 2730-2734, 2004.|
|4||*||Manson et al, "Core semantics of multithreaded Java", ACM JAVA pp. 29-38, 2001.|
|5||*||Matsuzaki et al, "A multithread processor architecture based on the continuation model", IEEE IWIA, pp. 1-8, 2005.|
|6||Roth et al, "Speculative Data-Driven Multithreading", Jan. 2001, pp. 1-12.|
|7||*||Schwan et al, "Multiprocessor real time threads", ACM SIGOPS, vol. 26, issues 1, pp. 54-65, 1992.|
|8||*||Serrano et al, "Scheme fair threads", ACM PPDP, pp. 203-214, 2004.|
|9||*||Tardieu et al, "Scheduling independent threads and exceptions in SHIM" ACM EMSOFT, pp. 142-151, 2006.|
|10||Young et al, "Static Correlated Branch Prediction", 1999, pp. 111-159.|
|11||Young et al, A Comparative Analysis of Schemes for Correlated Branch Prediction, Jun. 1995, pp. 1-11.|
|12||Zilles et al, "Understanding the Backward Slices of Performance Degrading Instructions", Jun. 2000.|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7707543 *||Nov 22, 2005||Apr 27, 2010||Siemens Aktiengesellschaft||Architecture for a computer-based development environment with self-contained components and a threading model|
|US8423750 *||May 12, 2010||Apr 16, 2013||International Business Machines Corporation||Hardware assist thread for increasing code parallelism|
|US8650384 *||Apr 27, 2010||Feb 11, 2014||Samsung Electronics Co., Ltd.||Method and system for dynamically parallelizing application program|
|US9037837||Apr 3, 2012||May 19, 2015||International Business Machines Corporation||Hardware assist thread for increasing code parallelism|
|US9189277||Jul 22, 2013||Nov 17, 2015||Samsung Electronics Co., Ltd.||Method and system for dynamically parallelizing application program|
|US20060136921 *||Nov 22, 2005||Jun 22, 2006||Detlef Becker||Architecture for a computer-based development environment with self-contained components and a threading model|
|US20100281489 *||Apr 27, 2010||Nov 4, 2010||Samsung Electronics Co., Ltd.||Method and system for dynamically parallelizing application program|
|US20110283095 *||May 12, 2010||Nov 17, 2011||International Business Machines Corporation||Hardware Assist Thread for Increasing Code Parallelism|
|U.S. Classification||717/141, 717/149, 717/150, 712/E09.082|
|International Classification||G06F9/44, G06F9/45|
|Cooperative Classification||G06F8/445, G06F9/4425|
|European Classification||G06F8/445, G06F9/44F1A|
|Jun 22, 2004||AS||Assignment|
Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ILEY, ADAM D;SCOTT, JOHN J R;REEL/FRAME:014761/0388
Effective date: 20031112
|Oct 24, 2011||REMI||Maintenance fee reminder mailed|
|Mar 9, 2012||FPAY||Fee payment|
Year of fee payment: 4
|Mar 9, 2012||SULP||Surcharge for late payment|
|Apr 4, 2012||AS||Assignment|
Owner name: FACEBOOK, INC., CALIFORNIA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:INTERNATIONAL BUSINESS MACHINES CORPORATION;REEL/FRAME:027991/0554
Effective date: 20120327
|Aug 26, 2015||FPAY||Fee payment|
Year of fee payment: 8